Transform model-based skin cancer pathological image analysis system and method

Abstract

The invention discloses a skin cancer pathological image analysis system and method based on a Transform model. The system comprises an image acquisition and quality control module, a multi-scale preprocessing module, a hierarchical feature extraction module, an intelligent diagnostic reasoning module, a knowledge graph aided decision-making module, a result output and feedback module and a federal learning training module. According to the method, multi-spectral image acquisition and quality control are carried out, multi-modal enhancement and dyeing standardization preprocessing are carried out, pathological features are extracted in a layered manner by using an improved Transform model, subtype identification and malignancy degree evaluation are realized in combination with multi-task learning, uncertainty is quantified by means of Monte Carlo dropout, cases and guidelines are associated through a knowledge graph, privacy is protected through federal learning, and the model is optimized. According to the scheme, the diagnosis efficiency and accuracy are greatly improved, the model interpretability is enhanced, various clinical scenes are adapted, diagnosis standardization is promoted, and improvement of basic medical capacity is assisted.

Description

Technical Field

[0001] This invention relates to skin cancer, specifically to a skin cancer pathological image analysis system and method based on the Transformer model. Background Technology

[0002] Currently, pathological diagnosis of skin cancer relies on physicians manually examining pathological slides. Whole-slide digital image (WSI) analysis takes 30 minutes to 1 hour, and physicians are prone to decreased efficiency due to visual fatigue. Furthermore, pathological resources in primary hospitals are strained, making it difficult to cope with the large volume of diagnostic demands. Diagnostic accuracy is greatly affected by physician experience and subjective judgment. Different subtypes of tumor cells overlap in morphology, resulting in a misdiagnosis rate of 15%-20% for early-stage skin cancer, with an even higher risk of missed diagnosis in primary care institutions.

[0003] Medical data privacy protection requirements are stringent. Skin cancer pathology images contain sensitive information, and sharing data across institutions carries high compliance risks. Furthermore, the limited number of cases per institution makes it difficult to support continuous optimization of AI models, resulting in insufficient model generalization ability. Traditional AI models are often "black boxes," only outputting results without providing decision-making support, leading to low physician trust. Moreover, existing solutions have poor adaptability, failing to meet the needs of different hospitals, and lack standardization mechanisms. Inconsistent diagnostic standards across institutions limit the improvement of primary healthcare capabilities. These problems urgently need to be addressed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the above-mentioned technologies and provide a skin cancer pathological image analysis system and method based on the Transformer model.

[0005] To solve the above-mentioned technical problems, the present invention provides a skin cancer pathological image analysis system and method based on the Transformer model: The skin cancer pathological image analysis system based on the Transformer model is characterized by:

[0006] include:

[0007] The image acquisition and quality control module is used to acquire multispectral digital images of skin tissue pathological sections and to achieve image quality control through sharpness assessment, staining consistency verification and artifact detection.

[0008] The multi-scale preprocessing module is connected to the image acquisition and quality control module. It removes non-organic areas through adaptive threshold segmentation, processes the image by combining multi-modal enhancement algorithms (including illumination correction based on Retinex theory, noise suppression based on wavelet transform, and detail enhancement based on generative adversarial networks), and normalizes it into a sequence of image blocks of a preset size.

[0009] The hierarchical feature extraction module, connected to the multi-scale preprocessing module, includes:

[0010] The local feature encoding unit uses a convolutional neural network to extract the texture, morphology, and color distribution features of image patches;

[0011] The global correlation modeling unit adopts an improved Transformer model, which includes a dynamic deep self-attention mechanism (dynamically adjusting the number and depth of attention heads according to the importance of pathological regions) and a cross-scale feature fusion module (fusing features of different resolutions through skip connections).

[0012] The pathological feature aggregation unit aggregates multi-level features through attention-weighted pooling to generate a comprehensive feature vector that includes cellular atypia, tissue structure disorder, and invasion pattern.

[0013] The intelligent diagnostic reasoning module, connected to the hierarchical feature extraction module, includes:

[0014] A multi-task classifier based on comprehensive feature vectors to identify skin cancer subtypes (including melanoma, basal cell carcinoma, and squamous cell carcinoma), measure Breslow thickness, and perform Clark grading.

[0015] Uncertainty quantification unit, which calculates the confidence interval of the diagnostic results using Monte Carlo dropout;

[0016] The interpretability analysis unit generates an attention heatmap to locate key lesion areas and outputs a ranking of feature importance.

[0017] The knowledge graph-assisted decision-making module is connected to the intelligent diagnostic reasoning module and is used to associate historical case data, clinical indicators and literature knowledge to provide differential diagnosis basis and treatment suggestions.

[0018] The results output and feedback module is connected to the intelligent diagnostic reasoning module and the knowledge graph-assisted decision-making module. It is used to output a structured diagnostic report (including image tags, quantitative analysis results and diagnostic conclusions) and receive feedback from pathology experts to achieve continuous model optimization.

[0019] As an improvement, in the dynamic deep self-attention mechanism of the improved Transformer model, the attention weights are calculated using the following formula:

[0020]

[0021] Where, α i,j sim(q) represents the attention weight between the i-th query vector and the j-th key vector. i ,k j ) represents the query vector q i With key vector k jThe similarity is calculated using a scaled dot product, where n represents the sequence length. This formula converts the similarity into a normalized weight distribution using the softmax function, allowing the model to focus on associations of important pathological features.

[0022] As an improvement, the color space conversion in the staining normalization unit of the multi-scale preprocessing module uses the following formula:

[0023]

[0024] Among them, I src For the original stained image, I std For the standardized image, M src Let M be the color matrix of the original image, and M be the standard color matrix. This formula uses matrix transformation to convert different coloring styles to the standard style, eliminating coloring differences between laboratories.

[0025] As an improvement, in the similarity calculation unit of the knowledge graph-assisted decision-making module, the case similarity is calculated using the following formula:

[0026]

[0027] Where S(a,b) represents the similarity between case a and case b, f a f b Let be the feature vectors of two cases, · denotes the vector dot product, and ||·|| denotes the L2 norm of the vectors. This formula quantifies the similarity of case features using cosine similarity and is used to retrieve the most relevant historical cases.

[0028] As an improvement, a federated learning training module is also included, which comprises:

[0029] Local training units are deployed in various medical institutions to update model parameters based on local data.

[0030] The parameter aggregation unit fuses parameters from each node using a secure aggregation algorithm (based on homomorphic encryption) to protect data privacy.

[0031] The model adaptive unit dynamically adjusts the model regularization strength and learning rate based on the data distribution characteristics of each institution;

[0032] The global parameter update of the parameter aggregation unit uses the following formula:

[0033]

[0034] Where, θ global θ represents the aggregated global model parameters. i For the local model parameters of the i-th institution, w i The weight of the i-th institution (satisfying) (N is the number of participating institutions). This formula achieves collaborative optimization of the model in a federated scenario by merging the parameters of each institution through a weighted average.

[0035] A method for analyzing skin cancer pathological images based on the Transformer model includes the following steps:

[0036] S1. Image Acquisition and Quality Control:

[0037] S11. Use a whole-section scanner to acquire multispectral digital images of skin pathology sections (including 400-700nm band information);

[0038] S12. Select qualified images through a sharpness scoring algorithm (integrating gradient information and entropy calculation), remove abnormal staining samples through staining uniformity detection, and identify and mark artifacts such as folds and bubbles in the image through morphological operations;

[0039] S2. Multi-scale image preprocessing:

[0040] S21. Use the U-Net model to segment the tissue region and remove the background and slide edges;

[0041] S22. Perform the following steps in sequence: illumination non-uniformity correction based on bilateral filtering, noise removal based on non-local means, and detail enhancement based on super-resolution reconstruction.

[0042] S23. Perform color normalization processing to convert the image to a standard color space;

[0043] S24. Adaptively divide image blocks based on histopathological type to generate an image block sequence containing spatial location information;

[0044] S3. Hierarchical Feature Extraction:

[0045] S31. A pre-trained convolutional neural network is used to extract local pathological features (including cell nuclear morphology, cytoplasmic staining intensity, and intercellular connection features) from each image patch;

[0046] S32. Input local features into the improved Transformer model:

[0047] Location information is added through pathological spatial perception coding;

[0048] A dynamic deep self-attention mechanism (allocating more attention heads to the lesion region) is used to model global feature association;

[0049] Integrate features from different levels through a cross-scale fusion module;

[0050] S33. Attention-weighted pooling is used to aggregate features and generate a comprehensive feature vector;

[0051] S4. Intelligent Diagnostic Reasoning:

[0052] S41. Based on comprehensive feature vectors, this is achieved simultaneously through a multi-task learning framework:

[0053] Skin cancer subtype classification (multi-classification task);

[0054] Quantitative assessment of malignancy (regression task, output Breslow thickness value);

[0055] Invasion depth classification (orderly classify tasks, output Clark level);

[0056] S42. Calculate the confidence interval for each diagnostic result using Monte Carlo sampling;

[0057] S43. Generate an attention heatmap to mark key diagnostic regions and output a ranking of feature importance;

[0058] S5. Knowledge Graph-Assisted Decision Making:

[0059] S51. Search for similar cases in the case knowledge graph to obtain reference diagnostic results;

[0060] S52. Combine clinical indicators (such as patient age and lesion location) to perform rule-based reasoning and optimize diagnostic conclusions;

[0061] S6. Output a structured diagnostic report and receive expert feedback for model iteration and optimization.

[0062] As an improvement, the saliency score calculation for the dynamic deep self-attention mechanism in step S32 adopts the following formula:

[0063] S(p)=β·var(p)+(1-β)·d(p,μ norm )

[0064] Where S(p) is the significance score of image patch p, var(p) is the variance of the image patch features (reflecting feature complexity), and d(p,μ) is the feature score of the image patch. norm ) represents the mean μ of image patch features and normal tissue features. norm The Euclidean distance (reflecting the degree of anomaly) is used, and β is the weighting coefficient (ranging from [0,1]). This formula combines feature complexity and anomaly degree to evaluate the diagnostic importance of image patches and is used to dynamically allocate attention resources.

[0065] As an improvement, the loss function of the multi-task learning framework in step S41 adopts the formula:

[0066] L=w1L ce +w2L l1 +w3L ord

[0067] Where L is the total loss, L ce For cross-entropy loss (used for subtype classification), L l1 For L1 loss (used in Breslow thickness measurement), L ord The formula uses ordered cross-entropy loss (for Clark classification), where w1, w2, and w3 are task weights (satisfying w1 + w2 + w3 = 1). This formula achieves collaborative optimization of multiple diagnostic objectives by weighted fusion of multi-task losses.

[0068] As an improvement, the uncertainty quantification in step S42 adopts the following formula:

[0069]

[0070] Where CI represents the 95% confidence interval. Let σ be the mean of the diagnostic results, σ ​​be the standard deviation of the Monte Carlo sampling results, and 1.96 be the critical value of the 95% confidence interval of the normal distribution. This formula calculates the confidence range of the diagnostic results based on statistical principles, quantifying the uncertainty of the model prediction.

[0071] As an improvement, a federated model training step is also included:

[0072] S01. Each participating institution initializes its local model based on its local labeled dataset (including pathological images and corresponding diagnostic labels);

[0073] S02. Local model training: The improved AdamW optimizer (with gradient clipping mechanism) is used to minimize the loss function, and the model parameter update amount is saved after each training round;

[0074] S03. Encrypted parameter upload: The parameter update is encrypted using the Paillier homomorphic encryption algorithm and then uploaded to the federated server;

[0075] S04. Global Parameter Aggregation: The federated server uses a weighted average (the weight is proportional to the amount of data from each institution) to aggregate encrypted parameters and generate global model parameters;

[0076] S05. Model Distribution and Iteration: Distribute the decrypted global parameters to each institution, update the local model, and repeat steps S02-S04 until the model converges.

[0077] S06. Model Personalization Adjustment: Each institution fine-tunes the global model based on a small amount of local data to adapt to the characteristics of local data distribution;

[0078] The gradient clipping mechanism in step S02 uses the following formula:

[0079]

[0080] Where g is the original gradient, g′ is the clipped gradient, ||g|| is the L2 norm of the gradient, and θ is the clipping threshold. This formula prevents gradient explosion during training by limiting the gradient magnitude, thus ensuring the stability of model training.

[0081] The advantages of this invention compared to existing technologies are as follows: This technical solution can significantly improve the efficiency of skin cancer pathological diagnosis. Traditional manual analysis of a whole-slice digital image takes 30 minutes to 1 hour. This solution, through fully automated processing, reduces the analysis time of a single slice to less than 1 / 10 of the traditional process. It can also automatically generate structured reports, which can improve the efficiency of physicians' effective diagnosis by 3-5 times and alleviate the shortage of pathological resources.

[0082] The diagnostic accuracy is significantly improved. Multispectral imaging and refined preprocessing provide high-quality data, hierarchical feature extraction comprehensively captures pathological features, and multi-task learning optimizes diagnostic accuracy. In a test set of 5,000 cases, the subtype classification accuracy reached 96.8%, and the Breslow thickness measurement error was less than 0.1 mm, reducing the risk of missed diagnosis and misdiagnosis.

[0083] Its federated learning module protects medical data privacy, allowing institutions to train locally without sharing raw data. Parameters are uploaded in encrypted form, while the module also collaboratively optimizes the model, improving generalization ability. The model is highly interpretable, generating heatmaps and feature importance rankings, and combining knowledge graphs to link cases and guidelines, thus enhancing clinical acceptance.

[0084] The solution is also adaptable to diverse clinical scenarios, with configurable modules that support integration with hospital systems. It can be continuously iterated and optimized, and it promotes diagnostic standardization, narrows the diagnostic gap between institutions, and helps improve the capabilities of primary healthcare. Attached Figure Description

[0085] Figure 1 This is a system framework diagram of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0086] Figure 2 This is a flowchart of the image acquisition and quality control module of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0087] Figure 3 This is a flowchart of the multi-scale preprocessing module of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0088] Figure 4 This is a flowchart of the hierarchical feature extraction module of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0089] Figure 5This is a flowchart of the intelligent diagnostic reasoning module of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0090] Figure 6 This is a flowchart of the knowledge graph-assisted decision-making module of the skin cancer pathological image analysis system and method based on the Transformer model of this invention.

[0091] Figure 7 This is a flowchart of the result output and feedback module of the skin cancer pathological image analysis system and method based on the Transformer model of the present invention.

[0092] Figure 8 This is a flowchart illustrating the overall process of the skin cancer pathological image analysis system and method based on the Transformer model of this invention. Detailed Implementation

[0093] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings, which illustrate embodiments of the present application. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of this application will be thorough and complete.

[0094] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0095] It is understood that spatial relation terms such as "below," "under," "below," "below," "above," "over," etc., can be used here to describe the relationship between one element or feature shown in the figure and other elements or features. It should be understood that, in addition to the orientation shown in the figure, spatial relation terms also include different orientations of the device in use and operation. For example, if the device in the figure is flipped, the element or feature described as "below" or "under" or "below" of the other element or feature will be oriented "over" the other element or feature. Therefore, the exemplary terms "below" and "under" can include both upper and lower orientations. Furthermore, the device may also include other orientations, such as being rotated 90 degrees or other orientations, and the spatial descriptive terms used herein will be interpreted accordingly.

[0096] It should be noted that when one element is considered to be "connected" to another element, it can be directly connected to the other element or connected to the other element through an intermediary element. In the following embodiments, "connection" should be understood as "electrical connection," "communication connection," etc., if the connected circuits, modules, units, etc., have the transmission of electrical signals or data between them.

[0097] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising,” “including,” or “having,” etc., specify the presence of the stated feature, whole, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof.

[0098] Referring to the accompanying drawings, a skin cancer pathological image analysis system and method based on the Transformer model are described. The skin cancer pathological image analysis system based on the Transformer model is characterized by:

[0099] include:

[0100] The image acquisition and quality control module is used to acquire multispectral digital images of skin tissue pathological sections and to achieve image quality control through sharpness assessment, staining consistency verification and artifact detection.

[0101] The multi-scale preprocessing module is connected to the image acquisition and quality control module. It removes non-organic areas through adaptive threshold segmentation, processes the image by combining multi-modal enhancement algorithms (including illumination correction based on Retinex theory, noise suppression based on wavelet transform, and detail enhancement based on generative adversarial networks), and normalizes it into a sequence of image blocks of a preset size.

[0102] The hierarchical feature extraction module, connected to the multi-scale preprocessing module, includes:

[0103] The local feature encoding unit uses a convolutional neural network to extract the texture, morphology, and color distribution features of image patches;

[0104] The global correlation modeling unit adopts an improved Transformer model, which includes a dynamic deep self-attention mechanism (dynamically adjusting the number and depth of attention heads according to the importance of pathological regions) and a cross-scale feature fusion module (fusing features of different resolutions through skip connections).

[0105] The pathological feature aggregation unit aggregates multi-level features through attention-weighted pooling to generate a comprehensive feature vector that includes cellular atypia, tissue structure disorder, and invasion pattern.

[0106] The intelligent diagnostic reasoning module, connected to the hierarchical feature extraction module, includes:

[0107] A multi-task classifier based on comprehensive feature vectors to identify skin cancer subtypes (including melanoma, basal cell carcinoma, and squamous cell carcinoma), measure Breslow thickness, and perform Clark grading.

[0108] Uncertainty quantification unit, which calculates the confidence interval of the diagnostic results using Monte Carlo dropout;

[0109] The interpretability analysis unit generates an attention heatmap to locate key lesion areas and outputs a ranking of feature importance.

[0110] The knowledge graph-assisted decision-making module is connected to the intelligent diagnostic reasoning module and is used to associate historical case data, clinical indicators and literature knowledge to provide differential diagnosis basis and treatment suggestions.

[0111] The results output and feedback module is connected to the intelligent diagnostic reasoning module and the knowledge graph-assisted decision-making module. It is used to output a structured diagnostic report (including image tags, quantitative analysis results and diagnostic conclusions) and receive feedback from pathology experts to achieve continuous model optimization.

[0112] In the improved Transformer model's dynamic deep self-attention mechanism, the attention weights are calculated using the following formula:

[0113]

[0114] Where, α i,j sim(q) represents the attention weight between the i-th query vector and the j-th key vector. i ,k j ) represents the query vector q i With key vector k j The similarity is calculated using a scaled dot product, where n represents the sequence length. This formula converts the similarity into a normalized weight distribution using the softmax function, allowing the model to focus on associations of important pathological features.

[0115] In the color normalization unit of the multi-scale preprocessing module, the color space conversion uses the following formula:

[0116]

[0117] Among them, I src For the original stained image, I std For the standardized image, M srcLet M be the color matrix of the original image, and M be the standard color matrix. This formula uses matrix transformation to convert different coloring styles to the standard style, eliminating coloring differences between laboratories.

[0118] In the similarity calculation unit of the knowledge graph-assisted decision-making module, the case similarity is calculated using the following formula:

[0119]

[0120] Where S(a,b) represents the similarity between case a and case b, f a f b Let be the feature vectors of two cases, · denotes the vector dot product, and ||·|| denotes the L2 norm of the vectors. This formula quantifies the similarity of case features using cosine similarity and is used to retrieve the most relevant historical cases.

[0121] It also includes a federated learning training module, which comprises:

[0122] Local training units are deployed in various medical institutions to update model parameters based on local data.

[0123] The parameter aggregation unit fuses parameters from each node using a secure aggregation algorithm (based on homomorphic encryption) to protect data privacy.

[0124] The model adaptive unit dynamically adjusts the model regularization strength and learning rate based on the data distribution characteristics of each institution;

[0125] The global parameter update of the parameter aggregation unit uses the following formula:

[0126]

[0127] Where, θ global θ represents the aggregated global model parameters. i For the local model parameters of the i-th institution, w i The weight of the i-th institution (satisfying) (N is the number of participating institutions). This formula achieves collaborative optimization of the model in a federated scenario by merging the parameters of each institution through a weighted average.

[0128] A method for analyzing skin cancer pathological images based on the Transformer model includes the following steps:

[0129] S1. Image Acquisition and Quality Control:

[0130] S11. Use a whole-section scanner to acquire multispectral digital images of skin pathology sections (including 400-700nm band information);

[0131] S12. Select qualified images through a sharpness scoring algorithm (integrating gradient information and entropy calculation), remove abnormal staining samples through staining uniformity detection, and identify and mark artifacts such as folds and bubbles in the image through morphological operations;

[0132] S2. Multi-scale image preprocessing:

[0133] S21. Use the U-Net model to segment the tissue region and remove the background and slide edges;

[0134] S22. Perform the following steps in sequence: illumination non-uniformity correction based on bilateral filtering, noise removal based on non-local means, and detail enhancement based on super-resolution reconstruction.

[0135] S23. Perform color normalization processing to convert the image to a standard color space;

[0136] S24. Adaptively divide image blocks based on histopathological type to generate an image block sequence containing spatial location information;

[0137] S3. Hierarchical Feature Extraction:

[0138] S31. A pre-trained convolutional neural network is used to extract local pathological features (including cell nuclear morphology, cytoplasmic staining intensity, and intercellular connection features) from each image patch;

[0139] S32. Input local features into the improved Transformer model:

[0140] Location information is added through pathological spatial perception coding;

[0141] A dynamic deep self-attention mechanism (allocating more attention heads to the lesion region) is used to model global feature association;

[0142] Integrate features from different levels through a cross-scale fusion module;

[0143] S33. Attention-weighted pooling is used to aggregate features and generate a comprehensive feature vector;

[0144] S4. Intelligent Diagnostic Reasoning:

[0145] S41. Based on comprehensive feature vectors, this is achieved simultaneously through a multi-task learning framework:

[0146] Skin cancer subtype classification (multi-classification task);

[0147] Quantitative assessment of malignancy (regression task, output Breslow thickness value);

[0148] Invasion depth classification (orderly classify tasks, output Clark level);

[0149] S42. Calculate the confidence interval for each diagnostic result using Monte Carlo sampling;

[0150] S43. Generate an attention heatmap to mark key diagnostic regions and output a ranking of feature importance;

[0151] S5. Knowledge Graph-Assisted Decision Making:

[0152] S51. Search for similar cases in the case knowledge graph to obtain reference diagnostic results;

[0153] S52. Combine clinical indicators (such as patient age and lesion location) to perform rule-based reasoning and optimize diagnostic conclusions;

[0154] S6. Output a structured diagnostic report and receive expert feedback for model iteration and optimization.

[0155] The saliency score of the dynamic deep self-attention mechanism in step S32 is calculated using the following formula:

[0156] S(p)=β·var(p)+(1-β)·d(p,μ norm )

[0157] Where S(p) is the significance score of image patch p, var(p) is the variance of the image patch features (reflecting feature complexity), and d(p,μ) is the feature score of the image patch. norm ) represents the mean μ of image patch features and normal tissue features. norm The Euclidean distance (reflecting the degree of anomaly) is used, and β is the weighting coefficient (ranging from [0,1]). This formula combines feature complexity and anomaly degree to evaluate the diagnostic importance of image patches and is used to dynamically allocate attention resources.

[0158] The loss function for the multi-task learning framework in step S41 uses the following formula:

[0159] L=w1L ce +w2L l1 +w3L ord

[0160] Where L is the total loss, L ce For cross-entropy loss (used for subtype classification), L l1 For L1 loss (used in Breslow thickness measurement), L ord The formula uses ordered cross-entropy loss (for Clark classification), where w1, w2, and w3 are task weights (satisfying w1 + w2 + w3 = 1). This formula achieves collaborative optimization of multiple diagnostic objectives by weighted fusion of multi-task losses.

[0161] The uncertainty quantification in step S42 uses the following formula:

[0162]

[0163] Where CI represents the 95% confidence interval. Let σ be the mean of the diagnostic results, σ ​​be the standard deviation of the Monte Carlo sampling results, and 1.96 be the critical value of the 95% confidence interval of the normal distribution. This formula calculates the confidence range of the diagnostic results based on statistical principles, quantifying the uncertainty of the model prediction.

[0164] It also includes the model federated training step:

[0165] S01. Each participating institution initializes its local model based on its local labeled dataset (including pathological images and corresponding diagnostic labels);

[0166] S02. Local model training: The improved AdamW optimizer (with gradient clipping mechanism) is used to minimize the loss function, and the model parameter update amount is saved after each training round;

[0167] S03. Encrypted parameter upload: The parameter update is encrypted using the Paillier homomorphic encryption algorithm and then uploaded to the federated server;

[0168] S04. Global Parameter Aggregation: The federated server uses a weighted average (the weight is proportional to the amount of data from each institution) to aggregate encrypted parameters and generate global model parameters;

[0169] S05. Model Distribution and Iteration: Distribute the decrypted global parameters to each institution, update the local model, and repeat steps S02-S04 until the model converges.

[0170] S06. Model Personalization Adjustment: Each institution fine-tunes the global model based on a small amount of local data to adapt to the characteristics of local data distribution;

[0171] The gradient clipping mechanism in step S02 uses the following formula:

[0172]

[0173] Where g is the original gradient, g′ is the clipped gradient, ||g|| is the L2 norm of the gradient, and θ is the clipping threshold. This formula limits the gradient...

[0174] I. System Overall Architecture

[0175] This invention discloses a skin cancer pathology image analysis system and method based on the Transformer model, aiming to achieve automated and high-precision analysis of skin cancer pathology images through artificial intelligence technology, providing reliable assistance for clinical diagnosis. The system integrates functional modules such as multispectral image acquisition, multi-scale preprocessing, hierarchical feature extraction, intelligent diagnostic reasoning, knowledge graph-assisted decision-making, and result feedback optimization, forming a complete skin cancer pathology analysis solution.

[0176] The overall workflow of the system is as follows: First, high-quality digital images of skin tissue pathological sections are acquired through the image acquisition and quality control module; then, the images are standardized by the multi-scale preprocessing module; next, the pathological features in the images are extracted by the hierarchical feature extraction module; then, the intelligent diagnostic reasoning module makes a diagnosis based on the extracted features; the knowledge graph-assisted decision-making module further optimizes the diagnostic results; finally, the result output and feedback module outputs a diagnostic report and receives expert feedback to continuously optimize system performance.

[0177] II. Detailed Implementation Methods of Each Module

[0178] 2.1 Image Acquisition and Quality Control Module

[0179] The main function of this module is to acquire multispectral digital images of skin tissue pathological sections and to strictly control the image quality to ensure the reliability of subsequent analysis.

[0180] For image acquisition, a whole-slice scanner was used to scan skin pathology sections to obtain multispectral digital images containing information in the 400-700nm wavelength range. Multispectral imaging can capture more pathological information and, compared to traditional RGB images, helps to more accurately distinguish different types of cells and tissues.

[0181] Image quality control comprises three key steps: sharpness assessment, color consistency verification, and artifact detection. Sharpness assessment is achieved by fusing gradient information and entropy calculations. Gradient information reflects the edge sharpness of the image, while entropy reflects the information richness of the image; combining the two effectively assesses the image's sharpness. Color consistency verification analyzes the color distribution characteristics of the image, eliminating samples with abnormal coloring to ensure that subsequent analysis is not affected by color differences. Artifact detection uses morphological operations to identify and label interfering factors such as folds and bubbles in the image, providing a reference for subsequent processing.

[0182] 2.2 Multi-scale preprocessing module

[0183] The multi-scale preprocessing module is connected to the image acquisition and quality control module and is responsible for performing a series of processes on the acquired images to prepare for subsequent feature extraction.

[0184] First, non-tissue regions are removed through adaptive thresholding. This process utilizes the difference in grayscale values ​​between tissue and background to automatically determine the segmentation threshold, accurately separating tissue regions from the background and removing interference from the background and slide edges.

[0185] Next, a multimodal enhancement algorithm is used to process the image, including illumination correction based on Retinex theory, noise suppression based on wavelet transform, and detail enhancement based on generative adversarial networks. Illumination correction can solve the problem of uneven illumination in the image, making the brightness distribution of the image more uniform; noise suppression can remove noise interference in the image and improve image quality; detail enhancement can highlight the fine structure in the image, which is helpful for subsequent feature extraction.

[0186] Then, color normalization is performed to convert the image to a standard color space. The color space conversion uses the following formula:

[0187]

[0188] In this formula, I src Represents the original stained image, I std M represents the standardized image. src M is the color matrix of the original image, and M is the standard color matrix. Through this matrix transformation, stained images from different laboratories and batches can be converted into a unified standard style, effectively eliminating staining differences between laboratories and improving the consistency of subsequent analyses.

[0189] Finally, image blocks are adaptively divided based on histopathological type to generate a sequence of image blocks containing spatial location information. For lesion areas, 128×128 pixel image blocks are used to capture more detailed information; for normal areas, 256×256 pixel image blocks are used to improve processing efficiency while ensuring information integrity.

[0190] 2.3 Hierarchical Feature Extraction Module

[0191] The hierarchical feature extraction module is connected to the multi-scale preprocessing module and is responsible for extracting valuable pathological features from the preprocessed image, including three steps: local feature encoding, global correlation modeling, and pathological feature aggregation.

[0192] The local feature encoding unit uses a pre-trained convolutional neural network to extract local pathological features from each image patch. These features include cell nuclear morphology, cytoplasmic staining intensity, and intercellular connection characteristics. Convolutional neural networks possess powerful feature extraction capabilities, automatically learning local feature patterns in images, thus laying the foundation for subsequent global correlation modeling.

[0193] The global correlation modeling unit employs an improved Transformer model, which includes a dynamic deep self-attention mechanism and a cross-scale feature fusion module. The dynamic deep self-attention mechanism dynamically adjusts the number and depth of attention heads based on the importance of pathological regions, allowing the model to focus more on important pathological areas. The attention weights are calculated using the following formula:

[0194]

[0195] In the formula, α i,j sim(q) represents the attention weight between the i-th query vector and the j-th key vector. i ,k j ) represents the query vector q i With key vector k j The similarity is calculated using a scaled dot product, where n represents the sequence length. This formula converts the similarity into a normalized weight distribution using the softmax function, enabling the model to focus on important pathological feature associations and improving the targeting and effectiveness of feature extraction.

[0196] The cross-scale feature fusion module fuses features of different resolutions through skip connections, enabling it to comprehensively utilize image information at different scales and capture more comprehensive pathological features.

[0197] The pathological feature aggregation unit aggregates multi-level features through attention-weighted pooling to generate a comprehensive feature vector that includes cellular atypia, tissue structure disorder, and invasion patterns. Attention-weighted pooling can assign different weights according to the importance of features, so that important features occupy a larger proportion in the comprehensive feature vector, thereby improving the representativeness of the feature vector.

[0198] 2.4 Intelligent Diagnostic Reasoning Module

[0199] The intelligent diagnostic reasoning module is connected to the hierarchical feature extraction module. Based on the extracted comprehensive feature vector, it performs diagnostic reasoning for skin cancer, including multi-task classification, uncertainty quantification, and interpretability analysis.

[0200] The multi-task classifier can simultaneously identify skin cancer subtypes (including melanoma, basal cell carcinoma, and squamous cell carcinoma), measure Breslow thickness, and perform Clark grading. The multi-task learning framework fully leverages the correlation between different tasks to improve the performance of each task. The loss function of this framework uses the formula:

[0201] L=w1L ce +w2L l1 +w3L ord

[0202] Where L is the total loss, L ceFor cross-entropy loss (used for subtype classification), L l1 For L1 loss (used in Breslow thickness measurement), L ord The ordered cross-entropy loss is used (for Clark classification), and w1, w2, and w3 are task weights (satisfying w1 + w2 + w3 = 1). By weighted fusion of multi-task losses, collaborative optimization of multiple diagnostic objectives can be achieved, enabling the model to achieve good performance on multiple tasks.

[0203] The uncertainty quantification unit calculates the confidence interval of the diagnostic results using Monte Carlo dropout, employing the formula:

[0204]

[0205] In the formula, CI represents the 95% confidence interval. Let σ be the mean of the diagnostic results, σ ​​be the standard deviation of the Monte Carlo sampling results, and 1.96 be the critical value of the 95% confidence interval of the normal distribution. This formula calculates the confidence range of the diagnostic results based on statistical principles, quantifies the uncertainty of model predictions, and provides clinicians with more comprehensive reference information.

[0206] The interpretability analysis unit generates an attention heatmap to locate key lesion areas and outputs a ranking of feature importance. The attention heatmap can visually show the areas the model focuses on during diagnosis, helping doctors understand the model's decision-making basis; the feature importance ranking can reveal which features have a greater impact on the diagnostic results, providing clues for pathological research.

[0207] 2.5 Knowledge Graph-Assisted Decision-Making Module

[0208] The knowledge graph-assisted decision-making module is connected to the intelligent diagnostic reasoning module to associate historical case data, clinical indicators, and literature knowledge, providing a basis for differential diagnosis and treatment suggestions.

[0209] This module includes a case knowledge graph construction unit, a similarity calculation unit, and a rule-based reasoning engine. The case knowledge graph construction unit stores structured knowledge in the form of entities (lesion features, diagnostic results, treatment plans) and relationships (causal associations, co-occurrence relationships), building a comprehensive case knowledge network.

[0210] The similarity calculation unit uses a graph neural network to calculate the feature similarity between the current case and historical cases, using the formula:

[0211]

[0212] Where S(a,b) represents the similarity between case a and case b, f a f bLet be the feature vectors of the two cases, · denote the vector dot product, and ||·|| denote the L2 norm of the vectors. This formula uses cosine similarity to quantify the similarity of case features, and is used to retrieve the most relevant historical cases, providing a reference for the diagnosis of the current case.

[0213] The rule-based reasoning engine constructs a decision rule base based on pathological diagnostic guidelines to achieve consistency verification of diagnostic results. By combining rule-based reasoning with clinical indicators (such as patient age and lesion location), diagnostic conclusions can be further optimized, improving the accuracy and reliability of diagnoses.

[0214] 2.6 Results Output and Feedback Module

[0215] The results output and feedback module is connected to the intelligent diagnostic reasoning module and the knowledge graph-assisted decision-making module. It is used to output structured diagnostic reports (including image labels, quantitative analysis results and diagnostic conclusions) and receive feedback from pathology experts to achieve continuous model optimization.

[0216] Structured diagnostic reports present diagnostic results clearly and comprehensively, facilitating physician review and comprehension. Pathologists can evaluate and revise the diagnostic results, inputting feedback into the system. The system then adjusts model parameters based on this feedback, continuously improving diagnostic performance and achieving ongoing model optimization.

[0217] 2.7 Federated Learning Training Module

[0218] The system also includes a federated learning training module, which contains local training units, parameter aggregation units, and model adaptation units.

[0219] Local training units are deployed across various medical institutions, updating model parameters based on local data. This avoids data leakage and protects patient privacy. During local model training, an improved AdamW optimizer (with gradient pruning) is used to minimize the loss function, and the updated model parameters are saved after each training round. The gradient pruning mechanism uses the following formula:

[0220]

[0221] Where g is the original gradient, g′ is the clipped gradient, ||g|| is the L2 norm of the gradient, and θ is the clipping threshold. This formula prevents gradient explosion during training by limiting the gradient magnitude, thus ensuring the stability of model training.

[0222] The parameter aggregation unit fuses parameters from each node using a secure aggregation algorithm (based on homomorphic encryption) to protect data privacy. Global parameter updates are performed using the following formula:

[0223]

[0224] Where, θglobal θ represents the aggregated global model parameters. i For the local model parameters of the i-th institution, w i The weight of the i-th institution (satisfying) N represents the number of participating institutions. This formula achieves collaborative optimization of the model in a federated scenario by merging the parameters of each institution through weighted averaging, enabling the model to fully utilize the data resources of each institution and improve the model's generalization ability.

[0225] The model adaptive unit dynamically adjusts the model regularization strength and learning rate according to the data distribution characteristics of each institution, so that the model can better adapt to the data distribution of different institutions and improve the application effect of the model in each institution.

[0226] III. Method Implementation Steps

[0227] The skin cancer pathological image analysis method based on the Transformer model of the present invention includes the following steps:

[0228] 3.1 Image Acquisition and Quality Control

[0229] S11. Multispectral digital images (containing 400-700nm wavelength information) of skin pathology sections are acquired using a whole-section scanner. Multispectral images provide richer spectral information, which helps to identify pathological features more accurately.

[0230] S12. Qualified images are selected using a sharpness scoring algorithm (integrating gradient information and entropy calculation), abnormally stained samples are removed using staining homogeneity detection, and artifacts such as folds and bubbles in the images are identified and marked using morphological operations. This step ensures that the image quality used for subsequent analysis meets the requirements and reduces the impact of noise and interference on the analysis results.

[0231] 3.2 Multi-scale image preprocessing

[0232] S21. The U-Net model is used to segment the tissue region and remove the background and slide edges. The U-Net model has good image segmentation performance and can accurately segment the tissue region.

[0233] S22. Perform the following steps in sequence: illumination non-uniformity correction based on bilateral filtering, noise removal based on non-local means, and detail enhancement based on super-resolution reconstruction. Illumination non-uniformity correction can make the image brightness distribution more uniform; noise removal can improve the signal-to-noise ratio of the image; and detail enhancement can highlight the fine structures in the image, providing a better foundation for subsequent feature extraction.

[0234] S23. Perform color normalization to convert the image to a standard color space. As mentioned earlier, this step eliminates the influence of different coloring conditions on the image and improves image consistency.

[0235] S24. Image blocks are adaptively divided based on histopathological type to generate a sequence of image blocks containing spatial location information. Using image blocks of different sizes according to the pathological characteristics of different regions can improve processing efficiency while ensuring analytical accuracy.

[0236] 3.3 Hierarchical Feature Extraction

[0237] S31. A pre-trained convolutional neural network is used to extract local pathological features (including cell nuclear morphology, cytoplasmic staining intensity, and intercellular connection features) from each image patch. The pre-trained convolutional neural network has learned a large number of image feature patterns and can quickly and accurately extract local pathological features.

[0238] S32. Input local features into the improved Transformer model:

[0239] By adding location information through pathological spatial perception coding, the model can consider the spatial positional relationships of image patches;

[0240] A dynamic deep self-attention mechanism (allocating more attention heads to the lesion region) is employed to model global feature association. The saliency score of the dynamic deep self-attention mechanism is calculated using the following formula:

[0241] S(p)=β·var(p)+(1-β)·d(p,μ norm )

[0242] In the formula, S(p) is the significance score of image patch p, var(p) is the variance of the image patch features (reflecting feature complexity), and d(p,μ) is the feature score of the image patch. norm ) represents the mean μ of image patch features and normal tissue features. norm The Euclidean distance (reflecting the degree of abnormality) is used, and β is the weighting coefficient (ranging from [0,1]). This formula combines feature complexity and the degree of abnormality to evaluate the diagnostic importance of image patches, and is used to dynamically allocate attention resources, making the model pay more attention to important information such as lesion areas;

[0243] By integrating features from different levels through a cross-scale fusion module, and making comprehensive use of feature information at different scales, the comprehensiveness and representativeness of features are improved.

[0244] S33. Attention-weighted pooling is used to aggregate features and generate a comprehensive feature vector. By assigning different weights to different features, important features occupy a larger proportion in the comprehensive feature vector, thereby improving the quality of the feature vector.

[0245] 3.4 Intelligent Diagnostic Reasoning

[0246] S41. Based on comprehensive feature vectors, this is achieved simultaneously through a multi-task learning framework:

[0247] Skin cancer subtype classification (multi-classification task);

[0248] Quantitative assessment of malignancy (regression task, output Breslow thickness value);

[0249] Invasion depth classification (ordered classification of tasks, output Clark level).

[0250] Multi-task learning can fully utilize the correlation between different tasks and improve the overall performance of the model.

[0251] S42. Calculate the confidence intervals for each diagnostic result using Monte Carlo sampling, as shown in the formula, to quantify the uncertainty of the model's predictions and provide doctors with a more comprehensive diagnostic reference.

[0252] S43. Generate an attention heatmap to mark key diagnostic regions and output a ranking of feature importance to improve the interpretability of the model and help doctors understand the model's decision-making process.

[0253] 3.5 Knowledge Graph-Assisted Decision Making

[0254] S51. Search for similar cases in the case knowledge graph to obtain reference diagnostic results. By calculating the similarity between cases, find the historical case most similar to the current case to provide a reference for diagnosis.

[0255] S52. Optimize diagnostic conclusions by combining clinical indicators (such as patient age and lesion location) with rule-based reasoning. Integrating clinical indicators with case knowledge can further improve diagnostic accuracy.

[0256] 3.6 Output a structured diagnostic report and receive expert feedback for model iteration and optimization.

[0257] The system outputs a structured diagnostic report containing image labels, quantitative analysis results, and diagnostic conclusions, making it convenient for doctors to use. Simultaneously, it receives feedback from pathology experts for iterative model optimization, continuously improving the system's diagnostic performance.

[0258] 3.7 Model Federated Training Steps

[0259] S01. Each participating institution initializes its local model based on its local labeled dataset (which includes pathological images and corresponding diagnostic labels).

[0260] S02. Local Model Training: An improved AdamW optimizer (with gradient pruning mechanism) is used to minimize the loss function, and the model parameter updates are saved after each training round. The gradient pruning mechanism ensures the stability of the training process.

[0261] S03. Encrypted Parameter Upload: The parameter update is encrypted using the Paillier homomorphic encryption algorithm before being uploaded to the federated server to protect data privacy.

[0262] S04. Global Parameter Aggregation: The federated server uses a weighted average (the weight is proportional to the amount of data from each institution) to aggregate encrypted parameters and generate global model parameters, as shown in the formula, to achieve collaborative optimization of the model.

[0263] S05. Model Distribution and Iteration: Distribute the decrypted global parameters to each institution, update the local model, and repeat steps S02-S04 until the model converges.

[0264] S06. Model Personalization Adjustment: Each institution fine-tunes the global model based on a small amount of local data to adapt to the characteristics of local data distribution and improve the model's application effect locally.

[0265] IV. Beneficial Effects of the Invention

[0266] The skin cancer pathological image analysis system and method based on the Transformer model of the present invention have the following beneficial effects:

[0267] By employing multispectral image acquisition and multiscale preprocessing techniques, high-quality, standardized pathological images can be obtained, providing a reliable data foundation for subsequent analysis.

[0268] By combining convolutional neural networks and an improved Transformer model for hierarchical feature extraction, we can capture both local pathological features and model global feature associations, thereby improving the comprehensiveness and accuracy of feature extraction.

[0269] By introducing a multi-task learning framework, multiple diagnostic tasks such as skin cancer subtype identification, malignancy assessment, and invasion depth grading can be achieved simultaneously, improving diagnostic efficiency and accuracy.

[0270] By quantifying uncertainty and analyzing interpretability, the reliability and transparency of the model are improved, which helps doctors better understand and use the model's diagnostic results.

[0271] The knowledge graph-based decision support module can link historical cases and clinical knowledge, providing more references for diagnosis and further improving diagnostic accuracy.

[0272] The federated learning training module enables the collaborative use of data from multiple institutions while protecting data privacy, thereby improving the model's generalization ability.

[0273] The results output and feedback module enables standardized output of diagnostic reports and continuous optimization of the model, allowing the system to continuously adapt to practical application needs.

[0274] In summary, this invention provides an efficient, accurate, and reliable auxiliary tool for the pathological diagnosis of skin cancer, and has significant clinical application value.

[0275] Significantly improves the efficiency of skin cancer pathology diagnosis and alleviates the workload of clinical pathologists:

[0276] In the traditional pathological diagnosis process for skin cancer, pathologists need to examine each pathological slide area by area, manually identifying abnormalities in cell morphology and tissue structure, determining the tumor subtype, and assessing the degree of malignancy—a process that not only relies on the physician's extensive experience but also consumes a significant amount of time. For example, with whole-slide digital imaging (WSI), a single WSI image with a resolution of tens of thousands of pixels typically requires 30 minutes to an hour for complete analysis, and prolonged observation can easily lead to decreased efficiency due to visual fatigue.

[0277] This technical solution addresses this pain point at its root through fully automated processing: First, the image acquisition and quality control module uses a multispectral whole-slice scanner, which can acquire complete pathological images in the 400-700nm band in a single pass, eliminating the need for physicians to manually adjust scanning parameters. Second, the multi-scale preprocessing module uses automated algorithms such as adaptive threshold segmentation, illumination correction, and staining standardization to automatically remove background interference and unify staining styles across different laboratories, saving physicians the steps of manually cropping and adjusting images. Most importantly, the hierarchical feature extraction and intelligent diagnostic reasoning module can complete feature extraction and analysis of WSI images within minutes, while simultaneously outputting key diagnostic indicators such as skin cancer subtypes (melanoma, basal cell carcinoma, etc.), Breslow thickness, and Clark grade, reducing the analysis time for a single slice to less than 1 / 10 of the traditional process.

[0278] Furthermore, the results output module in the solution can automatically generate structured reports containing image labels, quantitative data, and diagnostic conclusions. Physicians do not need to manually organize diagnostic results; they only need to review and confirm based on the report, further reducing the time spent on non-diagnostic work. For pathology departments that need to process dozens of slides daily, this solution can improve the effective diagnostic efficiency of physicians by 3-5 times, significantly alleviating the problem of strained clinical pathology resources.

[0279] Significantly improves diagnostic accuracy and reduces the risk of missed or misdiagnosed cases:

[0280] The core challenges in pathological diagnosis of skin cancer lie in the overlapping cellular morphologies of different tumor subtypes (e.g., differentiating early-stage melanoma from benign nevus cells), the reliance on subtle tissue structural changes for assessing tumor malignancy (e.g., precise measurement of the depth of tumor invasion is required for Breslow thickness measurement), and the susceptibility of diagnostic results to the influence of physician experience and subjective judgment. Clinical data shows that the misdiagnosis rate for early-stage skin cancer using traditional manual diagnosis can reach 15%-20%, with an even higher risk of missed diagnosis, particularly in primary healthcare institutions where physicians often lack sufficient experience.

[0281] This technical solution elevates diagnostic accuracy to a new level through multi-dimensional technological innovation:

[0282] Multispectral imaging and refined preprocessing: Compared with traditional RGB images, 400-700nm multispectral images can capture more cell staining details (such as nuclear staining intensity and cytoplasmic texture differences). Illumination correction based on Retinex theory and staining standardization based on the improved Reinhard algorithm can eliminate interference factors such as uneven illumination during slide scanning and laboratory staining differences, providing a high-quality and consistent data foundation for subsequent analysis and avoiding feature misjudgment due to image quality issues.

[0283] The comprehensiveness of hierarchical feature extraction: The local feature encoding unit, through a pre-trained convolutional neural network, can accurately extract microscopic features such as cell nuclear morphology (e.g., nucleolar size, nucleocytoplasmic ratio) and cytoplasmic staining intensity; the improved Transformer model of the global association modeling unit, through a dynamic deep self-attention mechanism (e.g., formula...) It automatically focuses on the lesion area and models the global correlation between cells and tissues (such as the infiltration pattern of tumor cells and the distribution of lymphocytes), avoiding the omission of key features due to the limited field of view during manual observation.

[0284] Multi-task learning and uncertainty quantification: The intelligent diagnostic reasoning module adopts a multi-task learning framework (loss function L = w1L). ce +w2L l1 +w3L ord Simultaneously, it optimizes three tasks: subtype classification, thickness measurement, and grading assessment, leveraging the correlation between tasks to improve overall diagnostic accuracy—for example, through collaborative learning of Breslow thickness and Clark grading, it avoids bias in assessments based on a single indicator; and the confidence interval calculation for Monte Carlo dropout (formula) This tool can quantify the confidence level of diagnostic results, mark low-confidence results (such as suspected early-stage tumors), remind physicians to review them carefully, and effectively reduce the risk of missed or misdiagnosed cases.

[0285] Experimental results show that, in a test set containing 5,000 skin cancer pathology images, the subtype classification accuracy reached 96.8%, the Breslow thickness measurement error was less than 0.1 mm, and the Clark grading consistency (Kappa value) reached 0.92, which is significantly better than the performance of traditional manual diagnosis and single algorithm models.

[0286] Protecting medical data privacy and enabling multi-institutional collaborative optimization:

[0287] Protecting the privacy of medical data is one of the core challenges of artificial intelligence in clinical applications. Skin cancer pathology images contain sensitive pathological information of patients, and directly sharing data across institutions poses compliance risks (such as violating the Personal Information Protection Law and the Medical Data Security Guidelines). At the same time, the number of cases in a single medical institution is limited, making it difficult to support the continuous optimization of the model, resulting in insufficient generalization ability of the model across different institutions.

[0288] The federated learning training module of this technical solution perfectly resolves this contradiction:

[0289] Local training and privacy protection: Each medical institution's local training unit updates model parameters only based on its own data, without uploading original pathological images or labeled data to third-party servers, thus preventing data leakage at the source; before uploading parameters, the parameter update amount is encrypted using the Paillier homomorphic encryption algorithm, ensuring that even if intercepted during transmission, attackers cannot parse the original data information, fully complying with medical data privacy protection requirements.

[0290] Global collaboration and generalization capabilities are improved: the parameter aggregation unit uses a weighted average algorithm (formula) This integrates the encryption parameters of each institution into global model parameters—weights w. i The model is proportional to the amount of data from each institution, ensuring the contribution of large sample institutions while avoiding the bias of small sample institutions. The fused global model is then redistributed to each institution, and through multiple rounds of iterative optimization, the model can learn the pathological characteristics of skin cancer in different regions and populations (such as the morphological differences of melanoma in different ethnic groups), significantly improving its generalization ability.

[0291] Enhancing model interpretability and improving clinical acceptance: Traditional artificial intelligence models are often called "black boxes"—they can only output diagnostic results but cannot explain "why the conclusion was reached." This leads to insufficient trust in the model results by pathologists and makes it difficult to truly integrate them into the clinical diagnostic process.

[0292] This technical solution completely breaks down the "black box" barrier through the interpretability analysis unit and the knowledge graph-assisted decision-making module:

[0293] Attention Heatmap and Feature Importance Ranking: When the model outputs diagnostic results, it simultaneously generates an attention heatmap—through the weight distribution of a dynamic deep self-attention mechanism (such as formula α). i,j The visualization visually marks the lesion areas that the model focuses on (such as abnormal cell aggregation areas and tumor infiltration boundaries). Physicians can directly observe the corresponding areas by referring to the heat map to verify the rationality of the model's judgment. At the same time, the module will also output the ranking of feature importance (such as the contribution ratio of "nuclear atypia" and "tissue structure disorder" to the diagnostic results), clearly showing the decision basis of the model and making the diagnostic process "visible and traceable".

[0294] Adaptable to diverse clinical scenarios, supporting continuous iterative optimization:

[0295] Different medical institutions have different needs for pathological diagnosis: tertiary hospitals need to handle complex and rare cases, while primary hospitals focus more on rapid screening of common subtypes; at the same time, with the development of pathological diagnostic technology (such as new staining techniques and the discovery of new subtypes), the model needs to have the ability to be continuously updated.

[0296] This technical solution achieves adaptation and continuous optimization to diverse scenarios through modular design and feedback mechanisms:

[0297] Module configurability: The system's functional modules support parameter adjustment—for example, the multi-scale preprocessing module can adjust the image block size (128×128 pixels or 256×256 pixels for lesion areas) according to the hospital's slide type (such as paraffin slides or frozen slides), and the intelligent diagnostic reasoning module can enable / disable a certain diagnostic task according to clinical needs (such as disabling the thickness measurement function when only subtype classification is needed), meeting the personalized needs of different hospitals.

[0298] Expert feedback and model iteration: The results output and feedback module allows pathologists to correct the model's diagnostic results (such as marking "misdiagnosed cases" and "missed areas"). Feedback information is automatically recorded and used for model updates. For example, when the misdiagnosis rate of a certain rare subtype is high, the system will add the feature data of such cases to the training set and optimize the model parameters through local fine-tuning. At the same time, the model adaptive unit can dynamically adjust the regularization strength and learning rate according to the data distribution characteristics of each institution (such as a high proportion of basal cell carcinoma cases in a certain hospital), so that the model can better adapt to the local case characteristics.

[0299] In addition, the solution supports integration with existing hospital pathology information systems (PACS / HIS), allowing direct reading of pathological images and clinical data from the system without requiring physicians to manually upload data, further lowering the barrier to clinical application and achieving a "plug-and-play" adaptation effect.

[0300] Promoting the standardization of skin cancer diagnosis and helping to improve the capabilities of primary healthcare:

[0301] The lack of standardization in the diagnosis of skin cancer is another problem currently faced in clinical practice: different physicians have different criteria for judging pathological features (such as the definition of "cellular atypia"), which may lead to different diagnostic results for the same case in different hospitals; especially in primary healthcare institutions, where physicians have limited experience and find it difficult to master unified diagnostic standards, resulting in inconsistent diagnostic quality.

[0302] This technical solution promotes diagnostic standardization in the following ways:

[0303] Standardization of staining and feature extraction: A staining standardization algorithm for a multi-scale preprocessing module (formula) This standardizes staining styles across different laboratories, avoiding misjudgments due to staining differences. The hierarchical feature extraction module, based on a pre-trained model, employs unified feature definitions (e.g., "nuclear area > 50 μm"). 2 (Consider it as abnormal) to ensure that different institutions have consistent standards for extracting pathological features.

[0304] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A skin cancer pathological image analysis system based on the Transformer model, characterized in that: include: The image acquisition and quality control module is used to acquire multispectral digital images of skin tissue pathological sections and to achieve image quality control through sharpness assessment, staining consistency verification and artifact detection. The multi-scale preprocessing module is connected to the image acquisition and quality control module. It removes non-organic areas through adaptive threshold segmentation, processes the image by combining multi-modal enhancement algorithms (including illumination correction based on Retinex theory, noise suppression based on wavelet transform, and detail enhancement based on generative adversarial networks), and normalizes it into a sequence of image blocks of a preset size. The hierarchical feature extraction module, connected to the multi-scale preprocessing module, includes: The local feature encoding unit uses a convolutional neural network to extract the texture, morphology, and color distribution features of image patches; The global correlation modeling unit adopts an improved Transformer model, which includes a dynamic deep self-attention mechanism (dynamically adjusting the number and depth of attention heads according to the importance of pathological regions) and a cross-scale feature fusion module (fusing features of different resolutions through skip connections). The pathological feature aggregation unit aggregates multi-level features through attention-weighted pooling to generate a comprehensive feature vector that includes cellular atypia, tissue structure disorder, and invasion pattern. The intelligent diagnostic reasoning module, connected to the hierarchical feature extraction module, includes: A multi-task classifier based on comprehensive feature vectors to identify skin cancer subtypes (including melanoma, basal cell carcinoma, and squamous cell carcinoma), measure Breslow thickness, and perform Clark grading. Uncertainty quantification unit, which calculates the confidence interval of the diagnostic results using Monte Carlo dropout; The interpretability analysis unit generates an attention heatmap to locate key lesion areas and outputs a ranking of feature importance. The knowledge graph-assisted decision-making module is connected to the intelligent diagnostic reasoning module and is used to associate historical case data, clinical indicators and literature knowledge to provide differential diagnosis basis and treatment suggestions. The results output and feedback module is connected to the intelligent diagnostic reasoning module and the knowledge graph-assisted decision-making module. It is used to output a structured diagnostic report (including image tags, quantitative analysis results and diagnostic conclusions) and receive feedback from pathology experts to achieve continuous model optimization.

2. The skin cancer pathological image analysis system based on the Transformer model according to claim 1, characterized in that: In the improved Transformer model's dynamic deep self-attention mechanism, the attention weights are calculated using the following formula: Where, α i,j sim(q) represents the attention weight between the i-th query vector and the j-th key vector. i ,k j ) represents the query vector q i With key vector k j The similarity is calculated using scaled dot product, where n represents the sequence length; This formula uses the softmax function to convert similarity into a normalized weight distribution, enabling the model to focus on associations of important pathological features.

3. The skin cancer pathological image analysis system based on the Transformer model according to claim 1, characterized in that: In the color normalization unit of the multi-scale preprocessing module, the color space conversion uses the following formula: Among them, I src For the original stained image, I std For the standardized image, M src M is the color matrix of the original image, and M is the standard color matrix; This formula uses matrix transformation to convert different staining styles to a standard style, eliminating staining differences between laboratories.

4. The skin cancer pathological image analysis system based on the Transformer model according to claim 1, characterized in that: In the similarity calculation unit of the knowledge graph-assisted decision-making module, the case similarity is calculated using the following formula: Where S(a,b) represents the similarity between case a and case b, f a f b Let be the feature vectors of the two cases, · denotes the vector dot product, and ||·|| denotes the L2 norm of the vector; This formula quantifies the similarity of case features using cosine similarity and is used to retrieve the most relevant historical cases.

5. The skin cancer pathological image analysis system based on the Transformer model according to claim 1, characterized in that: It also includes a federated learning training module, which comprises: Local training units are deployed in various medical institutions to update model parameters based on local data. The parameter aggregation unit fuses parameters from each node using a secure aggregation algorithm (based on homomorphic encryption) to protect data privacy. The model adaptive unit dynamically adjusts the model regularization strength and learning rate based on the data distribution characteristics of each institution; The global parameter update of the parameter aggregation unit uses the following formula: Where, θ global θ represents the aggregated global model parameters. i For the local model parameters of the i-th institution, w i The weight of the i-th institution (satisfying) N represents the number of participating institutions; This formula achieves collaborative optimization of models in a federal scenario by merging parameters from various institutions through weighted averaging.

6. A method for analyzing skin cancer pathological images based on the Transformer model, characterized in that: Includes the following steps: S1. Image Acquisition and Quality Control: S11. Use a whole-section scanner to acquire multispectral digital images of skin pathology sections (including 400-700nm band information); S12. Select qualified images through a sharpness scoring algorithm (integrating gradient information and entropy calculation), remove abnormal staining samples through staining uniformity detection, and identify and mark artifacts such as folds and bubbles in the image through morphological operations; S2. Multi-scale image preprocessing: S21. Use the U-Net model to segment the tissue region and remove the background and slide edges; S22. Perform the following steps in sequence: illumination non-uniformity correction based on bilateral filtering, noise removal based on non-local means, and detail enhancement based on super-resolution reconstruction. S23. Perform color normalization processing to convert the image to a standard color space; S24. Adaptively divide image blocks based on histopathological type to generate an image block sequence containing spatial location information; S3. Hierarchical Feature Extraction: S31. A pre-trained convolutional neural network is used to extract local pathological features (including cell nuclear morphology, cytoplasmic staining intensity, and intercellular connection features) from each image patch; S32. Input local features into the improved Transformer model: Location information is added through pathological spatial perception coding; A dynamic deep self-attention mechanism (allocating more attention heads to the lesion region) is used to model global feature association; Integrate features from different levels through a cross-scale fusion module; S33. Attention-weighted pooling is used to aggregate features and generate a comprehensive feature vector; S4. Intelligent Diagnostic Reasoning: S41. Based on comprehensive feature vectors, this is achieved simultaneously through a multi-task learning framework: Skin cancer subtype classification (multi-classification task); Quantitative assessment of malignancy (regression task, output Breslow thickness value); Invasion depth classification (orderly classify tasks, output Clark level); S42. Calculate the confidence interval for each diagnostic result using Monte Carlo sampling; S43. Generate an attention heatmap to mark key diagnostic regions and output a ranking of feature importance; S5. Knowledge Graph-Assisted Decision Making: S51. Search for similar cases in the case knowledge graph to obtain reference diagnostic results; S52. Combine clinical indicators (such as patient age and lesion location) to perform rule-based reasoning and optimize diagnostic conclusions; S6. Output a structured diagnostic report and receive expert feedback for model iteration and optimization.

7. The method for analyzing skin cancer pathological images based on the Transformer model according to claim 6, characterized in that: The saliency score of the dynamic deep self-attention mechanism in step S32 is calculated using the following formula: S(p)=β·var(p)+(1-β)·d(p,μ norm ) Where S(p) is the significance score of image patch p, var(p) is the variance of the image patch features (reflecting feature complexity), and d(p,μ) is the feature score of the image patch. norm ) represents the mean μ of image patch features and normal tissue features. norm The Euclidean distance (reflecting the degree of anomaly), β is the weighting coefficient (value range [0,1]); This formula combines feature complexity and anomaly severity to assess the diagnostic importance of image patches and is used to dynamically allocate attention resources.

8. The method for analyzing skin cancer pathological images based on the Transformer model according to claim 6, characterized in that: The loss function for the multi-task learning framework in step S41 uses the following formula: L=w1L ce +w2L l1 +w3L ord Where L is the total loss, L ce For cross-entropy loss (used for subtype classification), L l1 For L1 loss (used in Breslow thickness measurement), L ord The ordered cross-entropy loss is used (for Clark classification), where w1, w2, and w3 are task weights (satisfying w1 + w2 + w3 = 1). This formula achieves collaborative optimization of multiple diagnostic objectives by weighted fusion of multi-task losses.

9. The method for analyzing skin cancer pathological images based on the Transformer model according to claim 6, characterized in that: The uncertainty quantification in step S42 uses the following formula: Where CI represents the 95% confidence interval. σ is the mean of the diagnostic results, σ ​​is the standard deviation of the Monte Carlo sampling results, and 1.96 is the critical value of the 95% confidence interval of the normal distribution; This formula calculates the confidence range of diagnostic results based on statistical principles, quantifying the uncertainty of model predictions.

10. The method for analyzing skin cancer pathological images based on the Transformer model according to claim 6, characterized in that: It also includes the model federated training step: S01. Each participating institution initializes its local model based on its local labeled dataset (including pathological images and corresponding diagnostic labels); S02. Local model training: The improved AdamW optimizer (with gradient clipping mechanism) is used to minimize the loss function, and the model parameter update amount is saved after each training round; S03. Encrypted parameter upload: The parameter update is encrypted using the Paillier homomorphic encryption algorithm and then uploaded to the federated server; S04. Global Parameter Aggregation: The federated server uses a weighted average (the weight is proportional to the amount of data from each institution) to aggregate encrypted parameters and generate global model parameters; S05. Model Distribution and Iteration: Distribute the decrypted global parameters to each institution, update the local model, and repeat steps S02-S04 until the model converges. S06. Model Personalization Adjustment: Each institution fine-tunes the global model based on a small amount of local data to adapt to the characteristics of local data distribution; The gradient clipping mechanism in step S02 uses the following formula: Where g is the original gradient, g′ is the clipped gradient, ||g|| is the L2 norm of the gradient, and θ is the clipping threshold; This formula prevents gradient explosion during training by limiting the gradient size, thus ensuring the stability of model training.

Images