Auxiliary film reading method and system based on artificial intelligence

Abstract

The invention discloses an auxiliary film reading method and system based on artificial intelligence, and the method comprises the steps: 1, collecting a pathological WSI, an electronic medical record, detection data and equipment parameters, correcting the equipment difference through adaptive dyeing normalization, and constructing a structured data package associated with an ID-timestamp of a patient; 2, developing a dynamic branch CNN, migrating teacher model knowledge through knowledge distillation, and introducing federated learning; step 3, the edge generates a thermodynamic diagram to mark a suspicious area, and the cloud outputs a structured report; step 4, constructing a normal tissue feature space by the variational auto-encoder, detecting abnormal slices and triggering expert re-checking; a reverse automatic encoder generates a pseudo-health image to compare and position a pathological area, and dynamic weight adjustment balances the federal learning convergence speed; 5, integrating the thermodynamic diagram, the gene data and the clinical indexes by a three-dimensional platform, and supporting multi-dimensional superposition display; webGL realizes browser end rendering, and NLP automatically generates a report abstract marked with a key evidence chain and is in butt joint with an international diagnosis and treatment guide.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of auxiliary reading of films, and in particular to an auxiliary reading method and system based on artificial intelligence. BACKGROUND

[0002] In the primary medical system, pathological diagnosis is the core basis for tumor screening, disease typing and treatment scheme making, but due to the problems of lack of professional pathologists, uneven equipment performance and dependence on manual diagnosis process, primary hospitals generally face the difficulty of low reading efficiency.

[0003] Taking a county hospital in Hebei Province as an example, the pathology department of the hospital has only 2 doctors, and 40-60 pathological sections need to be processed every day, and single diagnosis needs 15-20 minutes, which leads to an average waiting time of more than 24 hours for patients, and some complex cases are easily misjudged as benign due to atypical cell morphology, and the misdiagnosis rate of 12% found by the superior hospital review seriously affects the timely medical diagnosis of patients, and therefore an auxiliary reading method and system based on artificial intelligence are proposed. SUMMARY

[0004] The application aims to solve the problems in the prior art and provides an auxiliary reading method and system based on artificial intelligence.

[0005] In order to achieve the above-mentioned purpose, the application adopts the following technical scheme: An auxiliary reading method based on artificial intelligence comprises the following steps: Step 1: through a cross-modal data fusion engine, pathological section WSI images, electronic medical record texts, laboratory test data (such as blood routine, tumor markers) and equipment parameters (such as scanner model, staining batch) are synchronously collected, and a self-adaptive staining normalization algorithm is used to automatically correct the staining differences of different equipment or batches, so that the input data are consistent; at the same time, through a multi-modal data alignment module, the patient ID and the time stamp are associated, and a structured data package is constructed, thereby providing a unified and high-quality data basis for subsequent analysis; Step 2: a dynamic branch convolutional neural network is developed, the model branches are automatically switched according to the computing power of the primary equipment, such as enabling a lightweight branch MobileNetV3 for low-end equipment and enabling a high-precision branch ResNet-152 for high-end equipment; the knowledge of the high-precision branch is migrated to the lightweight branch through knowledge distillation, so that the accuracy on the low-end equipment is high; at the same time, a federated learning framework is introduced, which allows the primary hospital to jointly train the model without domain data, thereby improving the recognition ability of common cases such as gastritis and cervicitis; Step 3: Deploy edge nodes (such as NVIDIA Jetson devices) in primary hospitals to handle slide preprocessing, preliminary screening (such as completing initial screening of tumor areas within 30 seconds), and data encryption; cloud servers (such as Alibaba Cloud ECS) handle complex model inference, multi-institution data comparison, and long-term storage; edge nodes generate heat maps to mark suspicious areas, and the cloud outputs structured diagnostic reports, including diagnostic evidence, a database of similar cases, and literature support, forming a collaborative architecture of "edge initial screening - cloud-based detailed diagnosis" to improve diagnostic efficiency and reduce latency; Step 4: Construct a normal tissue feature space using a variational autoencoder, detect slices that deviate significantly from the normal pattern, automatically trigger the expert review process, and include the case in the dynamic training dataset; at the same time, use an inverse autoencoder to generate pseudo-healthy images, compare the differences between the original image and the reconstructed image to accurately locate the pathological area; use a dynamic weight adjustment algorithm to balance the model convergence speed of each primary hospital in federated learning, and avoid model bias caused by data silos. Step 5: Develop a 3D visualization interactive platform that integrates pathological slide heatmaps, gene testing data, and clinical indicators, supporting multi-dimensional overlay display. Doctors can adjust the transparency using sliders to intuitively verify the AI's judgment basis and retrieve recommended solutions from international treatment guidelines with one click. The platform uses WebGL technology to achieve 3D rendering on the browser side, supporting smooth scaling and rotation of million-pixel slides. It also integrates a natural language processing interface to automatically generate diagnostic report summaries annotated with key evidence chains, improving doctors' efficiency in understanding AI decision-making.

[0006] The above technical solution further includes: Furthermore, the automatic correction of staining differences between different devices or batches using the adaptive staining normalization algorithm includes the following steps: Identify key variables that cause staining differences: batch of staining solution (such as pH fluctuations in hematoxylin-eosin staining solution), equipment characteristics (spectral response differences between different brands of scanners), operating procedures (errors in staining time ±15 seconds), and environmental factors (changes in temperature and humidity leading to changes in staining diffusion coefficient); construct a staining difference fingerprint database, and collect typical staining characteristics of different devices / batches through hyperspectral imaging technology, such as RGB intensity distribution curves of cell nuclei and cytoplasm, to form a traceable staining difference map. The original RGB image is converted to the CIELAB color space, which is based on the human visual perception model and can separate luminance (L channel) and chromaticity (a, b* channels) information, avoiding color distortion caused by direct operation in the RGB space; the staining feature vector is extracted, the Lab value distribution of each pixel is calculated, and a staining feature matrix is ​​generated, such as the mean ± standard deviation of the a channel in the cell nucleus region and the skewness of the b* channel in the interstitial region. A dynamic statistical model is constructed, a reference staining pattern is selected, and 1000 cases are randomly selected from slices produced by standard staining procedures (such as those produced by a laboratory certified by ISO 13485) to form a benchmark dataset. The mean, covariance matrix and principal component analysis characteristics of the Lab* distribution are calculated. A staining correction mapping function is established, and the color mapping relationship between the benchmark dataset and the image to be corrected is learned through a nonlinear regression model, such as a radial basis function network, to generate a transformation function from the biased staining pattern to the standard pattern. The scanner serial number or staining solution batch number is automatically matched with the pre-stored calibration parameters. If it is a new device / batch, the fast self-learning mode is activated, and the first 50 slides are collected for model fine-tuning. A mapping function is applied to each pixel for color correction, focusing on adjusting the contrast between the cell nucleus (purple tone in H&E staining) and the cytoplasm (pink tone) to ensure that the nucleus-cytoplasm ratio meets the pathological diagnostic criteria, such as NCR>0.8 in the diagnosis of breast cancer. Calculate the ΔE value (color difference) between the corrected image and the baseline dataset, requiring ΔE < 1.5 (compliant with the International Association of Pathologists (IAP) standard); randomly select 200 slides each before and after correction, and have 3 senior pathologists conduct diagnostic consistency evaluation to ensure that the false negative rate of the corrected image is reduced by ≥ 40%; automatically incorporate the validated correction parameters into the staining fingerprint database to form a continuously optimized closed-loop system.

[0007] Furthermore, the development of the dynamic branch convolutional neural network, which automatically switches model branches based on the computing power of the underlying device, and transfers knowledge from the high-precision branch to the lightweight branch through knowledge distillation, includes the following steps: Real-time device computing power metrics, including CPU core count, GPU model (such as Adreno 650 / Mali-G76), memory capacity, and free memory ratio, can be obtained through APIs (such as Android NDK or Linux / proc / cpuinfo). Establish a three-tier computing power evaluation system: Lightweight (RAM < 4GB, no dedicated GPU): Adapted to ultra-lightweight models such as MobileNetV3 / ShuffleNetV2; Standard level (RAM 4-8GB, integrated GPU): Compatible with medium-sized models such as EfficientNet-Lite / ResNet-18; High-precision version (RAM>8GB, dedicated GPU): Adapted to large models such as ResNet-152 / Inception-v4.

[0008] The corresponding branch is automatically loaded based on the detection results. For example, on the Rockchip RK3399 chip (dual Cortex-A72 + quad Cortex-A53), the model size is compressed from 217MB for the high-precision branch to 3.2MB for the lightweight branch, and the inference speed is increased by 12 times (from 470ms to 39ms). Design the teacher-student model architecture: Teacher model (high-precision branch): Employs a deep convolutional network (such as ResNet-152), trained on a cloud server, to extract high-dimensional features and generate soft labels; soft labels contain relative probabilities between categories (such as breast cancer probability 0.7 vs benign 0.3), providing richer information than hard labels (0 or 1); Student model (lightweight branch): Based on the MobileNetV3 architecture optimization, it reduces the amount of computation through depthwise separable convolution, with only 2.3% of the number of parameters of the teacher model (5M vs 62M), while maintaining 94.7% diagnostic consistency (Kappa coefficient 0.89). Knowledge distillation and transfer: Output layer distillation: Aligning the output distributions of the teacher and student models using the KL divergence loss function.

[0009] in, λ is the total loss function for knowledge distillation, which combines the real label loss and the teacher model knowledge transfer loss; λ is the balancing coefficient (usually set to 0.7), which controls the weight ratio of the real label loss and the teacher model knowledge transfer loss. is the cross-entropy loss function, which measures the difference between the student model output and the true label; The true label of the sample, such as "benign" or "malignant", is represented in one-hot encoding form; The output probability distribution (soft label) of the student model represents the predicted probability of the model for each category; KL divergence measures the difference in distribution between the teacher model output and the student model output. The output probability distribution of the teacher model contains richer information on relative probabilities between categories than hard labels; Intermediate layer feature distillation: Introducing feature alignment loss forces the feature maps of intermediate layers in the student model (such as the output of the 10th convolutional layer) to be similar to the feature maps of the corresponding layers in the teacher model.

[0010] in, The feature alignment loss function forces the feature maps of the intermediate layers of the student model to be similar to the feature maps of the corresponding layers of the teacher model; N is the number of channels in the feature map (e.g., the number of channels in the output of the 10th convolutional layer is 64). This is the feature map of the i-th channel in the intermediate layer of the teacher model (e.g., the 5th channel output from the 10th convolution layer). The feature map of the i-th channel in the intermediate layer of the student model (with the same layer structure as the teacher model); The teacher model and student model share the bottom feature extraction layer (such as the first 5 convolutional layers), while the high-level branches are processed independently. During training, the teacher model has fixed parameters, while the student model updates weights through backpropagation using distillation loss. To address the characteristics of primary pathological data, such as small sample size and class imbalance, enhancement methods are adopted, including staining normalization (such as CIELAB spatial correction in step one), random occlusion of cell regions (simulating slide stains or folds), and multi-scale scaling (0.5x-2x random scaling), to adapt to different scanner resolutions. When the device is used for the first time, all branches are preloaded, and subsequent switching is performed quickly based on the computing power test results (switching time is less than 500ms). For example, on the Huawei MatePad Pro (Kirin 990 chip), when the model switches from the lightweight branch to the standard branch, the inference latency of the first frame only increases by 120ms. The lightweight branch performs initial inference on the local device, while the high-precision branch performs complex analysis on the cloud server. The joint update of model parameters is achieved through a federated learning framework (such as TensorFlow Federated), so that the basic data does not leave the domain.

[0011] Furthermore, the edge nodes generate heatmaps to mark suspicious areas, and the cloud outputs a structured diagnostic report, including the following steps: The edge nodes first perform adaptive thresholding on the WSI (whole slice image) to remove background noise such as slide stains and bubbles, and standardize the image resolution (e.g., to 0.5 μm / pixel at 20× magnification). Based on the dynamically switched lightweight branch (e.g., MobileNetV3) in step two, the preprocessed slices are initially screened, focusing on detecting the following three types of anomalies: Morphological abnormalities: The size and shape of the cell nucleus deviate from the normal range, such as the diameter of the nucleus of breast cancer cells being greater than 15 μm; Abnormal distribution: Disordered cell arrangement patterns, such as adenocarcinoma cells being distributed in solid nests; Abnormal staining intensity: The staining intensity of a specific marker (such as Ki-67) exceeds the threshold, such as a positive rate greater than 30%; Grad-CAM (Gradient Weighted Class Activation Mapping) is used to calculate the attention distribution of the model on the input image through backpropagation, generating a heatmap; in the heatmap: Red areas: Regions of high interest to the model, such as areas with dense cell nuclei; Blue area: Background organization that the model considers normal; Based on the heatmap intensity threshold, the boundaries of suspicious areas are automatically marked and labeled, such as rectangles or free curves. At the same time, the coordinates, area and feature description of the area are recorded, such as "cell clusters with nucleocytoplasmic ratio greater than 0.8". Heatmaps and annotation information are encrypted with AES-256 to ensure data transmission security; the encrypted data packets are compressed to 15% of the original data size (using a lossless compression algorithm) to adapt to the bandwidth limitations of grassroots networks, such as an average upload speed of less than 10Mbps; based on the risk level of suspicious areas, high-risk cases are prioritized for uploading to the cloud, and the data transmission queue is dynamically adjusted to ensure that emergency cases are synchronized to the cloud within 30 seconds. The high-precision branch (such as ResNet-152) in step two of cloud deployment performs secondary verification on suspicious areas uploaded by edge nodes, focusing on solving the following issues: False positive filtering: Eliminating misjudgments caused by slide folding or uneven staining, such as misjudging normal tissue as tumor; Subtype classification: Suspicious areas are further classified, such as "adenocarcinoma" being further divided into "invasive ductal carcinoma" or "mucinous adenocarcinoma".

[0012] The current case is compared with similar cases in the cloud database, such as the same pathological type and the same age group, to generate a similarity ranking, such as the IDs and diagnosis results of the top 5 similar cases, to assist doctors in differential diagnosis. Generate structured diagnostic reports based on international pathology report templates, such as CAP (College of American Pathologists Standard). The automatically generated structured report includes the following modules: Diagnostic conclusion: Determine the pathological type and grade, such as "invasive ductal carcinoma of the breast, grade II"; Diagnostic criteria: List the key features that support the diagnosis, such as "significant nuclear atypia, with more than 10 mitotic figures per 10 HPF"; Recommendation: Further examinations are recommended, such as immunohistochemical testing of ER, PR, and HER2 status, or treatment regimens, such as neoadjuvant chemotherapy.

[0013] The report embeds heatmaps and annotation boxes, allowing doctors to adjust the transparency using a slider to visually verify the AI's judgment criteria, such as "red areas correspond to nuclear atypia, and blue areas represent normal stroma"; the report automatically links to the latest treatment guidelines, indicating the degree of match between recommended treatments and the current diagnosis.

[0014] Furthermore, the process of constructing a normal tissue feature space through a variational autoencoder, detecting slices that significantly deviate from the normal pattern, automatically triggering an expert review process, and incorporating the case into a dynamic training dataset includes the following steps: Collect large-scale normal tissue sections, covering different organs (breast, lung, thyroid, etc.) and populations (age, sex); perform standardized processing on the sections, such as staining normalization in step one, to ensure the consistency of input data; Variational autoencoder training: Encoder: Compresses high-dimensional slice images into low-dimensional latent space representations (e.g., 16-dimensional), and forces the latent distribution to approximate a standard normal distribution through KL divergence loss; Decoder: Reconstructs slices from the latent space, minimizes reconstruction errors (such as mean square error MSE), and ensures that the encoder captures key features of normal tissue; Feature space visualization: By using t-SNE or UMAP dimensionality reduction, observe the aggregation of normal tissues in the latent space, such as normal breast tissue forming a tight cluster, while cancerous tissue is scattered on the periphery; Dual detection: Reconstruction error threshold: Calculate the difference between the input slice and the VAE reconstructed slice (e.g., SSIM index). When SSIM < 0.7, it is marked as abnormal (normal tissue SSIM > 0.85). Latent space deviation: Calculate the degree of deviation of the distribution of the input slice in the latent space from the normal pattern (mean ± 3σ). An alarm is triggered when the deviation is > 0.5. Generate a heatmap to display abnormal areas and label specific features, such as "abnormal cell nucleus size" or "increased interstitial density"; output an anomaly confidence score, and automatically enter the review process when the score is greater than 70. Based on the type of abnormality (e.g., breast, lung) and the degree of urgency (e.g., high-risk cases are prioritized), slides are assigned to pathology experts in the corresponding fields. Experts view the slides, heat maps, and AI analysis reports through remote consultation platforms, such as the provincial pathology service platform, and complete the review within 15 minutes. After the expert confirms the diagnosis, a review report is automatically generated, which includes the expert's opinion, the revised diagnosis and its basis. The review results are fed back to the primary hospitals in real time and the patient's electronic medical record is updated simultaneously. Abnormal cases confirmed by experts are labeled as true cases, such as "invasive ductal carcinoma of the breast," and stored in a dynamic dataset along with the original slides, heatmaps, and review reports. The dataset is categorized by organ, pathological type, and population characteristics to form a structured knowledge base. The VAE model is fine-tuned monthly with new data, focusing on optimizing the false positive rate, such as reducing the proportion of breast fibroadenomas misdiagnosed as cancer from 12% to 3%. Incremental learning techniques are used to ensure that historical knowledge is not forgotten when the model is updated. False positive / false negative monitoring: Track the model's performance in the production environment, triggering an alarm when the false positive rate is >5% or the false negative rate is >2%, automatically tracing back problematic cases; generating monthly quality reports, comparing the consistency between the model and expert diagnoses (e.g., the Kappa coefficient improves from 0.82 to 0.91).

[0015] Furthermore, the method employs a reverse autoencoder to generate pseudo-healthy images, and compares the differences between the original and reconstructed images to accurately locate pathological regions; a dynamic weight adjustment algorithm is used to balance the model convergence speed of each primary hospital in the federated learning process, including the following steps: The inverse autoencoder accurately locates pathological areas, employing a VAE-GAN (Variational Autoencoder-Generative Adversarial Network) fusion architecture, in which: Encoder: Compresses the original pathological image (e.g., 1024×1024 pixels) into a low-dimensional latent space representation (e.g., 128 dimensions), capturing the statistical features of normal tissue; Generator: Reconstructs images from the latent space and generates highly realistic pseudo-healthy images (SSIM index > 0.85) through adversarial training (playing against the discriminator). Discriminator: Distinguishes between real, normal images and generated fake healthy images, forcing the generator to produce more realistic results; VAE-GAN is trained using large-scale normal tissue slices to ensure that the generator can accurately reconstruct normal tissue structures (such as gland arrangement and cell distribution); a small number of pathological slices are introduced, and conditional generation techniques such as adding pathological labels as input enable the generator to generate pseudo-healthy images corresponding to the pathological type. The original pathological image and the generated pseudo-healthy image are spatially aligned, and rotation and translation differences are corrected by affine transformation to ensure pixel-level correspondence. Difference calculation: Structural differences: Calculate the SSIM (structural similarity) index between two images, and mark low SSIM regions (e.g., less than 0.7) as potential pathological regions; Feature Differences: Extract cell-level features, such as nuclear size and nucleocytoplasmic ratio, and compare the feature distribution of the original image with that of the pseudo-healthy image. Regions with differences exceeding a threshold, such as a nucleocytoplasmic ratio greater than 0.8, are marked. The differential regions are mapped onto a heatmap, with red indicating high differential and blue indicating low differential, and then overlaid onto the original image to visually display the boundaries of pathological regions such as breast cancer cell clusters; The dynamic weight adjustment algorithm balances the convergence speed of federated learning. The data distribution of primary hospitals varies greatly. For example, county hospitals mainly have breast cancer cases, while city hospitals mainly have lung cancer cases. This results in the model converging quickly on local data, but the generalization ability of the global model is poor. Some hospitals use high-end GPUs, while others rely solely on CPUs, resulting in a large difference in model update frequency (from daily updates to weekly updates). Dynamic weight adjustments are performed. Each hospital records the model's loss value on the local validation set, such as cross-entropy loss. When the loss value decreases by less than 1% for three consecutive rounds, it is judged as "local convergence". The gradient updates of all hospitals are aggregated, and the variance of the gradient direction is calculated. If the variance is greater than 0.1, it is judged as "global non-convergence" using the Welford algorithm. For hospitals with slow convergence speeds that rely solely on CPU updates, increase their gradient weights to make their updates have a greater impact on the global model; for hospitals with fast convergence speeds that use GPU updates, decrease their gradient weights to prevent them from dominating the global model's direction; Dynamic balancing formula:

[0016] in, Let i be the gradient variance of hospital i, and ensure that hospitals with high variance (non-converged) receive higher weights.

[0017] Furthermore, the platform employs WebGL technology for browser-side 3D rendering and integrates a natural language processing interface to automatically generate a diagnostic report summary annotated with key evidence chains, including the following steps: This system integrates pathological slide heatmaps (two-dimensional), gene testing data (one-dimensional mutation frequency), and clinical indicators such as patient age and tumor marker levels to construct a three-dimensional data model. For example, it integrates HER2 immunohistochemical heatmaps of breast cancer cases with ER / PR gene expression data and patient CA15-3 levels into the same three-dimensional coordinate system. Utilizing the browser's built-in GPU acceleration (no plugins required), the following functions are achieved through vertex shaders and fragment shaders: 3D rotation and zoom: Supports mouse or touch operation to achieve 360° rotation of pathological slides (such as observing tumor boundaries) and multi-level zoom (from tissue level to cell level). Transparency overlay: By adjusting the layer transparency, heatmaps (red high-risk areas) and gene data (blue low-expression areas) can be displayed simultaneously, helping doctors identify key evidence chains, such as the co-location of HER2 positive regions and TP53 mutations; Dynamic slide browsing: Simulates the movement of the microscope stage, allowing users to quickly switch between different areas of the WSI via keyboard or gestures, such as from the tumor center to the edge, improving diagnostic efficiency; The AI-generated heatmap (with red markings for suspicious areas) is overlaid on the original pathological slide in a semi-transparent manner. Doctors can adjust the transparency of the heatmap using a slider (e.g., 30% transparency to observe the underlying cell structure). High-risk area boundaries (e.g., breast cancer cell clusters) are automatically marked, and characteristic parameters of the area are displayed, such as a nucleocytoplasmic ratio greater than 0.8 and a Ki-67 positivity rate greater than 30%. Pathological features (e.g., cellular atypia), genetic data (e.g., BRCA1 mutation), and clinical indicators (e.g., elevated CA125) are presented in the form of three-dimensional bar charts or scatter plots. Clicking on the legend allows you to jump to the corresponding evidence source; for example, clicking on the BRCA1 mutation bar chart will take you to the gene testing report. The NLP interface receives the following data output from the 3D platform: Pathological features: parameters of suspicious areas marked on the thermogram, such as nuclear size and staining intensity; Genetic data: mutation sites, expression levels, such as HER2 positivity defined as IHC 3+ or FISH amplification; Clinical indicators: tumor markers, patient history, such as family history of breast cancer; Through rule engines (such as regular expression matching) and pre-trained models, core evidence supporting the diagnosis is extracted, such as "significant nuclear atypia, mitotic figures >10 / 10HPF" (pathological features), "HER2 gene amplification, FISH ratio = 2.8" (genetic data), and "patient's CA15-3 level is 120U / mL (normal <30U / mL)" (clinical indicators). Based on international pathology reporting standards (such as the CAP protocol), generate an abstract containing the following modules: Diagnostic conclusion: Determine the pathological type and grade, such as "invasive ductal carcinoma of the breast, grade II"; Diagnostic criteria: Prioritize key evidence, such as "HER2 positive (level of evidence Ia)"; Recommendation: Further examination (such as PD-L1 testing) or treatment options (such as targeted therapy) are recommended. Transform structured data into natural language using a pre-trained model to ensure the report meets the following requirements: Readability: Avoid overuse of technical terms, such as replacing "high mitotic figures" with "actively growing tumor cells"; Compliance: Citify the latest treatment guidelines and indicate the degree of matching between the recommended treatment and the current diagnosis, such as "Complies with NCCN Guideline Class I Recommendation".

[0018] An AI-based assisted film viewing system includes the following steps: Data Foundation Layer: By integrating pathology slide scanners, electronic medical record systems, and laboratory information systems, high-resolution WSI images, patient clinical texts, and test data are acquired simultaneously. CIELAB staining normalization (ΔE<1.5) and adaptive threshold segmentation algorithms are used to remove noise and standardize image resolution to 20× magnification (0.5μm / pixel). Finally, a three-dimensional index library of patient ID-timestamp-data type is constructed, and the storage space of a single WSI case is reduced to 800MB through lossless compression, providing a high-quality, standardized data foundation for subsequent analysis. Core algorithm layer: Deploys a dynamic branch convolutional neural network, automatically switches model branches according to device computing power, and transfers high-precision branch knowledge to lightweight branches through knowledge distillation (KL divergence loss function); In the federated learning framework, the dynamic weight adjustment algorithm balances the convergence speed of primary hospitals, and differential privacy protection (σ=0.1) ensures data security; Variational autoencoder constructs a normal tissue feature space, detects abnormal slices through SSIM exponent (greater than 0.85), automatically triggers expert review and updates the dynamic training dataset; Application Service Layer: Edge nodes complete slice preprocessing (such as adaptive threshold segmentation) and preliminary screening (filtering 78.3% of negative cases); cloud servers handle complex model inference and multi-institution data comparison to generate structured diagnostic reports; the WebGL engine renders millions of voxel data (frame rate > 60fps), supports the overlay of heatmaps and original images (adjustable transparency) and the display of three-dimensional evidence chains (such as the correlation between cytoplasmic ratio, HER2 amplification, and clinical indicators); the NLP interface extracts key evidence (such as "mitotic figures > 10 / 10HPF"), generates structured report summaries that conform to CAP standards, and marks the degree of matching with treatment guidelines (such as "conforming to NCCN Class I recommendations"). User interaction layer: The doctor workstation supports pathologists in verifying AI judgment criteria through a 3D platform (such as adjusting the transparency of heatmaps with sliders), and clinicians can quickly view report summaries and formulate treatment plans; the expert review platform is connected to the provincial remote consultation system, completing the review of high-risk cases within 15 minutes and providing feedback to the grassroots level; the system management backend monitors model performance through a quality control dashboard (such as triggering alarms when the false negative rate is greater than 2%), and the resource scheduling center dynamically allocates edge-cloud computing resources (such as GPU cluster load balancing) to ensure service stability and compliance.

[0019] The present invention has the following beneficial effects: In this invention, a cross-modal data fusion engine and adaptive staining normalization are used to construct structured data packets, eliminating preprocessing time consumption caused by device differences. A dynamic branch convolutional neural network is deployed, combined with knowledge distillation and federated learning frameworks, to achieve adaptive model switching based on computing power and joint training of local data, avoiding data transmission delays. Edge nodes complete slice preprocessing, preliminary screening, and data encryption, while the cloud server handles complex inference and multi-institution comparisons to generate structured reports. Variational autoencoders are used to detect abnormal slices and trigger expert review, while inverse autoencoders accurately locate pathological areas. Dynamic weight adjustment balances the convergence speed of federated learning. Finally, WebGL 3D rendering and NLP report generation support heatmap overlay verification and key evidence chain annotation, greatly shortening report generation time and ultimately achieving a dual breakthrough in improving slide reading efficiency and quality assurance. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the steps of an AI-based assisted image reading method proposed in this invention. Figure 2 This is a system block diagram of an artificial intelligence-based assisted image reading system proposed in this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 As shown, the present invention is an artificial intelligence-based assisted image reading method and system, comprising: An AI-based assisted film viewing method includes: Step 1: Through a cross-modal data fusion engine, pathological slide WSI images, electronic medical record text, laboratory test data (such as complete blood count and tumor markers), and equipment parameters (such as scanner model and staining batch) are simultaneously acquired. An adaptive staining normalization algorithm is used to automatically correct staining differences between different devices or batches, ensuring consistency of input data. At the same time, through a multimodal data alignment module, patient IDs and timestamps are associated to construct structured data packages, providing a unified and high-quality data foundation for subsequent analysis. Step 2: Develop a dynamic branch convolutional neural network that automatically switches model branches based on the computing power of the grassroots devices. For example, low-end devices use the lightweight branch MobileNetV3, while high-end devices use the high-precision branch ResNet-152. Knowledge distillation is used to transfer knowledge from the high-precision branch to the lightweight branch, resulting in higher accuracy on low-end devices. At the same time, a federated learning framework is introduced, allowing grassroots hospitals to jointly train the model without leaving their local data domain, thereby improving their ability to identify common cases at the grassroots level, such as gastritis and cervicitis. Step 3: Deploy edge nodes (such as NVIDIA Jetson devices) in primary hospitals to handle slide preprocessing, preliminary screening (such as completing initial screening of tumor areas within 30 seconds), and data encryption; cloud servers (such as Alibaba Cloud ECS) handle complex model inference, multi-institution data comparison, and long-term storage; edge nodes generate heat maps to mark suspicious areas, and the cloud outputs structured diagnostic reports, including diagnostic evidence, a database of similar cases, and literature support, forming a collaborative architecture of "edge initial screening - cloud-based detailed diagnosis" to improve diagnostic efficiency and reduce latency; Step 4: Construct a normal tissue feature space using a variational autoencoder, detect slices that deviate significantly from the normal pattern, automatically trigger the expert review process, and include the case in the dynamic training dataset; at the same time, use an inverse autoencoder to generate pseudo-healthy images, compare the differences between the original image and the reconstructed image to accurately locate the pathological area; use a dynamic weight adjustment algorithm to balance the model convergence speed of each primary hospital in federated learning, and avoid model bias caused by data silos. Step 5: Develop a 3D visualization interactive platform that integrates pathological slide heatmaps, gene testing data, and clinical indicators, supporting multi-dimensional overlay display. Doctors can adjust the transparency using sliders to intuitively verify the AI's judgment basis and retrieve recommended solutions from international treatment guidelines with one click. The platform uses WebGL technology to achieve 3D rendering on the browser side, supporting smooth scaling and rotation of million-pixel slides. It also integrates a natural language processing interface to automatically generate diagnostic report summaries annotated with key evidence chains, improving doctors' efficiency in understanding AI decision-making.

[0023] In one embodiment, the automatic correction of staining differences between different devices or batches using an adaptive staining normalization algorithm includes the following steps: Identify key variables that cause staining differences: batch of staining solution (such as pH fluctuations in hematoxylin-eosin staining solution), equipment characteristics (spectral response differences between different brands of scanners), operating procedures (errors in staining time ±15 seconds), and environmental factors (changes in temperature and humidity leading to changes in staining diffusion coefficient); construct a staining difference fingerprint database, and collect typical staining characteristics of different devices / batches through hyperspectral imaging technology, such as RGB intensity distribution curves of cell nuclei and cytoplasm, to form a traceable staining difference map. The original RGB image is converted to the CIELAB color space, which is based on the human visual perception model and can separate luminance (L channel) and chromaticity (a, b* channels) information, avoiding color distortion caused by direct operation in the RGB space; the staining feature vector is extracted, the Lab value distribution of each pixel is calculated, and a staining feature matrix is ​​generated, such as the mean ± standard deviation of the a channel in the cell nucleus region and the skewness of the b* channel in the interstitial region. A dynamic statistical model is constructed, a reference staining pattern is selected, and 1000 cases are randomly selected from slices produced by standard staining procedures (such as those produced by a laboratory certified by ISO 13485) to form a benchmark dataset. The mean, covariance matrix and principal component analysis characteristics of the Lab* distribution are calculated. A staining correction mapping function is established, and the color mapping relationship between the benchmark dataset and the image to be corrected is learned through a nonlinear regression model, such as a radial basis function network, to generate a transformation function from the biased staining pattern to the standard pattern. The scanner serial number or staining solution batch number is automatically matched with the pre-stored calibration parameters. If it is a new device / batch, the fast self-learning mode is activated, and the first 50 slides are collected for model fine-tuning. A mapping function is applied to each pixel for color correction, focusing on adjusting the contrast between the cell nucleus (purple tone in H&E staining) and the cytoplasm (pink tone) to ensure that the nucleus-cytoplasm ratio meets the pathological diagnostic criteria, such as NCR>0.8 in the diagnosis of breast cancer. Calculate the ΔE value (color difference) between the corrected image and the baseline dataset, requiring ΔE < 1.5 (compliant with the International Association of Pathologists (IAP) standard); randomly select 200 slides each before and after correction, and have 3 senior pathologists conduct diagnostic consistency evaluation to ensure that the false negative rate of the corrected image is reduced by ≥ 40%; automatically incorporate the validated correction parameters into the staining fingerprint database to form a continuously optimized closed-loop system.

[0024] In one embodiment, the development of a dynamically branched convolutional neural network, which automatically switches model branches based on the computing power of the underlying device and transfers knowledge from the high-precision branch to the lightweight branch through knowledge distillation, includes the following steps: Real-time device computing power metrics, including CPU core count, GPU model (such as Adreno 650 / Mali-G76), memory capacity, and free memory ratio, can be obtained through APIs (such as Android NDK or Linux / proc / cpuinfo). Establish a three-tier computing power evaluation system: Lightweight (RAM < 4GB, no dedicated GPU): Adapted to ultra-lightweight models such as MobileNetV3 / ShuffleNetV2; Standard level (RAM 4-8GB, integrated GPU): Compatible with medium-sized models such as EfficientNet-Lite / ResNet-18; High-precision version (RAM>8GB, dedicated GPU): Adapted to large models such as ResNet-152 / Inception-v4.

[0025] The corresponding branch is automatically loaded based on the detection results. For example, on the Rockchip RK3399 chip (dual Cortex-A72 + quad Cortex-A53), the model size is compressed from 217MB for the high-precision branch to 3.2MB for the lightweight branch, and the inference speed is increased by 12 times (from 470ms to 39ms). Design the teacher-student model architecture: Teacher model (high-precision branch): Employs a deep convolutional network (such as ResNet-152), trained on a cloud server, to extract high-dimensional features and generate soft labels; soft labels contain relative probabilities between categories (such as breast cancer probability 0.7 vs benign 0.3), providing richer information than hard labels (0 or 1); Student model (lightweight branch): Based on the MobileNetV3 architecture optimization, it reduces the amount of computation through depthwise separable convolution, with only 2.3% of the number of parameters of the teacher model (5M vs 62M), while maintaining 94.7% diagnostic consistency (Kappa coefficient 0.89). Knowledge distillation and transfer: Output layer distillation: Aligning the output distributions of the teacher and student models using the KL divergence loss function.

[0026] in, λ is the total loss function for knowledge distillation, which combines the real label loss and the teacher model knowledge transfer loss; λ is the balancing coefficient (usually set to 0.7), which controls the weight ratio of the real label loss and the teacher model knowledge transfer loss. is the cross-entropy loss function, which measures the difference between the student model output and the true label; The true label of the sample, such as "benign" or "malignant", is represented in one-hot encoding form; The output probability distribution (soft label) of the student model represents the predicted probability of the model for each category; KL divergence measures the difference in distribution between the teacher model output and the student model output. The output probability distribution of the teacher model contains richer information on relative probabilities between categories than hard labels; Intermediate layer feature distillation: Introducing feature alignment loss forces the feature maps of intermediate layers in the student model (such as the output of the 10th convolutional layer) to be similar to the feature maps of the corresponding layers in the teacher model.

[0027] in, The feature alignment loss function forces the feature maps of the intermediate layers of the student model to be similar to the feature maps of the corresponding layers of the teacher model; N is the number of channels in the feature map (e.g., the number of channels in the output of the 10th convolutional layer is 64). This is the feature map of the i-th channel in the intermediate layer of the teacher model (e.g., the 5th channel output from the 10th convolution layer). The feature map of the i-th channel in the intermediate layer of the student model (with the same layer structure as the teacher model); The teacher model and student model share the bottom feature extraction layer (such as the first 5 convolutional layers), while the high-level branches are processed independently. During training, the teacher model has fixed parameters, while the student model updates weights through backpropagation using distillation loss. To address the characteristics of primary pathological data, such as small sample size and class imbalance, enhancement methods are adopted, including staining normalization (such as CIELAB spatial correction in step one), random occlusion of cell regions (simulating slide stains or folds), and multi-scale scaling (0.5x-2x random scaling), to adapt to different scanner resolutions. When the device is used for the first time, all branches are preloaded, and subsequent switching is performed quickly based on the computing power test results (switching time is less than 500ms). For example, on the Huawei MatePad Pro (Kirin 990 chip), when the model switches from the lightweight branch to the standard branch, the inference latency of the first frame only increases by 120ms. The lightweight branch performs initial inference on the local device, while the high-precision branch performs complex analysis on the cloud server. The joint update of model parameters is achieved through a federated learning framework (such as TensorFlow Federated), so that the basic data does not leave the domain.

[0028] In one embodiment, the edge node generates a heatmap to mark suspicious areas, and the cloud outputs a structured diagnostic report, including the following steps: The edge nodes first perform adaptive thresholding on the WSI (whole slice image) to remove background noise such as slide stains and bubbles, and standardize the image resolution (e.g., to 0.5 μm / pixel at 20× magnification). Based on the dynamically switched lightweight branch (e.g., MobileNetV3) in step two, the preprocessed slices are initially screened, focusing on detecting the following three types of anomalies: Morphological abnormalities: The size and shape of the cell nucleus deviate from the normal range, such as the diameter of the nucleus of breast cancer cells being greater than 15 μm; Abnormal distribution: Disordered cell arrangement patterns, such as adenocarcinoma cells being distributed in solid nests; Abnormal staining intensity: The staining intensity of a specific marker (such as Ki-67) exceeds the threshold, such as a positive rate greater than 30%; Grad-CAM (Gradient Weighted Class Activation Mapping) is used to calculate the attention distribution of the model on the input image through backpropagation, generating a heatmap; in the heatmap: Red areas: Regions of high interest to the model, such as areas with dense cell nuclei; Blue area: Background organization that the model considers normal; Based on the heatmap intensity threshold, the boundaries of suspicious areas are automatically marked and labeled, such as rectangles or free curves. At the same time, the coordinates, area and feature description of the area are recorded, such as "cell clusters with nucleocytoplasmic ratio greater than 0.8". Heatmaps and annotation information are encrypted with AES-256 to ensure data transmission security; the encrypted data packets are compressed to 15% of the original data size (using a lossless compression algorithm) to adapt to the bandwidth limitations of grassroots networks, such as an average upload speed of less than 10Mbps; based on the risk level of suspicious areas, high-risk cases are prioritized for uploading to the cloud, and the data transmission queue is dynamically adjusted to ensure that emergency cases are synchronized to the cloud within 30 seconds. The high-precision branch (such as ResNet-152) in step two of cloud deployment performs secondary verification on suspicious areas uploaded by edge nodes, focusing on solving the following issues: False positive filtering: Eliminating misjudgments caused by slide folding or uneven staining, such as misjudging normal tissue as tumor; Subtype classification: Suspicious areas are further classified, such as "adenocarcinoma" being further divided into "invasive ductal carcinoma" or "mucinous adenocarcinoma".

[0029] The current case is compared with similar cases in the cloud database, such as the same pathological type and the same age group, to generate a similarity ranking, such as the IDs and diagnosis results of the top 5 similar cases, to assist doctors in differential diagnosis. Generate structured diagnostic reports based on international pathology report templates, such as CAP (College of American Pathologists Standard). The automatically generated structured report includes the following modules: Diagnostic conclusion: Determine the pathological type and grade, such as "invasive ductal carcinoma of the breast, grade II"; Diagnostic criteria: List the key features that support the diagnosis, such as "significant nuclear atypia, with more than 10 mitotic figures per 10 HPF"; Recommendation: Further examinations are recommended, such as immunohistochemical testing of ER, PR, and HER2 status, or treatment regimens, such as neoadjuvant chemotherapy.

[0030] The report embeds heatmaps and annotation boxes, allowing doctors to adjust the transparency using a slider to visually verify the AI's judgment criteria, such as "red areas correspond to nuclear atypia, and blue areas represent normal stroma"; the report automatically links to the latest treatment guidelines, indicating the degree of match between recommended treatments and the current diagnosis.

[0031] In one embodiment, the process of constructing a normal tissue feature space through a variational autoencoder, detecting slices that significantly deviate from the normal pattern, automatically triggering an expert review process, and incorporating the case into a dynamic training dataset includes the following steps: Collect large-scale normal tissue sections, covering different organs (breast, lung, thyroid, etc.) and populations (age, sex); perform standardized processing on the sections, such as staining normalization in step one, to ensure the consistency of input data; Variational autoencoder training: Encoder: Compresses high-dimensional slice images into low-dimensional latent space representations (e.g., 16-dimensional), and forces the latent distribution to approximate a standard normal distribution through KL divergence loss; Decoder: Reconstructs slices from the latent space, minimizes reconstruction errors (such as mean square error MSE), and ensures that the encoder captures key features of normal tissue; Feature space visualization: By using t-SNE or UMAP dimensionality reduction, observe the aggregation of normal tissues in the latent space, such as normal breast tissue forming a tight cluster, while cancerous tissue is scattered on the periphery; Dual detection: Reconstruction error threshold: Calculate the difference between the input slice and the VAE reconstructed slice (e.g., SSIM index). When SSIM < 0.7, it is marked as abnormal (normal tissue SSIM > 0.85). Latent space deviation: Calculate the degree of deviation of the distribution of the input slice in the latent space from the normal pattern (mean ± 3σ). An alarm is triggered when the deviation is > 0.5. Generate a heatmap to display abnormal areas and label specific features, such as "abnormal cell nucleus size" or "increased interstitial density"; output an anomaly confidence score, and automatically enter the review process when the score is greater than 70. Based on the type of abnormality (e.g., breast, lung) and the degree of urgency (e.g., high-risk cases are prioritized), slides are assigned to pathology experts in the corresponding fields. Experts view the slides, heat maps, and AI analysis reports through remote consultation platforms, such as the provincial pathology service platform, and complete the review within 15 minutes. After the expert confirms the diagnosis, a review report is automatically generated, which includes the expert's opinion, the revised diagnosis and its basis. The review results are fed back to the primary hospitals in real time and the patient's electronic medical record is updated simultaneously. Abnormal cases confirmed by experts are labeled as true cases, such as "invasive ductal carcinoma of the breast," and stored in a dynamic dataset along with the original slides, heatmaps, and review reports. The dataset is categorized by organ, pathological type, and population characteristics to form a structured knowledge base. The VAE model is fine-tuned monthly with new data, focusing on optimizing the false positive rate, such as reducing the proportion of breast fibroadenomas misdiagnosed as cancer from 12% to 3%. Incremental learning techniques are used to ensure that historical knowledge is not forgotten when the model is updated. False positive / false negative monitoring: Track the model's performance in the production environment, triggering an alarm when the false positive rate is >5% or the false negative rate is >2%, automatically tracing back problematic cases; generating monthly quality reports, comparing the consistency between the model and expert diagnoses (e.g., the Kappa coefficient improves from 0.82 to 0.91).

[0032] In one embodiment, the method of generating pseudo-healthy images using an inverse autoencoder and accurately locating pathological regions by comparing the differences between the original and reconstructed images; and balancing the model convergence speed of various primary hospitals in federated learning through a dynamic weight adjustment algorithm, including the following steps: The inverse autoencoder accurately locates pathological areas, employing a VAE-GAN (Variational Autoencoder-Generative Adversarial Network) fusion architecture, in which: Encoder: Compresses the original pathological image (e.g., 1024×1024 pixels) into a low-dimensional latent space representation (e.g., 128 dimensions), capturing the statistical features of normal tissue; Generator: Reconstructs images from the latent space and generates highly realistic pseudo-healthy images (SSIM index > 0.85) through adversarial training (playing against the discriminator). Discriminator: Distinguishes between real, normal images and generated fake healthy images, forcing the generator to produce more realistic results; VAE-GAN is trained using large-scale normal tissue slices to ensure that the generator can accurately reconstruct normal tissue structures (such as gland arrangement and cell distribution); a small number of pathological slices are introduced, and conditional generation techniques such as adding pathological labels as input enable the generator to generate pseudo-healthy images corresponding to the pathological type. The original pathological image and the generated pseudo-healthy image are spatially aligned, and rotation and translation differences are corrected by affine transformation to ensure pixel-level correspondence. Difference calculation: Structural differences: Calculate the SSIM (structural similarity) index between two images, and mark low SSIM regions (e.g., less than 0.7) as potential pathological regions; Feature Differences: Extract cell-level features, such as nuclear size and nucleocytoplasmic ratio, and compare the feature distribution of the original image with that of the pseudo-healthy image. Regions with differences exceeding a threshold, such as a nucleocytoplasmic ratio greater than 0.8, are marked. The differential regions are mapped onto a heatmap, with red indicating high differential and blue indicating low differential, and then overlaid onto the original image to visually display the boundaries of pathological regions such as breast cancer cell clusters; The dynamic weight adjustment algorithm balances the convergence speed of federated learning. The data distribution of primary hospitals varies greatly. For example, county hospitals mainly have breast cancer cases, while city hospitals mainly have lung cancer cases. This results in the model converging quickly on local data, but the generalization ability of the global model is poor. Some hospitals use high-end GPUs, while others rely solely on CPUs, resulting in a large difference in model update frequency (from daily updates to weekly updates). Dynamic weight adjustments are performed. Each hospital records the model's loss value on the local validation set, such as cross-entropy loss. When the loss value decreases by less than 1% for three consecutive rounds, it is judged as "local convergence". The gradient updates of all hospitals are aggregated, and the variance of the gradient direction is calculated. If the variance is greater than 0.1, it is judged as "global non-convergence" using the Welford algorithm. For hospitals with slow convergence speeds that rely solely on CPU updates, increase their gradient weights to make their updates have a greater impact on the global model; for hospitals with fast convergence speeds that use GPU updates, decrease their gradient weights to prevent them from dominating the global model's direction; Dynamic balancing formula:

[0033] in, Let i be the gradient variance of hospital i, and ensure that hospitals with high variance (non-converged) receive higher weights.

[0034] In one embodiment, the platform uses WebGL technology for browser-side 3D rendering and integrates a natural language processing interface to automatically generate a diagnostic report summary annotated with key evidence chains, including the following steps: This system integrates pathological slide heatmaps (two-dimensional), gene testing data (one-dimensional mutation frequency), and clinical indicators such as patient age and tumor marker levels to construct a three-dimensional data model. For example, it integrates HER2 immunohistochemical heatmaps of breast cancer cases with ER / PR gene expression data and patient CA15-3 levels into the same three-dimensional coordinate system. Utilizing the browser's built-in GPU acceleration (no plugins required), the following functions are achieved through vertex shaders and fragment shaders: 3D rotation and zoom: Supports mouse or touch operation to achieve 360° rotation of pathological slides (such as observing tumor boundaries) and multi-level zoom (from tissue level to cell level). Transparency overlay: By adjusting the layer transparency, heatmaps (red high-risk areas) and gene data (blue low-expression areas) can be displayed simultaneously, helping doctors identify key evidence chains, such as the co-location of HER2 positive regions and TP53 mutations; Dynamic slide browsing: Simulates the movement of the microscope stage, allowing users to quickly switch between different areas of the WSI via keyboard or gestures, such as from the tumor center to the edge, improving diagnostic efficiency; The AI-generated heatmap (with red markings for suspicious areas) is overlaid on the original pathological slide in a semi-transparent manner. Doctors can adjust the transparency of the heatmap using a slider (e.g., 30% transparency to observe the underlying cell structure). High-risk area boundaries (e.g., breast cancer cell clusters) are automatically marked, and characteristic parameters of the area are displayed, such as a nucleocytoplasmic ratio greater than 0.8 and a Ki-67 positivity rate greater than 30%. Pathological features (e.g., cellular atypia), genetic data (e.g., BRCA1 mutation), and clinical indicators (e.g., elevated CA125) are presented in the form of three-dimensional bar charts or scatter plots. Clicking on the legend allows you to jump to the corresponding evidence source; for example, clicking on the BRCA1 mutation bar chart will take you to the gene testing report. The NLP interface receives the following data output from the 3D platform: Pathological features: parameters of suspicious areas marked on the thermogram, such as nuclear size and staining intensity; Genetic data: mutation sites, expression levels, such as HER2 positivity defined as IHC 3+ or FISH amplification; Clinical indicators: tumor markers, patient history, such as family history of breast cancer; Through rule engines (such as regular expression matching) and pre-trained models, core evidence supporting the diagnosis is extracted, such as "significant nuclear atypia, mitotic figures >10 / 10HPF" (pathological features), "HER2 gene amplification, FISH ratio = 2.8" (genetic data), and "patient's CA15-3 level is 120U / mL (normal <30U / mL)" (clinical indicators). Based on international pathology reporting standards (such as the CAP protocol), generate an abstract containing the following modules: Diagnostic conclusion: Determine the pathological type and grade, such as "invasive ductal carcinoma of the breast, grade II"; Diagnostic criteria: Prioritize key evidence, such as "HER2 positive (level of evidence Ia)"; Recommendation: Further examination (such as PD-L1 testing) or treatment options (such as targeted therapy) are recommended. Transform structured data into natural language using a pre-trained model to ensure the report meets the following requirements: Readability: Avoid overuse of technical terms, such as replacing "high mitotic figures" with "actively growing tumor cells"; Compliance: Citify the latest treatment guidelines and indicate the degree of matching between the recommended treatment and the current diagnosis, such as "Complies with NCCN Guideline Class I Recommendation".

[0035] An AI-based assisted film viewing system includes the following steps: Data Foundation Layer: By integrating pathology slide scanners, electronic medical record systems, and laboratory information systems, high-resolution WSI images, patient clinical texts, and test data are acquired simultaneously. CIELAB staining normalization (ΔE<1.5) and adaptive threshold segmentation algorithms are used to remove noise and standardize image resolution to 20× magnification (0.5μm / pixel). Finally, a three-dimensional index library of patient ID-timestamp-data type is constructed, and the storage space of a single WSI case is reduced to 800MB through lossless compression, providing a high-quality, standardized data foundation for subsequent analysis. Core algorithm layer: Deploys a dynamic branch convolutional neural network, automatically switches model branches according to device computing power, and transfers high-precision branch knowledge to lightweight branches through knowledge distillation (KL divergence loss function); In the federated learning framework, the dynamic weight adjustment algorithm balances the convergence speed of primary hospitals, and differential privacy protection (σ=0.1) ensures data security; Variational autoencoder constructs a normal tissue feature space, detects abnormal slices through SSIM exponent (greater than 0.85), automatically triggers expert review and updates the dynamic training dataset; Application Service Layer: Edge nodes complete slice preprocessing (such as adaptive threshold segmentation) and preliminary screening (filtering 78.3% of negative cases); cloud servers handle complex model inference and multi-institution data comparison to generate structured diagnostic reports; the WebGL engine renders millions of voxel data (frame rate > 60fps), supports the overlay of heatmaps and original images (adjustable transparency) and the display of three-dimensional evidence chains (such as the correlation between cytoplasmic ratio, HER2 amplification, and clinical indicators); the NLP interface extracts key evidence (such as "mitotic figures > 10 / 10HPF"), generates structured report summaries that conform to CAP standards, and marks the degree of matching with treatment guidelines (such as "conforming to NCCN Class I recommendations"). User interaction layer: The doctor workstation supports pathologists in verifying AI judgment criteria through a 3D platform (such as adjusting the transparency of heatmaps with sliders), and clinicians can quickly view report summaries and formulate treatment plans; the expert review platform is connected to the provincial remote consultation system, completing the review of high-risk cases within 15 minutes and providing feedback to the grassroots level; the system management backend monitors model performance through a quality control dashboard (such as triggering alarms when the false negative rate is greater than 2%), and the resource scheduling center dynamically allocates edge-cloud computing resources (such as GPU cluster load balancing) to ensure service stability and compliance.

[0036] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An artificial intelligence-based assisted reading method, characterized by, Comprise: Step one: through the cross-modal data fusion engine, synchronous acquisition of pathological section WSI image, electronic medical record text, laboratory test data and equipment parameters, and utilize adaptive staining normalization algorithm to automatically correct the staining difference of different equipment or batch; at the same time, through multi-modal data alignment module, associate patient ID and time stamp, construct structured data package, provide unified, high-quality data basis for subsequent analysis; Step two: develop dynamic branch convolutional neural network, automatically switch model branch according to basic equipment computing power, migrate the knowledge of high-precision branch to lightweight branch through knowledge distillation; introduce federal learning framework, allow hospitals to jointly train models without domain data; Step three: deploy edge nodes in primary hospitals, responsible for slice preprocessing, preliminary screening and data encryption; cloud server undertakes complex model reasoning, multi-institution data comparison and long-term storage; edge node generates heat map to mark suspicious areas, and cloud output structured diagnosis report; Step four: build normal tissue feature space through variational autoencoder, detect slices significantly deviating from normal mode, automatically trigger expert review process, and include the case in dynamic training dataset; at the same time, generate pseudo healthy images using reverse autoencoder, accurately locate pathological areas by comparing differences between original images and reconstructed images; through dynamic weight adjustment algorithm, balance the model convergence speed of each primary hospital in federal learning, avoid model deviation caused by data silos; Step five: develop three-dimensional visualization interaction platform, integrate pathological section heat map, gene detection data and clinical indicators, support multi-dimensional overlay display; doctors adjust transparency through sliding bar, intuitively verify AI judgment basis, and one-key call international diagnosis guideline recommended scheme; the platform uses WebGL technology for browser-side three-dimensional rendering, and integrates natural language processing interface to automatically generate diagnosis report abstract with labeled key evidence chain.

2. The method of claim 1, wherein, The adaptive staining normalization algorithm automatically corrects the staining difference of different equipment or batch, comprising the following steps: Identify key variables that cause staining difference, including staining solution batch, equipment characteristics, operation process and environmental factors, construct staining difference fingerprint library, collect typical staining characteristics of different equipment / batch through hyperspectral imaging technology, form traceable staining difference spectrum; Convert the original RGB image to CIELAB color space, which is based on human visual perception model, can separate brightness and chrominance information, avoid color distortion caused by direct operation in RGB space; extract staining feature vector, calculate Lab value distribution of each pixel point, generate staining feature matrix; Build a dynamic statistical model, select a reference staining mode, randomly select 1000 cases from the slices produced by the standard staining process to form a benchmark dataset, calculate the mean, covariance matrix and principal component analysis features of the Lab* distribution; establish a staining correction mapping function, learn the color mapping relationship between the benchmark dataset and the image to be corrected through a nonlinear regression model, generate a conversion function from the deviated staining mode to the standard mode; Automatic matching of pre-stored correction parameters through scanner serial number or staining solution batch number, if new equipment / batch, start fast self-learning mode; apply mapping function to each pixel for color correction, focus on adjusting the contrast between nucleus and cytoplasm to make the nucleus-cytoplasm ratio meet the pathological diagnosis standard; Calculate the ΔE value of the corrected image and the reference data set, requiring ΔE to be less than 1.5; randomly select 200 cases of pre-correction and post-correction sections, and conduct diagnostic consistency evaluation by three senior pathologists to reduce the false negative rate of the corrected image, and automatically include the corrected parameters that pass the verification into the staining fingerprint library to form a closed loop of continuous optimization. 3.The AI-based assisted reading method of claim 1, wherein, The dynamic branch convolutional neural network is developed, the model branches are automatically switched according to the computing power of the basic equipment, and the knowledge of the high-precision branch is migrated to the lightweight branch through knowledge distillation, including the following steps: Real-time acquisition of device computing power indicators including CPU core number, GPU model, memory capacity and free memory ratio through API; Establish a three-level computing power evaluation system: Lightweight: adapt to MobileNetV3 / ShuffleNetV2 ultra-lightweight model; Standard level: adapt to EfficientNet-Lite / ResNet-18 medium-sized model; High-precision level: adapt to ResNet-152 / Inception-v4 large model; According to the test results, the corresponding branch is automatically loaded; Design teacher-student model architecture: Teacher model: use deep convolutional network, train on cloud server, used to extract high-dimensional features and generate soft labels; soft labels contain relative probabilities between classes, providing more information than hard labels; Student model: based on MobileNetV3 architecture optimization, reduce computation through depth separable convolution; Knowledge distillation migration: Output layer distillation: use KL divergence loss function to align the output distribution of the teacher model and the student model: wherein, is the total loss function of knowledge distillation, combining the real label loss and the teacher model knowledge transfer loss; λ is the balance coefficient, controlling the weight ratio of the real label loss and the teacher model knowledge transfer loss; is the cross-entropy loss function, measuring the difference between the student model output and the real label; is the real label of the sample, represented in the form of one-hot encoding; is the output probability distribution of the student model, representing the prediction probability of each class by the model; is the KL divergence, measuring the distribution difference between the teacher model output and the student model output; is the output probability distribution of the teacher model, containing richer relative probability information between classes than the hard label. Intermediate layer feature distillation: introduce feature alignment loss to force the feature maps of the intermediate layers of the student model to be similar to the feature maps of the corresponding layers of the teacher model: wherein, is a feature alignment loss function, forcing the feature maps of the intermediate layers of the student model to be similar to the feature maps of the corresponding layers of the teacher model; N is the number of channels of the feature maps; is the feature map of the i-th channel of the intermediate layer of the teacher model; is the feature map of the i-th channel of the intermediate layer of the student model; The teacher model and the student model share the bottom feature extraction layer, and the high-level branches are processed independently; during training, the teacher model fixes the parameters, and the student model updates the weights through distillation loss back propagation; according to the characteristics of basic pathological data, use staining normalization, cell region random occlusion and multi-scale scaling to adapt to different scanner resolution enhancement methods; All branches are preloaded when the device is first used, and subsequent switching is based on computing power test results; lightweight branch performs preliminary inference on local device, high-precision branch performs complex analysis on cloud server, and model parameters are updated through federated learning framework, so that basic data does not leave the domain. 4.The method of claim 1, wherein, The edge node generates a heat map to mark suspicious areas, and the cloud outputs a structured diagnostic report, including the following steps: The edge node first performs adaptive threshold segmentation on the WSI to remove background noise and standardize the image resolution; based on the lightweight branch switched in step two, the preprocessed section is preliminarily screened, focusing on detecting the following three types of abnormalities: Morphological abnormalities: cell nucleus size and shape deviate from normal range; Distribution abnormalities: cell arrangement pattern is disorderly; Abnormal staining intensity: the staining intensity of a specific marker exceeds a threshold value; Using Grad-CAM technology, the attention distribution of the input image is calculated by backpropagation of the model, and a heat map is generated; the red area in the heat map is the area that the model pays high attention to, and the blue area is the background tissue that the model considers normal; According to the intensity threshold of the heat map, the suspicious area boundary is automatically marked, and the annotation box is generated, and the coordinates, area and feature description of the area are recorded; AES-256 encryption is performed on the heat map and annotation information to improve data transmission security; the encrypted data packet volume is compressed to 15% of the original data, which adapts to the bandwidth limit of the basic network; according to the risk level of the suspicious area, the data transmission queue is dynamically adjusted, so that the emergency cases are synchronized to the cloud within 30 seconds; The high-precision branch in step two of cloud deployment performs secondary verification on the suspicious areas uploaded by the edge node, focusing on solving the problems of false positive filtering and subtype classification; the false positive filtering is to exclude the misjudgment caused by slice folding and uneven staining; the subtype classification is a more detailed classification of the suspicious area; the current case is compared with the same type of cases in the cloud database to generate a similarity ranking, which assists doctors in differential diagnosis; Based on the international pathological report template, a structured report is automatically generated, including the following modules: Diagnosis conclusion: clearly state the pathological type and grade; Diagnosis basis: list the key features supporting the diagnosis; Suggestion: recommend further examination items or treatment plan; Embed the heat map and annotation box in the report, and the doctor can adjust the transparency through the slider to intuitively verify the AI judgment basis; the report automatically associates with the latest diagnosis and treatment guidelines, and marks the matching degree of the recommended plan with the current diagnosis.

5. The method of claim 1, wherein the method further comprises: The normal tissue feature space is constructed by the variational autoencoder, the slices significantly deviating from the normal mode are detected, the expert review process is automatically triggered, and the case is included in the dynamic training data set, including the following steps: Collect a large number of normal tissue slices, covering different organs and populations; standardize the slices to make the input data consistent; Train the variational autoencoder: Encoder: compress high-dimensional slice images into low-dimensional latent space representation, and force the latent distribution to approach the standard normal distribution through KL divergence loss; Decoder: reconstruct the slice from the latent space, minimize the reconstruction error, and make the encoder capture the key features of normal tissue; Feature space visualization: observe the clustering of normal tissue in the latent space through t-SNE or UMAP dimensionality reduction; Double detection: Reconstruction error threshold: calculate the difference between the input slice and the VAE reconstructed slice, and mark it as abnormal when the SSIM is less than 0.7; Latent space deviation: calculate the deviation of the input slice distribution in the latent space from the normal mode, and trigger an alarm when the deviation is greater than 0.5; Generate a heat map to display the abnormal area and mark the specific features; output an abnormal confidence score, and automatically enter the review process when the score is greater than 70 points; According to the type and severity of the anomaly, the slice is assigned to a pathologist in the corresponding field; the expert reviews the slice, heat map and AI analysis report through the remote consultation platform, and completes the review within 15 minutes; after the expert confirms the diagnosis, an automatic review report is generated, and the review result is fed back to the primary hospital in real time, and the patient's electronic medical record is updated synchronously; The abnormal cases confirmed by experts are labeled as true labels, and are stored together with the original slice, heat map and review report into a dynamic dataset; the dataset is classified by organ, pathology type and population characteristics to form a structured knowledge base; the VAE model is fine-tuned with new data every month, focusing on optimizing the false positive rate; incremental learning technology is used to ensure that the model does not forget historical knowledge when updated; False positive / false negative monitoring: track the performance of the model in the production environment, trigger an alarm when the false positive rate is greater than 5% or the false negative rate is greater than 2%, and automatically backtrack the problem cases; generate a monthly quality report to compare the consistency of the model and the expert's diagnosis.

6. The method of claim 1, wherein, The reverse autoencoder generates pseudo-healthy images, which accurately locate the pathological area by comparing the differences between the original image and the reconstructed image; through a dynamic weight adjustment algorithm, the model convergence speed of each primary hospital in federated learning is balanced, including the following steps: The reverse autoencoder accurately locates the pathological area, using a VAE-GAN fusion architecture, which includes: Encoder: compresses the original pathological image into a low-dimensional latent space representation, capturing the statistical features of normal tissue; Generator: reconstructs the image from the latent space, generating pseudo-healthy images with high realism through adversarial training; Discriminator: distinguishes between real normal images and generated pseudo-healthy images, forcing the generator to generate more realistic results; Train VAE-GAN with large-scale normal tissue slices to ensure that the generator can accurately reconstruct normal tissue structures; introduce a small amount of pathological slices to enable the generator to generate pseudo-healthy images of corresponding pathological types through conditional generation technology; spatially align the original pathological image with the generated pseudo-healthy image; Calculate the difference: Structural difference: calculate the SSIM index of the two images, and mark the low SSIM area as a potential pathological area; Feature difference: extract cell-level features and compare the feature distribution of the original image and the pseudo-healthy image, and mark the area with a difference greater than the threshold; Map the difference area to a heat map and overlay it on the original image to visually display the pathological area boundary; The dynamic weight adjustment algorithm balances the convergence speed of federated learning; the data distribution of primary hospitals is different, resulting in fast convergence of the model on local data but poor generalization of the global model; some hospitals use high-end GPUs, while others rely only on CPUs, resulting in large differences in model update frequency; Perform dynamic weight adjustment, record the loss value of the model on the local validation set for each hospital, and when the loss value decreases by less than 1% for three consecutive rounds, determine that it is "locally converged"; aggregate all hospital gradient updates and calculate the variance of the gradient direction, with a variance greater than 0.1 indicating "global non-convergence"; For hospitals with slow convergence speed, increase their gradient weight to make their updates have a greater impact on the global model; for hospitals with fast convergence speed, reduce their gradient weight to prevent them from dominating the global model direction; dynamic balance formula: where, is the gradient variance for hospital i, giving higher weight to hospitals with higher variance.

7. The method of claim 1, wherein, The platform uses WebGL technology for browser-side three-dimensional rendering, while integrating a natural language processing interface to automatically generate a diagnostic report summary with annotated key evidence chains, including the following steps: Integrate pathological section heat map, gene detection data and clinical indicators to build a three-dimensional data model; use the built-in GPU acceleration function of the browser to realize the following functions through vertex and fragment shaders: Three-dimensional rotation and scaling: support mouse or touch control to realize 360° rotation and multi-level scaling of pathological sections; Transparency overlay: adjust the transparency of the layers to simultaneously display the heat map and gene data, assisting doctors in identifying key evidence chains; Dynamic slice browsing: simulate the movement of the microscope stage to quickly switch between different areas of the WSI through the keyboard or gestures, improving diagnostic efficiency; Overlay the AI-generated heat map on the original pathological section in a semi-transparent manner; doctors can adjust the heat map transparency through a slider; automatically mark the boundaries of high-risk areas and display the characteristic parameters of the area; present pathological features, gene data and clinical indicators in the form of three-dimensional column charts or scatter plots, supporting clicking on the legend to jump to the corresponding evidence source; On mainstream browsers, real-time rendering of million-level voxel data ensures smooth interaction experience; supports PC, tablet and mobile browser access, adapting to the diverse equipment conditions of primary hospitals; The NLP interface receives the following data output by the three-dimensional platform: Pathological features: parameters of suspicious areas marked by the heat map; Gene data: mutation sites, expression levels; Clinical indicators: tumor markers, patient history; Extract core evidence supporting diagnosis through rule engines and pre-trained models; Based on international pathological report standards, generate an abstract containing the following modules: Diagnosis conclusion: clearly state the pathological type and grade; Diagnosis basis: sort key evidence by priority; Recommendations: recommend further tests or treatment options; Convert structured data to natural language using pre-trained models to avoid excessive use of professional terminology, reference the latest diagnostic guidelines, and label the matching degree of recommended options to the current diagnosis. 8.The artificial intelligence-based assisted reading method according to claim 1, wherein the artificial intelligence-based assisted reading system is characterized by, Includes: Data foundation layer: integrate pathological section scanners, electronic medical record systems and laboratory information systems to synchronize the collection of high-resolution WSI images, patient clinical text and detection data; use CIELAB dye normalization and adaptive threshold segmentation algorithms to remove noise and standardize image resolution to 20x magnification; finally, build a three-dimensional index library with patient ID-time stamp-data type, reduce the single instance WSI storage space to 800MB through lossless compression, and provide a high-quality, standardized data foundation for subsequent analysis; Core algorithm layer: deploy a dynamic branch convolutional neural network to automatically switch model branches based on device computing power, and migrate high-precision branch knowledge to lightweight branches through knowledge distillation; in the federated learning framework, the dynamic weight adjustment algorithm balances the convergence speed of primary hospitals, and differential privacy protects data security; a variational autoencoder constructs a normal tissue feature space, detects abnormal sections through SSIM index, automatically triggers expert review and updates the dynamic training dataset; Application service layer: edge nodes complete slice preprocessing and preliminary screening, cloud servers undertake complex model reasoning and multi-institution data comparison, and generate structured diagnostic reports; WebGL engine renders million-level voxel data, supports heat map and original image overlay, and displays three-dimensional evidence chain; NLP interface extracts key evidence, generates structured report abstracts conforming to CAP standards, and labels diagnosis and treatment guideline matching degree; User interaction layer: physician workstation supports pathologists to verify AI judgment basis through three-dimensional platform, and clinical physicians to quickly view report abstracts and develop plans; expert review platform accesses provincial remote consultation system, quickly reviews high-risk cases, and feeds back to primary level; system management background monitors model performance through quality control board, and resource scheduling center dynamically allocates edge-cloud computing resources.

Images