Cervical cell pathology panoramic image diagnosis method based on multi-agent cooperation

Abstract

The invention discloses a cervical cell pathology panoramic image diagnosis method based on multi-agent collaboration, and the method comprises a task scheduling agent, a preprocessing agent, a model selection agent and a result analysis agent. And a multi-task and high-precision diagnosis framework oriented to the cervical cell pathological panoramic image is constructed. Compared with a traditional assembly line type system and a general medical agent lacking domain knowledge, the method can perform rapid and automatic processing on massive cervical cell panoramic images, and completes key tasks such as abnormal region positioning, cell segmentation, cell classification and preliminary diagnosis suggestion generation in sequence; and intelligent auxiliary diagnosis of the cervical cell pathological image is realized. The method can be used as an auxiliary tool for a pathologist, the diagnosis efficiency and the result consistency are remarkably improved, the influence of subjective difference of manual film reading on a diagnosis conclusion is effectively reduced, and then popularization and precision of early diagnosis and early treatment of cervical cancer are promoted.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical image processing and artificial intelligence auxiliary diagnosis, and particularly relates to a cervical cell pathological panoramic image diagnosis method based on multi-agent cooperation. BACKGROUND

[0002] Cervical cancer is one of the major threats to global women's health, and early screening is a key means to prevent and control cervical cancer. At present, cervical cytology examination (Thinprep Cytology Test, TCT) has become a major screening method. However, the cervical cell pathological panoramic image (Whole Slide Image, WSI) usually has a resolution of gigapixel level, and the abnormal cell morphology is highly similar, which poses a great challenge to efficient and accurate automated diagnosis. The traditional diagnosis process highly depends on manual operation, and the pathologist first browses the entire WSI at a lower resolution to identify suspicious areas, then switches to high resolution to analyze the suspicious areas in detail to confirm the lesion cells, and finally forms a diagnosis result. This process is very labor-intensive and requires rich professional knowledge to effectively interpret complex visual information. In recent years, the rapid development of deep learning technology has greatly promoted the automation of cervical cell pathological image analysis. However, existing automated diagnosis systems usually adopt a modular pipeline structure, relying on the high coupling between cell detection, cell classification and other modules, which leads to a lack of universality. At the same time, existing systems can usually only handle single tasks, and have weak data and knowledge sharing capabilities, and the information flow between internal modules and data sources is not smooth, making it difficult to ensure the accuracy of the diagnosis result. These problems make the traditional system inefficient in processing complex cell pathological image diagnosis, and unable to meet the requirements of modern medicine for diagnosis accuracy and response speed.

[0003] Multi-agent system (MAS) provides a highly potential architecture solution for solving the trade-off between diagnosis accuracy and system universality by integrating multiple specialized models and real-world application program interfaces. In medical applications, this architecture has shown significant advantages by decoupling complex processes and integrating specialized models. Specifically, it uses task specialization mechanism to avoid the load bottleneck of monolithic systems, such as the existing WSI-Agents framework which attempts to handle diagnosis, morphological analysis and report generation by specialized agents respectively; at the same time, the collaborative interaction between agents can simulate the decision-making logic of human pathologists, such as the PathFinder system which integrates diagnosis, navigation and diagnosis agents to reproduce the "global browsing-detailed analysis" clinical workflow. This architecture combines domain-specific models and verification mechanisms, indicating the direction of improving the diagnosis accuracy of specific pathological tasks while maintaining the universality of the system.

[0004] However, although the multi-agent architecture itself has the above advantages, when it is directly applied to the diagnosis of complex cervical cell pathology panoramic images, the existing multi-agent method still faces severe challenges. First, the current medical multi-agent method often covers a variety of imaging modes too broadly, resulting in the dilution of professional knowledge for specific pathological fields. This lack of field depth of generality results in a diagnosis accuracy of only 40%-50% on the specific task of cytopathology. Second, even the dedicated multi-agent method lacks efficient processing capability for terapixel-level ultra-high resolution WSI.

[0005] In view of the technical bottlenecks that the above technical generality and professionalism are difficult to balance, and it is difficult to accurately and efficiently process ultra-high resolution WSI, the present application proposes a cervical cell pathology panoramic image diagnosis method based on multi-agent collaboration. The method constructs a high-precision diagnosis framework with multi-task processing capability through the deep collaboration of task scheduling agents, preprocessing agents, model selection agents and result analysis agents. This framework can simulate the cognitive decision-making process of cytopathologists "global browsing - key analysis - comprehensive diagnosis", and realize the iterative reasoning from preliminary screening to fine interpretation. This method can not only effectively diagnose terapixel-level cervical cell WSI, but also explicitly integrate cytopathology field knowledge into the automated processing process, realize key tasks such as abnormal area positioning, cell segmentation, cell classification and preliminary diagnosis suggestion generation, and thus improve the classification accuracy and diagnosis reliability of cervical cell pathology panoramic images. SUMMARY

[0006] The present application proposes a cervical cell pathology panoramic image diagnosis method based on multi-agent collaboration to solve the problem of single task type and low diagnosis accuracy of existing automated diagnosis technology in the cervical cell pathology WSI diagnosis process.

[0007] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0008] A cervical cell pathology panoramic image diagnosis method based on multi-agent collaboration, comprising a task scheduling agent, a preprocessing agent, a model selection agent, and a result analysis agent, wherein the task scheduling agent coordinates the preprocessing agent, the model selection agent, and the result analysis agent to jointly complete the cervical cell pathology WSI classification task.

[0009] As an implementation manner, the technical scheme can be implemented by the following steps:

[0010] S1, system input:

[0011] S11, the system receives a cervical cell pathological image and a prompt word, or only receives the prompt word. The cervical cell pathological image includes a cervical single cell image, a cervical Patch (WSI section) level image, and a cervical WSI.

[0012] S2, a task scheduling agent:

[0013] S21, the task scheduling agent calls a Multimodal Large Language Model (MLLM) GPT-4o to perform high-level semantic understanding and intent analysis on the input image type, text prompt information, and historical interaction records. If the input is a cervical WSI, the system first down-samples to generate a compressed image as input. The MLLM generates corresponding scalar scores for the three features based on the analysis results, and calculates a priority score based on a pre-set priority calculation formula to determine the combination of agents that should be started for this task. The priority calculation formula is as follows:

[0014] priority = f (ImageType, TextHint, Historty) = a x I + b x T + g x H (1)

[0015] where I represents the scalar score corresponding to the image type generated by the MLLM, T represents the scalar score of the text prompt generated by the MLLM, H represents the scalar score of the historical interaction generated by the MLLM, a, b, and g are weight hyperparameters learned by the agent through statistical learning, and are used to balance the influence of the three factors on task scheduling.

[0016] S3, a preprocessing agent:

[0017] S31, after reading the original cervical WSI, first parse the pyramid image structure information from the image metadata. The metadata includes the total number of resolution levels of the WSI, the image width w l , the image height h l , the down-sampling ratio d l , and the resolution level index l, etc. Based on the above parameters, the system can accurately obtain the spatial dimensions of the WSI at different resolutions, laying a foundation for subsequent optimal level selection, Patch segmentation, and feature extraction;

[0018] S32, the preprocessing agent pre-sets the target Patch number N target for subsequent analysis. For each resolution level l, calculate the number of Patches N l generated by sliding window segmentation at that resolution level. The expression is as follows:

[0019]

[0020] where w l and h l denote the width and height of the l-th resolution level, respectively, and psdenotes the patch size.

[0021] The best level is then selected by minimizing the error between the patch number and the target. The expression is as follows:

[0022] l * = arg l min |N l -N target | (3)

[0023] where N l is the number of patches generated by the sliding window segmentation at the l-th level, N target is the target number of patches pre-set by the preprocessing agent for subsequent analysis, and l * denotes the WSI resolution level that best fits the subsequent task.

[0024] S33, determine the optimal level l * After that, the image is segmented into patches using the sliding window strategy with a step size of ps, and the background proportion of each patch is calculated by the background determination model. If the background proportion exceeds the set threshold, the patch is automatically skipped to reduce invalid calculations.

[0025] S34, perform reference cell screening on the detected cell instances;

[0026] The reference cell screening includes the following steps:

[0027] The preprocessing agent first automatically removes low-quality cells in the fuzzy, broken, or artifact regions based on cell area, nuclear contour integrity, staining saturation, and focus clarity. Then, based on cell morphological consistency, nuclear-cytoplasmic ratio stability, and texture distribution characteristics, it selects and constructs a "reference cell set" for global alignment among the remaining high-quality cells.

[0028] S35, perform feature normalization processing on the input image;

[0029] The feature normalization processing includes the following steps:

[0030] The preprocessing agent calculates the mean vector and variance in the feature space of the reference cell set, and uses it as a normalization reference to standardize the feature vectors of all cells in the current WSI, thereby eliminating feature offsets caused by imaging styles, staining changes, and slice differences between different WSIs, and providing consistent and stable input features for the subsequent model selection agent's cell classification and feature reasoning.

[0031] S36, for the Patch image and single cell image, perform rotation, flipping, color disturbance and other enhancement operations.

[0032] S4, model selection agent:

[0033] S41, when the task is abnormal cell detection, the model selection agent calls the YOLOv5 model from the system model library to infer the input image, and outputs the boundary box, cell category and corresponding confidence of each candidate cell;

[0034] S42, when the task is instance segmentation, the model selection agent will call the SOLOv2 network for segmentation inference, and output the mask (mask or pixel-level region) of the cell. Then, according to the area range of the cell, the candidate cells are screened out to ensure the quality of the effective candidate samples;

[0035] S43, when the task is single cell classification, the model selection agent will call the ResNet50 network for classification inference, and output the class probability of each cell.

[0036] S5, result analysis agent:

[0037] S51, after the model selection agent completes cell classification or feature extraction, the result analysis agent calculates the comprehensive score of each single cell and the probability of the cell being predicted as each category. The categories include: negative / without epithelial lesion or malignant (NILM), atypical squamous cells of uncertain significance (ASC-US), atypical squamous cells that cannot be ruled out high-grade lesions (ASC-H), low-grade squamous intraepithelial lesions (LSIL), and high-grade squamous intraepithelial lesions (HSIL).

[0038] The calculation of the comprehensive score of each single cell and the pathological category probability includes steps S52-S55:

[0039] S52, for each cell, its features are represented as a column vector X = [x1, x2,..., x n ] T , and the linear score is calculated using the trained weight row vector W = [w1, w2,..., w n ]. The expression is as follows:

[0040] s = W·X (4)

[0041] Where s = W·X is the vector inner product, that is x i represents the i-th feature value, w i is the corresponding weight coefficient, and s is the linear comprehensive score of the cell.

[0042] S53, mapping the linear score to a probability value by a mapping function f(s) as the comprehensive score of the cell. The expression is as follows:

[0043] p = f(s) (5)

[0044] where p ∈ (0, 1) represents the comprehensive score of the cell.

[0045] S54, for each pathological category c, the category-specific weight w k (c) The normalized features f k are weighted and summed to obtain the weighted score under the category. The expression is as follows:

[0046]

[0047] where k represents the feature index, f k is the kth normalized feature value, map(·) is the feature mapping function, is the importance weight of the feature under category c.

[0048] S55, the weighted score is mapped to the probability that the cell belongs to the pathological category c by using the sigmoid function, and the expression is as follows:

[0049]

[0050] where a c is the category adjustment parameter, which is a scaling coefficient learned independently for each pathological category during the model training stage, used to adjust the sensitivity or decision boundary of the category score, so that each category has better distinguishability when mapping to probability, and p (c) ∈ (0, 1) represents the probability that the cell belongs to category c.

[0051] S56, after the calculation of the single-cell comprehensive score and the category probability, the result analysis agent fuses the pathological category probabilities of all detected cells in the whole WSI to generate the WSI-level diagnostic result;

[0052] The fusion of the pathological category probabilities of all detected cells in the whole WSI to generate the WSI-level diagnostic result includes steps S57-S58:

[0053] S57, the probability p of each pathological category c of all M cells in the WSI is weighted and averaged to obtain the WSI-level diagnostic probability, and the expression is as follows:

[0054]

[0055] where Pi,c represents the probability that the i-th cell belongs to the pathological category c, and M is the total number of cells involved in the diagnosis of the WSI, is the comprehensive diagnostic probability of the WSI in category c.

[0056] S58, according to the obtained WSI level probability The result analysis agent selects the category with the highest probability as the final diagnostic result according to the maximum likelihood principle, and judges the category of the WSI according to the threshold set for different pathological types to achieve more refined diagnostic output.

[0057] S59, the result analysis agent calls MLM to integrate the results and generate reports according to the initial input granularity determined by the task scheduling agent. The initial input granularity includes cervical single cell images, cervical patch level images, and cervical WSI level images. If the input is WSI, the result analysis agent takes the WSI level diagnostic result, the key evidence cell image coordinates, and the final probability of each category as the input data of the MLM, and the MLM converts the result into a natural language diagnostic report conforming to the clinical specification based on the preset clinical diagnostic report template; if the input is a patch level image, the patch level cell classification statistical data, the suspected lesion area coordinates and the model confidence are taken as the input of the MLM, and the MLM generates an analysis report of the patch based on the local area analysis template; if the input is a single cell image, the final category probability and feature description of the single cell are taken as the input of the MLM, and the MLM generates a morphological description and classification judgment report of the cell based on the cell morphology analysis template. Finally, the result analysis agent generates and outputs the final report and the corresponding key evidence according to any of the above modes.

[0058] Inventive effects

[0059] The application provides a cervical cell pathological panoramic image diagnosis method based on multi-agent cooperation. The method takes a task scheduling agent, a preprocessing agent, a model selection agent and a result analysis agent as cores, and through cooperative decision and information interaction among the multi-agents, a multi-task and high-precision diagnosis framework for cervical cell pathological panoramic images is constructed. Compared with a traditional pipeline system and a general medical intelligent agent lacking of field knowledge, the application can quickly and automatically process a large number of cervical cell panoramic images, sequentially complete key tasks such as abnormal area positioning, cell segmentation, cell classification and preliminary diagnosis suggestion generation, and realizes intelligent auxiliary diagnosis of cervical cell pathological images. The method of the application can be used as an auxiliary tool for pathologists, significantly improves the diagnosis efficiency and result consistency, and effectively reduces the influence of human reading subjective differences on the diagnosis conclusion. The application organically integrates artificial intelligence and medical treatment, has important application value in a cervical cancer large-scale screening scene, and is especially helpful for relieving the shortage of pathologists in areas with relatively limited medical resources, and promoting the popularization and precision of early diagnosis and treatment of cervical cancer. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 : overall architecture diagram of one embodiment of the application;

[0061] Figure 2 : system effect diagram of one embodiment of the application. DETAILED DESCRIPTION

[0062] To make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments of the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0063] Embodiment 1:

[0064] The application is a cervical cell pathological panoramic image diagnosis method based on multi-agent cooperation, which generally contains four core agent modules of a task scheduling agent, a preprocessing agent, a model selection agent and a result analysis agent. The agents communicate with each other through a message bus, cooperatively run in a center scheduling structure, and realize intelligent diagnosis and report generation of single cell images, patch images and WSI images or text input.

[0065] S1, system input:

[0066] S11, the system receives a cervical cell pathology image and a prompt word, or only receives the prompt word. The cervical cell pathology image includes a cervical cell single cell image, a cervical cell patch-level image, and a cervical cell pathology WSI.

[0067] S2, a task scheduling agent:

[0068] S21, the task scheduling agent calls the multi-modal large model GPT-4o to perform high-level semantic understanding and intent analysis on the input image type, text prompt information, and historical interaction records. If the input is a cervical WSI, the system first down-samples to generate a compressed image as input. The MLLM generates corresponding scalar scores for the three features based on the analysis results, and generates a priority score by combining a pre-set priority calculation formula to determine the combination of intelligent agents that should be started for this task.

[0069] S3, a preprocessing agent:

[0070] S31, after reading the original cervical WSI, the pyramid image structure information is parsed from the image metadata.

[0071] S32, determine the WSI resolution level most suitable for cell detection and subsequent analysis;

[0072] S33, after determining the optimal level, use a sliding window strategy to split the image Patch with a step size ps, and calculate the background proportion of each Patch through a background determination model. If the background proportion exceeds the set threshold, the Patch is automatically skipped to reduce invalid calculations;

[0073] S34, perform reference cell screening on the detected cell instances;

[0074] S35, perform feature normalization on the input image;

[0075] S36, for Patch images and single cell images, perform rotation, flipping, color perturbation, and other enhancement operations according to the tasks to be performed subsequently.

[0076] S4, model selection agent:

[0077] S41, when the task is abnormal cell detection, the model selection agent calls the YOLOv5 model to perform inference on the input image, outputting the bounding box, cell class, and corresponding confidence of each candidate cell;

[0078] S42, when the task is instance segmentation, the model selection agent calls the SOLOv2 network for segmentation inference and outputs the mask of the cell. Then, according to the area range of the cell, the candidate cells are screened out to ensure the quality of the effective candidate samples;

[0079] S43, when the task is single-cell classification, the task scheduling intelligent agent will call the ResNet50 network for classification inference, outputting the class probability of each cell.

[0080] S5, result analysis intelligent agent:

[0081] S51, after the model selection intelligent agent completes cell classification or feature extraction, the single-cell level comprehensive score of each cell is calculated, and the probability of the cell being predicted as each pathological category is calculated;

[0082] The steps for calculating the comprehensive score and pathological category probability of a single cell include S52-S55:

[0083] S52, calculate the linear score for each cell;

[0084] S53, map the linear score to a probability value through a mapping function;

[0085] S54, for each pathological category, the normalized features are weighted and summed using category-specific weights to obtain the weighted score under the category;

[0086] S55, use the sigmoid function to map the weighted score to the probability that the cell belongs to the pathological category;

[0087] S56, after the single-cell score calculation is completed, the system fuses the probabilities of all detected cells in the entire WSI to generate a WSI-level diagnostic result;

[0088] The steps for fusing the pathological category probabilities of all detected cells in the entire WSI to generate a WSI-level diagnostic result include S57-S58:

[0089] S57, perform a weighted average of the probability of each pathological category for all cells in the WSI to obtain the WSI-level diagnostic probability;

[0090] S58, according to the obtained WSI-level probability, the system selects the category with the highest probability as the final diagnostic result using the maximum likelihood principle, and judges the category of the WSI according to the threshold set for different pathological types to achieve more refined diagnostic output;

[0091] S59、Result analysis agent calls the MLLM according to the initial input granularity determined by the task scheduling agent to generate the result integration and report. The initial input granularity includes: cervical single cell image, cervical patch-level image, and cervical WSI-level image. If the input is WSI, the result analysis agent takes the WSI-level diagnosis result, key evidence cell image coordinates, and final probability of each category as the input data of the MLLM, and the MLLM converts the result into a natural language diagnosis report conforming to the clinical standard based on the preset clinical diagnosis report template; if the input is a patch-level image, the MLLM takes the patch-level cell classification statistical data, suspected lesion area coordinates, and model confidence as the input, and generates an analysis report for the patch based on the local area analysis template; if the input is a single cell image, the MLLM takes the final category probability and feature description of the single cell as the input, and generates a morphological description and classification judgment report for the cell based on the cell morphology analysis template. Finally, the result analysis agent generates and outputs the final report and the corresponding key evidence according to any of the above modes.

[0092] The embodiments of the present application will be described in detail below:

[0093] The embodiments of the present application are a cervical cell pathology panoramic image diagnosis method based on multi-agent cooperation. The multi-agent method of the present application is applied to complete the classification diagnosis of cervical cell panoramic images, the classification and segmentation of single cell images, the detection of abnormal cells in patch images, and the intelligent question and answer task of input text. Please refer to Figure 1 Read the following steps, which are the system architecture diagram of the embodiment.

[0094] S1、System input:

[0095] S11、The system receives cervical cell pathology images and prompt words, or only receives prompt words. The cervical cell pathology images include cervical single cell images, cervical patch-level images, and cervical WSI.

[0096] S2、Task scheduling agent:

[0097] S21、The task scheduling agent calls the multi-modal large model GPT-4o to perform high-level semantic understanding and intent analysis on the input image type, text prompt information, and historical interaction record. If the input is cervical WSI, the system first down-samples to generate a compressed image as input. The MLLM generates corresponding scalar scores for the three features based on the analysis result, and generates a priority score by combining a preset priority calculation formula to determine the combination of agents that should be started for this task. The priority calculation formula is as follows:

[0098] priority = f(ImageType, TextHint, Historty) = a x I + b x T + g x H (9)

[0099] where I represents the scalar score corresponding to the image type generated by MLLM parsing, T represents the scalar score of the text hint generated by MLLM parsing, H represents the scalar score of the historical interaction generated by MLLM parsing, a, b, g are weight hyperparameters learned by the agent through statistical learning, and are used to balance the influence of the three factors on task scheduling.

[0100] S3, preprocessing agent:

[0101] S31, after reading the original cervical WSI, first parse the pyramid image structure information from the image metadata. The metadata contains the total number of resolution levels of the WSI, the image width w l , image height h l , down-sampling ratio d l and resolution level index l and other parameters. Based on the above parameters, the system can accurately know the spatial size of the WSI at different resolutions, which lays the foundation for subsequent optimal level selection, patch segmentation and feature extraction;

[0102] S32, the preprocessing agent pre-sets the target patch number N target for subsequent analysis. For each resolution level l, calculate the number of patches N l generated by sliding window segmentation at this resolution level. The expression is as follows:

[0103]

[0104] where w l and h l represent the width and height of the lth resolution level, respectively, and ps represents the patch size.

[0105] Then the best level is selected by minimizing the error between the patch number and the target. The expression is as follows:

[0106] l * = arg l min | N l - N target | (11)

[0107] where N l is the number of patches generated by sliding window segmentation at level l, N target is the target patch number pre-set by the preprocessing agent for subsequent analysis, and l * represents the WSI resolution level most suitable for subsequent tasks.

[0108] S33, determining an optimal level l * Afterwards, the image is cut into patches with a sliding window strategy with a step size ps, and the background proportion of each patch is calculated by a background determination model. If the background proportion exceeds a set threshold, the patch is automatically skipped to reduce invalid calculations.

[0109] S34, performing reference cell screening on the detected cell instances;

[0110] The reference cell screening includes the following steps:

[0111] The preprocessing agent first automatically removes low-quality cells in blurred, broken, or artifact regions according to cell area, nuclear contour integrity, staining saturation, and focus clarity. Then, based on cell morphological consistency, nuclear-cytoplasmic ratio stability, and texture distribution characteristics, a "reference cell set" for global alignment is constructed from the remaining high-quality cells.

[0112] S35, performing feature normalization processing on the input image;

[0113] The feature normalization processing includes the following steps:

[0114] The preprocessing agent calculates the mean vector and variance in the feature space of the reference cell set, and uses them as a normalization reference to standardize the feature vectors of all cells in the current WSI, thereby eliminating feature offsets caused by imaging styles, staining changes, and slice differences between different WSI, and providing consistent and stable input features for subsequent model selection agent cell classification and feature reasoning.

[0115] S36, performing rotation, flipping, color perturbation, and other enhancement operations on the patch image and single cell image.

[0116] S4, model selection agent:

[0117] S41, when the task is abnormal cell detection, the model selection agent calls the YOLOv5 model from the system model library to perform inference on the input image, and outputs the bounding box, cell class, and corresponding confidence of each candidate cell;

[0118] S42, when the task is instance segmentation, the model selection agent calls the SOLOv2 network for segmentation inference and outputs the mask of the cell. Then, the candidate cells are screened according to the cell area range to ensure the quality of the effective candidate samples;

[0119] S43, when the task is single cell classification, the model selection agent calls the ResNet50 network for classification inference and outputs the class probability of each cell.

[0120] S5, Results Analysis Agent:

[0121] S51. After the model selects an agent to complete cell classification or feature extraction, the result analysis agent calculates the comprehensive score of each cell and the probability that the cell is predicted to belong to each category. The categories include: Negative / No intraepithelial lesion or malignancy (NILM), Atypical squamous cells of indeterminate significance (ASC-US), Atypical squamous cells that cannot rule out high-grade lesions (ASC-H), Low-grade squamous intraepithelial lesion (LSIL), and High-grade squamous intraepithelial lesion (HSIL).

[0122] The calculation of the overall score and pathological category probability of a single cell includes steps S52 to S55:

[0123] S52. For each cell, its features are represented as a column vector X = [x1, x2, ..., x...]. n ] T Using the weighted row vector W = [w1, w2, ..., w2] obtained during training n Calculate the linear score. The expression is as follows:

[0124] s=W·X (12)

[0125] Where s = W·X is the vector dot product, i.e. x i Let w represent the i-th eigenvalue. i is the corresponding weighting coefficient, and s is the linear composite score of the cell.

[0126] S53. The linear score is mapped to a probability value using the mapping function f(s), which serves as the cell's overall score. The expression is as follows:

[0127] p = f(s) (13)

[0128] Where p∈(0,1) represents the overall score of the cell.

[0129] S54. For each pathological category c, utilize the category-specific weights w learned by the cell classification model during the training phase through backpropagation. k (c) For normalized feature f k Perform a weighted summation to obtain the weighted score for that category. The expression is as follows:

[0130]

[0131] Where k represents the feature index, f k Let k be the k-th normalized eigenvalue, and map(·) be the feature mapping function. This represents the importance weight of the feature under category c.

[0132] S55. Using the sigmoid function, the weighted score is mapped to the probability that the cell belongs to pathological category c. The expression is as follows:

[0133]

[0134] Where, α c The category adjustment parameter is a scaling factor learned independently for each pathological category during model training. It is used to adjust the sensitivity or decision boundary of the category score, making each category more discriminative during probability mapping. (c) ∈(0,1) represents the probability that the cell belongs to category c.

[0135] S56. After the single-cell comprehensive score and category probability calculation is completed, the result analysis agent fuses the pathological category probabilities of all detected cells in the entire WSI to generate WSI-level diagnostic results.

[0136] The process of fusing the pathological category probabilities of all detected cells in the entire WSI to generate a WSI-level diagnostic result includes the following steps: S57-S58:

[0137] S57. Probability c for each pathological category of all M cells in WSI The weighted average is used to obtain the diagnostic probability of WSI level, as shown in the following expression:

[0138]

[0139] in, Let represent the probability that the i-th cell belongs to pathological category c, and M be the total number of cells involved in diagnosis in WSI. This represents the overall diagnostic probability of the WSI under category c.

[0140] S58. Based on the obtained WSI level probability The results analysis agent uses the maximum likelihood principle to select the category with the highest probability as the final diagnosis result, and judges the category of WSI according to the threshold set for different pathological types, so as to achieve more refined diagnostic output.

[0141] S59. The result analysis agent invokes MLLM to integrate results and generate reports based on the initial input granularity determined by the task scheduling agent. The initial input granularity includes: cervical single-cell images, cervical patch-level images, and cervical WSI-level images. If the input is WSI, the result analysis agent uses the WSI-level diagnostic results, key evidence cell image coordinates, and the final probability of each category as input data to MLLM. MLLM, based on a preset clinical diagnostic report template, converts the results into a natural language diagnostic report conforming to clinical standards. If the input is a patch-level image, the cell classification statistics, suspected lesion area coordinates, and model confidence level at the patch level are used as input to MLLM. MLLM generates an analysis report for the patch based on a local region analysis template. If the input is a single-cell image, the final category probability and feature description of the single cell are used as input to MLLM. MLLM generates a morphological description and classification judgment report for the cell based on a cell morphology analysis template. Finally, the result analysis agent generates and outputs the final report and corresponding key evidence according to any of the above modes.

[0142] To demonstrate the effectiveness of this invention, various input combinations of prompt words and image types are provided, and this embodiment can provide answers in all of them. Please refer to [link / reference]. Figure 2 This is a diagram illustrating the effect of one embodiment of the present invention.

[0143] As demonstrated by the above embodiments, the cervical cytopathology panoramic image diagnostic method based on multi-agent collaboration proposed in this invention can simulate the actual diagnostic process of a cytopathologist, completing multiple tasks such as abnormal area localization, cell segmentation, cell classification, and preliminary diagnostic suggestion generation. This collaborative architecture not only efficiently processes gigapixel-level cervical WSI images but also achieves a good balance between versatility and specialization, organically integrating knowledge from the field of cytopathology into the automated processing flow, thereby significantly improving the classification accuracy and diagnostic reliability of cervical cytopathology panoramic images.

[0144] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A diagnostic method for cervical cytology based on multi-agent collaborative whole-slide image (WSI) imaging, characterized in that, It includes a task scheduling agent, a preprocessing agent, a model selection agent, and a result analysis agent. S1, System Input: S11. The system receives cervical cell pathology images and prompts, or only prompts. The cervical cell pathology images include cervical single-cell images, cervical patch (WSI slice) level images, and cervical WSI. S2, Task Scheduling Agent: S21. The task scheduling agent invokes the Multimodal Large Language Model (MLLM) GPT-4o to perform advanced semantic understanding and intent analysis on the input image type, text prompts, and historical interaction records. If the input is a cervical WSI image, the system first downsamples and generates a compressed image as input. Based on the analysis results, MLLM generates corresponding scalar scores for these three features and combines them with a preset priority calculation formula to generate a priority score that determines the combination of agents to be activated for this task. The priority calculation formula is as follows: priority=f(ImageType,TextHint,Historty)=α×I+β×T+γ×H (1) Where I represents the scalar score corresponding to the image type generated by MLLM parsing, T represents the scalar score of the text prompt generated by MLLM parsing, H represents the scalar score of the historical interactions generated by MLLM parsing, and α, β, γ are weight hyperparameters, which are learned by the agent through statistical methods. S3, Preprocessing Agent: S31. After reading the original cervical WSI, firstly, the pyramid-shaped image structure information is parsed from the image metadata. The metadata includes the total number of resolution levels of the WSI and the image width w corresponding to each resolution level. l Image height h l Downsampling ratio d l And parameters such as resolution level index l. S32. The preprocessing agent selects the optimal resolution level of WSI based on the type of task to be performed subsequently. * ; S33. Determine the optimal level. * Then, the image is segmented into patches using a sliding window strategy with a step size of ps, and the background ratio of each patch is calculated by a background determination model. If the background ratio exceeds a set threshold, the patch is automatically skipped. S34. Perform baseline cell screening on the detected cell instances; S35. Perform feature normalization processing on the input image; S36. For patch images and single-cell images, perform enhancement operations such as rotation, flipping, and color perturbation. S4. Model Selection of Agents: S41. When the task is abnormal cell detection, the model selects an agent to call the YOLOv5 model from the system model library to perform inference on the input image and output the bounding box, cell category and corresponding confidence score of each candidate cell. S42. When the task is instance segmentation, the model selects an agent to call the SOLOv2 network for segmentation inference and outputs the cell mask (mask or pixel-level region). S43. When the task is single-cell classification, the model selects an agent to call the ResNet50 network for classification reasoning and outputs the class probability of each cell. S5, Results Analysis Agent: S51. After the model selects an agent to complete cell classification or feature extraction, the result analysis agent calculates the comprehensive score of each cell and the probability that the cell is predicted to be in each category. The categories include: negative / no intraepithelial lesion or malignancy (NILM), atypical squamous cells of indeterminate significance (ASC-US), atypical squamous cells that cannot rule out high-grade lesions (ASC-H), low-grade squamous intraepithelial lesion (LSIL), and high-grade squamous intraepithelial lesion (HSIL). The calculation of the overall score and pathological category probability of a single cell includes steps S52 to S55: S52. For each cell, its features are represented as a column vector X = [x1, x2, ..., x...]. n ] T Using the weighted row vector W = [w1, w2, ..., w2] obtained during training n Calculate the linear score. The expression is as follows: s=W·X (2) Where s = W·X is the vector dot product, i.e. x i Let w represent the i-th eigenvalue. i is the corresponding weighting coefficient, and s is the linear composite score of the cell. S53. The linear score is mapped to a probability value using the mapping function f(s), which serves as the cell's overall score. The expression is as follows: p=f(s) (3) Where p∈(0,1) represents the overall score of the cell. S54. For each pathological category c, utilize the category-specific weights w learned by the cell classification model during the training phase through backpropagation. k (c) For normalized feature f k Perform a weighted summation to obtain the weighted score for that category. The expression is as follows: Where k represents the feature index, f k Let k be the k-th normalized eigenvalue, and map(·) be the feature mapping function. This represents the importance weight of the feature under category c. S55. Use the sigmoid function to map the weighted score to the probability that the cell belongs to pathological category c. The expression is as follows: Where, α c The category adjustment parameter is a scaling factor learned independently for each pathological category during model training. It is used to adjust the sensitivity or decision boundary of the category score, making each category more discriminative during probability mapping. (c) ∈(0,1) represents the probability that the cell belongs to category c. S56. After the single-cell comprehensive score and category probability calculation is completed, the result analysis agent fuses the pathological category probabilities of all detected cells in the entire WSI to generate WSI-level diagnostic results. The process of fusing the pathological category probabilities of all detected cells in the entire WSI to generate a WSI-level diagnostic result includes the following steps: S57-S58: S57. Probability c for each pathological category of all M cells in WSI A weighted average is then applied to obtain the diagnostic probability at the WSI level. The expression is as follows: in, Let represent the probability that the i-th cell belongs to pathological category c, and M be the total number of cells involved in diagnosis in WSI. This represents the overall diagnostic probability of the WSI under category c. S58. Based on the obtained WSI level probability The results analysis agent uses the maximum likelihood principle to select the category with the highest probability as the final diagnosis result, and judges the category of WSI according to the threshold set for different pathological types, so as to achieve more refined diagnostic output. S59. The result analysis agent calls MLLM to integrate results and generate reports based on the initial input granularity determined by the task scheduling agent. The initial input granularity includes: cervical single-cell images, cervical patch-level images, and cervical WSI-level images. If the input is WSI, the result analysis agent uses the WSI-level diagnostic results, key evidence cell image coordinates, and the final probability of each category as input data to MLLM. MLLM converts the results into a natural language diagnostic report that conforms to clinical standards based on a preset clinical diagnostic report template. If the input is a patch-level image, the cell classification statistics, suspected lesion area coordinates, and model confidence level at the patch level are used as input to MLLM. MLLM generates an analysis report for the patch based on a local region analysis template. If the input is a single-cell image, the final category probability and feature description of the single cell are used as input to MLLM. MLLM generates a morphological description and classification judgment report for the cell based on a cell morphology analysis template. Finally, the result analysis agent generates and outputs the final report and corresponding key evidence according to any of the above modes.

2. The method of selecting the optimal WSI resolution level for subsequent tasks as described in claim 1, characterized in that, The method for selecting the optimal resolution level in step S32 is as follows: The preprocessing agent pre-sets the number N of target patches for subsequent analysis. target For each resolution level l, calculate the number N of patches generated by the downward window segmentation at that resolution level. l The expression is as follows: Among them, w l and h l These represent the width and height of the l-th resolution layer, respectively, and ps represents the patch size. Then, the optimal level is selected by minimizing the error between the number of patches and the target. The expression is as follows: l * =angry l my∣N l -N target ∣ (8) Where, N l N represents the number of patches generated by the downward window segmentation at level l. target The number of target patches for subsequent analysis is pre-defined for the preprocessing agent. * This indicates the WSI resolution level best suited for subsequent tasks.

3. The method of screening baseline cells for detected cell instances as described in claim 1, characterized in that, The method for screening the baseline cells in step S34 is as follows: The preprocessing agent first automatically removes low-quality cells from blurry, fragmented, or artifact areas based on cell area, nuclear outline integrity, staining saturation, and focus clarity. Then, among the remaining high-quality cells, a "benchmark cell set" is constructed based on cell morphological consistency, nuclear-cytoplasmic ratio stability, and texture distribution characteristics.

4. The feature normalization processing of the input image as described in claim 1, characterized in that, The feature normalization method described in step S34 is as follows: The preprocessing agent calculates the mean vector and variance in the feature space of the baseline cell set, and uses these as a normalization reference to standardize the feature vectors of all cells in this WSI.

Images