Slide image processing method, device and equipment based on dynamic consensus

Abstract

Embodiments of the present disclosure disclose a slide image processing method, device and equipment based on dynamic consensus. A specific embodiment of the method comprises: generating a multi-modal instruction package according to an obtained original cervical cell image and a pre-trained visual language model; generating an image processing result data package according to an image processing agent group, the multi-modal instruction package and cervical cell retrieval knowledge information; generating a first image processing result according to the image processing result data package and historical weight information; determining a category number mapping relationship according to the obtained first image processing results; dynamically dividing a preset storage space according to the category number mapping relationship to obtain a storage space corresponding to the original cervical cell image; and storing and processing the original cervical cell image according to the storage space and the first image processing result. This embodiment reduces the storage resources consumed, shortens the time consumed for overall classified storage, and reduces the computing resources consumed.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to a method, apparatus, and device for slide image processing based on dynamic consensus. Background Technology

[0002] In the field of cell morphology research, to achieve digital management of a large number of slide images (e.g., cervical cell slide images), it is first necessary to classify the different types of slide images. Then, the slide images are stored in a structured manner based on the classification results. Currently, the common approach to classifying and storing slide images is as follows: first, simple deep learning single models or ensemble multi-models are used to classify the slide images. Then, the slide images are stored based on the classification results.

[0003] However, in practice, it has been found that when classifying and storing slide images using the above method, the following technical problems often arise: Cell morphology is complex, and different cell classes exhibit high intra- and inter-class similarity. Existing single-model deep learning systems typically employ a single architecture for classification, resulting in poor robustness when handling cell classes with ambiguous boundaries. Furthermore, these models are prone to model illusions under uncertain conditions, generating artificially high confidence scores and leading to low classification accuracy. Due to classification errors, a large number of slide images are incorrectly stored, requiring secondary classification and storage, which consumes significant storage resources for intermediate results and is time-consuming for slide image classification. In addition, existing multi-model ensembles often use static weights, requiring the parallel execution of multiple large models, resulting in high computational resource consumption.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose a slide image processing method, apparatus, electronic device, and computer-readable medium based on dynamic consensus to solve one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide a slide image processing method based on dynamic consensus. The method includes: in response to detecting image processing request information corresponding to each target slide in each target slide, performing the following steps based on the acquired original cervical cell image: generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing intelligent agent group, a shared memory knowledge base, and a private memory knowledge base; generating an image processing result data package based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information; generating image processing coefficient information and image processing probability information based on the image processing result data package and historical weight information; in response to detecting that the image processing coefficient information satisfies preset constraints, generating a first image processing result based on the image processing probability information; determining a category quantity mapping relationship based on each obtained first image processing result; dynamically dividing a preset storage space according to the category quantity mapping relationship to obtain a storage space corresponding to the original cervical cell image; and storing the original cervical cell image based on the storage space and the first image processing result.

[0008] Secondly, some embodiments of this disclosure provide a slide image processing apparatus based on dynamic consensus, comprising: an execution unit configured to, in response to detecting image processing request information corresponding to each target slide in each target slide, perform the following steps based on the acquired raw cervical cell image: generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing intelligent agent group, a shared memory knowledge base, and a private memory knowledge base; generating an image processing result data package based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information; and generating an image processing result data package based on the above... The image processing result data packet and historical weight information are used to generate image processing coefficient information and image processing probability information; in response to detecting that the image processing coefficient information meets preset constraints, a first image processing result is generated based on the image processing probability information; a determining unit is configured to determine a category quantity mapping relationship based on each obtained first image processing result; a partitioning unit is configured to dynamically partition a preset storage space based on the category quantity mapping relationship to obtain a storage space corresponding to the original cervical cell image; and a storage unit is configured to perform storage processing on the original cervical cell image based on the storage space and the first image processing result.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any of the implementations of the first or second aspect.

[0011] The above-described embodiments of this disclosure have the following beneficial effects: A slide image processing method based on dynamic consensus, as described in some embodiments of this disclosure, can improve classification accuracy, reduce storage resource consumption, shorten the overall classification storage time, and reduce computational resource consumption. The reasons for poor classification accuracy, high storage resource consumption, long overall classification storage time, and high computational resource consumption are as follows: Cell morphology is complex, and there is high intra- and inter-class similarity between different cell types. Existing deep learning single models typically use a single architecture for classification, which has poor robustness when dealing with cell types with blurred boundaries. Furthermore, the model is prone to model illusions under uncertain conditions, generating artificially high confidence levels, resulting in low classification accuracy. Due to classification errors, a large number of slide images are incorrectly stored, requiring secondary classification and storage of the slide images, leading to high storage resource consumption for storing intermediate results and long classification time for slide images. In addition, existing multi-model ensembles often use static weights, requiring the parallel operation of multiple large models, resulting in high computational resource consumption. Based on this, some embodiments of the slide image processing method based on dynamic consensus of this disclosure, in response to detecting image processing request information corresponding to each target slide in each target slide, perform the following steps according to the acquired original cervical cell image: First, a multimodal instruction packet corresponding to the above-mentioned image processing request information is generated according to a pre-trained visual language model. Thus, a multimodal instruction packet corresponding to the above-mentioned original cervical cell image can be obtained. Then, cervical cell retrieval knowledge information is generated according to the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base. Thus, retrieval knowledge can be obtained by searching from the shared memory knowledge base and the private memory knowledge base respectively. Next, an image processing result data packet is generated according to the image processing intelligent agent group, the multimodal instruction packet, and the cervical cell retrieval knowledge information. Thus, an image processing result data packet corresponding to the above-mentioned original cervical cell image can be obtained. Next, image processing coefficient information and image processing probability information are generated according to the above-mentioned image processing result data packet and historical weight information. Thus, the probability distribution and classification consistency score corresponding to the above-mentioned image processing result data packet can be obtained. Next, in response to the detection that the image processing coefficient information satisfies the preset constraints, a first image processing result is generated based on the image processing probability information. Thus, a classification result corresponding to the original cervical cell image can be obtained. Then, based on each obtained first image processing result, a category number mapping relationship is determined. Thus, a category number mapping relationship corresponding to each first image processing result can be obtained. Next, based on the above category number mapping relationship, a preset storage space is dynamically divided to obtain a storage space corresponding to the original cervical cell image. Thus, a storage space corresponding to the original cervical cell image can be obtained.Finally, based on the aforementioned storage space and the results of the first image processing, the original cervical cell images are stored. Because classification is not performed using a single architecture, but rather based on a pre-trained visual language model, the original cervical cell images are structurally analyzed to obtain multimodal instruction packages, and classification is then performed based on these multimodal instruction packages. Furthermore, because classification is based on cervical cell retrieval knowledge information obtained from shared and private memory knowledge bases, it can simultaneously integrate authoritative standard knowledge within the domain and the model's own personalized error experience. Moreover, because historical weight information is introduced, dynamic weighting and evaluation of the classification results from multiple models can be achieved. Therefore, classification accuracy can be improved, storage resources consumed can be reduced, the overall classification and storage time can be shortened, and computational resources consumed can be reduced. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the slide image processing method based on dynamic consensus according to the present disclosure; Figure 2 These are schematic diagrams of some embodiments of the slide image processing apparatus based on dynamic consensus according to this disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Figure 1 A flow 100 of some embodiments of the slide image processing method based on dynamic consensus according to this disclosure is shown. This slide image processing method based on dynamic consensus includes the following steps: Step 101: In response to detecting image processing request information for each target slide in each target slide, the following steps are performed based on the acquired raw cervical cell image: Step 1011: Generate a multimodal instruction packet corresponding to the image processing request information based on the pre-trained visual language model.

[0021] In some embodiments, the execution entity (e.g., a computing device) of the slide image processing method based on dynamic consensus can generate a multimodal instruction package corresponding to the image processing request information according to a pre-trained visual language model. Each target slide can represent a slide corresponding to the original cervical cell image. The image processing request information can represent an instruction to classify the original cervical cell image. The original cervical cell image can represent a digital image of cervical cells after liquid-based thin-layer preparation or Papanicolaou staining. The original cervical cell image can include a global view and a local view. The pre-trained visual language model can represent a fine-tuned visual language model. For example, the pre-trained visual language model can be a CLIP. The pre-trained visual language model can include a visual encoder and a decoder. The visual encoder of the pre-trained visual language model can take the original cervical cell image as input and output a multi-scale visual feature vector. The decoder of the pre-trained visual language model can take the multi-scale visual feature vector as input and output a morphological text description and an initial classification probability vector. The pre-trained visual language model can be trained in batches. The multimodal instruction package can represent the data package obtained after structured packaging of the multi-scale visual feature vectors, morphological text descriptions, and initial classification probability vectors. The multi-scale visual feature vectors can represent feature vectors extracted from the original cervical cell image through multi-layer convolution. The morphological text descriptions can represent statements describing the pathological features of the multi-scale visual feature vectors; for example, the morphological text description could be "increased nuclear-cytoplasmic ratio, irregular nuclear membrane." The initial classification probability vector can represent the normalized probability distribution output by the pre-trained visual language model for a preset category. The preset category can represent a pre-defined category describing cell state. For example, the preset category could be AGC type cells.

[0022] In some optional implementations of certain embodiments, the execution entity may generate a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model through the following steps: The first step involves generating multi-scale visual feature vectors based on the original cervical cell image and the visual encoder included in the pre-trained visual language model. In practice, the executing entity can input the original cervical cell image into the visual encoder included in the pre-trained visual language model to obtain the multi-scale visual feature vectors.

[0023] The second step involves generating a morphological text description and an initial classification probability vector based on the aforementioned multi-scale visual feature vectors and the decoder included in the pre-trained visual language model. In practice, the executing entity can input the aforementioned multi-scale visual feature vectors into the decoder included in the pre-trained visual language model to obtain the morphological text description and the initial classification probability vector.

[0024] The third step involves structuring the aforementioned multi-scale visual feature vectors, morphological text descriptions, and initial classification probability vectors to obtain a multimodal instruction package. In practice, the executing entity can use data encapsulation to structurally package the aforementioned multi-scale visual feature vectors, morphological text descriptions, and initial classification probability vectors to obtain the multimodal instruction package.

[0025] Step 1012: Generate cervical cell retrieval knowledge information based on the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base.

[0026] In some embodiments, the aforementioned executing entity can generate cervical cell retrieval knowledge information based on an image processing intelligent agent group, a shared memory knowledge base, and a private memory knowledge base. The image processing intelligent agent group can include a predetermined number of structurally identical classification decision agents. This predetermined number can be 6. The classification decision agent can represent a large-scale language model possessing basic knowledge and dialogue capabilities in the field of cervical cytology. The classification decision agent can be Llama-2-70B. The image processing intelligent agent group can take multimodal instruction packets and cervical cell retrieval knowledge information as input and output image processing result data packets. The classification decision agent can take reflection instructions as input and output structured reflection text. The classification decision agent can include one input embedding layer, 80 Transformer layers, and one output layer. Each of the 80 Transformer layers can include one grouped query attention layer, one feedforward neural network, two layers of normalization, and two layers of residual connections. The shared memory knowledge base can represent a database storing decision criteria and reference cases in the field of cell morphology. The aforementioned private memory knowledge base can represent a vector library storing the reflective memory records of the aforementioned image processing agent group. It should be noted that the aforementioned reflective memory records can be empty. The aforementioned cervical cell retrieval knowledge information can represent decision criteria, typical case information, and the aforementioned reflective memory records. The aforementioned decision criteria can represent the standards of knowledge in the corresponding cell morphology field. The aforementioned typical case information can represent referable cases in the field of cell morphology. The aforementioned referential cases can include images of cells with arbitrary morphological characteristics, morphological text descriptions, cell categories, and decision-making criteria.

[0027] In addressing the technical problems mentioned above, when classifying and storing cell slide images from different sources (e.g., different staining schemes) for the intended application scenario, the following technical problem often arises: Slide images from different sources exhibit significant domain differences. Different staining schemes (e.g., Papanicolaou staining and HE staining) result in variations in the visual feature distribution of the cell nucleus and cytoplasm. Feature representations extracted by agents trained on a single data domain become biased, leading to batch classification errors. This necessitates a secondary classification and storage of the slide images. If agents that make incorrect predictions are not promptly forced to reflect on their errors, summarize the causes, and provide corrective suggestions, the secondary classification will still output incorrect results, leading to poor classification accuracy, high computational resource consumption, and long classification time. To meet the following requirements for this application scenario: real-time learning and correction of error patterns, and accurate correction of batches of continuous errors, we have decided to adopt the following solution: Optionally, the aforementioned implementing entity may also perform the following steps to update the aforementioned private memory knowledge base: The first step, based on each classification decision agent in the image processing agent group described above, is to perform the following steps: The first sub-step involves obtaining the prediction result data of the classification decision agent based on the confidence distribution included in the image processing result data package. This prediction result data includes a prediction category and a prediction probability. The prediction category represents the classification result of the classification decision agent on the original cervical cell image. The specific type of the prediction category is not limited here; for example, the prediction category could be ASC-US. The prediction probability represents the probability corresponding to the prediction category. In practice, firstly, the executing agent can determine the category with the highest probability in the confidence distribution included in the image processing result data package as the prediction category. Then, the probability corresponding to the prediction category is determined as the prediction probability. Finally, the prediction category and the prediction probability are determined as the prediction result data.

[0028] The second sub-step, in response to determining that the predicted result data meets preset triggering conditions, involves structurally packaging the original cervical cell image, the multimodal instruction package, the true label information, and the predicted probability to obtain reflective context information. The preset triggering conditions may include an inconsistency between the predicted category and the true label information in the predicted result data, an image processing coefficient information less than a first preset threshold, and a predicted probability greater than a second preset threshold. The true label information can represent the correct category corresponding to the original cervical cell image. The first preset threshold can be 0.75. The second preset threshold can be 0.8. The reflective context information can represent a structured data package containing the original cervical cell image, the multimodal instruction package, the true label information, and the predicted probability. In practice, the executing entity can encapsulate the original cervical cell image, the multimodal instruction package, the true label information, and the predicted probability into JSON format to obtain the reflective context information.

[0029] The third sub-step involves constructing a reflection instruction based on the aforementioned reflection context information and the target reflection template. The target reflection template can represent a structured text framework containing placeholders and a target output format. For example, the target reflection template could be: "Please output the reflection content in the following target output format: Morphological text description: {xxx}; Predicted category: {xxx}; Correct category: {xxx}; Error cause analysis: {xxx}; Improvement suggestions: {xxx}". The target output format can represent JSON format. The reflection instruction can represent the text obtained by filling the target reflection template with the aforementioned reflection context information. In practice, the executing entity can construct the reflection instruction by replacing strings to fill the target reflection template with the aforementioned reflection context information.

[0030] The fourth sub-step involves generating structured reflection text based on the aforementioned reflection instructions and the classification decision-making agent. This structured reflection text represents the text output by the classification decision-making agent corresponding to the aforementioned reflection instructions. For example, the structured reflection text could be: "Morphological text description: {Moderately increased nucleocytoplasmic ratio, smooth nuclear membrane}; Predicted category: {LSIL}; Correct category: {ASC-US}; Error cause analysis: {Over-reliance on the single feature of nucleocytoplasmic ratio, ignoring key distinguishing features such as uniform chromatin distribution in the nucleus and a smooth, unnotched nuclear membrane, which conform to ASC-US rather than LSIL}; Improvement suggestion: {Nucleocytoplasmic ratio, chromatin distribution, and nuclear membrane regularity should be evaluated simultaneously to avoid relying solely on a single feature for classification}." In practice, the executing agent can input the aforementioned reflection instructions into the classification decision-making agent to obtain the structured reflection text.

[0031] The fifth sub-step involves vectorizing and extracting keywords from the structured reflective text to obtain a structured reflective vector and a keyword tag set. The structured reflective vector represents the numerical representation of the structured reflective text in the semantic space. The keyword tags in the keyword tag set represent the words extracted from the structured reflective text. The specific content of the keyword tags is not limited here; for example, a keyword tag could be chromatin roughness. In practice, firstly, the executing entity can encode the structured reflective text into a vector using a text encoder to obtain the structured reflective vector. Then, the TextRank algorithm is used to extract keywords from the structured reflective text to obtain the keyword tag set. The specific type of text encoder is not limited here; for example, a CLIP text encoder can be used.

[0032] The sixth sub-step involves identifying the aforementioned structured reflection vector, keyword tag set, and structured reflection text as the reflection memory record corresponding to the aforementioned classification decision agent. The reflection memory record can represent a structured record corresponding to the aforementioned classification decision agent. The reflection memory record may contain the aforementioned structured reflection vector, keyword tag set, and structured reflection text.

[0033] The second step is to store the aforementioned reflective memory records in the aforementioned private memory knowledge base in order to update the aforementioned private memory knowledge base.

[0034] The above-described technical solution, as an inventive point of this disclosure, solves technical problem two: "poor classification accuracy, high computational resource consumption, and long time consumption for classifying slide images." The reasons for this are as follows: slide images from different sources exhibit significant domain differences; different staining schemes (e.g., Papanicolaou staining and HE staining) cause differences in the visual feature distribution of cell nuclei and cytoplasm. Feature representations extracted by agents trained on a single data domain will shift, resulting in batches of classification errors. This necessitates reclassifying and storing the slide images. If agents that make incorrect predictions are not promptly subjected to forced reflection to summarize the causes of errors and provide correction suggestions, incorrect classification results will still be output during reclassification, leading to poor classification accuracy, high computational resource consumption, and long time consumption for classifying slide images. Solving these factors can improve classification accuracy, reduce computational resource consumption, and shorten the time required for classifying slide images. To achieve this effect, the slide image processing method based on dynamic consensus disclosed herein first packages the prediction results of the mispredicting classification decision agent into structured reflection context information, and constructs reflection instructions by combining them with a target reflection template. Therefore, the classification decision agent can be guided to reflect on itself according to the reflection instructions, summarizing the reasons for the prediction errors and improvement suggestions, resulting in structured reflection text. Finally, based on the structured reflection text, a reflection memory record corresponding to the mispredicting classification decision agent is constructed and stored in its private memory bank. This allows for the optimization of the mispredicting classification decision agent, and classification can be performed based on the optimized agent. This improves classification accuracy, reduces computational resources, and shortens the time required to classify slide images.

[0035] In some optional implementations of certain embodiments, the aforementioned execution entity can generate cervical cell retrieval knowledge information based on the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base through the following steps: The first step is to extract and process the shared memory knowledge base based on the image processing intelligent agent group mentioned above to obtain decision criteria and typical case information.

[0036] The second step involves extracting and processing the aforementioned private memory knowledge base to obtain reflective memory records. In practice, the executing entity can use knowledge retrieval enhancement techniques to extract and process the aforementioned private memory knowledge base to obtain reflective memory records. For example, knowledge retrieval enhancement techniques could be dense retrieval techniques.

[0037] The third step is to identify the above decision-making criteria, the above typical case information, and the above reflection and memory records as cervical cell retrieval knowledge information.

[0038] In addressing the technical problems mentioned above, when classifying and storing slide images of problematic cells with ambiguous chromatin distribution and atypical nuclear membrane morphology in the intended application scenario, the following technical problem often arises: The morphological boundaries of problematic cells are blurred. Retrieving images based solely on single-modal feature vectors often contains significant noise, leading to poor retrieval accuracy. This results in a large amount of similar but inaccurate data being used as a reference for classifying the target slide image, further reducing classification accuracy. Due to classification errors, many slide images are incorrectly stored, requiring secondary classification and storage, which consumes significant storage resources and is time-consuming. To meet the following requirements of this application scenario: accurate retrieval adaptable to multimodal ambiguous features and precise classification decisions for problematic cells, we have decided to adopt the following solution: Optionally, the aforementioned executing entity may further perform the following steps: extracting and processing the shared memory knowledge base from the aforementioned image processing intelligent agent group to obtain decision criteria and typical case information; and storing and processing the aforementioned original cervical cell image based on the aforementioned decision criteria and typical case information. The first step involves fusing the aforementioned multi-scale visual feature vectors and morphological text descriptions to obtain a multimodal query vector. This multimodal query vector represents the fused multi-scale visual feature vectors and morphological text descriptions used for cross-modal retrieval. In practice, the executing entity can use the cross-attention layer of a Transformer to fuse the multi-scale visual feature vectors and morphological text descriptions to obtain the multimodal query vector.

[0039] The second step involves keyword extraction processing of the multimodal query vectors to obtain a set of morphological keywords. These morphological keywords represent terms extracted from the multimodal query vectors that describe the morphological features of cells. The specific content of these morphological keywords is not limited; for example, a morphological keyword could be "irregular nuclear membrane." In practice, the executing entity first constructs a vector index based on a preset terminology set using the Annoy vector index library. The preset terms in this set represent pre-acquired terms describing cell morphology. This preset terminology set may include, but is not limited to, any of the following: irregular nuclear membrane, thickened chromatin, prominent nucleoli. Then, the multimodal query vectors are input into the vector index, retrieving a preset number of index IDs most similar to the multimodal query vectors. These index IDs represent unique numbers assigned to each vector in the Annoy vector index library. The specific value of this preset number is not limited and can be adjusted according to actual needs. Next, each morphological keyword corresponding to the index ID is searched from the preset terminology set. Finally, the obtained morphological keywords are defined as the morphological keyword set.

[0040] The third step involves preliminary extraction processing of the shared memory knowledge base based on the aforementioned multimodal query vector and morphological keyword set, yielding relevant standard provisions. These relevant standard provisions characterize the standards included in the shared memory knowledge base that correspond to the aforementioned multimodal query vector and morphological keyword set. The specific type of these standards is not limited here; for example, a standard could be TBS (The Bethesda system). In practice, firstly, the executing entity can use the IndexFlatIP function of the FAISS library to retrieve a predetermined number of relevant standard provisions from the shared memory knowledge base that are most similar to the aforementioned multimodal query vector. Then, using full-text search technology, a predetermined number of relevant standard provisions matching the aforementioned morphological keyword set are retrieved from the shared memory knowledge base. Finally, a weighted fusion algorithm is used to fuse and deduplicate the predetermined number of relevant standard provisions obtained above, yielding the relevant standard provisions. For example, the weighted fusion algorithm could be a weighted score fusion algorithm.

[0041] The fourth step involves generating an identification query instruction based on the aforementioned set of morphological keywords and the relevant standard provisions. This identification query instruction represents an instruction to extract information from the shared memory knowledge base. In practice, the executing entity can generate the identification query instruction based on the IF-THEN rule, the aforementioned set of morphological keywords, and the relevant standard provisions. For example, the rule could be: "If the set of morphological keywords contains 'nuclear membrane irregularity' and the relevant standard provisions contain 'LSIL,' then generate the identification query instruction: 'Retrieve standards and cases that distinguish LSIL from nuclear membrane integrity with reactive changes.'"

[0042] Fifth, based on the aforementioned identification query command, deep extraction processing is performed on the shared memory knowledge base to obtain a deep retrieval result set. This deep retrieval result set includes positive examples, negative examples, and positive example criteria. Positive examples represent correctly classified cases similar to the original cervical cell images. Negative examples represent incorrectly classified cases similar to the original cervical cell images. Positive example criteria represent standards supporting the positive and negative examples (e.g., the nuclear membrane integrity standard in the TBS system for distinguishing LSIL from reactive changes). In practice, firstly, the executing entity can use a hybrid retrieval algorithm to search the shared memory knowledge base according to the aforementioned identification query command, obtaining positive examples, negative examples, and positive example criteria respectively. Then, the retrieved positive examples, negative examples, and positive example criteria are determined as the deep retrieval result set. Here, the specific type of the hybrid retrieval algorithm is not limited and can be adjusted according to actual needs.

[0043] Step 6: Based on the slide metadata, filter the above-mentioned deep search result set to obtain a refined result set. The slide metadata can characterize data describing the preparation method and preparation time of the target slide. Here, the specific details of the preparation method and preparation time are not limited; for example, the preparation method could be Papanicolaou staining, and the preparation time could be September 2025. The refined result set represents the filtered deep search result set. In practice, the executing entity can delete content in the deep search result set that differs from the slide metadata to obtain the refined result set.

[0044] Step 7: Based on the refined result set, determine the decision criteria and typical case information, and store the original cervical cell images according to these criteria and typical case information. In practice, the implementing entity can use the positive examples from the refined result set as the decision criteria and the positive and negative examples from the refined result set as typical case information. It should be noted that the specific implementation steps for storing the original cervical cell images based on the decision criteria and typical case information can be found in steps 1012-104, and will not be repeated here.

[0045] The above technical solution, as an inventive point of this disclosure, solves technical problem three: "resulting in low classification accuracy, high storage resource consumption, and long time consumption for classifying and storing slide images." The reasons for low classification accuracy, high storage resource consumption, and long time consumption for classifying and storing slide images are as follows: the morphological boundaries of problematic cells are blurred; retrieval based solely on single-modal feature vectors often contains a large amount of noise, leading to poor retrieval accuracy; a large amount of data similar to but inaccurate to the current sample is used as a reference for classifying the target slide image, resulting in low classification accuracy; due to classification errors, a large number of slide images are incorrectly stored, requiring secondary classification and storage, resulting in high storage resource consumption for storing intermediate results and long time consumption for classifying and storing slide images. Solving these factors can improve classification accuracy, reduce storage resource consumption, and shorten the time consumption for classifying and storing slide images. To achieve this effect, the slide image processing method disclosed herein, based on dynamic consensus, constructs a multimodal query vector for retrieval by fusing multi-scale visual feature vectors and morphological text descriptions, thereby reducing the interference of single-modal noise. Furthermore, by constructing identification query instructions based on a set of morphological keywords and relevant standard clauses, it can accurately retrieve positive examples, negative examples, and positive example criteria similar to the current sample from a shared memory knowledge base. In addition, slide metadata is introduced to precisely filter the aforementioned positive examples, negative examples, and positive example criteria, removing unnecessary reference data and determining the final retrieved decision criteria and typical case information. Therefore, it can improve the accuracy of slide image classification, reduce storage resource consumption, and shorten the time spent classifying and storing slide images.

[0046] Step 1013: Generate an image processing result data package based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information.

[0047] In some embodiments, the executing entity can generate an image processing result data package based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information. The image processing result data package may include a decision result, a decision basis, and a confidence distribution. The decision result can characterize the classification result of the decision classification intelligent agent group on the original cervical cell image according to the multimodal instruction package. The decision basis can characterize the standard on which the decision result is obtained. The confidence distribution can characterize the probability vector corresponding to each preset category of the original cervical cell image. In practice, the executing entity can input the multimodal instruction package and the cervical cell retrieval knowledge information into the image processing intelligent agent group to obtain the image processing result data package.

[0048] Optionally, after step 103, the aforementioned executing entity may also perform the following steps: The first step involves constructing output constraint instructions in response to the detection that the classification decision agent in the aforementioned image processing agent group is a closed-source agent. Here, the closed-source agent can represent a classification decision agent that cannot directly read the output layer. The output constraint instructions can represent structured instructions that control the closed-source agent to output results in a preset format. The specific type of the preset format results is not limited here; for example, the preset format results can be in JSON format. In practice, the executing entity can construct output constraint instructions using ReAct hint technology.

[0049] The second step involves generating a closed-source confidence distribution based on the aforementioned closed-source agent and the output constraint instructions. This closed-source confidence distribution characterizes the probability distribution in JSON format obtained after processing by the closed-source agent. In practice, the executing entity can input the output constraint instructions into the closed-source agent to obtain the closed-source confidence distribution.

[0050] The third step involves detecting that the classification decision agent in the aforementioned image processing agent group is an open-source agent, and then obtaining the open-source raw output values. Here, the open-source agent can represent a classification decision agent that can directly read the output layer or internal representation. The open-source raw output values ​​can represent the scores of the original output of the classification decision agent corresponding to the aforementioned preset categories. In practice, the executing entity can use hook technology to extract and process the output layer of the open-source agent to obtain the open-source raw output values.

[0051] The fourth step is to generate an open-source confidence distribution based on the aforementioned raw open-source output values. This open-source confidence distribution characterizes the probability distribution in JSON format obtained after processing by the open-source agent. In practice, the executing agent can use the Softmax function to normalize the raw open-source output values ​​to obtain the open-source confidence distribution.

[0052] The fifth step is to determine the aforementioned closed-source confidence distribution or the aforementioned open-source confidence distribution as the confidence distribution. It should be noted that the aforementioned closed-source confidence distribution or the aforementioned open-source confidence distribution can be empty.

[0053] Step 1014: Generate image processing coefficient information and image processing probability information based on the image processing result data packet and historical weight information.

[0054] In some embodiments, the execution entity can generate image processing coefficient information and image processing probability information based on the image processing result data package and historical weight information. The historical weight information can represent the exponential moving average weights of each classification decision agent in the image processing agent group before processing the current sample. The current sample can represent the sample currently being classified. The sample can represent the original cervical cell image. The image processing coefficient information can represent the Fleiss' Kappa index, which measures the consistency between the various decision results output by the image processing agent group. The image processing probability information can represent the probability distribution of the original cervical cell image belonging to the preset category. As an example, the weighted voting formula can be: .

[0055] Among them, the above This can represent the aforementioned preset categories. The above... The classification results that can characterize the above-mentioned original cervical cell images are as follows: The probability of that. The above. An index that can characterize the above classification decision agent, for example, in response to determining A value of 1 indicates that the classification decision agent is the first classification decision agent. The above... This can represent the sequence number of the current sample. The above... This can represent the sequence number of the sample preceding the current sample. The classification result of the original cervical cell image output by the i-th classification decision agent can be characterized as follows: The probability of that. The above. It can characterize the first Historical weight information of each classification decision agent.

[0056] The first step is to generate image processing probability information based on historical weight information, the aforementioned confidence distribution, and the aforementioned weighted voting formula. In practice, the executing entity can input the aforementioned historical weight information and the aforementioned confidence distribution into the aforementioned weighted voting formula to obtain the image processing probability information.

[0057] The second step is to generate image processing coefficient information based on the above decision results. In practice, the executing entity can use the Fleiss' Kappa method to perform statistical calculations on the decision results to obtain the image processing coefficient information.

[0058] Step 1015: In response to detecting that the image processing coefficient information meets the preset constraints, a first image processing result is generated based on the image processing probability information.

[0059] In some embodiments, in response to detecting that the image processing coefficient information satisfies a preset constraint, the executing entity can generate a first image processing result based on the image processing probability information. The preset constraint can be that the image processing coefficient information is greater than or equal to a preset threshold. The preset threshold can be 0.75. The first image processing result can characterize a preset category corresponding to the highest probability in the image processing probability information. In practice, the executing entity can determine the preset category corresponding to the highest probability in the image processing probability information as the first image processing result.

[0060] Optionally, after step 1015, the aforementioned executing entity may also perform the following steps: The first step involves generating a second image processing result based on the arbitration decision agent, the image processing result data package, and the exponential moving average weights, in response to the detection that the image processing coefficient information does not meet the preset constraints. The arbitration decision agent can represent an independent classification decision agent with a weight higher than the preset weight. The preset weight can be 0.6. The arbitration decision agent can take the image processing result data package and the exponential moving average weights as input and the second image processing result as output. The arbitration decision agent can be trained in batches. It should be noted that the structure of the arbitration decision agent is the same as that of the classification decision agent, and will not be elaborated upon here. The second image processing result can include error cause information and a ruling result. The error cause information can represent the reason why the classification decision agent with a weight lower than the preset weights outputs an incorrect preset category. The ruling result can represent the correct preset category obtained by classifying the original cervical cell image. In practice, the executing entity can input the image processing result data package and the exponential moving average weights into the arbitration decision agent to obtain the second image processing result.

[0061] The second step involves generating updated weights based on the first image processing result, the second image processing result, the ground truth label information, and the weight update function. These updated weights represent the exponentially moving average weights updated by each classification decision agent in the image processing agent group after processing the current sample. In practice, the executing agent can input the first image processing result, the second image processing result, and the ground truth label information into the weight update function to obtain the updated weights. As an example, the weight update function can be: .

[0062] Among them, the above It can characterize the first The classification decision-making agent, after processing the first... Update weights after each sample. (The above) The learning rate can be used to characterize the EMA. (The above...) It can characterize the first Each classification decision-making agent, when processing the current sample, performs... The predicted probability. The above. This can represent the above-mentioned real label information.

[0063] The third step is to store the updated weights in the historical weight information.

[0064] Step 102: Determine the category quantity mapping relationship based on the obtained first image processing results.

[0065] In some embodiments, the execution entity can determine the category quantity mapping relationship based on the obtained first image processing results. This category quantity mapping relationship characterizes the mapping relationship between the categories and quantities of the first image processing results. For example, the category quantity mapping relationship could be "Category 1: LSIL; Quantity: 50; Category 2: ASC-US; Quantity: 100". In practice, firstly, the execution entity can iterate through the first image processing results to obtain all the categories contained in each first image processing result. Then, using an initialized hash table, the total number of categories is accumulated to determine the category quantity mapping relationship.

[0066] Step 103: Based on the category quantity mapping relationship, the preset storage space is dynamically divided to obtain the storage space of the corresponding original cervical cell image.

[0067] In some embodiments, the execution entity can dynamically divide the preset storage space according to the above-mentioned category quantity mapping relationship to obtain the storage space corresponding to the above-mentioned original cervical cell images. The preset storage space can represent logical storage resources used to store different categories of original cervical cell images. Here, the specific type of the above-mentioned logical storage resources is not limited; for example, logical storage resources can be directories with quotas. The above-mentioned storage space can represent the logical storage resources allocated to the above-mentioned original cervical cell images. In practice, the execution entity can use a proportional allocation algorithm to dynamically divide the preset storage space according to the category quantity mapping relationship to obtain the storage space corresponding to the original cervical cell images.

[0068] Step 104: Based on the storage space and the first image processing result, store the original cervical cell image.

[0069] In some embodiments, the execution entity may store the original cervical cell image based on the storage space and the first image processing result. In practice, the execution entity may store the original cervical cell image in the storage space based on the first image processing result.

[0070] The above-described embodiments of this disclosure have the following beneficial effects: A slide image processing method based on dynamic consensus, as described in some embodiments of this disclosure, can improve classification accuracy, reduce storage resource consumption, shorten the overall classification storage time, and reduce computational resource consumption. The reasons for poor classification accuracy, high storage resource consumption, long overall classification storage time, and high computational resource consumption are as follows: Cell morphology is complex, and there is high intra- and inter-class similarity between different cell types. Existing deep learning single models typically use a single architecture for classification, which has poor robustness when dealing with cell types with blurred boundaries. Furthermore, the model is prone to model illusions under uncertain conditions, generating artificially high confidence levels, resulting in low classification accuracy. Due to classification errors, a large number of slide images are incorrectly stored, requiring secondary classification and storage of the slide images, leading to high storage resource consumption for storing intermediate results and long classification time for slide images. In addition, existing multi-model ensembles often use static weights, requiring the parallel operation of multiple large models, resulting in high computational resource consumption. Based on this, some embodiments of the slide image processing method based on dynamic consensus of this disclosure, in response to detecting image processing request information corresponding to each target slide in each target slide, perform the following steps according to the acquired original cervical cell image: First, a multimodal instruction packet corresponding to the above-mentioned image processing request information is generated according to a pre-trained visual language model. Thus, a multimodal instruction packet corresponding to the above-mentioned original cervical cell image can be obtained. Then, cervical cell retrieval knowledge information is generated according to the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base. Thus, retrieval knowledge can be obtained by searching from the shared memory knowledge base and the private memory knowledge base respectively. Next, an image processing result data packet is generated according to the image processing intelligent agent group, the multimodal instruction packet, and the cervical cell retrieval knowledge information. Thus, an image processing result data packet corresponding to the above-mentioned original cervical cell image can be obtained. Next, image processing coefficient information and image processing probability information are generated according to the above-mentioned image processing result data packet and historical weight information. Thus, the probability distribution and classification consistency score corresponding to the above-mentioned image processing result data packet can be obtained. Next, in response to the detection that the image processing coefficient information satisfies the preset constraints, a first image processing result is generated based on the image processing probability information. Thus, a classification result corresponding to the original cervical cell image can be obtained. Then, based on each obtained first image processing result, a category number mapping relationship is determined. Thus, a category number mapping relationship corresponding to each first image processing result can be obtained. Next, based on the above category number mapping relationship, a preset storage space is dynamically divided to obtain a storage space corresponding to the original cervical cell image. Thus, a storage space corresponding to the original cervical cell image can be obtained.Finally, based on the aforementioned storage space and the results of the first image processing, the original cervical cell images are stored. Because classification is not performed using a single architecture, but rather based on a pre-trained visual language model, the original cervical cell images are structurally analyzed to obtain multimodal instruction packages, and classification is then performed based on these multimodal instruction packages. Furthermore, because classification is based on cervical cell retrieval knowledge information obtained from shared and private memory knowledge bases, it can simultaneously integrate authoritative standard knowledge within the domain and the model's own personalized error experience. Moreover, because historical weight information is introduced, dynamic weighting and evaluation of the classification results from multiple models can be achieved. Therefore, classification accuracy can be improved, storage resources consumed can be reduced, the overall classification and storage time can be shortened, and computational resources consumed can be reduced.

[0071] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a slide image processing method based on dynamic consensus. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0072] like Figure 2 As shown, a slide image processing apparatus 200 based on dynamic consensus in some embodiments includes: an execution unit 201, a determination unit 202, a partitioning unit 203, and a storage unit 204. The execution unit 201 is configured to, in response to detecting image processing request information corresponding to each target slide in each target slide, perform the following steps based on the acquired raw cervical cell image: generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing intelligent agent group, a shared memory knowledge base, and a private memory knowledge base; and generating an image processing function based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information. The image processing result data packet is processed; image processing coefficient information and image processing probability information are generated based on the image processing result data packet and historical weight information; in response to the detection that the image processing coefficient information meets the preset constraint conditions, a first image processing result is generated based on the image processing probability information; the determining unit 202 is configured to determine the category quantity mapping relationship based on each of the obtained first image processing results; the partitioning unit 203 is configured to dynamically partition the preset storage space according to the category quantity mapping relationship to obtain the storage space corresponding to the original cervical cell image; the storage unit 204 is configured to perform storage processing on the original cervical cell image based on the storage space and the first image processing result.

[0073] It is understandable that the units described in the slide image processing apparatus 200 based on dynamic consensus are related to the reference... Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the slide image processing apparatus 200 based on dynamic consensus and the units contained therein, and will not be repeated here.

[0074] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0075] like Figure 3 As shown, the electronic device 300 may include a processing unit 301 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0076] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0077] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0078] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0079] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0080] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs. When the aforementioned one or more programs are executed by the electronic device, the electronic device, in response to detecting image processing request information corresponding to each target slide in each target slide, performs the following steps based on the acquired raw cervical cell image: generating a multimodal instruction packet corresponding to the aforementioned image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing intelligent agent group, a shared memory knowledge base, and a private memory knowledge base; generating an image processing result data packet based on the aforementioned image processing intelligent agent group, the aforementioned multimodal instruction packet, and the aforementioned cervical cell retrieval knowledge information; generating image processing coefficient information and image processing probability information based on the aforementioned image processing result data packet and historical weight information; in response to detecting that the aforementioned image processing coefficient information satisfies preset constraints, generating a first image processing result based on the aforementioned image processing probability information; determining a category quantity mapping relationship based on each obtained first image processing result; dynamically dividing a preset storage space based on the aforementioned category quantity mapping relationship to obtain a storage space corresponding to the aforementioned raw cervical cell image; and storing and processing the aforementioned raw cervical cell image based on the aforementioned storage space and the aforementioned first image processing result.

[0081] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0083] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be located in a processor, for example, and can be described as: an execution unit, a determination unit, a partitioning unit, and a storage unit. The names of these units do not necessarily limit the unit itself. For example, an execution unit can be described as a unit that, in response to detecting image processing request information for each target slide in each target slide, performs the following steps based on the acquired raw cervical cell image: generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing agent group, a shared memory knowledge base, and a private memory knowledge base; generating an image processing result data package based on the image processing agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information; generating image processing coefficient information and image processing probability information based on the image processing result data package and historical weight information; and generating a first image processing result based on the image processing probability information in response to detecting that the image processing coefficient information satisfies preset constraints.

[0084] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0085] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A slide image processing method based on dynamic consensus, comprising: In response to the detection of image processing request information for each target slide in each target slide, the following steps are performed based on the acquired raw cervical cell images: Based on a pre-trained visual language model, a multimodal instruction packet corresponding to the image processing request information is generated; Based on the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base, cervical cell retrieval knowledge information is generated. Based on the image processing intelligent agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information, an image processing result data package is generated; Based on the image processing result data packet and historical weight information, image processing coefficient information and image processing probability information are generated; In response to detecting that the image processing coefficient information meets preset constraints, a first image processing result is generated based on the image processing probability information; Based on the processing results of each first image, determine the mapping relationship of the number of categories; Based on the category quantity mapping relationship, the preset storage space is dynamically divided to obtain the storage space corresponding to the original cervical cell image; The original cervical cell image is stored and processed according to the storage space and the first image processing result.

2. The method according to claim 1, wherein, The step of generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model includes: Based on the original cervical cell image and the visual encoder included in the pre-trained visual language model, a multi-scale visual feature vector is generated. Based on the multi-scale visual feature vector and the decoder included in the pre-trained visual language model, a morphological text description and an initial classification probability vector are generated. The multi-scale visual feature vector, the morphological text description, and the initial classification probability vector are structured to obtain a multimodal instruction package.

3. The method according to claim 1, wherein, The process of generating cervical cell retrieval knowledge information based on the image processing intelligent agent group, the shared memory knowledge base, and the private memory knowledge base includes: Based on the image processing intelligent agent group, the shared memory knowledge base is extracted and processed to obtain decision criteria and typical case information; Based on the image processing intelligent agent group, the private memory knowledge base is extracted and processed to obtain reflective memory records; The decision criteria, the typical case information, and the reflective memory records are identified as cervical cell retrieval knowledge information.

4. The method according to claim 1, wherein, The method further includes: In response to detecting that the classification decision agent in the image processing agent group is a closed-source agent, an output constraint instruction is constructed; Based on the closed-source agent and the output constraint instructions, a closed-source confidence distribution is generated; In response to detecting that the classification decision agent in the image processing agent group is an open-source agent, the open-source raw output value is obtained; Based on the original open-source output values, generate the open-source confidence distribution; The closed-source confidence distribution or the open-source confidence distribution is determined as a confidence distribution.

5. The method according to claim 1, wherein, The image processing result data package includes decision results, decision basis, and confidence distribution; as well as The step of generating image processing coefficient information and image processing probability information based on the image processing result data packet and historical weight information includes: Based on the historical weight information, the confidence distribution, and the weighted voting formula, image processing probability information is generated; Based on the decision results, image processing coefficient information is generated.

6. The method according to claim 1, wherein, The method further includes: In response to the detection that the image processing coefficient information does not meet the preset constraint conditions, a second image processing result is generated based on the arbitration decision agent, the image processing result data packet, and the exponential moving average weight. Based on the first image processing result, the second image processing result, the real label information, and the weight update function, update weights are generated; The updated weights are stored in the historical weight information.

7. A slide image processing device based on dynamic consensus, comprising: The execution unit is configured to, in response to detecting image processing request information for each target slide in each target slide, perform the following steps based on the acquired raw cervical cell images: generating a multimodal instruction package corresponding to the image processing request information based on a pre-trained visual language model; generating cervical cell retrieval knowledge information based on an image processing agent group, a shared memory knowledge base, and a private memory knowledge base; and generating an image processing result data package based on the image processing agent group, the multimodal instruction package, and the cervical cell retrieval knowledge information. Based on the image processing result data packet and historical weight information, image processing coefficient information and image processing probability information are generated; In response to detecting that the image processing coefficient information meets preset constraints, a first image processing result is generated based on the image processing probability information; The determining unit is configured to determine the category quantity mapping relationship based on the obtained first image processing results; The partitioning unit is configured to dynamically partition the preset storage space according to the category quantity mapping relationship to obtain the storage space corresponding to the original cervical cell image; The storage unit is configured to store the original cervical cell image based on the storage space and the first image processing result.

8. An electronic device, comprising: One or more processors; A storage device on which one or more programs are stored; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images