Bone marrow cell WSI region-of-interest automatic analysis positioning method and system

Abstract

The invention belongs to the technical field of medical image processing, and provides a bone marrow cell WSI region-of-interest automatic analysis positioning method and system, and the method comprises the steps: obtaining a full glass slide bone marrow aspiration smear image (WSI); a pyramid network is adopted to zoom the full-glass-slide bone marrow aspiration smear image, and a low-resolution image is generated; designing a general area condition prompt strategy by combining bone marrow cytology medical priori knowledge, and calling a language model to construct text description of a bone marrow microparticle area for the low-resolution image; using a visual language model to align the text and the low-resolution image, and then adopting a feature visualization technology and based on gradient weighted channel selection to generate a semantic response graph; the dispersion degree of a semantic response region in the semantic response graph is calculated, and segmentation prompt information is generated by adopting a region perception and strategy self-adaption mechanism; and inputting segmentation prompt information and the low-resolution image into an SAM model, generating an ROI mask image through reasoning, and completing positioning of the bone marrow cell WSI region of interest.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to an automatic analysis and localization method and system for WSI regions of interest in bone marrow cells. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, the rapid development of artificial intelligence has accelerated the research on intelligent diagnosis and treatment and the construction of smart hospitals. Among them, the research and system development of intelligent diagnosis of hematological diseases based on whole-slide bone marrow aspiration smear images (WSI) has become an important component, which is beneficial for the early diagnosis and treatment of malignant hematological diseases such as acute myeloid leukemia and myelodysplastic syndrome.

[0004] In intelligent diagnostic systems for hematological diseases based on whole-slide bone marrow aspiration smear images (WSI), patches are typically extracted directly from the entire WSI and inferences are performed. This method relies on high-quality manual annotation by professional physicians and technicians, a time-consuming and costly process. Furthermore, WSI-based analysis and inference consume significant computational resources; each WSI may generate thousands of patches, leading to high storage and inference costs. Automated analysis and localization of regions of interest (ROIs) in pathological images has become an effective solution to these problems. However, current methods are difficult to apply universally due to variations in pathological knowledge. Therefore, how to quickly and accurately analyze and automatically locate ROIs in whole-slide bone marrow aspiration smear images, maintaining consistency with pathological knowledge and diagnostic procedures, thereby improving diagnostic accuracy and efficiency, has become a key technical challenge to be addressed in the development of intelligent diagnostic systems for hematological diseases. Summary of the Invention

[0005] To address the limitations of existing technologies, this invention provides an automatic analysis and localization method and system for regions of interest (ROIs) in bone marrow cells using the WSI (Wide Spot Injection) model. This addresses the challenges of ROI localization caused by high computational resource consumption and reliance on manual annotation in existing technologies, as well as maintaining consistency between the ROI and medical priors in whole-slide bone marrow aspiration smear images.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides an automated method for analyzing and locating regions of interest (ROIs) in bone marrow cells using Western bone marrow staining (WSI), comprising:

[0008] Obtain images of bone marrow aspiration smears from full glass slides;

[0009] The acquired image is scaled and preprocessed to obtain a low-resolution image;

[0010] A precise text description of the bone marrow microparticle region is constructed based on a general region conditional prompting strategy that combines visual pre-location and text generation.

[0011] Calculate the similarity between text and image features, and generate semantic response maps using feature visualization techniques;

[0012] Perform color space conversion on the semantic response map and create a mask for the red area;

[0013] Extract all semantic response regions, generate cue information from the selected semantic response regions, and convert the semantic response map into point cue or box cue to guide the SAM model for image segmentation.

[0014] As a further technical limitation, the image scaling preprocessing process is as follows: to address the issue of the large size of the whole slide bone marrow aspiration smear image, a pyramid network is used to scale the whole slide bone marrow aspiration smear image to generate a low-resolution image.

[0015] As a further technical limitation, the process of constructing an accurate text description of the bone marrow microparticle region based on the visual pre-location-text generation collaborative prompting strategy is as follows: perform bone marrow microparticle region pre-location on low-resolution images to provide clear visual anchors for a language model that incorporates medical prior knowledge, call the language model to generate an accurate text description of the bone marrow microparticle region in each smear, pointing to the peripheral region of interest of the bone marrow microparticle.

[0016] As a further technical limitation, the process of generating a semantic response map is as follows: input the text description and the low-resolution image into the visual language model, start the semantic alignment module, complete the semantic alignment by calculating the similarity between the text and image features, and then use feature visualization technology to generate a semantic response map based on gradient weighted channel selection.

[0017] As a further technical limitation, the process of generating semantic response region prompts is as follows: color space conversion and masking of the red area are performed on the semantic response map, all semantic response regions are extracted, the corresponding semantic response regions are extracted through the region awareness and policy adaptation mechanism, and finally prompts are generated in the selected semantic response regions.

[0018] This method can automatically convert semantic response maps into point or box prompts, thereby guiding the SAM model to perform image segmentation and generate ROI mask images.

[0019] Secondly, this invention provides an automated analysis and localization system for regions of interest (ROIs) in bone marrow cells using Western blotting (WSI), comprising the following modules:

[0020] The acquisition module is configured to acquire images of bone marrow aspiration smears from a full glass slide.

[0021] The preprocessing module is configured to reduce the dimensionality of the acquired whole-slide bone marrow aspiration smear images;

[0022] The image-text fusion module is configured to perform pre-localization of bone marrow microparticle regions on low-resolution images, clarify the target range of text descriptions, and combine prior knowledge of clinical diagnosis in bone marrow cytology with a language model adapted to medical text to generate precise text descriptions pointing to ROI-related regions for low-resolution images, thus achieving a preliminary fusion of medical prior knowledge and image features.

[0023] The semantic response map generation module is configured to receive a low-resolution image and ROI-related text description output by the image-text fusion module, input both into a feature visualization model based on a visual language model, achieve cross-modal alignment of text semantics and image features through the visual language model, and generate a semantic response map based on gradient-weighted channel selection.

[0024] The prompt information generation module is configured to receive the semantic response map, call the prompt generation sub-model based on the features of the semantic response map, calculate the red region dispersion index, dynamically select the prompt generation method through the policy adaptive mechanism, and finally generate segmented prompt information.

[0025] The region of interest localization module is configured to receive the combined prompt information output by the prompt information generation module and the low-resolution image output by the image-text fusion module, and input them into the prompt encoder and image encoder of the medically adapted SAM model, respectively; through the segmentation inference of the SAM model, a pixel-level ROI mask map of the whole slide bone marrow aspiration smear image is generated.

[0026] The third aspect of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0027] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the bone marrow cell smear region of interest localization method as described in the first aspect of the present invention.

[0028] The fourth aspect of the present invention provides an electronic device, which adopts the following technical solution:

[0029] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the bone marrow cell smear region of interest localization method as described in the first aspect of the present invention.

[0030] The fifth aspect of this invention provides a computer program product, which adopts the following technical solution:

[0031] A computer program product includes software code, wherein the program in the software code performs the steps in the bone marrow cell smear region of interest localization method as described in the first aspect of the present invention.

[0032] One or more technical solutions of the present invention have the following beneficial effects:

[0033] 1. This invention utilizes clinical experience in bone marrow cytology to design a text prompting strategy, embedding prior medical information such as the periphery of the bone marrow microparticle region under low magnification, the uniform distribution of nucleated cells, and the intact morphology into a multimodal framework. This guides the feature visualization model to generate a semantic response map, improving the accuracy and medical relevance of ROI localization.

[0034] 2. This invention proposes an adaptive region-aware prompting method based on the spatial distribution characteristics of semantic response regions in semantic response maps. This method can dynamically generate prompting points and guide the SAM model to complete high-precision segmentation, effectively handling complex situations with varying region sizes and dispersion, and improving segmentation quality and computational efficiency.

[0035] 3. This invention constructs an automated workflow that eliminates the need for high-quality manual annotation. It can efficiently process large-size bone marrow smear images, achieve high-quality ROI localization, provide reliable input for subsequent cell detection and classification, and significantly reduce computational overhead. Attached Figure Description

[0036] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0037] Figure 1 This is a flowchart illustrating an automatic analysis and localization method for WSI regions of interest in bone marrow cells according to Embodiment 1 of the present invention.

[0038] Figure 2 This is a schematic diagram of a method for automatic analysis and localization of regions of interest (WSI) in bone marrow cells according to Embodiment 1 of the present invention.

[0039] Figure 3 This is a schematic diagram of the prompt information generation method based on semantic response graph features in Embodiment 1 of the present invention. Detailed Implementation

[0040] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] Example 1

[0042] An embodiment of the present invention provides a method for locating regions of interest in bone marrow cell smears, such as... Figure 2 As shown, the specific method includes the following steps:

[0043] S1: Read image

[0044] Obtain images of bone marrow aspiration smears from full glass slides;

[0045] S2: Image scaling processing

[0046] The input image is a whole-slide bone marrow aspiration smear image obtained from scanning (approximately 170,000 × 88,000 pixels). A pyramid network is used to scale it down to obtain a low-resolution image (approximately 2,700 × 1,400 pixels).

[0047] S3: Generate text prompt message

[0048] A general region-based conditional suggestion strategy, designed incorporating prior medical knowledge, is employed. The process is as follows: First, low-resolution images generated by a pyramid network are input into a finely tuned YOLO object detection model. This model automatically detects and locates bone marrow microparticle regions within the low-resolution images and outputs bounding boxes (marked in red), providing clear region anchors for subsequent text suggestions. Then, a fixed text instruction, "Please describe the area in the redbox in a shorter sentence," guides the language model to generate a text description of the bone marrow microparticle region in each low-resolution smear image. This text description accurately points to the bone marrow microparticle region, and the area surrounding this region is typically a Region of Interest (ROI), providing a reliable medical association for subsequent semantic alignment and ROI localization.

[0049] S4: Generate semantic response graph

[0050] The gScoreCAM method, based on a visual language model, aligns the text description generated by S3 with the low-resolution image generated by S2, visualizes the features, and ultimately generates a semantic response map. This method calculates the gradient magnitudes of each channel in the image encoder, selects the k channels most sensitive to the target category (usually k=300), and then generates a mask image based on these channels. The specific formula for generating the mask image is as follows:

[0051]

[0052] in, Input image; , No. The activation map of each channel is upsampled to the size of the input image; Element-wise multiplication (Hadamard product); The generated first A mask image.

[0053] The similarity between the visual language model and the text prompt is calculated using the following formula:

[0054]

[0055] in, The forward propagation function of the model; , No. A similarity score between a mask image and a text prompt.

[0056] Finally, these scores are used as weights to weight and combine the channel activation maps using the following formula to generate a high-resolution heatmap.

[0057]

[0058] in, The number of channels selected; , No. Similarity scores of masked images The similarity scores are normalized to obtain the weights. , No. Upsampling activation map of each channel; Semantic response graph.

[0059] The background of the semantic response map is mainly blue-purple, interspersed with yellow, orange, and red areas. Visually, color reflects the importance of a region; that is, the higher the semantic response level corresponding to a color, the stronger the correlation between the region and the ROI. Among them, red areas usually represent regions of interest in the image.

[0060] S5: Generate segmentation prompt information

[0061] Based on the features of the semantic response graph after S4 processing, this invention proposes a method for generating prompt information based on the features of the semantic response graph, such as... Figure 2 As shown, the specific method includes the following steps:

[0062] Image preprocessing

[0063] After obtaining the semantic response map, perform color space conversion on it, create binary masks for the two red ranges respectively, and then merge them into a complete mask;

[0064] Semantic response region extraction

[0065] Extract the outline of the red high-response region from the mask image and define it as the semantic response region. After sorting by area, take the largest outline as the main focus region and calculate its bounding box and center point coordinates.

[0066] Generate prompt information

[0067] Extract the number of semantic response regions, average area ratio, average compactness, and maximum distance between primary and secondary regions to construct the red region dispersion index RRDM ( Where K is the number of regions. For the first Area percentage of each region; For the compactness of this area; This is the Euclidean distance between the largest secondary region and the primary region. This is a redundant item for the region. , , , (for hyperparameter weights); when RRDM > preset threshold When using the AAPP strategy (allocating cue points according to area ratio), when RRDM ≤ The SACP strategy (focusing on the largest region and allocating cue points to adjacent regions after expansion) is adopted, with a preset threshold. The reference range for the value was determined through a large number of bone marrow cell smear samples, which is [0.3, 0.5] (the specific value can be fine-tuned according to the actual sample data). The maximum region bounding box was saved simultaneously, and finally, information including point coordinates and bounding box prompts was generated in JSON format and input into the SAM model.

[0068] S6: Segmenting the ROI image

[0069] The SAM model is used to segment the final region of interest (ROI) mask. The low-resolution image generated in S2 and the prompt information generated in S5 are embedded into the image encoder and prompt encoder of the SAM model. The generated prompt information guides the SAM model to find the ROI in the low-resolution image of the whole slide bone marrow aspiration smear, and finally accurately segment the ROI mask.

[0070] Example 2

[0071] This embodiment provides an automated analysis and localization system for regions of interest (ROIs) in bone marrow cells using Western blotting (WSI), comprising the following modules:

[0072] The acquisition module is configured to acquire images of bone marrow aspiration smears from a full glass slide.

[0073] The preprocessing module is configured to reduce the dimensionality of the acquired whole-slide bone marrow aspiration smear images;

[0074] The image-text fusion module is configured to perform pre-localization of bone marrow microparticle regions on low-resolution images, clarify the target range of text descriptions, and combine prior knowledge of clinical diagnosis in bone marrow cytology to call a language model adapted to medical text to generate precise text descriptions pointing to ROI-related regions for low-resolution images, thus achieving the initial fusion of medical prior knowledge and image features.

[0075] The semantic response map generation module is configured to receive the low-resolution image and ROI-related text description output by the image-text fusion module, input both into the feature visualization model based on the visual language model, realize cross-modal alignment of text semantics and image features through the visual language model, and generate a semantic response map based on gradient-weighted channel selection.

[0076] The prompt information generation module is configured to receive the semantic response map, call the prompt generation sub-model based on the features of the semantic response map, calculate the red region dispersion index, dynamically select the prompt generation method through the policy adaptive mechanism, and finally generate segmented prompt information.

[0077] The region of interest localization module is configured to receive the combined prompt information output by the prompt information generation module and the low-resolution image output by the image-text fusion module, and input them into the prompt encoder and image encoder of the medically adapted SAM model, respectively; through the segmentation inference of the SAM model, a pixel-level ROI mask map of the whole slide bone marrow aspiration smear is generated.

[0078] Example 3

[0079] The purpose of this embodiment is to provide a computer-readable storage medium for storing computer programs to perform the method described in Embodiment 1.

[0080] The method in Embodiment 1 can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor. The software modules can reside in readily available storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not provided here.

[0081] Example 4

[0082] The purpose of this embodiment is to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, used to perform the method described in Embodiment 1. For the sake of brevity, further details are omitted here.

[0083] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0084] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0085] Various modifications and variations of this invention will be apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for automated analysis and localization of regions of interest (ROIs) in bone marrow cells using Western blotting (WSI), characterized in that... include: Obtain images of bone marrow aspiration smears from full glass slides; A pyramid network is used to scale the acquired images to generate low-resolution images; Pre-location of bone marrow microparticle regions is performed on low-resolution images. A general region condition prompting strategy is designed based on prior medical knowledge of bone marrow cytology. A language model is invoked to generate text descriptions of bone marrow microparticle regions for the low-resolution images. By leveraging the semantic alignment capability of visual language models, the text description is aligned with a low-resolution image, and a semantic response map is generated using feature visualization techniques. The process of generating prompt information based on the semantic response map features includes color space conversion and red area mask creation of the semantic response map, extraction of semantic response areas, and determination of prompt generation method through area awareness and policy adaptation mechanism. The prompt information and low-resolution image are input into the SAM model. Through the segmentation inference of the SAM model, a ROI mask image of the whole slide bone marrow aspiration smear is generated to complete the region of interest localization.

2. The method for automatic analysis and localization of regions of interest in bone marrow cells using WSI as described in claim 1, characterized in that, The process of scaling the acquired whole-slide bone marrow aspiration smear image using a pyramid network is as follows: In view of the large size of the whole-slide bone marrow aspiration smear image, the pyramid dimensionality reduction network is called based on the principle of multi-scale feature preservation to scale the high-resolution whole-slide bone marrow aspiration smear image to a low-resolution image, thereby reducing computational overhead while preserving the overall morphological features of bone marrow particles.

3. The method for automatic analysis and localization of regions of interest in bone marrow cells using WSI as described in claim 1, characterized in that, The process of designing a general region condition prompting strategy based on prior medical knowledge of bone marrow cytology and calling a language model to generate text descriptions is as follows: Using the medical prior of "uniform cell distribution around the bone marrow microparticle region, clear cytological details, and few dissolved or fragmented cells" as the core, pre-location of the bone marrow microparticle region is performed on low-resolution images. A fixed text prompt template, "Please describe the area in the red box in a shorter sentence.", guides the language model to generate a precise text description pointing to the bone marrow microparticle region for each low-resolution image. This text description needs to be associated with the medical feature attributes of the ROI.

4. The method for automatic analysis and localization of regions of interest in bone marrow cells using WSI as described in claim 1, characterized in that, The process of generating semantic response maps using a visual language model is as follows: Feature visualization is achieved using the gScoreCAM method based on a visual language model. First, the gradient of each channel in the image encoder is calculated, and the gradient of the channel most sensitive to the target category is selected. One channel ( (usually taken as 300), through the formula Generate the first Mask image of each channel (where For low-resolution images, For the first Activation map after upsampling of each channel; (This is element-wise multiplication); then using the formula Calculate the similarity score between the masked image and the text description. (For the model's forward propagation function); finally, through the formula Generate semantic response graph ( In the semantic response graph, the red areas correspond to the ROI high correlation areas, and the blue-purple areas are the background.

5. The method for automatic analysis and localization of regions of interest in bone marrow cells using WSI as described in claim 1, characterized in that, The process of generating prompt information based on semantic response map features is as follows: The first step is to perform color space conversion on the semantic response map, create binary masks for the two red regions respectively, and merge them into a complete red high-response region mask; The second step is to extract the semantic response region contour based on the mask, sort the contours by area and take the largest contour as the main region of interest, and calculate its bounding box and center point coordinates. The third step is to construct the red region dispersion index RRDM: ,(in For the number of regions, For the first Area percentage of each region For the compactness of this area, The Euclidean distance between the largest secondary region and the primary region. This is a redundant item for the region. , , , (for hyperparameter weights); when RRDM > preset threshold When using the AAPP strategy (allocating cue points according to area ratio), when RRDM ≤ The SACP strategy (focusing on the largest region and assigning cue points to adjacent regions after expansion) is adopted, and the main region bounding box is saved simultaneously, ultimately generating a combined cue information of "point cue coordinates + main region bounding box cue".

6. The method for automatic analysis and localization of regions of interest in bone marrow cells using WSI as described in claim 1, characterized in that, The process of inputting the prompt information and low-resolution image into the SAM model is as follows: First, a medically adapted SAM model is selected. Then, the low-resolution image is input into the image encoder of the model, and the combined prompt information is input into the prompt encoder of the model. The model generates a pixel-level ROI mask map through internal segmentation and reasoning of the input information. The region corresponding to the mask map is a clinically effective ROI that conforms to medical priors (i.e., a region with uniform cell distribution, clear cytological details, and few dissolved or broken cells).

7. An automated analysis and localization system for regions of interest (ROIs) in bone marrow cells using Western blotting (WSI), characterized in that, include: The acquisition module is configured to acquire images of bone marrow aspiration smears from a full glass slide. The preprocessing module is configured to reduce the dimensionality of the acquired whole-slide bone marrow aspiration smear images; The image-text fusion module is configured to perform pre-localization of bone marrow microparticle regions on low-resolution images, clarify the target range of text descriptions, and combine prior knowledge of clinical diagnosis in bone marrow cytology to call a language model adapted to medical text to generate precise text descriptions pointing to ROI-related regions for low-resolution images, thus achieving the initial fusion of medical prior knowledge and image features. The semantic response map generation module is configured to receive the low-resolution image and ROI-related text description output by the image-text fusion module, and input them into the feature visualization model based on the visual language model. Cross-modal alignment of text semantics and image features is achieved through a visual language model, and semantic response maps are generated based on gradient-weighted channel selection. The prompt information generation module is configured to receive the semantic response map, call the prompt generation sub-model based on the features of the semantic response map, calculate the red region dispersion index, dynamically select the prompt generation method through the policy adaptive mechanism, and finally generate segmented prompt information. The region of interest localization module is configured to receive the combined prompt information output by the prompt information generation module and the low-resolution image output by the image-text fusion module, and input them into the prompttencoder and image encoder of the medically adapted SAM model, respectively; through the segmentation inference of the SAM model, a pixel-level ROI mask map of the whole slide bone marrow aspiration smear image is generated.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps of a method for locating regions of interest in bone marrow cell smears as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the automatic analysis and localization method for WSI regions of interest in bone marrow cells as described in any one of claims 1-6.

10. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of an automatic analysis and localization method for WSI regions of interest in bone marrow cells as described in any one of claims 1-6.

Images