An information processing method, device and equipment for medical tissue morphological image analysis and a medium

Abstract

The application discloses a medical tissue morphology image analysis information processing method and device, equipment and medium, and relates to the technical field of information processing. The method comprises the following steps: obtaining a target medical image to be analyzed, extracting an effective tissue region from the target medical image, and dividing the effective tissue region into a plurality of image instance blocks; inputting each image instance block into a pre-trained tissue feature extraction model, outputting a tissue morphology feature vector corresponding to each image instance block, performing attention scoring on each tissue morphology feature vector through an instance detection module of a multi-instance learning network, and selecting a target image instance block containing a pathological feature region according to the attention scoring; and performing aggregation processing on a target tissue morphology feature vector corresponding to the target image instance block, generating and outputting an aggregated global feature representation. The method automatically identifies a key region in a medical image and generates a visual global feature representation.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to an information processing method, apparatus, equipment and medium for medical tissue morphology image analysis. Background Technology

[0002] Medical images (such as whole pathological slides and endoscopic images) often contain a lot of background noise, and the effective diagnostic area only accounts for a small part of the image. Traditional image analysis methods lack the ability to automatically and accurately extract the effective tissue area, resulting in a heavy computational burden and a lot of interference information in subsequent analysis steps.

[0003] Existing medical image feature extraction methods mostly employ models based on convolutional neural networks (CNNs). These models are limited by their local receptive fields, making it difficult to simultaneously capture both local details and global spatial topology. Especially for medical image analysis tasks requiring the integration of multi-scale information such as cell morphology and tissue arrangement, the feature representation capabilities of CNN models are limited, resulting in restricted analysis accuracy. Furthermore, in medical image analysis, key regions containing atypical cells or lesions are often scattered and morphologically variable. Traditional multi-instance learning methods use coarse aggregation strategies such as mean pooling or max pooling, which cannot effectively distinguish between key and redundant instances, easily losing crucial information and lacking interpretability of the analysis results. Moreover, existing medical image analysis methods typically require directly inputting the entire high-resolution image into the model, resulting in high computational cost, slow inference speed, and difficulty in deployment on resource-constrained clinical edge devices.

[0004] In summary, how to automatically identify key regions in medical images and generate efficient medical images with visualized global feature representations is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an information processing method, apparatus, device, and medium for medical tissue morphology image analysis, capable of automatically identifying key regions in medical images and generating efficient medical images with visualized global feature representations. The specific solution is as follows: In a first aspect, this application discloses an information processing method for medical tissue morphology image analysis, applied to a computer device, comprising: A target medical image to be analyzed is acquired, and an effective tissue region is extracted from the target medical image. Then, the effective tissue region is divided into multiple image instance blocks; wherein, the effective tissue region is an image region containing cells and / or tissues. Each of the aforementioned image instance blocks is input into a pre-trained tissue feature extraction model to output a tissue morphological feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; Each of the tissue morphology feature vectors is input into a multi-instance learning network, and the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, so as to filter out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores. The multi-instance learning network aggregates the target tissue morphological feature vectors corresponding to the target image instance blocks to generate and output the aggregated global feature representation that characterizes the tissue morphological distribution information in the target medical image.

[0006] Optionally, the step of extracting the effective tissue region from the target medical image and then dividing the effective tissue region into multiple image instance blocks includes: A preset image segmentation algorithm is used to separate background information from effective tissue in the target medical image, so as to retain the image region containing cells and / or tissue as the effective tissue region; wherein, the background information includes blank areas of the slide and air bubbles in the image; The effective tissue region is segmented into multiple image instance blocks according to a preset size using a non-overlapping segmentation process.

[0007] Optionally, before inputting each of the image instance blocks into the pre-trained tissue feature extraction model, the method further includes: Obtain an unlabeled medical image sample set; Each sample image in the medical image sample set is divided into multiple training instance blocks; Generate different views for each of the training instance blocks to obtain multiple different views corresponding to each training instance block; Different views of the same training instance block are input to the teacher network and student network of the DINOv2 network until the first feature similarity between the first output feature output by the student network and the second output feature output by the teacher network satisfies the first similarity condition. Different views of each training instance block are input to the student network until the second feature similarity between the output features output by the student network satisfies the second similarity condition. The ViT encoder of the student network of the DINOv2 network that meets the preset training termination conditions is used as a pre-trained organizational feature extraction model.

[0008] Optionally, the step of inputting each of the image instance blocks into a pre-trained tissue feature extraction model to output the tissue morphological feature vector corresponding to each image instance block includes: Each of the image instance blocks is input into a pre-trained tissue feature extraction model to linearly project each of the image instance blocks into a corresponding embedding vector, and position encoding is added to the embedding vector to obtain an embedding vector sequence carrying relative position information. The multi-head self-attention mechanism of the pre-trained tissue feature extraction model is used to perform global feature interaction on the embedded vector sequence carrying relative position information to generate a fused feature sequence containing feature vectors of each image patch and classification label feature vectors. The classification label feature vector of the fused feature sequence is then extracted as the tissue morphology feature vector of the image instance patch.

[0009] Optionally, before inputting each of the tissue morphological feature vectors into the multi-instance learning network, the method further includes: A medical image sample set is obtained, and each sample image is labeled with slice-level labels and region-level labels; wherein, the slice-level labels are category labels indicating medical image samples, and the region-level labels are coordinate location labels indicating the regions containing pathological features in medical image samples; Extract the effective organization region of each sample image and divide the effective organization region of the sample into multiple training instance blocks; Each training instance block is input into the pre-trained tissue feature extraction model to obtain the training tissue morphological feature vector corresponding to each training instance block; All training tissue morphological feature vectors corresponding to the same sample image are input into the initial multi-instance learning network. The instance detection module of the initial multi-instance learning network classifies and scores the morphological feature vectors of each training tissue. The target training instance block representing the target pathological feature region is selected from all training instance blocks. The target training image block is constructed based on the target training instance block and the region-level label. The attention scores corresponding to the target training image patches are weighted by the super-patch aggregation module of the initial multi-instance learning network to generate aggregated feature representations. The aggregated feature representation is input into the classification head to output the predicted category; The loss function is calculated based on the difference between the predicted category and the slice-level label, and the network parameters of the initial multi-instance learning network are updated by backpropagation based on the calculation result until the convergence condition is met to obtain the multi-instance learning network.

[0010] Optionally, the information processing method for medical tissue morphology image analysis further includes: Obtain clinical feedback data; wherein, the clinical feedback data includes review and correction information on the global feature representation and / or newly added unlabeled medical images; Based on the audit correction information, the multi-instance learning network is incrementally fine-tuned to update the network parameters of the multi-instance learning network. And / or, based on the newly added unlabeled medical images, perform self-supervised incremental training on the pre-trained tissue feature extraction model to update the model parameters of the pre-trained tissue feature extraction model.

[0011] Optionally, the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphological feature vectors to filter out target image instance blocks containing pathological feature regions from all the image instance blocks based on the attention scores, including: Attention scores are assigned to each tissue morphology feature vector through the gating attention mechanism of the instance detection module of the multi-instance learning network to obtain the original attention scores corresponding to each image instance block. The original attention scores are normalized using the softmax function to obtain the normalized attention scores for each image instance block. The normalized attention scores are sorted in descending order, and a preset number of target normalized attention scores are selected from the score sorting table in a forward-to-back order. The target image instance blocks corresponding to the target normalized attention scores are then selected.

[0012] Secondly, this application discloses an information processing device for medical tissue morphology image analysis, comprising: Image segmentation processing is used to acquire a target medical image to be analyzed, extract effective tissue regions from the target medical image, and then segment the effective tissue regions into multiple image instance blocks; wherein, the effective tissue regions are image regions containing cells and / or tissues; The tissue feature extraction module is used to input each of the image instance blocks into a pre-trained tissue feature extraction model to output the tissue morphological feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; The instance block filtering module is used to input each of the tissue morphology feature vectors into a multi-instance learning network, so that the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, and filters out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores. The result output module is used to aggregate the target tissue morphological feature vectors corresponding to the target image instance blocks through the multi-instance learning network, so as to generate and output the aggregated global feature representation for characterizing the tissue morphological distribution information in the target medical image.

[0013] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the steps of the aforementioned disclosed information processing method for medical tissue morphology image analysis.

[0014] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned disclosed information processing method for medical tissue morphology image analysis.

[0015] As can be seen, this application discloses a method for acquiring a target medical image to be analyzed, extracting effective tissue regions from the target medical image, and then segmenting the effective tissue regions into multiple image instance blocks; wherein, the effective tissue regions are image regions containing cells and / or tissues; each image instance block is input into a pre-trained tissue feature extraction model to output a tissue morphology feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; each tissue morphology feature vector is input into a multi-instance learning network to perform attention scoring on each tissue morphology feature vector through the instance detection module of the multi-instance learning network, so as to filter out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores; and the multi-instance learning network aggregates the target tissue morphology feature vectors corresponding to the target image instance blocks to generate and output the aggregated global feature representation for characterizing the tissue morphology distribution information in the target medical image. Therefore, by extracting effective tissue regions containing cells and / or tissues and segmenting them into multiple image instance blocks, background interference such as blank areas and bubbles on the slide is automatically removed, reducing subsequent computational redundancy. A pre-trained tissue feature extraction model based on the DINOv2 network outputs tissue morphological feature vectors corresponding to each instance block. An instance detection module of a multi-instance learning network generates attention scores based on a gated attention mechanism. Simultaneously, superpixel block sampling and convolution operations are used to expand the local receptive field, thereby capturing both local cell morphological details and global tissue spatial topology under weak supervision, overcoming the limitations of traditional CNN models. This approach overcomes the limitations of restricted receptive field and loss of cross-scale information. By implicitly filtering target image instance blocks containing pathological feature regions through attention scores, it abandons the coarse aggregation methods of traditional mean pooling or max pooling, and achieves automatic identification of key analysis regions. Finally, by using dot product aggregation, the attention scores and the classifier's output category logits are combined to dynamically allocate the contribution of instance blocks, generating an aggregated global feature representation. This approach retains both local key information and global context, while also preserving instance-level attention weights for spatial heatmap generation. Ultimately, it outputs a global feature representation that represents the morphological distribution information of tissues in medical images. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1This application discloses a flowchart of an information processing method for medical tissue morphology image analysis. Figure 2 This is a flowchart of the training iteration of a DINOv2 network and a multi-instance learning network for pathological image analysis disclosed in this application. Figure 3 This is a schematic diagram of the information processing device for medical tissue morphology image analysis disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] Medical images (such as whole pathological slides and endoscopic images) often contain a lot of background noise, and the effective diagnostic area only accounts for a small part of the image. Traditional image analysis methods lack the ability to automatically and accurately extract the effective tissue area, resulting in a heavy computational burden and a lot of interference information in subsequent analysis steps.

[0020] Existing medical image feature extraction methods mostly employ convolutional neural network (CNN) models. These models are limited by their local receptive field, making it difficult to simultaneously capture both local details and global spatial topology. Especially for medical image analysis tasks requiring the integration of multi-scale information such as cell morphology and tissue arrangement, the feature representation capabilities of CNN models are limited, resulting in restricted analysis accuracy. Furthermore, in medical image analysis, atypical cells and / or key pathological regions are often scattered and morphologically variable. Traditional multi-instance learning methods use coarse aggregation strategies such as mean pooling or max pooling, which cannot effectively distinguish between key and redundant instances, easily losing crucial information and lacking interpretability of the analysis results. Moreover, existing medical image analysis methods typically require directly inputting the entire high-resolution image into the model, resulting in high computational cost, slow inference speed, and difficulty in deployment on resource-constrained clinical edge devices.

[0021] Therefore, this invention provides an information processing scheme for medical tissue morphology image analysis, which can automatically identify key regions in medical images and generate efficient medical images with visualized global feature representations.

[0022] like Figure 1 As shown, this invention provides an information processing method for medical tissue morphology image analysis, applied to a computer device, comprising: Step S11: Obtain the target medical image to be analyzed, extract the effective tissue region from the target medical image, and then divide the effective tissue region into multiple image instance blocks; wherein, the effective tissue region is an image region containing cells and / or tissues.

[0023] In this embodiment, the target medical image to be analyzed is obtained through an image acquisition device or from a preset database. The target medical image can be a digital pathology whole slide image, an endoscopic image, an ultrasound image, or a computed tomography image, etc., without specific limitation. In the specific implementation scenario of this invention, the target medical image is a digital pathology whole slide image, for example, a colorectal H&E stained whole slide image. The image format includes, but is not limited to, common medical image formats such as TIFF, SVS, NDPI, and DICOM.

[0024] In this embodiment, a preset image segmentation algorithm is used to separate background information from effective tissue in the target medical image, retaining image regions containing cells and / or tissue as effective tissue regions. The background information includes blank areas on the slide and air bubbles in the image. The effective tissue regions are then segmented non-overlapping according to a preset size to obtain multiple segmented image instance blocks. It is understood that after acquiring the target medical image to be analyzed, the image is preprocessed, including the following steps: First, the preset image segmentation algorithm can be any one of threshold-based segmentation, edge-based segmentation, region-based segmentation, or deep learning-based semantic segmentation algorithms, without specific limitations. The image segmentation algorithm automatically identifies and removes invalid background information such as blank areas on the slide, stains, and air bubbles from the image, retaining only image regions containing cells and / or tissue as effective tissue regions.

[0025] For the analysis scenario of digital pathology whole slide images, a foreground-background separation method is proposed. Through color space conversion and multi-resolution analysis, the background and effective tissue are accurately separated, and blank areas and impurities are effectively removed.

[0026] Specifically, acquire a digital pathology whole-slice image in SVS or NDPI format, stained with H&E (nuclei appear blue-purple, cytoplasm appears pink), and magnified 40 times at a resolution. First, read the original color image and downsample it to generate a low-resolution thumbnail (e.g., magnified 5 times) to improve processing efficiency.

[0027] Because the effective tissue regions (cells, glands, stroma) in H&E staining images have high color saturation, while the background (blank areas on the slide) has extremely low saturation, this embodiment first converts the RGB (Red Green Blue) image to the HSV (Hue Saturation Value) color space and extracts its saturation channels. The maximum inter-class difference algorithm is then used to perform thresholding segmentation on this saturation channel image to obtain an initial binary mask.

[0028] Morphological optimization of the initial mask is performed: first, an opening operation is performed using a circular structuring element with a radius of R1 pixels to remove isolated noise points; then, a closing operation is performed using a circular structuring element with a radius of R2 pixels to fill small pores inside the tissue and connect adjacent fragments. Subsequently, connected regions in the binary mask are marked, the area of ​​each connected region is calculated, regions with an area smaller than a preset threshold are removed, and significant tissue regions are retained.

[0029] The tissue region contours selected at low resolution are mapped back to the corresponding coordinate space of the original high-resolution image. Within this range, the final effective tissue region image patch containing cells and / or tissues is extracted. This region image is kept in color for subsequent analysis.

[0030] In this embodiment, after extracting the effective tissue region, the effective tissue region is segmented into multiple image instance blocks of the same size. Specifically, the effective tissue region is segmented using a non-overlapping sliding window according to a preset fixed size. Each segmented image block is an independent image instance block with a preset size of 256×256 pixels or 512×512 pixels, without specific limitations. Taking 256×256 pixels as an example, starting from the upper left corner of the effective tissue region, the window slides sequentially along the horizontal and vertical directions with a step size of 256 pixels, cropping the effective tissue region into several non-overlapping 256×256 pixel image blocks. If the boundary size of the effective tissue region is not an integer multiple of 256, the boundary region with insufficient size is either filled with edges (e.g., zero padding or mirror filling) or directly discarded. Each image instance block corresponds to a local region in the original medical image and contains the cell or tissue morphology information of that local region.

[0031] In this way, through the above segmentation process, a high-resolution medical image is converted into a sequence of image instance blocks of a controllable number, which reduces the amount of data input to the model in a single run while preserving the integrity of the local structure.

[0032] Step S12: Input each of the image instance blocks into the pre-trained tissue feature extraction model to output the tissue morphology feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network.

[0033] In this embodiment, before inputting each of the image instance blocks into the pre-trained tissue feature extraction model, the method further includes: acquiring an unlabeled medical image sample set; dividing each sample image in the medical image sample set into multiple training instance blocks; generating different views for each training instance block to obtain multiple different views corresponding to each training instance block; inputting different views of the same training instance block into the teacher network and student network of the DINOv2 network until the first feature similarity between the first output feature output by the student network and the second output feature output by the teacher network satisfies the first similarity condition, and inputting different views of each training instance block into the student network until the second feature similarity between the output features output by the student network satisfies the second similarity condition; and using the ViT encoder of the student network of the DINOv2 network that meets the preset training termination conditions as the pre-trained tissue feature extraction model. It can be understood that the training process of the pre-trained tissue feature extraction model is entirely based on unlabeled data, using the DINOv2 (Distillation with NO labels version 2) self-supervised learning framework, without any manual labeling. The specific training steps are as follows: First, obtain an unlabeled medical image sample set. Specifically, collect a large number of unlabeled medical image samples, such as historical pathological whole slide images. The medical image samples can come from different cancer types (colorectal cancer, lung cancer, gastric cancer, breast cancer, etc.), different staining methods (H&E staining, immunohistochemical staining, etc.), and different magnifications. All samples do not require any manual labeling, including no slide-level labeling and no pixel-level labeling. The number of samples can be determined according to the actual situation, and there is no specific limit to this.

[0034] Secondly, each sample image in the medical image sample set is divided into multiple training instance blocks. Specifically, for each medical image sample, the effective tissue region is extracted according to the aforementioned tissue region extraction method of the target medical image, and the effective tissue region is divided into multiple non-overlapping training instance blocks according to a preset size. Each training instance block is used as an independent training sample, and the original image is converted into a training instance block sequence with uniform size.

[0035] Secondly, different views of each training instance block are generated. For each training instance block, multiple different views of the instance block are generated through random data augmentation techniques. Data augmentation methods include, but are not limited to, random cropping, random rotation, random horizontal flipping, random vertical flipping, Gaussian blur, random affine transformation, random color perturbation, and simulated coloring differences. Each training instance block is subjected to different combinations of random augmentations to generate at least two different views (e.g., global view and local view). These different views constitute positive sample pairs in self-supervised contrastive learning.

[0036] Then, a DINOv2 network is constructed, including a teacher network and a student network. Specifically, a DINOv2 network based on the Vision Transformer (ViT) architecture is initialized. This network contains two sub-networks with the same structure but different parameter update methods. One is the student network, whose parameters are updated through gradient descent and backpropagation, and is used to learn feature representations. The other is the teacher network, whose parameters are updated using the exponential moving average of the student network parameters. Centering and sharpening operations are introduced to stabilize training. The teacher network does not participate in gradient backpropagation and is used to generate stable target features. Both networks contain a ViT encoder and a projection head. The ViT encoder is used to extract image features, and the projection head is used to map the features to the embedding space for contrastive learning.

[0037] Then, the first feature similarity is trained to ensure consistency between different views (e.g., global view and local view) of the same training instance block. Specifically, two different views of the same training instance block are input into the student network and the teacher network respectively: the global view is input into the student network to obtain the first output feature; the local view is input into the teacher network to obtain the second output feature. Then, the first feature similarity between the first and second output features is calculated using cross-entropy loss or cosine similarity loss. The training objective is to maximize this similarity, that is, to make the output of the student network for the global view as close as possible to the output of the teacher network for the local view. Then, the parameters of the student network are updated through backpropagation, and the parameters of the teacher network are updated along with the student network parameters using EMA. This process is repeated until the first feature similarity meets the first similarity condition, which is that the loss value is below a preset threshold or no longer decreases for several consecutive rounds, ensuring that the model can identify the essential invariant features of the same instance block under different augmented views.

[0038] While training the first feature similarity, the second feature similarity is also trained. This trains the differences between different views of different training instance blocks. Different views of different training instance blocks are input into the student network; for example, the global view of instance block X and the local view of instance block Y are input into the student network (or different forward propagations of the same student network), resulting in two output features. The second feature similarity between these two output features is calculated. The training objective is to minimize this similarity, that is, to make the features of different instance blocks far apart in the embedding space. This step is usually trained jointly with the first feature similarity, using a contrastive loss function to simultaneously bring positive sample pairs (different views of the same instance block) closer together and push away negative sample pairs (views of different instance blocks). Training continues until the second feature similarity meets the second similarity condition, which is that the similarity of negative sample pairs is below a preset threshold.

[0039] Next, iterate the training until the preset training termination condition is met, and repeat the above training process. Use a large number of training instance blocks for batch training. The preset training termination condition can be reaching a preset number of training rounds, or the loss function no longer decreasing for multiple consecutive rounds on the validation set, or the model's performance on downstream tasks reaching the expected index. No specific restrictions are imposed on these conditions.

[0040] Then, the ViT encoder of the student network is extracted as a pre-trained tissue feature extraction model. Specifically, after training, the projection heads of the teacher network and the student network are discarded, and only the ViT encoder part of the student network is retained as a pre-trained tissue feature extraction model. The ViT encoder can output a high-dimensional tissue morphology feature vector for any input image instance block. This feature vector implicitly encodes information such as cell morphology, tissue spatial topology, and interstitial distribution.

[0041] In this embodiment, each image instance block is input into a pre-trained tissue feature extraction model to linearly project each image instance block into a corresponding embedding vector. Position encoding is added to the embedding vector to obtain an embedding vector sequence carrying relative position information. The multi-head self-attention mechanism of the pre-trained tissue feature extraction model performs global feature interaction on the embedding vector sequence carrying relative position information to generate a fused feature sequence containing feature vectors of each image patch and classification label feature vectors. The classification label feature vector of the fused feature sequence is then extracted as the tissue morphology feature vector of the image instance block. It can be understood that the image instance block is divided into multiple image patches. An image instance block of size H×W (e.g., 256×256) is uniformly divided into N non-overlapping small image patches, each image patch having a size of P×P (e.g., 16×16 pixels). The image instance block is converted into a sequence of N image patches, where N = (H / P) × (W / P). For image patches of 256×256 pixels and 16×16 pixels, there are N=16×16=256 image patches. Each image patch is flattened into a one-dimensional vector (length P×P×3, i.e., 16×16×3=768). Then, a trainable linear projection matrix (fully connected layer) is used to map this vector into a D-dimensional embedding vector (D is the preset feature dimension). The same linear projection operation is performed on all image patches to obtain N D-dimensional embedding vectors. Simultaneously, to preserve the spatial location information of the image patches in the original image, a learnable position encoding vector is added to each embedding vector. This position encoding vector is a learnable parameter with the same dimension as the embedding vector D, and it is optimized along with other network parameters during the model pre-training stage. The position encoding vector is added to the corresponding embedding vector to obtain an embedding vector sequence carrying relative position information, represented as {e1, e2, ..., e...}. N}, where e i ∈ D , The set is the set of real numbers. Additionally, a learnable classification label embedding vector e is added at the beginning of the embedding vector sequence. cls This marker is used to aggregate global information for the entire image instance block in subsequent encoding. Finally, the vector sequence input to the ViT encoder is [e cls e1, e2, ..., e NThe input sequence, consisting of (N+1) D-dimensional vectors carrying positional information, is fed into the ViT encoder of a pre-trained organizational feature extraction model. The ViT encoder comprises L identical stacked Transformer layers (e.g., L=12 layers). Each Transformer layer contains a Multi-Head Self-Attention (MHSA) module and a Feed-Forward Network (FFN) module, and is equipped with layer normalization and residual connections. The computation process for each Transformer layer is as follows: After layer normalization, the input sequence is fed into the MHSA module. This module calculates the attention weights between all vectors in the sequence, ensuring that the output feature at each position is a weighted aggregation of all vector features in the sequence, thus fusing global contextual information. The output of the self-attention module is residually connected to the input sequence to obtain an intermediate sequence. The intermediate sequence is then normalized and fed into the Feed-Forward Network to further enhance the feature representation. The output of the Feed-Forward Network is residually connected to the intermediate sequence to obtain the output sequence of that layer. After processing through L Transformer layers, the ViT encoder outputs a fused feature sequence Z = [z cls z1, z2, ..., z N ], where z cls The final features corresponding to the classification labels, z1~z N The final features corresponding to each image patch contain both detailed information about the local image patch (aggregating the context of surrounding patches through a self-attention mechanism) and classification labels z. cls It aggregates global information from the entire image instance patch. From the feature sequence Z output by the ViT encoder, the feature vector z corresponding to the classification label is extracted. cls , z cls The vector has a dimension of D, which integrates information from all local regions within the image instance block and can characterize the organizational morphological features of the entire image instance block. This vector is output as the organizational morphological feature vector of the image instance block.

[0042] In this way, the feature extraction process based on the ViT encoder, through image patch segmentation, linear projection, positional encoding, and multi-head self-attention mechanism, enables the model to simultaneously capture cell morphology and global spatial topology (glandular arrangement) within image instance blocks. ViT's global self-attention mechanism avoids the loss of spatial information caused by layer-by-layer downsampling, and can more accurately represent tissue morphological features.

[0043] Step S13: Input each of the tissue morphology feature vectors into a multi-instance learning network, so that the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, and select target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores.

[0044] In this embodiment, before inputting each of the tissue morphology feature vectors into the multi-instance learning network, the method further includes: acquiring a medical image sample set and labeling each sample image with slice-level and region-level labels; wherein, the slice-level labels are category labels indicating the medical image sample, and the region-level labels are coordinate position labels indicating the medical image sample containing pathological feature regions; extracting the effective tissue region of each sample image and dividing the effective tissue region into multiple training instance blocks; inputting each training instance block into the pre-trained tissue feature extraction model to obtain the training tissue morphology feature vector corresponding to each training instance block; and inputting all training tissue morphology feature vectors corresponding to the same sample image into the initial multi-instance learning network to achieve the desired results. The instance detection module of the initial multi-instance learning network classifies and scores the attention of each training tissue morphological feature vector, selects target training instance blocks representing the target pathological feature region from all training instance blocks, and constructs target training image blocks based on the target training instance blocks and the region-level labels. The super-patch aggregation module of the initial multi-instance learning network weights the attention scores corresponding to the target training image blocks to generate aggregated feature representations. The aggregated feature representations are input into a classification head to output predicted categories. A loss function is calculated based on the difference between the predicted category and the slice-level labels, and the network parameters of the initial multi-instance learning network are updated through backpropagation based on the calculation results until the convergence condition is met to obtain the multi-instance learning network. It can be understood that the pre-trained tissue feature extraction model is used to extract features from the training samples, and the extracted training tissue morphological feature vectors are used to train the MIL (Multiple Instance Learning) network to obtain the trained multi-instance learning network. The specific steps are as follows: H&E stained whole-slide images with pathological diagnosis are selected from clinical databases. Each sample image needs to be labeled with slice-level and region-level tags. Slice-level tags indicate the overall category of the medical image sample, such as "tumor" or "normal," or more specific subtype tags, such as "well-differentiated adenocarcinoma," "moderately differentiated adenocarcinoma," and "poorly differentiated adenocarcinoma." These tags are associated with the sample image in the form of image file names or separate annotation files. Region-level tags indicate the coordinate locations of the pathologically characteristic regions within the medical image sample. The entire training process does not require any pixel-level annotation of individual instance blocks within the image, i.e., lesion boundaries and cell types are not labeled. Then, for each sample image, the same image segmentation algorithm as the aforementioned image preprocessing is used to remove background elements such as blank areas and air bubbles on the slide, retaining the effective tissue region containing cells and / or tissues. Then, the effective tissue region is non-overlappingly segmented according to a preset size (256×256 pixels) to obtain multiple training instance blocks. Each sample image corresponds to an instance bag, and all training instance blocks within this bag constitute the instance set of that sample. Each training instance block is input into the pre-trained tissue feature extraction model (i.e., the ViT encoder of DINOv2). Following the feature extraction method, the model outputs a training tissue morphology feature vector corresponding to each training instance block. This feature vector is a high-dimensional vector that implicitly encodes information such as cell morphology and tissue spatial structure within the instance block. Thus, the instance package of each sample image is converted into a set of feature vectors. An initial multi-instance learning network is then constructed, specifically including an instance detection module. This module uses a gated attention mechanism to classify and score the attention of each training tissue image block, and normalizes the scores to between 0 and 1 using a Softmax function. The attention score implicitly reflects the contribution of the instance block to the final diagnosis; higher-scoring instance blocks typically correspond to regions containing atypical cells or typical lesions, thereby automatically filtering target instance blocks containing pathological features without explicit similarity calculation. Then, during the training of the instance detection module, different constraints are applied to instance packages of different natures. Specifically, for... For negative instance packets that do not contain any pathological feature images, the network imposes an attention consistency constraint, forcing the attention scores of each instance block within the packet to be distributed as evenly as possible. This prevents the instance detection module from forcibly learning false signals from meaningless backgrounds or noise. For positive instance packets that contain pathological feature images, the network allows attention to naturally focus on discriminative regions. In addition, when the training data also provides doctor-annotated region-level labels, the network uses this information to assist in instance selection. The specific selection process is as follows: the instance detection module combines the classification results and attention scores to select key pathological feature regions of clinical interest from all training tissue image blocks, and uses these regions together with the region-level labels to construct target training image blocks, enabling the model to more accurately locate pathology-related regions.

[0045] In addition, an instance drop-off module that does not introduce training parameters can be added to the network. During training, this module will randomly drop instance blocks with high attention scores with a certain probability. The purpose of this is to break the model's over-reliance on a few outstanding instances, forcing the network to learn effective features from more diverse instances in the bag, thereby improving the robustness and generalization ability of instance selection.

[0046] Then, the attention score is multiplied by the class logic value output by the classifier to generate an aggregated feature representation. This aggregation method can simultaneously consider the discriminative strength and class tendency of the instance. The final output global feature representation retains the prominent information of the lesion instance and also takes into account the tissue distribution pattern of the whole slice.

[0047] Furthermore, when region-level annotations exist, target training image patches can be input into the super-patch aggregation module. The super-patch aggregation module performs weighted processing on the attention score corresponding to each target training image patch. Specifically, for target training image patches that have been filtered with the assistance of region annotations, the super-patch aggregation module performs weighted processing on their corresponding attention scores to generate more discriminative aggregated feature representations. This module can effectively integrate cross-instance contextual information and further improve diagnostic accuracy.

[0048] Then, the aggregated global feature representation is output as the final diagnostic result through the classification head. Specifically, the aggregated bag-level feature representation is input into the classification head, which contains one or more fully connected layers, followed by a softmax activation function. The output is the predicted class probability distribution, and the class with the highest probability is taken as the predicted class of the sample image. Based on the difference between the predicted class and the labeled slice-level label, the loss function is calculated. The gradient of the loss function with respect to each learnable parameter in the initial multi-instance learning network (including the parameters of the instance detection module, the super-patch aggregation module, and the classification head) is calculated using the backpropagation algorithm, and the network parameters are updated using the optimizer. During batch training, a batch of sample images is used each time, and the above training steps are repeated. Multiple rounds of iteration are performed using all or part of the training samples. The preset convergence conditions are reaching a preset number of training rounds, or the loss function no longer decreasing for several consecutive rounds on the validation set, or the evaluation indicators such as classification accuracy reaching the expected threshold. When the convergence conditions are met, training is stopped, and the multi-instance learning network at this point is saved as a multi-instance learning network.

[0049] In this embodiment, after obtaining the tissue morphology feature vectors of all image instance blocks of the medical image to be analyzed, these vectors are input into the instance detection module of the multi-instance learning network to automatically filter out target image instance blocks containing pathological feature regions from all instance blocks. Specifically, the following steps are included: The attention score for each tissue morphological feature vector is calculated using the gated attention mechanism of the instance detection module in the multi-instance learning network to obtain the original attention score corresponding to each image instance block. The original attention scores are then normalized using a softmax function to obtain the normalized attention score for each image instance block. These normalized attention scores are then sorted from highest to lowest, and a predetermined number of target normalized attention scores are selected from the score sorting table in a front-to-back order. The target image instance block corresponding to each target normalized attention score is then identified. Essentially, the feature vectors of all instance blocks are input into the pre-trained instance detection module of the multi-instance learning network. This module, based on the gated attention mechanism, automatically calculates an attention score (between 0 and 1) for each instance block. The score directly reflects the likelihood that the corresponding instance block contains a pathological feature region. Based on a preset score threshold or sorting ratio, the module automatically selects the target image instance block from all instance blocks without manual intervention.

[0050] Through the above filtering, invalid instance blocks such as normal tissue and blurred edges can be automatically removed, and only key instance blocks containing atypical cells or lesion areas are retained. This reduces the computational load of the subsequent attention-weighted aggregation module, avoids interference from redundant information, and improves the accuracy of analysis.

[0051] Step S14: Aggregate the target tissue morphological feature vectors corresponding to the target image instance blocks through the multi-instance learning network to generate and output the aggregated global feature representation used to characterize the tissue morphological distribution information in the target medical image.

[0052] In this embodiment, the attention score and feature vector corresponding to the selected target image instance block are processed by the aggregation module. In the inference stage, the dot product aggregation method is used by default, that is, the dot product of the attention score and the class logic value output by the classifier is calculated to generate the global feature representation of the entire slice. This representation is then input into the classification head to output the final diagnosis result.

[0053] Furthermore, the attention scores corresponding to each image instance block output by the multi-instance learning network are obtained; based on the attention scores, each image instance block in the target medical image is visualized and mapped to generate an attention heatmap; the attention heatmap is then overlaid onto the target medical image or effective tissue region to indicate the degree of contribution of different regions in the target medical image to the diagnostic results. In this way, by retaining the attention score of each instance block and generating an attention heatmap accordingly, the heatmap is overlaid onto the original medical image or effective tissue region. The heatmap is used to mark the positional distribution of each target image instance block and its contribution to the global feature representation on the target medical image. For example, in the overlaid image, areas with a reddish / yellow tint represent key instance blocks with high contribution, while areas with a blue / black tint have lower contribution. This visually demonstrates the contribution of different regions to the diagnostic results, helping doctors understand the model's judgment criteria and improving clinical interpretability.

[0054] In this embodiment, clinical feedback data is acquired; wherein, the clinical feedback data includes review and correction information on the global feature representation and / or newly added unlabeled medical images; the multi-instance learning network is incrementally fine-tuned based on the review and correction information to update the network parameters of the multi-instance learning network; and / or, the pre-trained tissue feature extraction model is self-supervised incrementally trained based on the newly added unlabeled medical images to update the model parameters of the pre-trained tissue feature extraction model. It is understood that after the information processing method is applied to a clinical environment, clinical feedback data is further acquired. The clinical feedback data includes at least one of the following: review and correction information on the global feature representation, such as analysis results confirmed or corrected by doctors; newly added unlabeled medical images. Based on the review and correction information, the multi-instance learning network is incrementally fine-tuned: the reviewed and corrected data is used as a weak supervision signal, and the multi-instance learning network training method (only slice-level labels are required) is adopted to continue updating the parameters of the multi-instance learning network on small batches of data to adapt it to the new data distribution. Based on newly added unlabeled medical images, the pre-trained tissue feature extraction model undergoes self-supervised incremental training: following the self-supervised learning approach used to train the pre-trained tissue feature extraction model, the ViT encoder parameters of the DINOv2 network are continuously updated using the newly added unlabeled images, thereby continuously optimizing its feature extraction capabilities. These two update processes can be executed independently or collaboratively, forming a data flywheel closed loop. With the accumulation of clinical data, the feature extraction model and the multi-instance learning network continuously optimize in both directions, resulting in sustained improvements in model accuracy and generalization ability.

[0055] As can be seen, this application discloses a method for acquiring a target medical image to be analyzed, extracting effective tissue regions from the target medical image, and then segmenting the effective tissue regions into multiple image instance blocks; wherein, the effective tissue regions are image regions containing cells and / or tissues; each image instance block is input into a pre-trained tissue feature extraction model to output a tissue morphology feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; each tissue morphology feature vector is input into a multi-instance learning network, so that the instance detection module of the multi-instance learning network filters out target image instance blocks containing pathological feature regions from all the image instance blocks based on the similarity comparison between each tissue morphology feature vector; the multi-instance learning network performs attention weighting processing on the target tissue morphology feature vectors corresponding to the target image instance blocks to generate and output an aggregated global feature representation for characterizing the tissue morphology distribution information in the target medical image. Therefore, by extracting effective tissue regions containing cells and / or tissues and segmenting them into multiple image instance blocks, background interference such as blank areas and bubbles on the slide is automatically removed, reducing subsequent computational redundancy. A pre-trained tissue feature extraction model based on the DINOv2 network outputs tissue morphological feature vectors corresponding to each instance block. An instance detection module of a multi-instance learning network generates attention scores based on a gated attention mechanism. Simultaneously, superpixel block sampling and convolution operations are used to expand the local receptive field, thereby capturing both local cell morphological details and global tissue spatial topology under weak supervision, overcoming the limitations of traditional CNN models. This approach overcomes the limitations of restricted receptive field and loss of cross-scale information. By implicitly filtering target image instance blocks containing pathological feature regions through attention scores, it abandons the coarse aggregation methods of traditional mean pooling or max pooling, and achieves automatic identification of key analysis regions. Finally, by using dot product aggregation, the attention scores and the classifier's output category logits are combined to dynamically allocate the contribution of instance blocks, generating an aggregated global feature representation. This approach retains both local key information and global context, while also preserving instance-level attention weights for spatial heatmap generation. Ultimately, it outputs a global feature representation that represents the morphological distribution information of tissues in medical images.

[0056] like Figure 2 As shown, the overall training iteration process of the DINOv2 network and multi-instance learning network for pathological image analysis provided by this invention is as follows: Whole slide images (WSI) are obtained as the target medical images to be analyzed. The WSI images are then processed to separate the foreground and background. Image segmentation algorithms are used to remove invalid backgrounds such as blank areas and air bubbles on the slides, while retaining the effective tissue areas containing cells and / or tissues.

[0057] The separated effective tissue regions are segmented into multiple independent image instance blocks (Patches) according to a fixed size.

[0058] The segmented image instance blocks are input into the basic visual model built on DINOv2 Vision Transformer (ViT). This model is pre-trained on a large-scale pathological image database through self-supervised learning and can extract high-dimensional tissue morphology feature vectors (Patch features) for each instance block.

[0059] The extracted instance block feature vectors are input into the multi-instance learning module, which uses a gating attention mechanism and an aggregation mechanism to generate an aggregated global feature representation.

[0060] The output of the multi-instance learning module is reviewed by experts, and the review results (confirmed or corrected analysis conclusions) are given. The reviewed results can be used as WSI-level labels for subsequent model optimization.

[0061] Furthermore, the entire system forms a data flywheel iteration mechanism. Newly added unlabeled images in the pathology image database can be fed back into the DINOv2 self-supervised learning process to continuously optimize the feature extraction model. Data corrected by expert knowledge review (which can be stored in the pathology report database) can be used for incremental fine-tuning training of the multi-instance learning module. Through bidirectional collaborative optimization, the model continuously improves its accuracy and generalization ability as clinical data accumulates.

[0062] like Figure 3 As shown, the present invention provides an information processing device for medical tissue morphology image analysis, comprising: Image segmentation processing 11 is used to acquire the target medical image to be analyzed, extract the effective tissue region from the target medical image, and then segment the effective tissue region into multiple image instance blocks; wherein, the effective tissue region is an image region containing cells and / or tissues; The tissue feature extraction module 12 is used to input each of the image instance blocks into a pre-trained tissue feature extraction model to output the tissue morphological feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; The instance block filtering module 13 is used to input each of the tissue morphology feature vectors into a multi-instance learning network, so that the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, and filters out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores. The result output module 14 is used to aggregate the target tissue morphological feature vectors corresponding to the target image instance blocks through the multi-instance learning network, so as to generate and output the aggregated global feature representation for characterizing the tissue morphological distribution information in the target medical image.

[0063] As can be seen, the image segmentation module automatically extracts and segments effective tissue regions, removes background interference, and reduces computational redundancy; the tissue feature extraction module extracts highly expressive tissue morphological features based on the DINOv2 ViT architecture, including local details and global context; the instance block filtering module filters key instance blocks to avoid redundant information interference; and the result output module generates global feature representations through aggregation and retains instance-level attention weights, which facilitates visualization and tracing.

[0064] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0065] Figure 4 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the information processing method for medical tissue morphology image analysis disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0066] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0067] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0068] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0069] The operating system 221 manages and controls the various hardware devices and computer programs 222 on the electronic device 20 to enable the processor 21 to perform calculations and processing on the massive amounts of data 223 in the memory 22. It can be Windows Server, Netware, Unix, Linux, etc. The computer program 222, in addition to including a computer program capable of performing the information processing method for medical tissue morphology image analysis executed by the electronic device 20 as disclosed in any of the foregoing embodiments, may further include computer programs capable of performing other specific tasks. The data 223 may include data received by the electronic device from external devices, as well as data collected by its own input / output interface 25.

[0070] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned information processing method for medical tissue morphology image analysis. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0072] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly in hardware, software modules executed by a processor, or a combination of both. The software module may be located in random access memory (RAM), memory, read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, CD-ROMs (Compact Disc-Read Only Memory), or any other form of storage medium known in the art.

[0073] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0074] The solution provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An information processing method of medical tissue morphological image analysis, characterized by, Applied to computer devices, including: A target medical image to be analyzed is acquired, and an effective tissue region is extracted from the target medical image. Then, the effective tissue region is divided into multiple image instance blocks; wherein, the effective tissue region is an image region containing cells and / or tissues. Each of the aforementioned image instance blocks is input into a pre-trained tissue feature extraction model to output a tissue morphological feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; Each of the tissue morphology feature vectors is input into a multi-instance learning network, and the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, so as to filter out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores. The multi-instance learning network aggregates the target tissue morphological feature vectors corresponding to the target image instance blocks to generate and output the aggregated global feature representation that characterizes the tissue morphological distribution information in the target medical image.

2. The information processing method for medical tissue morphology image analysis according to claim 1, characterized in that, The step of extracting effective tissue regions from the target medical image and then dividing the effective tissue regions into multiple image instance blocks includes: A preset image segmentation algorithm is used to separate background information from effective tissue in the target medical image, so as to retain the image region containing cells and / or tissue as the effective tissue region; wherein, the background information includes blank areas of the slide and air bubbles in the image; The effective tissue region is segmented into multiple image instance blocks according to a preset size using a non-overlapping segmentation process.

3. The information processing method for medical tissue morphology image analysis according to claim 1, characterized in that, Before inputting each of the image instance blocks into the pre-trained tissue feature extraction model, the method further includes: Obtain an unlabeled medical image sample set; Each sample image in the medical image sample set is divided into multiple training instance blocks; Generate different views for each of the training instance blocks to obtain multiple different views corresponding to each training instance block; Different views of the same training instance block are input to the teacher network and student network of the DINOv2 network until the first feature similarity between the first output feature output by the student network and the second output feature output by the teacher network satisfies the first similarity condition. Different views of each training instance block are input to the student network until the second feature similarity between the output features output by the student network satisfies the second similarity condition. The ViT encoder of the student network of the DINOv2 network that meets the preset training termination conditions is used as a pre-trained organizational feature extraction model.

4. The information processing method for medical tissue morphology image analysis according to claim 3, characterized in that, The step of inputting each of the image instance blocks into a pre-trained tissue feature extraction model to output the tissue morphological feature vector corresponding to each image instance block includes: Each of the image instance blocks is input into a pre-trained tissue feature extraction model to linearly project each of the image instance blocks into a corresponding embedding vector, and position encoding is added to the embedding vector to obtain an embedding vector sequence carrying relative position information. The multi-head self-attention mechanism of the pre-trained tissue feature extraction model is used to perform global feature interaction on the embedded vector sequence carrying relative position information to generate a fused feature sequence containing feature vectors of each image patch and classification label feature vectors. The classification label feature vector of the fused feature sequence is then extracted as the tissue morphology feature vector of the image instance patch.

5. The information processing method for medical tissue morphology image analysis according to claim 1, characterized in that, Before inputting the morphological feature vectors of each of the tissues into the multi-instance learning network, the method further includes: A medical image sample set is obtained, and each sample image is labeled with slice-level labels and region-level labels; wherein, the slice-level labels are category labels indicating medical image samples, and the region-level labels are coordinate location labels indicating the regions containing pathological features in medical image samples; Extract the effective organization region of each sample image and divide the effective organization region of the sample into multiple training instance blocks; Each training instance block is input into the pre-trained tissue feature extraction model to obtain the training tissue morphological feature vector corresponding to each training instance block; All training tissue morphological feature vectors corresponding to the same sample image are input into the initial multi-instance learning network. The instance detection module of the initial multi-instance learning network classifies and scores the morphological feature vectors of each training tissue. The target training instance block representing the target pathological feature region is selected from all training instance blocks. The target training image block is constructed based on the target training instance block and the region-level label. The attention scores corresponding to the target training image patches are weighted by the super-patch aggregation module of the initial multi-instance learning network to generate aggregated feature representations. The aggregated feature representation is input into the classification head to output the predicted category; The loss function is calculated based on the difference between the predicted category and the slice-level label, and the network parameters of the initial multi-instance learning network are updated by backpropagation based on the calculation result until the convergence condition is met to obtain the multi-instance learning network.

6. The information processing method for medical tissue morphology image analysis according to claim 1, characterized in that, Also includes: Obtain clinical feedback data; wherein, the clinical feedback data includes review and correction information on the global feature representation and / or newly added unlabeled medical images; Based on the audit correction information, the multi-instance learning network is incrementally fine-tuned to update the network parameters of the multi-instance learning network. And / or, based on the newly added unlabeled medical images, perform self-supervised incremental training on the pre-trained tissue feature extraction model to update the model parameters of the pre-trained tissue feature extraction model.

7. The information processing method for medical tissue morphology image analysis according to claim 1, characterized in that, The instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors to filter out target image instance blocks containing pathological feature regions from all the image instance blocks based on the attention scores, including: Attention scores are assigned to each tissue morphology feature vector through the gating attention mechanism of the instance detection module of the multi-instance learning network to obtain the original attention scores corresponding to each image instance block. The original attention scores are normalized using the softmax function to obtain the normalized attention scores for each image instance block. The normalized attention scores are sorted in descending order, and a preset number of target normalized attention scores are selected from the score sorting table in a forward-to-back order. The target image instance blocks corresponding to the target normalized attention scores are then selected.

8. An information processing device for medical tissue morphology image analysis, characterized in that, include: Image segmentation processing is used to acquire a target medical image to be analyzed, extract effective tissue regions from the target medical image, and then segment the effective tissue regions into multiple image instance blocks; wherein, the effective tissue regions are image regions containing cells and / or tissues; The tissue feature extraction module is used to input each of the image instance blocks into a pre-trained tissue feature extraction model to output the tissue morphological feature vector corresponding to each image instance block; wherein, the pre-trained tissue feature extraction model is a model built based on the DINOv2 network; The instance block filtering module is used to input each of the tissue morphology feature vectors into a multi-instance learning network, so that the instance detection module of the multi-instance learning network performs attention scoring on each of the tissue morphology feature vectors, and filters out target image instance blocks containing pathological feature regions from all the image instance blocks according to the attention scores. The result output module is used to aggregate the target tissue morphological feature vectors corresponding to the target image instance blocks through the multi-instance learning network, so as to generate and output the aggregated global feature representation for characterizing the tissue morphological distribution information in the target medical image.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the information processing method for medical tissue morphology image analysis as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the information processing method for medical tissue morphology image analysis as described in any one of claims 1 to 7.

Images