Pathological evaluation method and device based on attention mechanism and density feature extraction

Abstract

The embodiment of the application provides a pathological evaluation method and device based on an attention mechanism and density feature extraction, which comprises the following steps: performing pathological semantic feature extraction on a pathological section image to obtain a pathological feature image; performing attention feature extraction of multiple disease categories on the pathological feature image, and performing disease category-oriented adjustment on the pathological feature image according to multiple attention weight matrices obtained by extraction to obtain a disease category information expression matrix; performing density feature extraction of multiple disease categories on each attention weight matrix respectively, and combining density feature tensors obtained from each attention weight matrix into a density feature information expression matrix; and performing pathological evaluation on each disease category according to the density feature information expression matrix and the disease category information expression matrix. The embodiment of the application can improve the pathological evaluation capability of tumor tissues.

Description

Technical Field

[0001] This application relates to the field of pathological evaluation technology, and in particular to a pathological evaluation method and apparatus. Background Technology

[0002] With the rapid development of artificial intelligence technology in the field of medical image analysis, deep learning-based pathological image-assisted diagnostic systems have been gradually applied to the pathological evaluation of common tumors such as prostate cancer, colorectal adenoma, and breast cancer. These systems determine the grade of tumors by judging the degree of morphological atypia of pathological tissue sections.

[0003] However, in actual clinical medicine, the grading / classification criteria for many tumors or pre-tumor lesions not only depend on the degree of morphological atypia of the lesions, but also highly depend on the proportion of different types of lesion areas in the tissue section. Therefore, current pathological image-assisted diagnostic systems cannot actually assist pathological assessment work very well. Pathological assessors still need to manually confirm whether the pathological assessment results output by the pathological image-assisted diagnostic system meet the actual clinical assessment standards. Summary of the Invention

[0004] This application provides a pathological assessment method and apparatus based on attention mechanism and density feature extraction, aiming to improve the pathological assessment capability of tumor tissues so that the pathological assessment process is more consistent with actual clinical assessment standards.

[0005] To achieve the above objectives, a first aspect of this application provides a pathological assessment method based on attention mechanisms and density feature extraction, the method comprising:

[0006] Pathological semantic features are extracted from pathological slide images to obtain pathological feature images;

[0007] Attention features for multiple disease categories are extracted from the pathological feature image, and disease category-guided adjustment is performed on the pathological feature image based on the extracted multiple attention weight matrices to obtain a disease category information expression matrix;

[0008] Density features of various disease categories are extracted from each of the attention weight matrices, and the density feature tensors extracted from each of the attention weight matrices are merged into a density feature information representation matrix.

[0009] Based on the density feature information expression matrix and the disease category information expression matrix, a pathological assessment is performed on each of the disease categories.

[0010] In one embodiment, the step of extracting pathological semantic features from the pathological slide image to obtain a pathological feature image includes:

[0011] The pathological slide images are preprocessed to obtain multiple pathological slide image blocks;

[0012] Pathological semantic features are extracted from multiple pathological slice image blocks to obtain multiple pathological feature sub-images;

[0013] The pathological feature image is obtained by reconstructing the feature image based on multiple pathological feature sub-images.

[0014] In one embodiment, the step of extracting attention features for multiple disease categories from the pathological feature image and adjusting the pathological feature image according to the extracted multiple attention weight matrices to obtain a disease category information representation matrix includes:

[0015] The pathological feature image is subjected to attention feature extraction for multiple disease categories to obtain multiple attention weight matrices, wherein the attention weight matrix includes the disease category attention weight for each disease category;

[0016] For each of the attention weight matrices, the pathological feature image is adjusted according to the attention weight of each disease category in the attention weight matrix to obtain a disease category information expression submatrix;

[0017] The disease category information representation matrix is ​​obtained by concatenating multiple sub-matrices based on disease category.

[0018] In one embodiment, the step of performing disease category-oriented adjustment on the pathological feature image according to the attention weight of each disease category in the attention weight matrix to obtain a disease category information representation submatrix includes:

[0019] For each disease category, the pathological feature image is subjected to disease category-guided weighting based on the disease category attention weight to generate a disease category information expression feature vector for that disease category.

[0020] The disease category information expression feature vectors of each disease category are concatenated to form the disease category information expression submatrix.

[0021] In one embodiment, the step of extracting density features for multiple disease categories from each of the attention weight matrices and merging the density feature tensors extracted from each of the attention weight matrices into a density feature information representation matrix includes:

[0022] For each attention weight matrix, density features of multiple disease categories are extracted from the attention weight matrix, and the density feature maps of each disease category are merged to obtain the density feature tensor;

[0023] The density feature tensors of each attention weight matrix are merged into the density feature information representation matrix.

[0024] In one embodiment, the step of extracting density features for multiple disease categories from the attention weight matrix and merging the resulting density feature maps for each disease category to obtain the density feature tensor includes:

[0025] For each of the disease categories, density features are extracted from the attention weights of the disease categories to obtain the density feature map of the disease category;

[0026] The density feature maps for each disease category are merged based on the disease category to obtain the density feature tensor.

[0027] In one embodiment, the step of performing pathological evaluation on each disease category based on the density feature information expression matrix and the disease category information expression matrix includes:

[0028] Pathological evaluation is performed based on the density feature information expression matrix and the disease category information expression matrix to obtain preliminary pathological evaluation results for each disease category;

[0029] The visibility of disease categories is evaluated based on the density feature information expression matrix, and the visibility evaluation results for each disease category are obtained.

[0030] For each disease category, in response to the preliminary pathological assessment result of the disease category indicating the presence of pathological tissue of the disease category on the pathological slide image, the preliminary pathological assessment result of the disease category is corrected according to the visibility assessment result of the disease category to obtain the target pathological assessment result of the disease category.

[0031] To achieve the above objectives, a second aspect of this application provides a pathological assessment device based on attention mechanisms and density feature extraction, comprising:

[0032] The pathological semantic feature extraction module is used to extract pathological semantic features from pathological slide images to obtain pathological feature images.

[0033] The disease category-oriented adjustment module is used to extract attention features of multiple disease categories from the pathological feature image, and to perform disease category-oriented adjustment on the pathological feature image according to the extracted multiple attention weight matrices to obtain a disease category information expression matrix.

[0034] The density feature extraction module extracts density features for various disease categories from each of the attention weight matrices, and merges the density feature tensors extracted from each of the attention weight matrices into a density feature information representation matrix.

[0035] The pathological assessment module is used to perform pathological assessment on each of the disease categories based on the density feature information expression matrix and the disease category information expression matrix.

[0036] To achieve the above objectives, a third aspect of the present application provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first aspect above.

[0037] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0038] This application provides a pathological assessment method and apparatus based on attention mechanisms and density feature extraction. The method includes: extracting pathological semantic features from a pathological slide image to obtain a pathological feature image; extracting attention features for multiple disease categories from the pathological feature image, and adjusting the pathological feature image according to multiple extracted attention weight matrices to obtain a disease category information expression matrix; extracting density features for multiple disease categories from each attention weight matrix, and merging the density feature tensors extracted from each attention weight matrix into a density feature information expression matrix; and performing pathological assessment for each disease category based on the density feature information expression matrix and the disease category information expression matrix. By extracting pathological semantic features from pathological slide images, we can obtain pathological feature images rich in pathological semantic feature information. Then, by extracting attention features for multiple disease categories from these images, we can enhance the expression of pathological semantic feature information for each disease category. Based on this, by extracting density features from each attention weight matrix, we can extract the density semantic feature expressions of pathological tissues of various disease categories in the pathological feature images from the enhanced pathological semantic feature information for each disease category. Thus, when performing pathological evaluation for each disease category based on the density feature information expression matrix and the disease category information expression matrix, the density features of each disease category can be incorporated into the evaluation process. This allows the pathological evaluation to take into account the influence of the density proportion of each disease category on the pathological slide tissue, thereby achieving a more granular pathological evaluation, improving the accuracy of pathological evaluation, and making the pathological evaluation process more aligned with actual clinical evaluation standards. Attached Figure Description

[0039] Figure 1 This is a schematic flowchart of a pathological assessment method based on attention mechanism and density feature extraction provided in one embodiment of this application;

[0040] Figure 2 yes Figure 1 A flowchart illustrating an embodiment of a sub-step of step 110;

[0041] Figure 3 yes Figure 1 A flowchart illustrating an embodiment of a sub-step of step 120;

[0042] Figure 4 yes Figure 1 A flowchart illustrating an embodiment of a sub-step of step 130;

[0043] Figure 5This is a schematic diagram of the architecture of a neural network model for implementing a pathological assessment method according to an embodiment of this application;

[0044] Figure 6 yes Figure 1 A flowchart illustrating an embodiment of a sub-step of step 140;

[0045] Figure 7 This is a schematic diagram of the architecture of a neural network model for implementing a pathological assessment method provided in another embodiment of this application;

[0046] Figure 8 This is a schematic diagram of the pathological assessment device based on attention mechanism and density feature extraction provided in the embodiments of this application;

[0047] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0049] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0050] In this application, the terms "furthermore," "exemplarily," or "optionally" are used as examples, illustrations, or descriptions and should not be construed as being more preferred or advantageous than other embodiments or designs. The use of terms such as "furthermore," "exemplarily," or "optionally" is intended to present the relevant concepts in a specific manner.

[0051] With the rapid development of artificial intelligence technology in the field of medical image analysis, deep learning-based pathological image-assisted diagnostic systems have been gradually applied to the pathological evaluation of common tumors such as prostate cancer, colorectal adenoma, and breast cancer. These systems determine the grade of tumors by judging the degree of morphological atypia of pathological tissue sections.

[0052] However, in actual clinical medicine, the grading / classification criteria for many tumors or pre-tumor lesions not only depend on the degree of morphological atypia of the lesions, but also highly depend on the proportion of different types of lesion areas in the tissue section. Therefore, current pathological image-assisted diagnostic systems cannot actually assist pathological assessment work very well. Pathological assessors still need to manually confirm whether the pathological assessment results output by the pathological image-assisted diagnostic system meet the actual clinical assessment standards.

[0053] To improve the ability to assess the pathology of tumor tissues and make the pathology assessment process more aligned with actual clinical assessment standards, this application provides a pathology assessment method, device, electronic device, and computer-readable storage medium based on attention mechanisms and density feature extraction. The method includes: extracting pathological semantic features from a pathological slide image to obtain a pathological feature image; extracting attention features for multiple disease categories from the pathological feature image and adjusting the pathological feature image according to multiple extracted attention weight matrices to obtain a disease category information expression matrix; extracting density features for multiple disease categories from each attention weight matrix and merging the density feature tensors extracted from each attention weight matrix into a density feature information expression matrix; and performing a pathological assessment for each disease category based on the density feature information expression matrix and the disease category information expression matrix. By extracting pathological semantic features from pathological slide images, we can obtain pathological feature images rich in pathological semantic feature information. Then, by extracting attention features for multiple disease categories from these images, we can enhance the expression of pathological semantic feature information for each disease category. Based on this, by extracting density features from each attention weight matrix, we can extract the density semantic feature expressions of pathological tissues of various disease categories in the pathological feature images from the enhanced pathological semantic feature information for each disease category. Thus, when performing pathological evaluation for each disease category based on the density feature information expression matrix and the disease category information expression matrix, the density features of each disease category can be incorporated into the evaluation process. This allows the pathological evaluation to take into account the influence of the density proportion of each disease category on the pathological slide tissue, thereby achieving a more granular pathological evaluation, improving the accuracy of pathological evaluation, and making the pathological evaluation process more aligned with actual clinical evaluation standards.

[0054] The first aspect of this application provides a pathological assessment method based on attention mechanism and density feature extraction. This pathological assessment method can be executed by a server, by an electro-observation device such as an electron microscope, or by both an electro-observation device and a server. In this application embodiment, the method is described using the server as an example. Other execution examples can be found in the following embodiments.

[0055] See Figure 1 , Figure 1 The flowchart of the pathological evaluation method provided in the embodiments of this application is illustrated. In the embodiments of this application, the pathological evaluation method may include the following steps.

[0056] Step 110: Extract pathological semantic features from the pathological slide images to obtain pathological feature images;

[0057] Step 120: Extract attention features for multiple disease categories from the pathological feature image, and adjust the pathological feature image according to the extracted multiple attention weight matrices to obtain the disease category information expression matrix;

[0058] Step 130: Extract density features for multiple disease categories from each attention weight matrix, and merge the density feature tensors extracted from each attention weight matrix into a density feature information representation matrix;

[0059] Step 140: Perform pathological evaluation for each disease category based on the density feature information expression matrix and the disease category information expression matrix.

[0060] In one embodiment, a whole slide image (WSI) refers to an image obtained by observing and imaging a pathological slide tissue using an electrophysiological observation device, and that contains the complete pathological slide tissue. The pathological slide image can be a fluorescence image of the pathological slide tissue, a bright-field image of the pathological slide tissue, etc., and is not specifically limited here.

[0061] In one embodiment, pathological semantic feature extraction refers to performing a generalized feature extraction operation on pathological slide images for pathological semantics. Since pathological semantic feature extraction is a generalized extraction operation, the extracted pathological feature image can contain various semantic feature information that has not been extracted through deep abstraction, such as color semantic feature information and pathological tissue morphology semantic feature information. Furthermore, pathological semantic feature extraction can be performed through a pre-trained backbone network. The backbone network can be any convolutional neural network or Transformer structure with strong feature abstraction capabilities, such as ResNet, EfficientNet, DenseNet, ConvNeXt, or Swin Transformer, etc., without specific limitations here.

[0062] See Figure 2 In one embodiment, step 110 may include the following sub-steps.

[0063] Step 210: Preprocess the pathological slide images to obtain multiple pathological slide image blocks;

[0064] Step 220: Extract pathological semantic features from multiple pathological slide image blocks to obtain multiple pathological feature sub-images;

[0065] Step 230: Reconstruct the feature images based on multiple pathological feature sub-images to obtain the pathological feature images.

[0066] In one embodiment, preprocessing may include image quality processing and segmentation processing. Image quality processing may include image smoothing, noise reduction, and staining correction of the pathological slide image, etc., without specific limitations.

[0067] In one embodiment, during the preprocessing of pathological slide images to obtain multiple pathological slide image blocks, the image of the pathological slide after quality processing can be cropped using a preset sliding window size to obtain multiple pathological slide image blocks of the same size. A pathological slide image block may contain a portion of the tissue from the pathological slide, or it may not contain any tissue from the slide, etc., and is not specifically limited here.

[0068] In one embodiment, during the process of extracting pathological semantic features from multiple pathological slide image blocks to obtain multiple pathological feature sub-images, the multiple pathological slide image blocks can be input into the backbone network one by one, so that the backbone network outputs the pathological feature sub-image corresponding to each pathological slide image. Whether a pathological feature sub-image contains pathological semantic feature information depends on whether the corresponding pathological slide image block contains a portion of the tissue from the pathological slide. When the pathological slide image block does not contain any tissue, the pathological feature sub-image does not contain any pathological semantic feature information.

[0069] In one embodiment, for each pathological feature sub-image, after pathological semantic feature extraction, the pathological feature sub-image is a multi-dimensional image, where some dimensions can represent different types of pathological semantic features, and the type of pathological semantic features depends on the feature extraction capability of the backbone network. For example, assuming the size of the pathological feature sub-image is H1×W1×L1, the L1 dimension can be used to represent pathological semantic features. The specific size of L1 depends on the feature extraction capability of the backbone network, and also represents the number of types of pathological semantic features that the current pathological feature sub-image can contain.

[0070] In one embodiment, during the process of reconstructing a pathological feature image based on multiple pathological feature sub-images to obtain a pathological feature image, the obtained pathological feature sub-images can be stitched together to obtain the pathological feature image. The stitching method is diverse. For example, during the process of cropping pathological slide image blocks, positioning information can be assigned to each pathological slide image block, and multiple pathological feature sub-images can be stitched together based on the positioning information previously assigned to the corresponding pathological slide image blocks to obtain the pathological feature image. Alternatively, stitching can be performed using some automatic registration algorithms, etc., and the specific method is not limited here.

[0071] In one embodiment, during the process of reconstructing a pathological feature image based on multiple pathological feature sub-images, the dimensional data representing a specific pathological semantic feature type in each pathological feature sub-image can be detected. Pathological feature sub-images lacking dimensional data of a specific pathological semantic feature type are identified, and the remaining pathological feature sub-images are stitched together to obtain the final pathological feature image. This method can exclude some pathological feature sub-images that lack subsequently needed pathological semantic feature information, thus removing useless pathological feature sub-images to a certain extent and improving the efficiency of subsequent pathological assessment.

[0072] Figure 2The illustrated embodiment extracts pathological semantic features from image blocks. Compared to extracting the pathological slide image as a whole, this reduces the impact of limited receptive field, thus enabling the full extraction of signal features of lesions. This preserves the feature spatial distribution information of the pathological feature sub-images, providing sufficient feature information to support subsequent recognition operations and helping to improve the accuracy of pathological assessment results.

[0073] In one embodiment, multiple disease categories can be multiple grades under one grading mechanism for a single disease (e.g., villous component classification of colorectal adenoma, Gleason classification of prostate tumor), multiple grades under multiple grading mechanisms for a single disease (e.g., villous component classification of colorectal adenoma and high / low grade of tumor tissue), or multiple grades under the respective grading mechanisms of multiple diseases, etc., and the specifics are not limited here. For example, multiple disease categories may include high-grade colorectal adenoma and colorectal adenoma with specific villous component classification (e.g., tubular adenoma, villous tubular adenoma, villous adenoma). That is, if the pathological slide image shows pathological tissue of colorectal glands, the pathological evaluation method of this application embodiment can simultaneously evaluate the grade of the patient's colorectal adenoma and the specific villous component classification, thereby assisting medical personnel in performing more accurate clinical interventions.

[0074] In one embodiment, attention feature extraction for multiple disease categories refers to the operation of extracting attention information for each disease category from the pathological feature image to obtain an attention weight matrix. Here, the attention weight matrix is ​​a matrix containing attention weight information for each targeted disease category. It should be noted that when there are multiple attention weight matrices, each attention weight matrix contains attention weight information for all targeted disease categories. Furthermore, the attention weight information for one or more disease categories in each attention weight matrix can be the same or different; no specific limitation is made here.

[0075] In one embodiment, disease category-oriented adjustment refers to the operation of introducing attention information for each disease category into the pathological feature image through an attention weight matrix, thereby highlighting the pathological semantic features of each disease category in the pathological feature image. It is important to note that in this embodiment, the process of performing disease category-oriented adjustment on the entire pathological feature image is based on multiple attention weight matrices. That is, the pathological semantic features of each disease category in the pathological feature image are adjusted by introducing multiple attention weight information. It is also important to note that the attention information in disease category-oriented adjustment can be normalized using normalization functions such as softmax to reduce the impact on the stability of attention weight values ​​and suppress interference from irrelevant regions on the global representation. Alternatively, it can be used directly; no specific limitation is made here.

[0076] In one embodiment, in the process of performing disease category-oriented adjustment on the pathological feature image based on the extracted multiple attention weight matrices to obtain the disease category information expression matrix, specifically, for each disease category, the attention weight information of the disease category in the multiple attention weight matrices can be weighted and merged first. Then, the target attention weight information corresponding to each disease category is merged to obtain the target attention weight matrix. Then, the pathological feature image is adjusted according to the target attention weight matrix to obtain the disease category information expression matrix.

[0077] See Figure 3 In one embodiment, step 120 may include the following sub-steps.

[0078] Step 310: Extract attention features for multiple disease categories from the pathological feature image to obtain multiple attention weight matrices, wherein the attention weight matrix includes the disease category attention weight for each disease category;

[0079] Step 320: For each attention weight matrix, perform disease category-oriented adjustment on the pathological feature image according to the attention weight of each disease category in the attention weight matrix to obtain a disease category information expression submatrix;

[0080] Step 330: Perform matrix concatenation based on disease category to obtain the disease category information representation matrix from multiple disease category information representation sub-matrices.

[0081] In one embodiment, during the extraction of attention features for multiple disease categories from a pathological feature image, multiple convolutional kernels with different attention feature extraction capabilities can be used to extract attention features for multiple disease categories from the pathological feature image, resulting in multiple attention weight matrices. By using multiple convolutional kernels with different attention extraction capabilities to extract attention features for multiple disease categories, attention information for each disease category under different attention subspaces can be extracted from the pathological feature image. This allows attention responses for each disease category to be assigned to the pathological feature image from multiple attention perspectives, enriching the pathological semantic expression of each disease category. This helps improve the adaptability to the heterogeneous structure of lesions corresponding to each disease category during pathological assessment, thereby effectively improving the pathological assessment capability for each disease category.

[0082] In one embodiment, in the process of adjusting the pathological feature image according to the attention weight of each disease category in the attention weight matrix to obtain the disease category information expression sub-matrix, specifically, for each disease category, the pathological feature image is first weighted according to the disease category attention weight to generate a disease category information expression feature vector. Then, the disease category information expression feature vectors of each disease category are concatenated to form the disease category information expression sub-matrix. By applying disease category-oriented weighting to the pathological feature image according to the disease category attention weight in an attention weight matrix, the attention information of each disease category can be summarized into the pathological feature image, strengthening the pathological semantic feature expression of each disease category, while suppressing the expression of irrelevant semantic feature information in the pathological feature image, effectively filtering out interference information in the pathological feature image.

[0083] For example, suppose the size of the attention weight matrix is ​​H1×W1×C (where C represents the specific number of disease categories), and the size of the pathological feature image is H1×W1×L1. In the process of obtaining the disease category information representation submatrix, the pathological feature image and the attention weight matrix are weighted element-wise along the spatial dimension of the disease category to obtain a category-specific weighted feature map for each disease category. The size of the category-specific weighted feature map is H1×W1×L1. Then, the category-specific weighted feature map is summed along the spatial dimension of the disease category to obtain the disease category information representation feature vector for each disease category. The disease category information representation feature vector is an L1-dimensional vector. The disease category information representation feature vectors of multiple disease categories are concatenated to obtain a C×L1-dimensional disease category information representation submatrix.

[0084] In one embodiment, in the process of concatenating multiple disease category information expression sub-matrices based on disease categories to obtain a disease category information expression matrix, the multiple disease category information expression sub-matrices can be concatenated along the spatial dimension where the disease categories are located to obtain the disease category information expression matrix.

[0085] For example, suppose the size of the disease category information representation submatrix is ​​C×L1 (the size of C represents the specific number of disease categories). In the process of splicing the disease category information representation submatrix, the disease category information representation submatrix is ​​spliced ​​along the spatial dimension where the disease categories are located to obtain a C×(L1×N) dimensional disease category information representation matrix.

[0086] exist Figure 3In this embodiment, a disease category-guided weighting of the pathological feature image is applied using an attention matrix. This allows for the separate aggregation of attention weight information for each disease category within an attention subspace. By concatenating multiple disease category information expression submatrices, the attention weight information for each disease category in different attention subspaces can be integrated. This enables the disease category information expression matrix to encode whether each disease category appears in the pathological slide image and to include the existence patterns of each disease category in different semantic dimensions. Consequently, this enriches and enhances the expression of the pathological semantic features of each disease category, facilitating a more granular pathological assessment of each disease category.

[0087] In one embodiment, density feature extraction for multiple disease categories refers to the operation of extracting density feature information for each disease category from the attention weight matrix to obtain a density feature tensor. Here, density feature information refers to the semantic expression information corresponding to the density proportion of cell tissue corresponding to a disease category on the pathological tissue section displayed in the pathological slide image. Furthermore, the density feature tensor is an information structure containing density feature information for each disease category. It should be noted that when there are multiple density feature tensors, each density feature tensor contains density feature information for all the targeted disease categories. The density feature information for one or more disease categories in each density feature tensor can be the same or different; no specific limitation is made here.

[0088] See Figure 4 In one embodiment, step 130 may include the following sub-steps.

[0089] Step 410: For each attention weight matrix, extract density features for multiple disease categories, and merge the density feature maps of each disease category to obtain a density feature tensor;

[0090] Step 420: Merge the density feature tensors of each attention weight matrix into a density feature information representation matrix.

[0091] In one embodiment, during the process of extracting density features for multiple disease categories from the attention weight matrix and merging the resulting density feature maps for each disease category to obtain a density feature tensor, the process can be as follows: first, density features are extracted from the attention weights for each disease category to obtain a density feature map for that disease category; then, the density feature maps for each disease category are merged based on the disease category to obtain a density feature tensor. Here, the density feature map refers to the image used to represent the local spatial response intensity distribution of its corresponding disease category in the pathological slide image.

[0092] In one embodiment, during the process of extracting density features from the attention weights of disease categories to obtain a density feature map of the disease category, the density features of the attention weights of the disease category can be extracted to obtain density feature information. Then, the density feature information is pooled to generate a density feature map. The pooling process can use methods such as adaptive pooling, and is not specifically limited here.

[0093] In one embodiment, merging the density feature maps of each disease category to obtain a density feature tensor refers to the operation of merging the density feature maps of each disease category along the spatial dimension of the disease category to obtain the density feature tensor. For example, assuming the size of the density feature map of each disease category is H2×W2, merging it along the spatial dimension of the disease category results in a density feature tensor with a size of H2×W2×C (where C represents the specific number of disease categories).

[0094] In the process of extracting density features for multiple disease categories from the attention weight matrix, density features are extracted separately for the attention weights of each disease category. This achieves channel decoupling of the attention weight matrix for each disease category, allowing the attention weight information of each disease category to be input as a separate channel. This enables density feature extraction for each disease category independently. This process maintains the spatial continuity and semantic consistency of the density feature information of each disease category while enhancing the perception of density signals for each disease category. Consequently, density feature extraction becomes more accurate, contributing to improved accuracy in fine-grained pathological assessment.

[0095] It is important to note that density feature maps for each disease category can be directly output to display the density proportion of the disease category in pathological tissue. These density feature maps can be output in the form of heatmaps, staining maps, etc.

[0096] It should be noted that there are various ways to merge the density feature tensors of each attention weight matrix into a density feature information representation matrix. For example, one could concatenate the density feature tensors of each attention weight matrix, or add the density feature tensors of each attention weight matrix according to certain weights, etc. No specific limit is set here.

[0097] In one embodiment, during the process of merging the density feature tensors of each attention weight matrix into a density feature information representation matrix, the density feature tensors of each attention weight matrix can be concatenated along the spatial dimension corresponding to the disease category to obtain the density feature information representation matrix. During the concatenation along the spatial dimension corresponding to the disease category, no processing may be performed on the density feature tensors of each attention weight matrix, or dimensionality reduction processing may be performed, etc., without specific limitations.

[0098] In one embodiment, before concatenating the density feature tensors of each attention weight matrix along the spatial dimension corresponding to the disease category, the density feature tensors of each attention weight matrix can be flattened along the spatial dimension corresponding to the disease category to obtain the density feature vector corresponding to each attention weight matrix. During the concatenation of the density feature tensors of each attention weight matrix along the spatial dimension corresponding to the disease category, the density feature vectors corresponding to each attention weight matrix are also concatenated along the spatial dimension corresponding to the disease category to obtain the density feature information representation matrix.

[0099] For example, assuming the size of the density feature map for each disease category is H²×W², then for each attention weight matrix, C density feature maps of size H²×W² can be extracted. These C density feature maps are then concatenated along the spatial dimension corresponding to the disease category to form a density feature tensor of size H²×W²×C to be reduced in dimension. Correspondingly, N attention weight matrices will have N density feature tensors of size H²×W²×C to be reduced in dimension. For each attention weight matrix, the density feature tensor to be reduced in dimension corresponding to the attention weight matrix is ​​flattened to obtain a density feature vector of size (H²*W²)×C. Then, the density feature vectors of each attention weight matrix are concatenated based on the spatial dimension corresponding to the disease category to obtain a density feature information representation matrix of size (H²*W²*N)×C.

[0100] In one embodiment, during the pathological evaluation of each disease category based on the density feature information expression matrix and the disease category information expression matrix, the density feature information expression matrix and the disease category information expression matrix can be merged into a target information expression matrix. Then, a classification probability analysis is performed on the target information expression matrix for multiple disease categories to obtain the probabilities of each disease category appearing on the pathological tissue in the pathological slide image. Based on the probabilities of each disease category, the target pathological evaluation result for each disease category is determined. By merging the density feature information expression matrix and the disease category information expression matrix, density feature information can be integrated into the already enhanced pathological semantic feature information of each disease category. This allows the pathological semantic feature information of each disease category to possess density feature attributes. In this way, the classification probability analysis process can take into account the density information of the pathological tissue, thereby enabling a more granular pathological evaluation of multiple disease categories.

[0101] It's important to note that the target pathological assessment results for each disease category can be output as text, via a mask, or other methods, without specific limitations here. For example, assuming the target pathological assessment result for a disease category indicates the presence of pathological tissue of that disease category, the location of the pathological tissue in the pathological slide image is determined based on the feature information of that disease category in the target information representation matrix. Then, this location is masked (e.g., highlighted, outlined with a border), and the masked pathological slide image is output.

[0102] In one embodiment, classification probability analysis for multiple disease categories can be performed using a pre-trained classifier, which can be a multilayer perceptron or a Transformer encoder structure, without being specifically limited here.

[0103] See Figure 5 The pathological evaluation method in this application embodiment can be based on Figure 5 The neural network model shown is used. The neural network model includes a preprocessing and image segmentation module, a backbone feature extraction network, a pathological semantic feature summarization module, an attention feature extraction module for multiple disease categories, a disease category-guided adjustment module, a density feature extraction module, and a classifier. Pathological slide images are input. Figure 5 Following the neural network model shown, the preprocessing and image segmentation module first preprocesses the pathological slide image and segments it into multiple pathological slide image blocks (i.e., Figure 5The process involves several steps: first, inputting multiple pathological slide image patches into the backbone feature extraction network; second, extracting pathological semantic features from each pathological slide image patch to obtain a corresponding pathological feature sub-image; and third, reconstructing the image using the pathological semantic feature aggregation module to obtain the pathological feature image. Finally, the pathological feature image is input into the attention feature extraction module, which extracts the pathological features through each attention feature extraction convolutional kernel (assuming it is...). Figure 5 The module extracts attention features for various disease categories from the pathological feature image using N attention weight matrices, resulting in N attention weight matrices. The disease category-oriented adjustment module then uses these N attention weight matrices to adjust the pathological feature image for disease category orientation, obtaining a disease category information representation matrix. Simultaneously, the attention feature extraction module outputs the obtained N attention weight matrices to the density feature extraction module. For each attention weight matrix, the density feature extraction module extracts density features to obtain a density feature map for each disease category, concatenating these maps into a density feature tensor. Next, the density feature extraction module flattens the density feature tensor corresponding to each attention weight matrix to obtain a density feature vector, concatenating these vectors into a density feature information representation matrix. Finally, the disease category information representation matrix and the density feature information representation matrix are concatenated to obtain the target information representation matrix. Finally, the classification probability analysis of each disease category is performed on the target information expression matrix by a classifier to obtain the probabilities of each disease category appearing on the pathological tissue in the pathological slide image, and the target pathological evaluation results of each disease category are output according to the probabilities of each disease category.

[0104] See Figure 6 In one embodiment, step 140 may include the following sub-steps.

[0105] Step 510: Perform pathological evaluation based on the density feature information expression matrix and the disease category information expression matrix to obtain preliminary pathological evaluation results for each disease category;

[0106] Step 520: Evaluate the visibility of disease categories based on the density feature information expression matrix to obtain the visibility evaluation results for each disease category;

[0107] Step 530: For each disease category, in response to the preliminary pathological assessment result of the disease category indicating the presence of pathological tissue of the disease category on the pathological slide image, the preliminary pathological assessment result of the disease category is corrected according to the visibility assessment result of the disease category to obtain the target pathological assessment result of the disease category.

[0108] In one embodiment, in the process of obtaining preliminary pathological assessment results for each disease category by performing pathological assessment based on the density feature information expression matrix and the disease category information expression matrix, the density feature information expression matrix and the disease category information expression matrix can be merged into a target information expression matrix first. Then, a classification probability analysis for multiple disease categories is performed based on the target information expression matrix to obtain the probabilities of each disease category appearing on the pathological tissue in the pathological slide image. The preliminary pathological assessment results for each disease category are determined based on the probabilities of each disease category.

[0109] In one embodiment, the visibility assessment result of a disease category refers to information used to represent the distribution ratio of pathological tissue of that disease category on the pathological tissue displayed on the pathological slide image.

[0110] In one embodiment, the process of correcting the preliminary pathological assessment result of a disease category based on the manifestation degree assessment result of the disease category can be done by directly correcting the preliminary pathological assessment result of the disease category based on the manifestation degree assessment result of the disease category, or by sorting and subtracting the manifestation degree assessment result of the disease category from the manifestation degree assessment results of other disease categories, and then correcting the preliminary pathological assessment result of the disease category based on the processing result, etc., and so on. The specific method is not limited here.

[0111] In one embodiment, for each disease category, in response to a preliminary pathological assessment result for the disease category not indicating the presence of pathological tissue of the disease category on the pathological slide image, the preliminary pathological assessment result is determined as the target pathological assessment result.

[0112] See Figure 7 , Figure 6 The embodiments are based on Figure 7 The neural network model shown is used for... Figure 7 Neural network models and Figure 5 Compared to the neural network model, this model includes an additional disease category visibility assessment module. This module analyzes the density feature information expression matrix to obtain the visibility assessment result for each disease category. For each disease category, if the preliminary pathological assessment result indicates the presence of pathological tissue of that disease category in the pathological slide image, the visibility assessment result of the disease category is used to correct the preliminary pathological assessment result, thus obtaining the target pathological assessment result.

[0113] It is important to note that the hyperparameters of the disease category visibility assessment module can be adjusted during training using the Hinge Ranking Loss function, as shown in the following formula:

[0114] Loss(x1,x2,y)=max(0,-y*(x1-x2)+margin),

[0115] Where x1 can represent the score of the positive sample;

[0116] x2 can represent the score of the negative sample;

[0117] y can represent a label, and y is usually +1 (meaning x1 should come before x2) or -1 (meaning x2 should come before x1).

[0118] The margin can represent the minimum desired difference between positive and negative samples, and is usually set to 1.

[0119] It should be noted that the output method of the target pathological assessment results can be the same as in the previous embodiment. For example, it can be output in the form of text or in the form of a mask, etc. There is no specific limitation here.

[0120] pass Figure 6 The illustrated embodiment enables the pathological assessment process to determine whether pathological tissues of each disease category appear on the displayed pathological slides based on pathological semantic feature information of each disease category with density feature attributes, and to correct the preliminary judgment results using the corresponding degree of visibility. This makes the target pathological assessment results of each disease category more interpretable and accurate, and also makes the pathological assessment process closer to the actual clinical pathological assessment process.

[0121] The following examples illustrate some practical implementations of step 530.

[0122] Example 1: A scenario in clinical diagnosis where the presence and significance of lesions must be met. For example, in determining high-grade colorectal adenomas, during clinical pathological evaluation, it is necessary to determine whether the pathological tissue section is indeed a high-grade colorectal adenoma based on the morphology and distribution density of the high-grade colorectal adenoma tissue. For the disease category of high-grade colorectal adenoma, assuming the manifestation probability corresponding to this disease category is Y_pred[c1], when Y_pred[c1] > τ (preset manifestation threshold), the preliminary pathological evaluation result indicates the presence of a high-grade adenoma in the pathological tissue section. At this point, the manifestation degree evaluation result of this disease category, ranking_score[c1], needs to intervene for correction. When ranking_score[c1] > η (preset manifestation degree threshold), it is finally determined that a high-grade adenoma exists in the pathological tissue section; when ranking_score[c1] ≤ η1, it is finally determined that a low-grade adenoma or atypical hyperplastic adenoma exists in the pathological tissue section.

[0123] Example 2: A structured diagnostic scenario in clinical diagnosis relying on the combination of lesion primary and secondary rankings. For example, the Gleason grading of the prostate. In actual clinical practice, the specific Gleason grading of the prostate represents different malignant states of prostate tumor tissue. Assuming the probability of a disease category with a Gleason score of 3 is Y_pred[c1], the probability of a disease category with a Gleason score of 4 is Y_pred[c2], and the probability of a disease category with a Gleason score of 5 is Y_pred[c3], when Y_pred[c1] > τ (preset display threshold), it is determined whether the display degree ranking_score[c1] of prostate tumor tissue with a Gleason score of 3 is greater than η1 (preset display degree threshold). If yes, then prostate tumor tissue with a Gleason score of 3 exists in the pathological tissue section; otherwise, prostate tumor tissue with a Gleason score of 3 does not exist in the pathological tissue section. The classification results of other Gleason scores are similar and will not be elaborated here. It is important to note that in actual clinical practice, a single prostate tumor tissue sample may contain prostate tumors of various malignant stages. To ensure the clinical readability of the target pathological assessment results for different Gleason scores, the prostate tumors can be ranked according to the degree of visibility of the identified malignant stages. This allows pathologists to quickly understand the proportion of prostate tumors with different Gleason scores in the current tissue sample. For example, ranking_score[c1] > ranking_score[c3] > ranking_score[c2] indicates that the current prostate tumor tissue sample contains the most prostate tumors with a Gleason score of 3, followed by those with a Gleason score of 5, and the fewest with a Gleason score of 4.

[0124] Example 3: A scenario in clinical diagnosis where the disease has an internal ranking structure. For example, in the villous component classification of colorectal adenomas, in actual clinical practice, the villous component classification of colorectal adenomas includes tubular adenomas, villous tubular adenomas, and villous adenomas. Because villous tubular adenomas have the morphological characteristics of both tubular and villous adenomas, it may lead to the assessment of both tubular and villous adenomas. In this case, ranking_score can be used to correct the discrepancy. Assuming the disease category manifestation probability of tubular adenoma is Y_pred[c1] and the disease category manifestation probability of villous adenoma is Y_pred[c2], when Y_pred[c1] > τ (preset manifestation threshold) and Y_pred[c2] > τ, determine whether abs(ranking_score[c1] - ranking_score[c2]) is less than η (preset manifestation threshold). If yes, it is determined that villous tubular adenoma exists in the current colorectal adenoma tissue slice. If no, determine the size between ranking_score[c1] and ranking_score[c2]. If ranking_score[c1] > ranking_score[c2], it indicates that tubular adenoma tissue exists in the colorectal adenoma tissue slice. If ranking_score[c1] < ranking_score[c2], it indicates that villous adenoma tissue exists in the colorectal adenoma tissue slice.

[0125] Example 4: Scenario of outputting fuzzy disease levels in clinical diagnosis. For example, based on the Ishak scoring system for liver cirrhosis, assuming the disease category manifestation probabilities corresponding to Ishak scores 1 to 6 are Y_pred[c1] to Y_pred[c6], when Y_pred[c1] > τ (preset manifestation threshold), the liver cirrhosis level is determined to be stage 1; when Y_pred[c2] > τ, the liver cirrhosis level is determined to be stage 2, and so on. When the disease category manifestation probabilities of two or more Ishak scores are both greater than τ, the ranking_score corresponding to the highest Ishak score is determined. It is then determined whether the difference between the ranking_scores of these two Ishak scores is less than η (preset manifestation threshold). If so, the output result is that the liver cirrhosis level is between these two levels; otherwise, the liver cirrhosis level corresponding to the highest Ishak score is output.

[0126] See Figure 8 The second aspect of this application provides a pathological assessment device based on attention mechanisms and density feature extraction, which can implement the pathological assessment method of the first aspect described above. The pathological assessment device 800 includes:

[0127] The pathological semantic feature extraction module 810 can be used to extract pathological semantic features from pathological slide images to obtain pathological feature images.

[0128] The disease category-oriented adjustment module 820 can be used to extract attention features of multiple disease categories from pathological feature images, and to perform disease category-oriented adjustment on the pathological feature images based on the extracted multiple attention weight matrices to obtain a disease category information expression matrix.

[0129] The density feature extraction module 830 can be used to extract density features for various disease categories from each attention weight matrix, and merge the density feature tensors extracted from each attention weight matrix into a density feature information expression matrix.

[0130] The pathological assessment module 840 can be used to perform pathological assessments on each disease category based on the density feature information expression matrix and the disease category information expression matrix.

[0131] The specific implementation of this pathological assessment device is basically the same as the specific implementation of the pathological assessment method described above, and will not be repeated here.

[0132] A third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described pathological assessment method based on attention mechanism and density feature extraction.

[0133] Please see Figure 9 , Figure 9 The hardware structure of an electronic device 900, according to another embodiment, is illustrated. The electronic device 900 includes:

[0134] The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0135] The memory 902 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the pathological assessment method based on attention mechanism and density feature extraction of the embodiments of this application.

[0136] The input / output interface 903 is used to implement information input and output;

[0137] The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0138] Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904);

[0139] The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0140] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned pathological assessment method based on attention mechanisms and density feature extraction.

[0141] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0143] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0145] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0146] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0147] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0148] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0149] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0151] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A pathological assessment method based on attention mechanisms and density feature extraction, characterized in that, The method includes: Pathological semantic features are extracted from pathological slide images to obtain pathological feature images; Attention features of multiple disease categories are extracted from the pathological feature image to obtain multiple attention weight matrices, wherein the attention weight matrix includes the disease category attention weight for each of the disease categories; For each attention weight matrix, the pathological feature image is adjusted according to the attention weight of each disease category in the attention weight matrix to obtain a disease category information expression submatrix; Multiple disease category information representation sub-matrices are concatenated based on disease category to obtain a disease category information representation matrix; Density features of various disease categories are extracted from each of the attention weight matrices, and the density feature tensors extracted from each of the attention weight matrices are merged into a density feature information representation matrix. Pathological evaluation is performed based on the density feature information expression matrix and the disease category information expression matrix to obtain preliminary pathological evaluation results for each disease category; The visibility of disease categories is evaluated based on the density feature information expression matrix, and the visibility evaluation results for each disease category are obtained. For each disease category, in response to the preliminary pathological assessment result of the disease category indicating the presence of pathological tissue of the disease category on the pathological slide image, the preliminary pathological assessment result of the disease category is corrected according to the visibility assessment result of the disease category to obtain the target pathological assessment result of the disease category.

2. The method according to claim 1, characterized in that, The extraction of pathological semantic features from pathological slide images to obtain pathological feature images includes: The pathological slide images are preprocessed to obtain multiple pathological slide image blocks; Pathological semantic features are extracted from multiple pathological slice image blocks to obtain multiple pathological feature sub-images; The pathological feature image is obtained by reconstructing the feature image based on multiple pathological feature sub-images.

3. The method according to claim 1, characterized in that, The step of performing disease category-guided adjustment on the pathological feature image based on the attention weight of each disease category in the attention weight matrix to obtain a disease category information representation sub-matrix includes: For each disease category, the pathological feature image is subjected to disease category-guided weighting based on the disease category attention weight to generate a disease category information expression feature vector for that disease category. The disease category information expression feature vectors of each disease category are concatenated to form the disease category information expression submatrix.

4. The method according to claim 1, characterized in that, The step of extracting density features for various disease categories from each of the attention weight matrices and merging the density feature tensors extracted from each of the attention weight matrices into a density feature information representation matrix includes: For each attention weight matrix, density features of multiple disease categories are extracted from the attention weight matrix, and the density feature maps of each disease category are merged to obtain the density feature tensor; The density feature tensors of each attention weight matrix are merged into the density feature information representation matrix.

5. The method according to claim 4, characterized in that, The process of extracting density features for multiple disease categories from the attention weight matrix and merging the resulting density feature maps for each disease category to obtain the density feature tensor includes: For each of the disease categories, density features are extracted from the attention weights of the disease categories to obtain the density feature map of the disease category; The density feature maps for each disease category are merged based on the disease category to obtain the density feature tensor.

6. A pathological assessment device based on attention mechanisms and density feature extraction, characterized in that, include: The pathological semantic feature extraction module is used to extract pathological semantic features from pathological slide images to obtain pathological feature images. The disease category-oriented adjustment module is used to extract attention features of multiple disease categories from the pathological feature image to obtain multiple attention weight matrices, wherein the attention weight matrix includes a disease category attention weight for each disease category; for each attention weight matrix, the pathological feature image is adjusted according to the attention weight of each disease category in the attention weight matrix to obtain a disease category information expression sub-matrix; the multiple disease category information expression sub-matrices are concatenated based on disease categories to obtain a disease category information expression matrix; The density feature extraction module extracts density features for various disease categories from each of the attention weight matrices, and merges the density feature tensors extracted from each of the attention weight matrices into a density feature information representation matrix. The pathological assessment module is used to perform pathological assessment based on the density feature information expression matrix and the disease category information expression matrix to obtain a preliminary pathological assessment result for each disease category; to perform a disease category visibility assessment based on the density feature information expression matrix to obtain a visibility assessment result for each disease category; for each disease category, in response to the preliminary pathological assessment result indicating the presence of pathological tissue of the disease category on the pathological slide image, the module corrects the preliminary pathological assessment result of the disease category based on the visibility assessment result of the disease category to obtain a target pathological assessment result for the disease category.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images