Gleason grading method and device for prostatic cancer pathological image, equipment and medium

Abstract

The embodiment of the invention provides a Grignson grading method and device for a prostate cancer pathological image, equipment and a medium. The method comprises the following steps: acquiring a prostate cancer pathological image which is a full-view digital slice image; dividing a non-blank area of the prostatic cancer pathological image into a plurality of non-overlapping examples; converting the instance into a high-dimensional feature vector; the high-dimensional feature vector is input to a pre-trained multi-branch hybrid supervised network model, slice-level representation is obtained, the multi-branch hybrid supervised network model comprises an attention backbone network, instance-level branches and slice-level branches, the slice-level branches comprise slice-level benign branches and slice-level malignant branches, and slice-level representation is obtained; the slice-level malignant branch comprises an instance compression module and a component extraction module; and inputting the slice level representation into a classifier to obtain a Grignson classification result of the prostatic cancer pathological image. Based on this, the embodiment of the invention can improve the accuracy of Gleason grading of the prostatic cancer pathological image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of medical image processing, and in particular, to a Gleason grading method, device, equipment and medium for prostate cancer pathological images. Background Art

[0002] Prostate cancer is the second most common cancer among men globally, with over 1 million diagnosis reports each year. Pathologists observe whole slide images (WSIs) and perform Gleason grading based on the structural growth patterns of tumors. However, there are significant inter-observer differences among pathologists, and inaccurate grading may lead to unnecessary treatments or missed serious diagnoses. As the basis and gold standard for tumor diagnosis, the accuracy of pathological image grading is crucial for formulating treatment plans and evaluating prognoses.

[0003] Based on multi-instance learning, the current mainstream Gleason grading methods mainly rely on the weakly supervised strategy of slide-level labels. However, due to the weakly supervised method that only uses slide-level labels, there is a lack of supervision information in gigabyte-scale WSIs, which often leads to a significant decline in performance and thus affects the accuracy of Gleason grading. Therefore, how to improve the accuracy of Gleason grading for prostate cancer pathological images has become an urgent technical problem to be solved. Summary of the Invention

[0004] The embodiments of the present invention provide a Gleason grading method, device, equipment and medium for prostate cancer pathological images, which can improve the accuracy of Gleason grading for prostate cancer pathological images.

[0005] In a first aspect, the embodiments of the present invention provide a Gleason grading method for prostate cancer pathological images, including:

[0006] Obtain a prostate cancer pathological image, where the prostate cancer pathological image is a whole slide image;

[0007] Divide the non-blank area of the prostate cancer pathological image into multiple non-overlapping instances;

[0008] Convert the instance into a high-dimensional feature vector;

[0009] Input the high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slide-level representation, where the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch and a slide-level branch, the slide-level branch includes a slide-level benign branch and a slide-level malignant branch, and the slide-level malignant branch includes an instance compression module and a component extraction module;

[0010] Input the slice-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathological image.

[0011] In some embodiments, the training method of the multi-branch hybrid supervision network model includes:

[0012] Obtain the slice-level labels of the prostate cancer pathological images in the training set;

[0013] Assign labels to each instance according to the cancer focus annotation of the prostate cancer pathological images in the training set to obtain instance-level labels;

[0014] Jointly supervise the learning of the attention backbone network with the instance-level labels and the slice-level labels, and calculate the instance-level loss and the slice-level loss;

[0015] Determine the overall loss function according to the instance-level loss and the slice-level loss;

[0016] Train the multi-branch hybrid supervision network model based on the overall loss function to obtain the trained multi-branch hybrid supervision network model.

[0017] In some embodiments, the calculation method of the instance-level loss includes:

[0018] Take the attention score of the instance to the slice-level malignant branch as the output of the instance-level branch;

[0019] Calculate the instance-level loss based on the output of the instance-level branch using the binary cross-entropy loss function.

[0020] In some embodiments, the calculation method of the slice-level loss includes:

[0021] Aggregate the attention scores of the instance and the instance to the slice-level benign branch as the slice-level representation;

[0022] Calculate the predicted probability of the benign class of the slice-level representation through the slice-level classifier;

[0023] After processing the outputs of all slice-level classifiers through the Softmax layer, obtain the probability distribution of different classes of the prostate cancer pathological image;

[0024] Calculate the slice-level loss based on the probability distribution.

[0025] In some embodiments, the step of inputting the high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slice-level representation includes:

[0026] Input the high-dimensional feature vector into the slice-level benign branch and the slice-level malignant branch respectively;

[0027] Obtain all instance features in the slice-level benign branch;

[0028] In the slice-level malignant branch, obtain target instances that contribute significantly to the slice-level malignant branch and determine the contribution degree of the target instances;

[0029] Calculate the attention score according to the contribution degree;

[0030] Map the attention score to the corresponding spatial position of the prostate cancer pathological image to obtain the slice-level representation.

[0031] In some embodiments, the instance compression module masks the target instances according to the attention score, and only retains the target instances with higher attention scores as the input of the slice-level malignant branch; reconstructs the masked target instances, and further extracts information at different scales using three different sizes of convolutional kernels, and flattens them after fusion as the input of the component extraction module.

[0032] In some embodiments, the component extraction module performs two-dimensional spatial position encoding on the compressed instances, and splices a learnable class token as the input of the Transformer, and uses the multi-head self-attention mechanism to learn the correlation between the remaining instances.

[0033] In a second aspect, an embodiment of the present invention further provides a Gleason grading device for prostate cancer pathological images, the device includes:

[0034] An acquisition module, configured to acquire a prostate cancer pathological image, where the prostate cancer pathological image is a whole-slide digital section image;

[0035] A division module, configured to divide the non-blank area of the prostate cancer pathological image into a plurality of non-overlapping instances;

[0036] A conversion module, configured to convert the instance into a high-dimensional feature vector;

[0037] A processing module, configured to input the high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slice-level representation, where the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch, and a slice-level branch, the slice-level branch includes a slice-level benign branch and a slice-level malignant branch, and the slice-level malignant branch includes an instance compression module and a component extraction module;

[0038] A grading module, configured to input the slice-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathological image.

[0039] In a third aspect, an embodiment of the present invention further provides an electronic device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the Gleason grading method for prostate cancer pathological images as described in the first aspect is implemented.

[0040] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for executing the Gleason grading method for prostate cancer pathological images as described in the first aspect.

[0041] According to the Gleason grading method, device, equipment, and medium for prostate cancer pathological images provided by the embodiments of the present invention, wherein the Gleason grading method for prostate cancer pathological images includes: obtaining a prostate cancer pathological image, which is a whole-slide digital section image; dividing the non-blank area of the prostate cancer pathological image into multiple non-overlapping instances; converting the instances into high-dimensional feature vectors; inputting the high-dimensional feature vectors into a pre-trained multi-branch hybrid supervision network model to obtain a slide-level representation, wherein the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch, and a slide-level branch, the slide-level branch includes a slide-level benign branch and a slide-level malignant branch, and the slide-level malignant branch includes an instance compression module and a component extraction module; inputting the slide-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathological image. The present invention aims to use the rich supervision information contained in the fine-grained instance-level labels to assist the model in performing more accurate Gleason grading by converting the limited pixel-level labels into instance-level labels and using the instance-level labels and slide-level labels to jointly supervise the model learning. Secondly, a multi-branch structure is constructed according to clinical practice experience to perform specific processing on different Gleason grades, aiming to reduce the consumption of computing resources while retaining the integrity of the cancer focus information to a greater extent, thereby improving the accuracy of the malignant grade. Based on this, the embodiments of the present invention can improve the accuracy of Gleason grading for prostate cancer pathological images. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flowchart of the Gleason grading method for prostate cancer pathological images provided by an embodiment of the present invention;

[0043] Figure 2 is a diagram of the WSI preprocessing process provided by an embodiment of the present invention;

[0044] Figure 3 is a diagram of the WSIs preprocessing process provided by an embodiment of the present invention;

[0045] Figure 4 is an overall structural schematic diagram of the multi-branch hybrid supervision network structure provided by an embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0047] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0048] It should be noted that although functional module division is performed in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different module division in the device or a different order in the flowchart. Terms such as "first" and "second" in the description, claims and the following drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence.

[0049] In the embodiments of the present invention, words such as "furthermore", "exemplarily" or "optionally" are used to represent examples, illustrations or explanations, and should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Using words such as "furthermore", "exemplarily" or "optionally" is intended to present related concepts in a specific manner.

[0050] In order to more conveniently describe the working principle of the embodiments of the present invention later, the following first gives an introduction to the related technical scenarios.

[0051] Prostate cancer is the second most common cancer among men globally, with more than 1 million diagnostic reports each year. Pathologists observe whole slide images (WSIs) and perform Gleason grading based on the structural growth pattern of tumors. However, there are significant inter-observer differences among pathologists, and inaccurate grading may lead to unnecessary treatments or missed serious diagnoses. As the basis and gold standard for tumor diagnosis, the accuracy of pathological image grading is crucial for formulating treatment plans and evaluating prognoses.

[0052] Based on multi-instance learning, the current mainstream Gleason grading methods mainly rely on the weakly supervised strategy of slide-level labels. However, due to the weakly supervised method that only uses slide-level labels, there is a lack of supervision information in gigabyte-scale WSIs, which often leads to a significant decline in performance, thus affecting the accuracy of Gleason grading. Therefore, how to improve the accuracy of Gleason grading for prostate cancer pathological images has become a technical problem to be solved urgently.

[0053] Based on this, the present invention provides a Gleason grading method and apparatus, device, and medium for prostate cancer pathological images. The Gleason grading method for prostate cancer pathological images includes: obtaining a prostate cancer pathological image, where the prostate cancer pathological image is a whole-slide digital section image; dividing the non-blank area of the prostate cancer pathological image into multiple non-overlapping instances; converting the instances into high-dimensional feature vectors; inputting the high-dimensional feature vectors into a pre-trained multi-branch hybrid supervision network model to obtain a slide-level representation, where the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch, and a slide-level branch, the slide-level branch includes a slide-level benign branch and a slide-level malignant branch, and the slide-level malignant branch includes an instance compression module and a component extraction module; inputting the slide-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathological image. The present invention aims to use the rich supervision information contained in the fine-grained instance-level labels to assist the model in performing more accurate Gleason grading by converting limited pixel-level labels into instance-level labels and using the instance-level labels and slide-level labels to jointly supervise model learning. Secondly, a multi-branch structure is constructed according to clinical practice experience to perform specific processing on different Gleason grades, aiming to reduce the consumption of computing resources while retaining the integrity of cancer lesion information to a greater extent, thereby improving the accuracy of the malignant grade. Based on this, the embodiments of the present invention can improve the accuracy of Gleason grading for prostate cancer pathological images.

[0054] The following further elaborates on the embodiments of the present invention in conjunction with the accompanying drawings.

[0055] As Figure 1 shown, Figure 1 FIG. 10 is a flowchart of a Gleason grading method for prostate cancer pathological images provided by an embodiment of the present invention. The Gleason grading method for prostate cancer pathological images may include, but is not limited to, steps S101 to S105.

[0056] Step S101, obtaining a prostate cancer pathological image, where the prostate cancer pathological image is a whole-slide digital section image;

[0057] Step S102, dividing the non-blank area of the prostate cancer pathological image into multiple non-overlapping instances;

[0058] Step S103, converting the instances into high-dimensional feature vectors;

[0059] Step S104, inputting the high-dimensional feature vectors into a pre-trained multi-branch hybrid supervision network model to obtain a slide-level representation, where the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch, and a slide-level branch, the slide-level branch includes a slide-level benign branch and a slide-level malignant branch, and the slide-level malignant branch includes an instance compression module and a component extraction module;

[0060] Step S105: Input the slice-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathological image.

[0061] It can be understood that the present invention proposes a multi-branch hybrid supervision method based on multi-instance learning for Gleason grading of prostate cancer pathological sections. Specifically, the objectives of the present invention include:

[0062] (1) Construction of a multi-branch hybrid supervision network structure: Through an attention backbone network with weight sharing, make full use of instance-level labels to enable the model to have sufficient supervision information, and be able to perform more fine-grained operations on different Gleason grades in the grading task, thereby improving the accuracy of grading.

[0063] (2) Construction of an instance compression module and a component extraction module: Design an instance compression module. By performing masking operations on irrelevant instances, only retain the instances that contribute more to the slice-level malignancy grade, aiming to reduce the model burden while retaining the integrity of the cancer foci to a greater extent. On this basis, design a component extraction module. After reconstructing the position encoding of relevant instances, use them as the input of the Transformer, and utilize the multi-head self-attention mechanism to learn the global information of relevant instances, aiming to further refine the component information of the cancer foci area in the slice.

[0064] By combining these two modules, namely the instance compression module and the component extraction module, the present invention can achieve more accurate Gleason grading of prostate cancer pathological sections.

[0065] It can be understood that the present invention constructs a multi-branch hybrid supervision method based on multi-instance learning for Gleason grading of prostate cancer pathological sections. First, convert the limited pixel-level labels into instance-level labels, and use the instance-level labels and slice-level labels to jointly supervise the model learning, aiming to use the rich supervision information contained in the fine-grained instance-level labels to assist the model in performing more accurate Gleason grading. Second, construct a multi-branch structure according to clinical practice experience to perform specific processing on different Gleason grades, aiming to reduce the consumption of computing resources while retaining the integrity of the cancer foci information to a greater extent, thereby improving the accuracy of the malignancy grade. The present invention shows good potential in achieving accurate grading of prostate cancer. The generated visualization results can provide positioning information, reduce the cost for doctors to observe the sections, and a more accurate Gleason grading method can provide effective second opinions for doctors, assisting doctors in making more accurate judgments and reasonable treatment plans.

[0066] It can be understood that the present invention first preprocesses the WSI, such as Figure 2As shown, the non - blank areas of the WSI are divided into multiple non - overlapping instances, and each instance is assigned a label according to the cancer foci annotation. After these instances are converted into high - dimensional feature vectors by the feature extractor, they are used as the input of the model. In the overall structure of the multi - branch hybrid supervision network model, as Figure 3 shown, the model jointly uses instance - level labels and slice - level labels to supervise the learning of the attention backbone network, and uses the rich instance - level supervision information to improve the prediction accuracy of the model at the slice level. In the slice - level benign branch, all instance features are considered, while in the slice - level malignant branch, only the instances that contribute significantly to the slice - level malignant branch are concerned. The attention scores are calculated according to their contribution degrees, and these scores are mapped to the corresponding spatial positions of the WSI to obtain an intuitive visual attention mask map. Finally, the slice - level malignant category is subdivided through two independent sub - branches to improve the accuracy of the confusion level.

[0067] It can be understood that for the instance - level branch, the instance passes through the attention layers U a ∈R 512×1024 and V a ∈R 512×1024 After processing, the classifier W a,m ∈R 2×512 is used to evaluate the attention scores of the instance to the slice - level benign and malignant branches respectively. Among them, A i,m represents the attention score of the i - th instance to the m - th branch, as shown in formula (1). Since the pixel - level annotation assigns benign and malignant instance - level labels y i ∈{0,1} to each super - pixel instance, it is regarded as a binary classification task. On this basis, the attention score of the instance to the slice - level malignant branch is used as the output A i,1 of the instance - level branch, and the binary cross - entropy loss function is used to calculate the instance - level loss L instance,i , as shown in formula (2).

[0068]

[0069] L instance,i = - [y i * log(A i,1 )+(1 - y i )* log(1 - A i,1 )] (2)

[0070] It is understandable that in the slice-level malignant branch, this paper regards the five Gleason grades as a category and assigns attention scores to each instance according to its contribution to this branch. To make more full use of pixel-level annotations and extract more effective supervision information, this paper designs an Instance Squeeze Module (ISM), which masks the instances according to the attention scores and only retains the instances with higher attention scores as the input of the slice-level malignant branch. The masked instances are reconstructed, and convolutional kernels of three different sizes are used to further extract information at different scales. After fusion, they are flattened as the input of the downstream network. In clinical practice, the determination method of Gleason grade mainly depends on the category and proportion of differentiated glands in the cancerous focus. Therefore, to better characterize the component information of the slice, this paper designs a Component Extraction Module (CEM), which performs two-dimensional spatial position encoding on the compressed instances and concatenates a learnable class token as the input of the Transformer, and uses the multi-head self-attention mechanism to learn the correlation between the remaining instances. The class token and instance token in the Transformer structure correspond to the slice-level label and instance-level label in the Gleason grading, respectively. In the Gleason grading system, GG2 (GP3+GP4) and GG3 (GP4+GP3) are highly similar and belong to the same category (GS7) in the Gleason scoring system. Therefore, this invention uses two classifiers with different complexities to predict the malignant category of the WSI. The detailed calculation is shown in Algorithm 1.

[0071]

[0072] It is understandable that in the slice-level benign branch, the instance h i T is aggregated with the attention score A i,0 of the instance to this branch as the slice-level representation, and the benign category prediction probability 0 ∈R 1×512 of this slice is calculated through the classifier W as shown in Formula (3). The outputs of all slice-level classifiers are processed by the Softmax layer to obtain the probability distribution of the six categories of the WSI where c ∈ {0, 1, 2, 3, 4, 5}. Finally, the calculation of the slice-level loss L slide is shown in Formula (4), and the overall loss L total as shown in Formula (5) is minimized to optimize the model parameters. Among them, γ and λ are set to 2 and 0.4 respectively, and a c ∈ {0.25, 0.175, 0.125, 0.1, 0.15, 0.2}.

[0073]

[0074] Based on this, compared with the prior art, the Gleason grading method for prostate cancer pathological images of the present invention has at least the following beneficial effects:

[0075] (1) The present invention constructs a multi-branch network structure according to clinical practice experience to perform specific processing on different grades, aiming to mine and make more full use of the rich supervision information in the instance labels, and improve the accuracy of the easily confused grades in Gleason grading.

[0076] (2) The present invention models according to the special judgment method of Gleason grading, and designs an instance compression module and a component extraction module, aiming to establish the model's ability to locate cancerous foci and improve the model's ability to extract global information of relevant instances.

[0077] It should be noted that the present invention models according to the special judgment method of Gleason grading for prostate cancer pathological sections, and proposes a hybrid supervision method based on multi-instance learning. In the existing Gleason grading technologies, although some existing methods use pruning strategies such as random masking and Top-k to reduce the burden of the model, and use the multi-head self-attention mechanism in the Transformer structure to capture global information, they often ignore the information integrity of cancerous foci in pathological sections and the characteristics of easy confusion of some malignant categories. In contrast, the present invention performs specific processing on different grades by introducing a multi-branch hybrid supervision network structure, makes more full use of the expensive supervision information in the instance-level labels, and retains more complete cancerous focus information while reducing the consumption of computing resources through a more reasonable attention pruning strategy. The designed instance compression module and component extraction module have excellent cancerous focus location ability and global information extraction ability, and can effectively improve the accuracy of Gleason grading in prostate cancer. Compared with the prior art, the present invention can provide effective second opinions for doctors during grading and provide cancerous focus location information during the observation of sections.

[0078] In addition, as Figure 4 shown, an embodiment of the present invention also discloses a Gleason grading device for prostate cancer pathological images, and the device includes:

[0079] An acquisition module 110, configured to acquire a prostate cancer pathological image, and the prostate cancer pathological image is a whole-slide digital section image;

[0080] A division module 120, configured to divide the non-blank area of the prostate cancer pathological image into a plurality of non-overlapping instances;

[0081] A conversion module 130, configured to convert the instance into a high-dimensional feature vector;

[0082] A processing module 140 is configured to input a high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slice-level representation. The multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch, and a slice-level branch. The slice-level branch includes a slice-level benign branch and a slice-level malignant branch. The slice-level malignant branch includes an instance compression module and a component extraction module.

[0083] A grading module 150 is configured to input the slice-level representation into a classifier to obtain a Gleason grading result of the prostate cancer pathological image.

[0084] The Gleason grading device for prostate cancer pathological images according to the embodiments of the present invention is used to execute the Gleason grading method for prostate cancer pathological images in the above embodiments. The specific processing process is the same as that of the Gleason grading method for prostate cancer pathological images in the above embodiments, and will not be described in detail here.

[0085] In addition, as Figure 5 shown, an embodiment of the present invention also discloses an electronic device, including: at least one processor 210; at least one memory 220 for storing at least one program; when the at least one program is executed by the at least one processor 210, the Gleason grading method for prostate cancer pathological images in any of the previous embodiments is implemented.

[0086] In addition, an embodiment of the present invention also discloses a computer-readable storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to execute the Gleason grading method for prostate cancer pathological images in any of the previous embodiments.

[0087] The system architecture and application scenarios described in the embodiments of the present invention are for more clearly illustrating the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art can know that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0088] Those of ordinary skill in the art can understand that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, and their appropriate combinations.

[0089] In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be executed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes but is not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that a communication medium typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0090] The terms "component", "module", "system", etc. as used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, an application running on a computing device and the computing device can both be components. One or more components may reside in a process or execution thread, and a component may be located on one computer or distributed between two or more computers. In addition, these components may execute from various computer-readable media having various data structures stored thereon. A component may communicate, for example, by signals according to one or more data packets (e.g., data from two components interacting with each other from a local system, a distributed system, or a network, such as the Internet interacting with other systems through signals).

Claims

1. A Gleason grading method for prostate cancer pathology images, comprising: Acquiring a prostate cancer pathology image, wherein the prostate cancer pathology image is a full-field digital slice image; Dividing the non-blank area of ​​the prostate cancer pathology image into a plurality of non-overlapping instances; Converting the instance into a high-dimensional feature vector; Inputting the high-dimensional feature vector into a pre-trained multi-branch hybrid supervised network model to obtain a slice-level representation, wherein the multi-branch hybrid supervised network model includes an attention backbone network, an instance-level branch and a slice-level branch, the slice-level branch includes a slice-level benign branch and a slice-level malignant branch, and the slice-level malignant branch includes an instance compression module and a component extraction module; The slice-level representation is input into a classifier to obtain the Gleason grading result of the prostate cancer pathology image.

2. The method according to claim 1, characterized in that: The training method of the multi-branch hybrid supervision network model includes: Obtain slice-level labels for prostate cancer pathology images in the training set; Assigning a label to each of the instances according to the cancer lesion annotations of the prostate cancer pathology images in the training set to obtain an instance-level label; Combine the instance-level label and the slice-level label to jointly supervise the learning of the attention backbone network, and calculate the instance-level loss and the slice-level loss; Determine an overall loss function based on the instance-level loss and the slice-level loss; The multi-branch hybrid supervised network model is trained based on the overall loss function to obtain the trained multi-branch hybrid supervised network model.

3. The method according to claim 2, characterized in that The method for calculating the instance-level loss includes: Taking the attention score of the instance to the slice-level malignant branch as the output of the instance-level branch; The instance-level loss is calculated based on the output of the instance-level branch using a binary cross entropy loss function.

4. The method according to claim 2, characterized in that The calculation method of the slice level loss includes: Aggregating the attention scores of the slice-level benign branches using the instance and the instance as a slice-level representation; Calculating the benign category prediction probability of the slice-level representation by a slice-level classifier; The outputs of all slice-level classifiers are processed by a Softmax layer to obtain the probability distribution of different categories of the prostate cancer pathology image; The slice-level loss is calculated based on the probability distribution.

5. The method according to claim 1, characterized in that The step of inputting the high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slice-level representation includes: inputting the high-dimensional feature vector into the slice-level benign branch and the slice-level malignant branch respectively; Acquire all instance features in the slice-level benign branch; In the slice-level malignant branch, obtaining a target instance that contributes significantly to the slice-level malignant branch and determining a contribution degree of the target instance; Calculate an attention score according to the contribution; The attention score is mapped to the corresponding spatial position of the prostate cancer pathology image to obtain the slice-level representation.

6. The method according to claim 1, characterized in that The instance compression module masks the target instance according to the attention score, and only retains the target instance with a higher attention score as the input of the slice-level malignant branch; the masked target instance is reconstructed, and three convolution kernels of different sizes are used to further extract information at different scales, which is flattened after fusion as the input of the component extraction module.

7. The method according to claim 1, characterized in that The component extraction module encodes the two-dimensional spatial position of the compressed instance, concatenates a learnable class token as the input of the Transformer, and uses the multi-head self-attention mechanism to learn the correlation between the remaining instances.

8. A Gleason grading device for prostate cancer pathology images, characterized in that: The device comprises: An acquisition module, used for acquiring a prostate cancer pathology image, wherein the prostate cancer pathology image is a full-field digital slice image; A division module, used for dividing the non-blank area of ​​the prostate cancer pathology image into a plurality of non-overlapping instances; A conversion module, used for converting the instance into a high-dimensional feature vector; A processing module, used for inputting the high-dimensional feature vector into a pre-trained multi-branch hybrid supervision network model to obtain a slice-level representation, wherein the multi-branch hybrid supervision network model includes an attention backbone network, an instance-level branch and a slice-level branch, the slice-level branch includes a slice-level benign branch and a slice-level malignant branch, and the slice-level malignant branch includes an instance compression module and a component extraction module; The grading module is used to input the slice-level representation into a classifier to obtain the Gleason grading result of the prostate cancer pathology image.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the Gleason grading method for prostate cancer pathology images as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing computer-executable instructions for executing the Gleason grading method for prostate cancer pathology images according to any one of claims 1 to 7.

Images