Weakly supervised classification methods and devices for histopathological sections

Abstract

This invention provides a weakly supervised classification method and apparatus for histopathological sections, comprising: acquiring tissue sections to be classified; inputting the tissue sections to be classified into a pre-trained classification model to obtain classification prediction results; wherein, the classification model is based on a deep neural network and trained using a massive number of tissue section samples to be classified, and the classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolutional classification module. This invention utilizes a deep network with a larger receptive field as its foundation, extracts and fuses information from different levels, and classifies the tissue sections to be classified based on this information. This allows for complete observation of the entire section structure while clearly seeing cell morphology, greatly improving the classification effect of histopathological sections.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a weakly supervised classification method and apparatus for histopathological sections. Background Technology

[0002] Histopathological slides differ from general medical images in that their image size is enormous, requiring a single slide to be sliced ​​into numerous patches before they can be input into a model for training. Existing techniques for histopathological classification primarily fall into two categories: supervised learning and weakly supervised learning. Supervised learning requires detailed annotation of regions based on a large amount of histopathological slide data, but annotation is often time-consuming and labor-intensive, thus limiting the availability of large-scale labeled data. Even when achieving good results with sufficient labeled data, it faces significant challenges in generalization. Weakly supervised learning, on the other hand, requires no annotation; it only needs the diagnostic results corresponding to the histopathological slides to infer the corresponding labels for classification. Current weakly supervised classification of histopathological slides typically employs a multi-instance learning approach. x patches from each slide are selected as representatives, input into the model to obtain one or more classification outputs, and then the outputs are integrated to obtain the classification result for that slide. This type of weakly supervised method still treats each patch independently, preventing the model from effectively observing the structural information of the entire slide, resulting in generally poor classification performance. Summary of the Invention

[0003] This invention provides a weakly supervised classification method and apparatus for histopathological sections, which addresses the shortcomings of existing technologies in section classification, achieving better section classification results.

[0004] This invention provides a weakly supervised classification method for histopathological sections, comprising:

[0005] Obtain tissue slices to be classified;

[0006] The tissue slices to be classified are input into a pre-trained classification model to obtain classification prediction results;

[0007] The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0008] According to the present invention, a weakly supervised classification method for histopathological sections is provided, which uses a large number of unclassified tissue section samples to train a classification model, specifically including:

[0009] Obtain a large number of tissue slice samples to be classified;

[0010] A tissue slice sample to be classified is input into the threshold segmentation module and cut according to the target cutting size to obtain a grid slice. A position matrix is ​​obtained based on all the grid slices.

[0011] All the mesh slices are input into the feature stitching module to perform feature map re-division and re-stitching to obtain the target feature map;

[0012] The position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size;

[0013] The target feature map and the weight matrix are input into the convolution classification module to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0014] The classification loss is calculated and backpropagated using the sample classification prediction results. This process is repeated for all tissue slice samples to be classified, and the classification model is updated iteratively in this way.

[0015] According to a weakly supervised classification method for histopathological sections provided by the present invention, the tissue section sample to be classified is input into a threshold segmentation module, and the method further includes the following steps beforehand:

[0016] The maximum value of the size of all the tissue slice samples to be classified is taken as the input size of the classification model;

[0017] Fill blank areas in the tissue slices to be classified that are smaller than the input size until they are the same size as the input size.

[0018] According to the present invention, a weakly supervised classification method for histopathological sections is provided, which obtains a position matrix based on all the grid sections, specifically including:

[0019] The grid slices are arranged according to their original positions;

[0020] For all the aforementioned mesh slices, mesh slices without tissue are filled with 0, mesh slices filled with tissue are filled with 1, and mesh slices with partial tissue are filled with the area ratio of the tissue to obtain the position matrix.

[0021] According to a weakly supervised classification method for histopathological sections provided by the present invention, the feature stitching module includes at least a Conv1 layer and Layer 2, Layer 3, Layer 4, and Layer 5 layers; all the mesh slices are input into the feature stitching module for feature map re-division and re-stitching to obtain the target feature map, specifically including:

[0022] S1: Input all the mesh slices into the Convolutional Conv1 layer and the Max Pooling layer at the same time, and concatenate the output results in the original order to obtain the basic feature map;

[0023] S2: The basic feature map is re-divided according to the second preset number and the division result is input into Layer2. The output result is reassembled in the original order to obtain the second feature map.

[0024] S3: Following the steps of re-division and re-segmentation in step S2, traverse Layer3, Layer4, and Layer5, and re-segment the output results in the original order to obtain the target feature map.

[0025] According to a weakly supervised classification method for histopathological sections provided by the present invention, the position matrix module includes a convolutional layer and a sigmoid layer; the position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size, specifically including:

[0026] Each element of the position matrix is ​​expanded proportionally in terms of the number of rows and columns to obtain an expanded position matrix of the target feature map size;

[0027] The expanded position matrix is ​​input into the convolutional layer and the sigmoid layer to obtain the probability value of each element of the expanded position matrix, thereby obtaining the weight matrix.

[0028] According to a weakly supervised classification method for histopathological sections provided by the present invention, the convolutional classification module includes a feature extraction layer and a fully connected layer; the target feature map and the weight matrix are input into the convolutional classification module to obtain the sample classification prediction result of the tissue section sample to be classified, specifically including:

[0029] The target feature map is weighted and fused with the weight matrix to obtain the full slice information features;

[0030] The full slice information features are sequentially input into the feature extraction layer and the fully connected layer to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0031] The present invention also provides a weakly supervised classification device for histopathological sections, comprising:

[0032] Acquisition unit, used to acquire tissue slices to be classified;

[0033] A classification unit is used to input the tissue slice to be classified into a pre-trained classification model to obtain a classification prediction result;

[0034] The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the weakly supervised classification method for histopathological sections as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the weakly supervised classification method for histopathological sections as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the weakly supervised classification method for histopathological sections as described above.

[0038] The present invention provides a weakly supervised classification method and apparatus for histopathological sections. The method involves acquiring tissue sections to be classified and inputting these sections into a pre-trained classification model to obtain classification prediction results. The classification model is based on a deep neural network and trained using a large number of tissue section samples. The model includes a threshold segmentation module, a feature concatenation module, a position matrix module, and a convolutional classification module. This invention utilizes a deep network with a larger receptive field as its foundation, extracting and fusing information from different levels. Based on this, it classifies the tissue sections, enabling observation of the entire section structure while clearly seeing cell morphology, thus significantly improving the classification effect of histopathological sections. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is one of the flowcharts of the weakly supervised classification method for histopathological sections provided by the present invention;

[0041] Figure 2 This is the second flowchart of the weakly supervised classification method for histopathological sections provided by the present invention;

[0042] Figure 3This is a schematic diagram of feature map re-division and re-assembly in an embodiment of the weakly supervised classification method for histopathological sections provided by the present invention;

[0043] Figure 4 This is a schematic diagram of the structure of the weakly supervised classification device for histopathological sections provided by the present invention;

[0044] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0045] Figure label:

[0046] 410: Acquisition unit; 420: Classification unit;

[0047] 510: Processor; 520: Communication interface; 530: Memory; 540: Communication bus. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0049] The following combination Figures 1-3 The present invention describes a weakly supervised classification method for histopathological sections. Figure 1 This is one of the flowcharts of the weakly supervised classification method for histopathological sections provided by the present invention, such as... Figure 1 Shown, including:

[0050] Step 110: Obtain tissue slices to be classified.

[0051] In practical applications, the tissue slices obtained need to be classified. It is important to note that the tissue slices for classification are in image format.

[0052] Step 120: Input the tissue slice to be classified into a pre-trained classification model to obtain the classification prediction result;

[0053] The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0054] The tissue slices to be classified are input into the trained classification model, which outputs the classification prediction results. It is important to note that the classification prediction results include positive and negative classes. A positive class indicates that the tissue slice contains lesions, while a negative class indicates that the tissue slice does not contain lesions.

[0055] Histopathological classification primarily involves observing tissue structure, followed by cell morphology. Therefore, being able to clearly observe cell morphology while simultaneously examining the entire tissue section structure would significantly improve the classification accuracy of histopathological sections. However, due to the large size of the sections, current techniques do not allow models to observe the entire structure of the section.

[0056] Because shallow convolutional neural networks extract information such as edges, colors, and textures from images, their receptive field is relatively small; while deep neural networks extract more abstract semantic information, their receptive field is larger. This invention uses deep neural networks as a foundation, and at the same time, it utilizes weakly supervised learning methods to extract richer information while effectively utilizing a large amount of data. It does not require time-consuming and laborious data labeling, and improves classification performance based on weakly supervised learning.

[0057] Based on the above embodiments, this method, which uses a massive number of tissue slice samples to be classified to train a classification model, specifically includes:

[0058] Obtain a large number of tissue slice samples to be classified;

[0059] A tissue slice sample to be classified is input into the threshold segmentation module and cut according to the target cutting size to obtain a grid slice. A position matrix is ​​obtained based on all the grid slices.

[0060] All the mesh slices are input into the feature stitching module to perform feature map re-division and re-stitching to obtain the target feature map;

[0061] The position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size;

[0062] The target feature map and the weight matrix are input into the convolution classification module to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0063] The classification loss is calculated and backpropagated using the sample classification prediction results. This process is repeated for all tissue slice samples to be classified, and the classification model is updated iteratively in this way.

[0064] Specifically, this invention is described based on the ResNet50 network architecture, but this does not imply any limitation on the invention. It includes Conv1, Layer2, Layer3, Layer4, and Layer5 of the architecture, which are used to extract information from different layers respectively.

[0065] In the training process of the classification model, based on the massive amount of tissue slice samples to be classified, the threshold segmentation module first performs patch cutting to generate input grid slices and a position matrix. The grid slices are then sequentially input into the Conv1 layer and each layer in the feature concatenation module, followed by feature concatenation and feature re-segmentation. This process iterates multiple times, outputting a feature map F. The position matrix is ​​directly expanded to the size of the feature map in the position matrix module and subjected to convolution and sigmoid operations, outputting a weight matrix W. The feature map and weight matrix are combined in the convolution classification module. Finally, the classification prediction of the slice sample is obtained through the convolutional model classification layer, and common classification loss calculations and backpropagation are performed. This iterative update process yields the classification model. The classification loss is the cross-entropy loss function.

[0066]

[0067] Among them, Y i y represents the probability that sample i is predicted to be of the positive class. i The label represents sample i, with 1 for positive class and 0 for negative class; backpropagation is the foundation of neural network model training. By calculating the gradient of the loss function with respect to each parameter in the network, the network parameters are updated to reduce the loss function and achieve the purpose of network learning.

[0068] Based on the above embodiments, in this method, before inputting the tissue slice sample to be classified into the threshold segmentation module, the method further includes:

[0069] The maximum value of the size of all the tissue slice samples to be classified is taken as the input size of the classification model;

[0070] Fill blank areas in the tissue slices to be classified that are smaller than the input size until they are the same size as the input size.

[0071] Specifically, before segmenting the input tissue slice samples to be classified, all tissue slice samples need to be standardized in size. Specifically, a threshold segmentation method is used to extract the location and size of the tissue in all slices. The maximum size (h, w) of all slices is taken as the input size of the classification model. For all slices with a size smaller than (h, w), assuming the slice size is (h2, w2), the slice is filled with blank areas starting from row h2+1 and column w2+1, and this filled area is designated as the background region. It is important to note that the threshold segmentation method requires first statistically analyzing the grayscale values ​​of all pixels in the slice data, and then determining the threshold based on this data distribution. This threshold is designed to separate the tissue region and the background region of the slice. After selecting the threshold, regions exceeding the threshold are assigned a value of 1 (indicating foreground containing tissue), and regions below the threshold are assigned a value of 0 (indicating background without tissue). This yields a threshold segmentation map of size (h, w), which is used to extract the tissue region and location matrix.

[0072] Based on the above embodiments, the method of obtaining the position matrix according to all the mesh slices specifically includes:

[0073] The grid slices are arranged according to their original positions;

[0074] For all the aforementioned mesh slices, mesh slices without tissue are filled with 0, mesh slices filled with tissue are filled with 1, and mesh slices with partial tissue are filled with the area ratio of the tissue to obtain the position matrix.

[0075] Specifically, in the training process of the classification model, a tissue slice sample to be classified is extracted, and the threshold segmentation module cuts it according to the target cutting size to obtain grid slices. In actual operation, the rows and columns of the tissue slice sample to be classified are cut into k1 and k2 blocks respectively. That is, for a tissue slice sample to be classified with a uniform size of (h, w), it is cut into k1*k2 grid slices, and the size of each grid slice is (h / k1, w / k2).

[0076] Since the tissue extent varies in each mesh slice, to obtain the position matrix, this invention fills mesh slices without tissue with 0, mesh slices full of tissue with 1, and mesh slices partially containing tissue with a percentage of the tissue's area, based on the results obtained from the threshold segmentation module. After filling, each mesh slice is used as an element of the matrix to obtain the position matrix. Further, the mesh slices are arranged according to their original positions, and a k1*k2 blank matrix is ​​constructed corresponding to the arranged results. Each element of the blank matrix corresponds to a mesh slice at a specific position in the arranged results. If the mesh slice contains only foreground tissue (i.e., the entire region of the mesh slice in the results obtained from the threshold segmentation module is assigned a value of 1), its corresponding blank matrix element is filled with 1. If the slice contains only background tissue (i.e., the entire region of the mesh slice in the results obtained from the threshold segmentation module is assigned a value of 0), its corresponding blank matrix element is filled with 0. If the mesh slice contains both foreground and background (i.e., the region of the mesh slice in the results obtained from the threshold segmentation module includes both 1 and 0), its corresponding blank matrix element is filled with a percentage of 1s relative to the total number of assigned values ​​to obtain the position matrix.

[0077] Based on the above embodiments, in this method, the feature stitching module includes at least a Conv1 layer and Layer2, Layer3, Layer4, and Layer5 layers; all the mesh slices are input to the feature stitching module for feature map re-division and re-stitching to obtain the target feature map, specifically including:

[0078] S1: Input all the mesh slices into the Convolutional Conv1 layer and the Max Pooling layer at the same time, and concatenate the output results in the original order to obtain the basic feature map;

[0079] S2: The basic feature map is re-divided according to the second preset number and the division result is input into Layer2. The output result is reassembled in the original order to obtain the second feature map.

[0080] S3: Following the steps of re-division and re-segmentation in step S2, traverse Layer3, Layer4, and Layer5, and re-segment the output results in the original order to obtain the target feature map.

[0081] Specifically, for the feature splicing module, such as Figure 2As shown, a grid slice of size (h / k1, w / k2) is simultaneously input into both the Convolutional Conv1 layer and the max pooling layer, reducing its size to 1 / 4, i.e., (h / 4k1, w / 4k2). The features are then concatenated in their original order to obtain a base feature map of size (h / 4, w / 4). This base feature map is then divided into (k1 / 2)*(k2 / 2) blocks and input into the residual block (Layer 2). The output second feature map remains unchanged at (h / 4, w / 4). This process of feature map division and re-concatenation (again, the feature map is divided into (k1 / 2)*(k2 / 2) blocks) and input into Layer 3 continues until i in Layer i exceeds 4. The output target feature map F has a size of (h / 32, w / 32).

[0082] Furthermore, in the feature stitching module, the patch size and number of patches after running at different layers are shown in Table 1:

[0083] Table 1. Number of output results and feature map size for different layers of the feature stitching module.

[0084]

[0085] It should be noted that feature concatenation and re-segmentation are performed according to the following... Figure 3 The logic shown is as follows, where each box on the left represents a feature to be stitched together, and the right side represents the division and stitching principles.

[0086] Based on the above embodiments, in this method, the position matrix module includes a convolutional layer and a sigmoid layer; the position matrix is ​​input to the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size, specifically including:

[0087] Each element of the position matrix is ​​expanded proportionally in terms of the number of rows and columns to obtain an expanded position matrix of the target feature map size;

[0088] The expanded position matrix is ​​input into the convolutional layer and the sigmoid layer to obtain the probability value of each element of the expanded position matrix, thereby obtaining the weight matrix.

[0089] Specifically, according to the size (h / 32, w / 32) of the output full-slice feature stitching map, each element of the (k1, k2) position matrix is ​​expanded by h / (32*k1) rows and w / (32*k2) columns to obtain an expanded position matrix of size (h / 32, w / 32).

[0090] In one embodiment, as shown in Tables 2 and 3, the 4-row, 3-column position matrix in Table 2 is expanded by 3 rows and 2 columns to become the 12-row, 6-column matrix in Table 3.

[0091] Table 2. Position matrix before expansion in one embodiment.

[0092] 0 0 0 0.1 0.6 0 0.2 0.5 0 0.3 0 0

[0093] Table 3. Expanded position matrix of one embodiment.

[0094]

[0095]

[0096] The expanded position matrix is ​​input into the convolutional and sigmoid layers of the position matrix module. The probability value of each element in the expanded position matrix is ​​calculated to obtain the final weight matrix. Furthermore, the expanded position matrix is ​​input into three identical convolutional modules and sigmoid layers to obtain the probability value for each position point, thus obtaining the weight matrix. It is important to note that since each convolutional module has a kernel size of 3*3, padding of 1, and stride of 1, its output and input sizes are the same.

[0097] Based on the above embodiments, in this method, the convolutional classification module includes a feature extraction layer and a fully connected layer; inputting the target feature map and the weight matrix into the convolutional classification module to obtain the sample classification prediction result of the tissue slice sample to be classified specifically includes:

[0098] The target feature map is weighted and fused with the weight matrix to obtain the full slice information features;

[0099] The full slice information features are sequentially input into the feature extraction layer and the fully connected layer to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0100] Specifically, such as Figure 2 As shown, the target feature map and weight matrix are fused to obtain the full-slice information features. It's important to note that the fusion of the target feature map F and the weight matrix W is obtained through the calculation of F(1+w). The weighted fusion operation yields richer full-slice feature information. It can be understood that the full-slice information features output by F(1+w) incorporate the tissue structure and cell morphology information of the entire slice. These full-slice information features are then input into Layer 5 to extract deeper information, and finally input into a fully connected FC layer to obtain the classification prediction for the slice.

[0101] It's important to note that the Layer 5 input for the full slice information features is the same as the Layer 5 of the feature concatenation module. To distinguish it from the feature concatenation module, this step refers to it as the feature extraction layer or Layer 5'. Layer 5' is used to extract the structural and semantic information of the full slice. Finally, the output of Layer 5' is fed into a fully connected layer (FC layer) to obtain the final output. The overall implementation flowchart is as follows: Figure 2 As shown.

[0102] This invention provides a weakly supervised classification method for histopathological sections. The method involves acquiring tissue sections to be classified and inputting these sections into a pre-trained classification model to obtain classification prediction results. The classification model is based on a deep neural network and trained using a massive number of tissue section samples. The model includes a threshold segmentation module, a feature concatenation module, a position matrix module, and a convolutional classification module. This invention utilizes a deep network with a larger receptive field as its foundation, extracting and fusing information from different levels. Based on this, it classifies the tissue sections, enabling a complete observation of the entire section structure while clearly seeing cell morphology, significantly improving the classification effect of histopathological sections.

[0103] The following describes the weakly supervised classification device for histopathological sections provided by the present invention. The weakly supervised classification device for histopathological sections described below can be referred to in correspondence with the weakly supervised classification method for histopathological sections described above. Figure 4 This is a schematic diagram of the structure of the weakly supervised classification device for histopathological sections provided by the present invention, as shown below. Figure 4 It includes an acquisition unit 410 and a classification unit 420, wherein,

[0104] Acquisition unit 410 is used to acquire tissue slices to be classified;

[0105] The classification unit 420 is used to input the tissue slice to be classified into a pre-trained classification model to obtain a classification prediction result;

[0106] The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0107] Based on the above embodiments, the device uses a large number of tissue slice samples to be classified to train a classification model, specifically including:

[0108] Obtain a large number of tissue slice samples to be classified;

[0109] A tissue slice sample to be classified is input into the threshold segmentation module and cut according to the target cutting size to obtain a grid slice. A position matrix is ​​obtained based on all the grid slices.

[0110] All the mesh slices are input into the feature stitching module to perform feature map re-division and re-stitching to obtain the target feature map;

[0111] The position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size;

[0112] The target feature map and the weight matrix are input into the convolution classification module to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0113] The classification loss is calculated and backpropagated using the sample classification prediction results. This process is repeated for all tissue slice samples to be classified, and the classification model is updated iteratively in this way.

[0114] Based on the above embodiments, in this device, before inputting the tissue slice sample to be classified into the threshold segmentation module, the following steps are also included:

[0115] The maximum value of the size of all the tissue slice samples to be classified is taken as the input size of the classification model;

[0116] Fill blank areas in the tissue slices to be classified that are smaller than the input size until they are the same size as the input size.

[0117] Based on the above embodiments, in this device, obtaining the position matrix according to all the said mesh slices specifically includes:

[0118] The grid slices are arranged according to their original positions;

[0119] For all the aforementioned mesh slices, mesh slices without tissue are filled with 0, mesh slices filled with tissue are filled with 1, and mesh slices with partial tissue are filled with the area ratio of the tissue to obtain the position matrix.

[0120] Based on the above embodiments, in this device, the feature stitching module includes at least a Conv1 layer and Layer2, Layer3, Layer4, and Layer5 layers; all the mesh slices are input to the feature stitching module for feature map re-division and re-stitching to obtain the target feature map, specifically including:

[0121] S1: Input all the mesh slices into the Convolutional Conv1 layer and the Max Pooling layer at the same time, and concatenate the output results in the original order to obtain the basic feature map;

[0122] S2: The basic feature map is re-divided according to the second preset number and the division result is input into Layer2. The output result is reassembled in the original order to obtain the second feature map.

[0123] S3: Following the steps of re-division and re-segmentation in step S2, traverse Layer3, Layer4, and Layer5, and re-segment the output results in the original order to obtain the target feature map.

[0124] Based on the above embodiments, in this device, the position matrix module includes a convolutional layer and a sigmoid layer; the position matrix is ​​input to the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size, specifically including:

[0125] Each element of the position matrix is ​​expanded proportionally in terms of the number of rows and columns to obtain an expanded position matrix of the target feature map size;

[0126] The expanded position matrix is ​​input into the convolutional layer and the sigmoid layer to obtain the probability value of each element of the expanded position matrix, thereby obtaining the weight matrix.

[0127] Based on the above embodiments, in this device, the convolutional classification module includes a feature extraction layer and a fully connected layer; the target feature map and the weight matrix are input into the convolutional classification module to obtain the sample classification prediction result of the tissue slice sample to be classified, specifically including:

[0128] The target feature map is weighted and fused with the weight matrix to obtain the full slice information features;

[0129] The full slice information features are sequentially input into the feature extraction layer and the fully connected layer to obtain the sample classification prediction result of the tissue slice sample to be classified.

[0130] The present invention provides a weakly supervised classification device for histopathological sections. This device acquires tissue sections to be classified and inputs them into a pre-trained classification model to obtain classification prediction results. The classification model is based on a deep neural network and trained using a massive number of tissue section samples. The model includes a threshold segmentation module, a feature concatenation module, a position matrix module, and a convolutional classification module. This invention utilizes a deep network with a larger receptive field as its foundation, extracting and fusing information from different levels. Based on this, it classifies the tissue sections, enabling complete observation of the entire section structure while clearly seeing cell morphology, significantly improving the classification effect of histopathological sections.

[0131] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a weakly supervised classification method for histopathological slides. This method includes: acquiring a tissue slide to be classified; inputting the tissue slide to be classified into a pre-trained classification model to obtain a classification prediction result; wherein the classification model is based on a deep neural network and trained using a large number of tissue slide samples to be classified, and the classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolutional classification module.

[0132] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the weakly supervised classification method for histopathological slides provided by the above methods. The method includes: acquiring a tissue slide to be classified; inputting the tissue slide to be classified into a pre-trained classification model to obtain a classification prediction result; wherein the classification model is based on a deep neural network and is trained using a large number of tissue slide samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a weakly supervised classification method for histopathological slides provided by the methods described above. The method includes: acquiring a tissue slide to be classified; inputting the tissue slide to be classified into a pre-trained classification model to obtain a classification prediction result; wherein the classification model is based on a deep neural network and is trained using a large number of tissue slide samples to be classified, and the classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weakly supervised classification method for histopathological sections, characterized in that, include: Obtain tissue slices to be classified; The tissue slices to be classified are input into a pre-trained classification model to obtain classification prediction results; The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module. A classification model is obtained by training a large number of tissue slice samples to be classified, specifically including: Obtain a large number of tissue slice samples to be classified; A tissue slice sample to be classified is input into the threshold segmentation module and cut according to the target cutting size to obtain a grid slice. A position matrix is ​​obtained based on all the grid slices. All the mesh slices are input into the feature stitching module to perform feature map re-division and re-stitching to obtain the target feature map; The position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size; The target feature map and the weight matrix are input into the convolution classification module to obtain the sample classification prediction result of the tissue slice sample to be classified. The classification loss is calculated and backpropagated using the sample classification prediction results. This process is repeated for all tissue slice samples to be classified, and the classification model is updated iteratively in this way.

2. The weakly supervised classification method for histopathological sections according to claim 1, characterized in that, The process of inputting the tissue slice sample to be classified into the threshold segmentation module, before which the following steps are also included: The maximum value of the size of all the tissue slice samples to be classified is taken as the input size of the classification model; Fill blank areas in the tissue slices to be classified that are smaller than the input size until they are the same size as the input size.

3. The weakly supervised classification method for histopathological sections according to claim 1, characterized in that, The position matrix is ​​obtained based on all the aforementioned mesh slices, specifically including: The grid slices are arranged according to their original positions; For all the aforementioned mesh slices, mesh slices without tissue are filled with 0, mesh slices filled with tissue are filled with 1, and mesh slices with partial tissue are filled with the area ratio of the tissue to obtain the position matrix.

4. The weakly supervised classification method for histopathological sections according to claim 1, characterized in that, The feature stitching module includes at least a Conv1 layer and Layer 2, Layer 3, Layer 4, and Layer 5. All the mesh slices are input into the feature stitching module for feature map re-division and re-stitching to obtain the target feature map, specifically including: S1: Input all the mesh slices into the Convolutional Conv1 layer and the Max Pooling layer at the same time, and concatenate the output results in the original order to obtain the basic feature map; S2: The basic feature map is re-divided according to the second preset number and the division result is input into Layer2. The output result is reassembled in the original order to obtain the second feature map. S3: Following the steps of re-division and re-segmentation in step S2, traverse Layer3, Layer4, and Layer5, and re-segment the output results in the original order to obtain the target feature map.

5. The weakly supervised classification method for histopathological sections according to claim 1, characterized in that, The position matrix module includes convolutional layers and sigmoid layers; the position matrix is ​​input into the position matrix module for expansion and probability calculation to obtain a weight matrix of the target feature map size, specifically including: Each element of the position matrix is ​​expanded proportionally in terms of the number of rows and columns to obtain an expanded position matrix of the target feature map size; The expanded position matrix is ​​input into the convolutional layer and the sigmoid layer to obtain the probability value of each element of the expanded position matrix, thereby obtaining the weight matrix.

6. The weakly supervised classification method for histopathological sections according to claim 1, characterized in that, The convolutional classification module includes a feature extraction layer and a fully connected layer; the target feature map and the weight matrix are input into the convolutional classification module to obtain the sample classification prediction result of the tissue slice sample to be classified, specifically including: The target feature map is weighted and fused with the weight matrix to obtain the full slice information features; The full slice information features are sequentially input into the feature extraction layer and the fully connected layer to obtain the sample classification prediction result of the tissue slice sample to be classified.

7. A weakly supervised classification device for histopathological sections, used to implement the method as described in any one of claims 1-6, characterized in that, include: Acquisition unit, used to acquire tissue slices to be classified; A classification unit is used to input the tissue slice to be classified into a pre-trained classification model to obtain a classification prediction result; The classification model is based on a deep neural network and is trained using a large number of tissue slice samples to be classified. The classification model includes a threshold segmentation module, a feature splicing module, a position matrix module, and a convolution classification module.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the weakly supervised classification method for histopathological sections as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the weakly supervised classification method for histopathological sections as described in any one of claims 1 to 6.

Images