Pathological picture processing method and device and electronic equipment

Abstract

The invention provides a pathological picture processing method and device and electronic equipment, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining a to-be-processed pathological picture, and cutting the to-be-processed pathological picture into a plurality of slice images with preset sizes according to a pre-configured size parameter; performing color processing on the slice images, and screening out at least one target slice image; performing graph segmentation processing on the target slice graph to obtain a target cell nucleus region; determining the target slice map of which the target cell nucleus region is greater than a preset threshold value as a feature slice map; performing feature extraction on the feature slice map to obtain feature information; and fusing the feature information to obtain a target feature matrix for performing classification processing on the to-be-processed pathological picture. According to the pathological picture processing method and apparatus, and the electronic device provided by the invention, the influence of the target cell nucleus content on the classification result can be eliminated, so that the classification accuracy and classification efficiency of the pathological picture are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method, device and electronic equipment for processing pathological images. Background Art

[0002] The image size of pathological slice WSI (Whole-Slide-Image) ranges from tens of M to several G, and the number of pixels in the original image can reach hundreds of millions. Therefore, traditional image analysis methods cannot effectively classify it.

[0003] At present, deep learning methods are mostly used to train models, and the trained models can effectively classify pathological images. Usually, most deep learning methods are based on labeling cancer cells or cancer areas, outlining feature areas, and then training deep models on image blocks after image segmentation, which can then be used to classify pathological sections. However, labeling requires professional operations and is time-consuming and costly, so it is difficult to improve the accuracy and efficiency of image classification. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a method, device and electronic device for processing pathological images to improve the classification accuracy of pathological sections and improve the classification efficiency.

[0005] In a first aspect, an embodiment of the present invention provides a method for processing a pathological image, the method comprising: obtaining a pathological image to be processed, and cropping the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; performing color processing on the plurality of slice images, and screening out at least one target slice image; performing graphic segmentation processing on each of the target slice images to obtain a target cell nucleus area contained in the target slice image; determining the target slice image whose target cell nucleus area is greater than a preset threshold as a feature slice image; performing feature extraction on the feature slice images to obtain feature information contained in each of the feature slice images; fusing the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

[0006] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation of the first aspect, wherein the above-mentioned step of performing color processing on multiple slice images and screening out at least one target slice image includes: converting each of the slice images into a single-channel image; using a pre-configured edge detection algorithm to perform edge detection on the single-channel image to screen out the slice images whose background ratio is greater than a preset threshold; and performing color normalization processing on at least one of the slice images obtained after screening to obtain at least one of the target slice images.

[0007] In combination with the first aspect, an embodiment of the present invention provides a second possible implementation of the first aspect, wherein the above-mentioned step of performing graphic segmentation processing on each of the target slice images to obtain the target cell nucleus area contained in the target slice image includes: inputting each of the target slice images into a pre-trained VIT model respectively; performing instance segmentation on the target cell nuclei contained in each of the target slice images through the VIT model to obtain the target cell nucleus area containing the target cell nuclei; wherein the VIT model is trained based on a sample set containing cell nucleus labels, and the cell nucleus labels include at least the following markings: the size, shape, overlapping boundaries and cell nucleus clustering of the cell nuclei.

[0008] In combination with the second possible implementation of the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein the above method also includes: obtaining a sample image; the sample image is a image containing a target cell nucleus; cropping the sample image into a plurality of sample slice images of preset sizes according to pre-configured size parameters; mapping each of the sample slice images to a one-dimensional vector to obtain a sample vector corresponding to each of the sample slice images; marking the cell nucleus features included in each of the sample vectors to add cell nucleus labels; generating the sample set based on the sample vector and the cell nucleus label added to the sample vector; and the sample set is used to train the VIT model.

[0009] In combination with the first aspect, an embodiment of the present invention provides a fourth possible implementation method of the first aspect, wherein the above-mentioned step of extracting features from the feature slice image to obtain the feature information contained in each feature slice image includes: inputting the feature slice image into a pre-trained feature extraction network; extracting the feature information contained in each feature slice image through the feature extraction network; wherein the feature extraction network is a pre-trained deep convolutional neural network.

[0010] In combination with the first aspect, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein the above method also includes: inputting the target feature matrix into a pre-trained classifier model; outputting classification information of the target feature matrix through the classifier model; wherein the classifier model is trained based on a multi-layer perceptron MLP model.

[0011] In combination with the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein the above-mentioned step of obtaining the pathological image to be processed includes: obtaining a pyramid structure model of the pathological image; and selecting a pathological image of a preset level from the pyramid structure model as the pathological image to be processed.

[0012] In a second aspect, an embodiment of the present invention further provides a device for processing a pathological image, the device comprising: a cropping module, used to obtain a pathological image to be processed, and crop the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; a pre-processing module, used to perform color processing on the plurality of slice images, and screen out at least one target slice image; a segmentation module, used to perform graphic segmentation processing on each of the target slice images, and obtain a target cell nucleus area contained in the target slice image; a screening module, used to determine the target slice image whose target cell nucleus area is greater than a preset threshold as a feature slice image; a feature extraction module, used to perform feature extraction on the feature slice image, and obtain feature information contained in each of the feature slice images; a feature fusion module, used to fuse the feature information, and obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

[0013] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first aspect is executed.

[0015] The embodiments of the present invention bring the following beneficial effects: The pathological image processing method, device and electronic device provided by the embodiments of the present invention can crop the pathological image to be processed into multiple slice images of preset sizes according to pre-configured size parameters; perform color processing on the multiple slice images, and screen out at least one target slice image; perform graphic segmentation processing on each target slice image to obtain the target cell nucleus area contained in the target slice image; determine the target slice image whose target cell nucleus area is greater than a preset threshold as a feature slice image; perform feature extraction on the feature slice image to obtain the feature information contained in each feature slice image; fuse the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix can be used to classify the pathological image to be processed. In addition, since the target slice image is screened in the above-mentioned processing process of the pathological image, the target feature matrix obtained subsequently can more accurately represent the classification information of the pathological image, and eliminate the influence of the target cell nucleus content on the classification result, thereby improving the classification accuracy and classification efficiency of the pathological image.

[0016] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A flowchart of a method for processing a pathological image provided by an embodiment of the present invention; Figure 2 A schematic diagram of color processing provided by an embodiment of the present invention; Figure 3 A flowchart of another method for processing pathological images provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a pathological image processing device provided by an embodiment of the present invention; Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] At present, considering that the labeling work of deep learning methods requires professional operation and is time-consuming and material-consuming, weakly supervised learning methods are mostly used in related technologies. This weakly supervised learning method does not require the specific lesion area to be labeled on the sample image. It only needs to know whether the sample image contains the lesion. No manual labeling is required, which greatly reduces the workload of pathologists and helps to improve the accuracy and efficiency of lesion separation.

[0022] In actual use, the weakly supervised learning methods currently used for training and prediction usually have three models, namely, the Graph Convolution Neural Network model (GCN), the Multiple Instance Learning model (MIL), and the VIT model (Vision Transformer).

[0023] Among them, the graph convolutional neural network model usually constructs a graph structure based on tissue characteristics, and directly extends the convolution operation to the graph structure for deep training and learning; the multi-instance learning model segments the pathological sections, learns and extracts the features of each image block obtained by segmentation, aggregates them after extraction, and then uses a classifier for prediction; the VIT model is based on the self-attention mechanism, and in the visual field, it treats each image block as an ordered, problematic, and interrelated column of data as input for learning and prediction.

[0024] However, the above-mentioned weakly supervised deep model method has a major flaw in classifying and predicting pathological sections. That is, when the tumor content is relatively low, since the learned features are based on the entire pathological section, useful information may be submerged in a large amount of useless data, which in turn affects the accuracy of pathological section classification to a certain extent.

[0025] Based on this, the pathological image processing method, device and electronic device provided by the embodiments of the present invention can effectively eliminate the prediction influence brought by the tumor content of the pathological section and more efficiently predict the type of the pathological section.

[0026] To facilitate understanding of this embodiment, a pathological image processing method disclosed in an embodiment of the present invention is first introduced in detail.

[0027] In a possible implementation, the present invention provides a method for processing a pathological image. Figure 1 A flowchart of a method for processing a pathological image is shown, the method comprising the following steps: Step S102, obtaining a pathological image to be processed, and cutting the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; Usually, pathological images are also called pathological sections or WSI (Whole-Slide-Image) images. Pathological images are pyramid structures, in which different levels correspond to different magnifications, thereby retaining image information at different magnifications. Therefore, different levels can be selected to view pathological images as needed to observe cell information at different levels.

[0028] For example, the OpenSlide Python package can be used to read a pathological image and read the basic information of the entire pathological image to select a pathological image of a preset level. Therefore, in the above step S102, when obtaining the pathological image to be processed, the pyramid structure model of the pathological image can be first obtained; then the pathological image of the preset level is selected from the pyramid structure model as the pathological image to be processed. Usually, the highest level, that is, the original pathological image is selected for processing, and the pathological image of the highest level usually contains complete cell nucleus information.

[0029] Step S104, performing color processing on the multiple slice images, and selecting at least one target slice image; The process of cutting the pathological image to be processed into a plurality of slice images of preset sizes in the above step S102 is actually for the purpose of screening the plurality of slice images in the above step S104.

[0030] Specifically, for the pathological image to be processed, the area containing the target cell nucleus to be identified also includes the area not containing the target cell nucleus. For example, the pathological image to be processed contains both tumor areas and areas without tumors. Through the cropping process in the above step S102 and the screening process in the above step S104, a certain number of slice images can be removed, such as slice images that do not contain any tumors, that is, useless data is reduced, thereby reducing the subsequent calculation amount.

[0031] Step S106, performing a graphic segmentation process on each target slice image to obtain a target cell nucleus region included in the target slice image; Step S108, determining the target slice image whose target cell nucleus area is larger than a preset threshold as a characteristic slice image; Among them, the process of the above-mentioned steps S106 and S108 is a process of further screening the target slice image, that is, the target cell nucleus area is obtained by graphic segmentation. If the area of ​​the target cell nucleus area in the target slice image accounts for a small proportion, it means that the target slice image has a small share in the classification process of the pathological image. Therefore, in the above-mentioned step S108, the target slice image with a target cell nucleus area greater than a preset threshold is selected as the feature slice image to facilitate subsequent processing.

[0032] Step S110, extracting features from the feature slice graph to obtain feature information contained in each feature slice graph; Step S112, the feature information is integrated to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

[0033] The pathological image processing method provided by the embodiment of the present invention can crop the pathological image to be processed into multiple slice images of preset sizes according to pre-configured size parameters; perform color processing on the multiple slice images, and screen out at least one target slice image; perform graphic segmentation processing on each target slice image to obtain the target cell nucleus area contained in the target slice image; determine the target slice image whose target cell nucleus area is greater than a preset threshold as a feature slice image; perform feature extraction on the feature slice image to obtain the feature information contained in each feature slice image; fuse the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix can be used to classify the pathological image to be processed. In addition, since the target slice image is screened in the above-mentioned processing process of the pathological image, the target feature matrix obtained subsequently can more accurately represent the classification information of the pathological image, and eliminate the influence of the target cell nucleus content on the classification result, thereby improving the classification accuracy and classification efficiency of the pathological image.

[0034] In actual use, in order to reduce the differences caused by the staining levels of pathological images, in an embodiment of the present invention, the color processing in the above step S104 is actually a color normalization process. Specifically, the above step S104 includes the following processes: converting each slice image into a single-channel image; using a pre-configured edge detection algorithm to perform edge detection on the single-channel image to filter out slice images with a background ratio greater than a preset threshold; performing color normalization processing on at least one slice image obtained after screening to obtain at least one target slice image.

[0035] For example, each slice image can be converted into a grayscale histogram first, and the edge detection algorithm can be used to perform edge detection on the grayscale histogram. This can filter out slice images with a background ratio greater than a preset threshold. The background usually does not contain the target cell nucleus. Therefore, the influence of useless data can be effectively filtered out. Then, color normalization processing is performed on at least one slice image obtained after screening to obtain at least one target slice image.

[0036] In specific implementation, the above color normalization process can be implemented with the help of the Transformer model (such as Vision Transformer) provided by the OpenSlide library, and then the pathological images can be standardized by color normalization. For ease of understanding, Figure 2 A schematic diagram of color processing is shown, wherein Figure 2 The left side (a) is the image before color processing, and the right side (b) is the image after color processing.

[0037] It can be seen from Figure 2 that the image after color normalization can effectively eliminate the difference caused by the staining level compared to the image before color normalization.

[0038] In actual use, the specific color processing method can be set according to actual use conditions, and the embodiment of the present invention does not limit this.

[0039] Furthermore, in an embodiment of the present invention, in the above step S106, when performing graphic segmentation processing on the target slice image, it is achieved with the help of a weakly supervised deep learning model. The weakly supervised deep learning model does not require manual labeling, which greatly reduces the workload of pathologists.

[0040] Specifically, for ease of understanding, Figure 1 On the basis of Figure 3 A flow chart of another method for processing pathological images is also shown, and the processing process of the pathological images is described in detail.

[0041] Step S302, obtaining a pathological image to be processed, and cutting the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; Step S304, performing color processing on the multiple slice images, and selecting at least one target slice image; Step S306, inputting each target slice image into a pre-trained VIT model, performing instance segmentation on the target cell nucleus contained in each target slice image through the VIT model, and obtaining a target cell nucleus region containing the target cell nucleus; The VIT model in the embodiment of the present invention is obtained by training based on a sample set containing cell nucleus labels, and the cell nucleus labels at least include the following marks: the size, shape, overlapping boundaries and cell nucleus clustering of the cell nucleus.

[0042] Specifically, the VIT model is a weakly supervised deep model. In the embodiment of the present invention, the trained VIT model is used to perform instance segmentation on the target slice image. Therefore, the VIT model in the embodiment of the present invention is also a segmentation model.

[0043] In actual use, when training the above-mentioned weakly supervised deep model, it is only necessary to make the weakly supervised deep model learn the size, shape, overlapping boundaries and cell nuclei, so as to realize instance segmentation of the target cell nuclei contained in the target slice image and obtain the target cell nucleus region containing the target cell nucleus.

[0044] Therefore, the training process of the above VIT model usually includes the following steps: (1) Obtain a sample image; the sample image is an image containing a target cell nucleus, such as a tumor cell image, a cancer cell image, etc.

[0045] (2) Crop the sample image into multiple sample slices of preset sizes according to pre-configured size parameters; The sample slices at this time may also be called Patches. For example, the sample images are divided into 1024x1024 size slices.

[0046] (3) Map each sample slice image to a one-dimensional vector to obtain a sample vector corresponding to each sample slice image; In this process, the flatten patches technology can be used, and the linear mapping technology can be used to convert the image data into data that can be processed by the VIT model.

[0047] (4) marking the cell nucleus features included in each sample vector to add cell nucleus labels; In this process, the position of the cell nucleus in the sample vector is actually encoded, and, generally, in the VIT model, an encoding module (Encoder module) and a decoding module ((5) Decode module) are also included, wherein the encoding module generally includes a multi-head attention mechanism module, and sets a normalization layer and an activation function, and the decoding module generally adopts a U-Net-shaped encoder-decoder network structure to achieve efficient and accurate nuclear instance segmentation. The specific encoding module and decoding module can be set according to the actual use situation, and the embodiment of the present invention does not limit this.

[0048] (5) Generate a sample set based on the sample vector and the cell nucleus label added to the sample vector; the sample set is used to train the VIT model.

[0049] The VIT model trained with the sample set described above can implement instance segmentation of the target cell nucleus contained in the target slice image in step S306, thereby obtaining a target cell nucleus region containing the target cell nucleus.

[0050] Step S308, determining the target slice image whose target cell nucleus area is larger than a preset threshold as a characteristic slice image; This step S308 is actually to classify the target slice images after instance segmentation. For example, the preset threshold can be set to 50%, that is, the target slice images with target cell nucleus areas greater than or equal to 50% of the entire target slice image area are retained as feature slice images for subsequent processing.

[0051] Step S310, inputting the feature slice graph into a pre-trained feature extraction network; Step S312, extracting feature information contained in each feature slice graph through a feature extraction network; Among them, the feature extraction network in the embodiment of the present invention is a pre-trained deep convolutional neural network.

[0052] In actual use, the deep convolutional neural network here can adopt the ResNet50 model. Usually, corresponding pathological slice images can be prepared as samples to train the ResNet50 model, and its output layer can be configured. For example, 1000 features can be output for each training sample. In this way, for the trained feature extraction network, in the above step S312, for each feature slice image input, feature information containing 1000 features can be output.

[0053] Step S314, fusing the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed; Step S316, inputting the target feature matrix into a pre-trained classifier model; outputting classification information of the target feature matrix through the classifier model; In actual use, since the pathological image to be processed is cropped in the above step S302, it is also necessary to fuse the feature information of each feature slice image in the above step S314 to obtain the target feature matrix corresponding to the entire pathological image to be processed. In step S316, the target feature matrix is ​​classified to obtain the type of the pathological image to be processed.

[0054] Furthermore, the classifier model in the embodiment of the present invention is obtained by training based on a multi-layer perceptron MLP model.

[0055] Specifically, when training the MLP model, the sample also needs to be cropped to obtain multiple small slices. Then, the above-mentioned feature extraction network is used to extract features for each small slice to obtain feature information. The feature information of all small slices is further fused to obtain a sample feature matrix. The label of each slice is then added to the sample label matrix, and the MLP model is then trained based on the sample label matrix.

[0056] In addition, when constructing the MLP model in the embodiment of the present invention, two hidden layers are usually set to prevent overfitting and improve the generalization ability of the model. In addition, each hidden layer, in addition to the fully connected layer, also includes a dropout output layer, which uses sigmoid and softmax functions for nonlinear transformation to obtain probability values, so that each neuron in the last layer is regarded as the probability of containing a certain label in the sample. In addition, when training the MLP model, the cross-entropy loss (Cross-Entropy Loss) is usually used as the loss function (Loss Function) of the classification task, and the Adam optimizer is used to improve on the basis of gradient descent to accelerate convergence and avoid falling into local optimality. The specific training method can be set according to the actual use situation, and the embodiment of the present invention does not limit this.

[0057] In actual use, the various models used in the embodiments of the present invention, such as the VIT model, feature extraction network and classifier model, etc., can constitute a pathological image segmentation model, which can perform instance segmentation, feature extraction and feature classification on the pathological images to be processed, thereby improving the classification accuracy of the pathological images.

[0058] Moreover, the VIT model is a weakly supervised deep model that does not require manual labeling, saving manpower and financial resources. A large number of pathological sections can be used for instance segmentation using both the RestNet50 feature extraction network and the VIT model. The two models work together to make up for their respective shortcomings. Moreover, the ViT model can obtain key areas from the global perspective, namely the target cell nucleus area mentioned above. As a leader in convolutional neural networks, RestNet50 can learn local features, thereby improving the classification accuracy of pathological images, which in turn helps to improve the ability to identify different tumors in the same tissue.

[0059] Further, based on the above embodiment, the embodiment of the present invention also provides a pathological image processing device, such as Figure 4 A schematic diagram of the structure of a pathological image processing device is shown, the device comprising: A cropping module 40 is used to obtain a pathological image to be processed, and crop the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; A preprocessing module 42, configured to perform color processing on the plurality of slice images and select at least one target slice image; A segmentation module 44 is used to perform a graphic segmentation process on each of the target slice images to obtain a target cell nucleus region included in the target slice image; A screening module 46, configured to determine the target slice image with a target cell nucleus area greater than a preset threshold as a characteristic slice image; A feature extraction module 48 is used to extract features from the feature slice graphs to obtain feature information contained in each feature slice graph; The feature fusion module 49 is used to fuse the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

[0060] The pathological image processing device provided in the embodiment of the present invention has the same technical features as the pathological image processing method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0061] Furthermore, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0062] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are executed.

[0063] Furthermore, an embodiment of the present invention also provides a schematic diagram of the structure of an electronic device, such as Figure 5 As shown, it is a schematic diagram of the structure of the electronic device, wherein the electronic device includes a processor 51 and a memory 50, the memory 50 stores computer executable instructions that can be executed by the processor 51, and the processor 51 executes the computer executable instructions to implement the above method.

[0064] exist Figure 5 In the illustrated embodiment, the electronic device further includes a bus 52 and a communication interface 53 , wherein the processor 51 , the communication interface 53 and the memory 50 are connected via the bus 52 .

[0065] Among them, the memory 50 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 53 (which can be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. can be used. The bus 52 can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 52 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0066] The processor 51 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 51 or the instruction in the form of software. The above processor 51 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor 51 reads the information in the memory and completes the above method in combination with its hardware.

[0067] The computer program product of the pathological image processing method, device and electronic device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method described in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0068] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0069] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0071] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0072] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for processing pathological images, characterized in that: The method comprises: Acquire a pathological image to be processed, and cut the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; Performing color processing on the plurality of slice images, and selecting at least one target slice image; Performing a graphic segmentation process on each of the target slice images to obtain a target cell nucleus region contained in the target slice image; Determine the target slice image whose target cell nucleus area is larger than a preset threshold as a characteristic slice image; Performing feature extraction on the feature slice graphs to obtain feature information contained in each of the feature slice graphs; The feature information is fused to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

2. The method according to claim 1, characterized in that: The step of color processing the plurality of slice images and selecting at least one target slice image comprises: Convert each of the slice images into a single-channel image; Performing edge detection on the single-channel image using a pre-configured edge detection algorithm to filter out the slice images whose background ratio is greater than a preset threshold; Color normalization is performed on at least one of the slice images obtained after screening to obtain at least one of the target slice images.

3. The method according to claim 1, characterized in that The step of performing a graphic segmentation process on each of the target slice images to obtain a target cell nucleus region contained in the target slice image comprises: Input each of the target slice images into a pre-trained VIT model; Performing instance segmentation on the target cell nucleus contained in each of the target slice images by using the VIT model to obtain a target cell nucleus region containing the target cell nucleus; The VIT model is obtained by training based on a sample set containing cell nucleus labels, and the cell nucleus labels at least include the following marks: the size, shape, overlapping boundaries and cell nucleus clustering of the cell nucleus.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining a sample image; the sample image is an image containing a target cell nucleus; Cropping the sample image into a plurality of sample slice images of preset sizes according to pre-configured size parameters; Mapping each of the sample slice images to a one-dimensional vector to obtain a sample vector corresponding to each of the sample slice images; Marking the cell nucleus features included in each of the sample vectors to add a cell nucleus label; The sample set is generated based on the sample vector and the cell nucleus label added to the sample vector; the sample set is used to train the VIT model.

5. The method according to claim 1, characterized in that: The step of extracting features from the feature slice graph to obtain feature information contained in each feature slice graph comprises: Inputting the feature slice graph into a pre-trained feature extraction network; Extracting feature information contained in each of the feature slice graphs through the feature extraction network; Wherein, the feature extraction network is a pre-trained deep convolutional neural network.

6. The method according to claim 1, characterized in that The method further comprises: Inputting the target feature matrix into a pre-trained classifier model; Outputting classification information of the target feature matrix through the classifier model; The classifier model is obtained by training based on a multi-layer perceptron (MLP) model.

7. The method according to claim 1, characterized in that The steps of obtaining the pathological image to be processed include: Obtain the pyramid structure model of pathological images; Pathological pictures of a preset level are selected from the pyramid structure model as pathological pictures to be processed.

8. A pathological image processing device, characterized in that: The device comprises: A cropping module, used for acquiring a pathological image to be processed, and cropping the pathological image to be processed into a plurality of slice images of preset sizes according to pre-configured size parameters; A preprocessing module, used for performing color processing on the plurality of slice images, and screening out at least one target slice image; A segmentation module, used for performing a graphic segmentation process on each of the target slice images to obtain a target cell nucleus region contained in the target slice image; A screening module, used for determining the target slice image whose target cell nucleus area is larger than a preset threshold as a characteristic slice image; A feature extraction module, used to extract features from the feature slice graphs to obtain feature information contained in each feature slice graph; The feature fusion module is used to fuse the feature information to obtain a target feature matrix corresponding to the pathological image to be processed, and the target feature matrix is ​​used to classify the pathological image to be processed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images