Image processing method, computer readable storage medium and computer equipment

Abstract

The invention discloses an image processing method, a computer readable storage medium and computer equipment. The method comprises: acquiring a pathological image; dividing the pathological image to obtain a plurality of block regions; carrying out ROI region segmentation on the plurality of block regions to obtain RoI regions and non-RoI regions respectively included in the plurality of block regions; based on the RoI regions and the non-RoI regions respectively included in the plurality of block regions, determining the area proportion of the RoI regions and the non-RoI regions in the pathological image; sampling an RoI region and a non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region; performing nucleus segmentation and classification on the target RoI region to obtain a first tumor cell proportion corresponding to the target RoI region, and performing nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell proportion corresponding to the target non-RoI region; and obtaining a target tumor cell proportion value of the pathological image based on the first tumor cell proportion, the second tumor cell proportion and the area proportion.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to an image processing method, a computer-readable storage medium, and a computer device. Background Art

[0002] In the existing analysis and processing of pathological images, for the interpretation of conventional pathological tissue sections, the current mainly adopted method is the manual calculation method of conventional pathologists. Due to the scarcity of pathologists, uneven diagnostic levels, and low automation of related equipment, the efficiency of pathological diagnosis is low and the consistency of calculation results is poor. In order to improve the automation degree of auxiliary diagnosis in pathological images and achieve accurate and rapid auxiliary diagnosis, various methods have been tried in digital pathology technology. Method 1: Perform enhancement, normalization, and other basic digital image processing on H&E stained images. However, when using this method, the accuracy of the processing results is relatively low. Method 2: Based on traditional image segmentation methods, such as filtering, moment descriptors, neighborhood analysis, etc. However, when using this method, the accuracy is also relatively low, and at the same time, the processing time required is long, resulting in low efficiency.

[0003] Therefore, in the related art, there is a problem of low accuracy when determining the tumor region in pathological images.

[0004] For the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of the present invention provide an image processing method, a computer-readable storage medium, and a computer device to at least solve the technical problem of low accuracy when determining the tumor region in pathological images in the related art.

[0006] According to one aspect of the embodiments of the present invention, an image processing method is provided, which includes: acquiring a pathological image; dividing the pathological image to obtain a plurality of sub-block regions; respectively performing Region of Interest (RoI) region segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively; determining the area ratio of the RoI regions to the non-RoI regions in the pathological image based on the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively; sampling the RoI regions and non-RoI regions in the pathological image to obtain target RoI regions and target non-RoI regions; performing nucleus segmentation and classification on the target RoI regions to obtain a first tumor cell proportion value corresponding to the target RoI regions, and performing nucleus segmentation and classification on the target non-RoI regions to obtain a second tumor cell proportion value corresponding to the target non-RoI regions; and obtaining a target tumor cell proportion value of the pathological image based on the first tumor cell proportion value, the second tumor cell proportion value, and the area ratio.

[0007] Optionally, the sampling performed on the RoI regions and non-RoI regions in the pathological image is upsampling.

[0008] Optionally, the step of obtaining the target tumor cell proportion value of the pathological image based on the first tumor cell proportion value, the second tumor cell proportion value, and the area ratio includes: determining a first weight corresponding to the first tumor cell proportion value and a second weight corresponding to the second tumor cell proportion value based on the area ratio; and determining the target tumor cell proportion value of the pathological image based on the first tumor cell proportion value and the first weight, and the second tumor cell proportion value and the second weight.

[0009] Optionally, the step of respectively performing Region of Interest (RoI) region segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively includes: using an RoI segmentation model to respectively perform Region of Interest (RoI) region segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively, where the RoI segmentation model is trained using a first data set, the first data set includes a plurality of first samples, and each first sample includes: a first sample pathological image, and the RoI region segmentation result corresponding to the first sample pathological image.

[0010] Optionally, the method further includes: adjusting the size of the first sample pathological images in the first data set to obtain first target sample pathological images, where the size of the first target sample pathological images is larger than the size of the RoI prediction region; and using the first target sample pathological images in the first data set for machine training to obtain the RoI segmentation model.

[0011] Optionally, the nuclear segmentation and classification of the target RoI region to obtain the first tumor cell proportion value corresponding to the target Rol region, and the nuclear segmentation and classification of the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region include: using a nuclear segmentation and classification model to perform nuclear segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and performing nuclear segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region, wherein the nuclear segmentation and classification model is trained using a second data set, the second data set includes a plurality of second samples, and the second sample includes: a second sample pathological image and the tumor cell proportion value corresponding to the second sample pathological image.

[0012] Optionally, the division of the pathological image to obtain a plurality of sub-regions, and the segmentation of the region of interest (RoI) for each of the plurality of sub-regions to obtain the RoI region and the non-RoI region included in each of the plurality of sub-regions include: magnifying the pathological image by a first predetermined multiple to obtain a first magnified image, where the first predetermined multiple is lower than the first multiple; dividing the first magnified image to obtain the plurality of sub-regions, and respectively performing segmentation of the region of interest (RoI) for each of the plurality of sub-regions to obtain the RoI region and the non-RoI region included in each of the plurality of sub-regions.

[0013] Optionally, the sampling of the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region, the nuclear segmentation and classification of the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and the nuclear segmentation and classification of the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region include: magnifying the pathological image including the RoI region and the non-RoI region by a second predetermined multiple to obtain a second magnified image, where the second predetermined multiple is higher than the second multiple, and the second multiple is greater than the first multiple; sampling the RoI region and the non-RoI region in the second magnified image to obtain a target RoI region and a target non-RoI region, and performing nuclear segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and performing nuclear segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region.

[0014] According to another aspect of the present invention, there is provided an image processing apparatus, including an acquisition module for acquiring a pathological image; a first partitioning module for partitioning the pathological image to obtain a plurality of sub-block regions; a second partitioning module for respectively performing region of interest (RoI) region segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively; a first determination module for determining the area ratio of the RoI region to the non-RoI region in the pathological image based on the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively; a sampling module for sampling the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region; a third partitioning module for performing nucleus segmentation and classification on the target RoI region to obtain a first tumor cell proportion value corresponding to the target RoI region, and performing nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell proportion value corresponding to the target non-RoI region; and a second determination module for obtaining a target tumor cell proportion value of the pathological image based on the first tumor cell proportion value, the second tumor cell proportion value, and the area ratio.

[0015] Optionally, for the sampling module, the sampling method for the RoI region and the non-RoI region in the pathological image is upsampling to obtain a target RoI region and a target non-RoI region.

[0016] According to still another aspect of the present invention, there is provided a computer-readable storage medium, which includes an executable program stored therein. When the executable program runs, it controls the device where the computer-readable storage medium is located to execute the image processing method described in any one of the above.

[0017] According to yet another aspect of the present invention, there is provided an electronic device, including: a memory storing an executable program; and a processor for running the program, where when the program runs, it executes the image processing method described in any one of the above.

[0018] In a specific embodiment of the present invention, the pathological image is partitioned to obtain an RoI region and a non-RoI region, sampled from the RoI region and the non-RoI region to obtain a target RoI region and a target non-RoI region, and a first tumor cell proportion value corresponding to the target RoI region and a second tumor cell proportion value corresponding to the target non-RoI region are obtained respectively. Then, based on the area ratio of the RoI region to the non-RoI region, and the above first tumor cell proportion value and second tumor cell proportion value, the target tumor cell proportion value of the pathological image is determined.

[0019] In another specific embodiment of the present invention, since the RoI region may be the region where tumor cells are more likely to be located, based on the differentiation and division of pathological images and the sampling process after regional division, the data processing volume is effectively reduced. Therefore, the efficiency of determining the tumor region in pathological images is effectively achieved.

[0020] In another specific embodiment of the present invention, when determining the target tumor cell proportion value, it is determined based on the first tumor cell proportion value corresponding to the target RoI region, the second tumor cell proportion value corresponding to the target non-RoI region, and the area ratio between the RoI region and the non-RoI region. Therefore, when determining the target tumor cell proportion value, the final target tumor cell proportion value of the pathological image is determined proportionally according to the area ratio of the regions, effectively improving the accuracy of determining the tumor region in the pathological image.

[0021] In another specific embodiment of the present invention, the present invention effectively avoids performing nucleus segmentation and classification processing on the whole pathological image, improves the processing efficiency, thereby realizing the technical effect of accurately discriminating categories according to the area ratio while improving the time processing efficiency, and providing a quantitative evaluation index for auxiliary diagnosis. Furthermore, the technical problem of inaccuracy in determining the tumor region in pathological images is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention, form a part of this application, and the schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0023] Figure 1 is a flowchart of an image processing method according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of an encoding and decoding network structure provided according to an alternative embodiment of the present invention;

[0025] Figure 3 is a schematic illustration of an encoding and decoding structure provided according to an alternative embodiment of the present invention Figure 1 ;

[0026] Figure 4 is a schematic illustration of an encoding and decoding structure provided according to an alternative embodiment of the present invention Figure 2 ;

[0027] Figure 5 is a schematic diagram of a cell segmentation model provided according to an alternative embodiment of the present invention;

[0028] Figure 6 is a flowchart of pathological image analysis of an image processing method according to an embodiment of the present invention;

[0029] Figure 7 It is a structural block diagram of a computer device according to an embodiment of the present invention. Specific embodiments

[0030] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some nouns or terms that appear in the process of describing the embodiments of the present application are applicable to the following explanations:

[0033] Tumor Cell Fraction (TCF for short): It is measured by the proportion of the number of tumor cells in the whole pathological image relative to the total number of cells.

[0034] Region of interest (RoI for short) model: This model is usually widely used in computer vision tasks, such as object detection, image segmentation, face recognition, etc. By clearly defining the region of interest, the amount of calculation can be reduced, the efficiency of recognition and processing can be improved, and it is also helpful to focus on the detailed analysis of specific regions. Different tasks and application fields may have different requirements and specifications for the definition of the RoI region model.

[0035] Coding and decoding network structure: The coding and decoding network structure is a deep learning network structure commonly used in image segmentation tasks. It consists of two parts: an encoder and a decoder. The encoder is used to extract the features of the input image, and the decoder is used to map the extracted features back to the original image size and generate pixel-level segmentation results.

[0036] Nucleus segmentation and classification model (Hover-net): It is a neural network model for nucleus segmentation and classification. The main goal of this model is to simultaneously achieve nucleus segmentation and type classification in histological images. One branch of Hover-Net is responsible for performing the nucleus segmentation task. It uses image segmentation technology to segment the nuclei in the image and form pixel-level segmentation results. The application fields of Hover-Net mainly involve medical image analysis, especially in the field of pathology, for identifying and analyzing nuclei in histological images. Its ability to simultaneously segment and classify nuclei makes it potentially valuable in automated pathology analysis and medical diagnosis.

[0037] According to an embodiment of the present invention, an embodiment of an image processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0038] Figure 1 is a flowchart of the image processing method according to an embodiment of the present invention. As Figure 1 shown, the method includes the following steps:

[0039] Step S102, obtain a pathological image.

[0040] As an optional embodiment, the execution subject of the method in this embodiment can be a terminal or a server for processing pathological images. For example, when applied to a terminal for processing pathological images, that is, when applied to a terminal, it can easily implement pathological image processing in a simple processing scenario; and when applied to a server, it can call the rich computing resources of the server, or a relatively larger and more accurate processing model, and thus can more accurately perform corresponding processing on the pathological image.

[0041] It should be noted that the types of the above terminals can be various. For example, it can be a mobile terminal with certain computing capabilities, or a fixed computer device with computing capabilities, etc. The types of the above servers can also be various. For example, it can be a local server or a virtual cloud server. The server can be a single computer device according to its computing power, or a computer cluster integrated by multiple computer devices. Preferably, the execution subject of the method in this embodiment can simply be a computer device used to identify tumor regions from pathological images.

[0042] As an alternative embodiment, the above pathological image refers to a tissue section image obtained by microscopic observation and shooting. This pathological image can show the cell structure, tissue structure and pathological conditions of the tissue. For example, the special staining in the above pathological image can be obtained by staining the tissue section with hematoxylin-eosin (H&E).

[0043] In addition, when obtaining a pathological image, various methods can be adopted. For example, a high-resolution pathological image can be obtained by using a digital camera and synthesizing images through multiple shootings. It is also possible to scan the tissue specimen on the glass slide in the pathology laboratory through a digital pathology microscopic scanning system to obtain a pathological image. It is also possible to obtain the required pathological image through the open data sets of some research institutions and medical institutions.

[0044] Step S104: Divide the pathological image to obtain multiple sub-block regions.

[0045] As an alternative embodiment, the above pathological image can be a pathological image with a resolution level of tens of billions of pixels. The sub-block regions refer to the sub-block regions obtained when dividing the above pathological image. When dividing, it can be divided according to certain division conditions. For example, it can be divided according to the required magnification. For example, the pathological image can be divided at a low magnification. For example, it can be carried out at a magnification of 5x. When dividing the pathological image, it can be an equal division, and the sizes of the obtained sub-block regions are equal. Of course, it can also be an unequal division, and the sizes of the obtained sub-block regions are not equal. Depending on the selected division method, the sizes of the obtained sub-block regions are different.

[0046] Step S106: Respectively perform region of interest (RoI) segmentation on multiple sub-block regions to obtain the RoI regions and non-RoI regions included in each of the multiple sub-block regions.

[0047] As an alternative embodiment, the above-mentioned region of interest (RoI) can be a tumor region or other regions of concern. When dividing the pathological image to obtain a plurality of sub-block regions and separately performing RoI segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively, according to the division operation and the operation requirements of RoI segmentation, the pathological image can be placed at the corresponding magnification for operation. For example, when dividing the pathological image to obtain a plurality of sub-block regions and separately performing RoI segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively, the following processing method can be adopted: magnify the pathological image by a first predetermined magnification to obtain a first magnified image, where the first predetermined magnification is lower than the first magnification, divide the first magnified image to obtain a plurality of sub-block regions, and separately perform RoI segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively. The first predetermined magnification can be at a magnification of 5x or 10x. Since the image resolution at a low magnification is smaller, the processing is faster, which can reduce the amount of calculation, reduce the memory consumption of the computing device, and improve the computing efficiency. In addition, the lower resolution will reduce the detailed information in the image and the influence of noise will also be correspondingly reduced, which helps to improve the accuracy of segmentation.

[0048] As an alternative embodiment, to improve the efficiency of region of interest segmentation, when performing RoI segmentation, an RoI segmentation model (RoI-segnet) can be used for specific RoI segmentation. For example, separately performing RoI segmentation on a plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively includes: using the RoI segmentation model to separately perform RoI segmentation on the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in the plurality of sub-block regions respectively, where the RoI segmentation model is trained using a first data set, the first data set includes a plurality of first samples, and the first sample includes: a first sample pathological image and the RoI region segmentation result corresponding to the first sample pathological image, where the RoI region segmentation result corresponding to the first sample pathological image can be obtained by manual annotation. Through the above RoI region segmentation model, the RoI region segmentation result of the pathological image can be obtained quickly and accurately.

[0049] As an alternative embodiment, to improve the training speed and accuracy of the RoI segmentation model. When performing machine training using the first sample pathological images in the first dataset, the size of the first sample pathological images in the first dataset can be adjusted to obtain first target sample pathological images, where the size of the first target sample pathological images is larger than the size of the RoI prediction region. Then, machine training is performed using the first target sample pathological images in the first dataset to obtain the RoI segmentation model. When adjusting the size of the first sample pathological images in the first dataset, the above-mentioned overlapping block method is adopted. This method not only ensures that there will be no ambiguity due to different segmentation results of multiple images, but also there will be no obvious stitching boundaries. Machine training is performed using the first target sample pathological images in the first dataset after the above adjustment to obtain the RoI segmentation model. Since during machine training, the RoI segmentation model can learn to more accurately segment the RoI region, therefore, the RoI segmentation model obtained by machine training based on the first dataset after the above adjustment can also be more accurate when segmenting the RoI region of the actual pathological image, that is, it can more accurately obtain the RoI region segmentation result of the actual pathological image. It should be noted that to further improve the segmentation accuracy of the pathological image, some preprocessing operations can be performed on the pathological image before actual segmentation. For example, operations such as denoising, enhancing contrast, and adjusting brightness can be performed on the pathological image to improve the segmentation accuracy.

[0050] Step S108: Based on the RoI regions and non-RoI regions respectively included in multiple block regions, determine the area ratio between the RoI region and the non-RoI region in the pathological image.

[0051] As an alternative embodiment, the area ratio between the RoI region and the non-RoI region in the above pathological image can be determined according to the areas of the RoI regions and non-RoI regions respectively included in multiple block regions. For example, the following direct area summation processing method can be adopted: Add up the areas of the RoI regions in multiple block regions to obtain the area of the RoI region in the entire pathological image; Add up the areas of the non-RoI regions in multiple block regions to obtain the area of the non-RoI region in the entire pathological image. The area ratio between the RoI region and the non-RoI region in the pathological image is obtained by the ratio of the area of the RoI region in the entire pathological image to the area of the non-RoI region in the entire pathological image.

[0052] Step S110: Sample the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region.

[0053] As an alternative embodiment, the above-mentioned target RoI region and target non-RoI region are sample blocks obtained by sampling the RoI region and non-RoI region in the pathological image. The sample blocks obtained by sampling can be the above-mentioned target RoI region and target non-RoI region. In addition, there are various ways to sample the RoI region and non-RoI region in the pathological image, such as random sampling, systematic sampling, stratified sampling, cluster sampling, etc.

[0054] Step S112: Perform nucleus segmentation and classification on the target RoI region to obtain a first tumor cell proportion value corresponding to the target RoI region, and perform nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell proportion value corresponding to the target non-RoI region.

[0055] As an alternative embodiment, the above-mentioned first tumor cell proportion value may refer to the proportion of tumor cells in the target RoI region among all the cells in the target RoI region, and the above-mentioned second tumor cell proportion value refers to the proportion of tumor cells in the target non-RoI region among all the cells in the target non-RoI region. Performing nucleus segmentation and classification on the above-mentioned target RoI region and target non-RoI region obtained by sampling can avoid performing cell segmentation and classification on the entire image region, thereby reducing the data processing time.

[0056] When performing nuclear segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target Rol region, and performing nuclear segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region, various methods can be adopted. For example, the following processing method can be used: Use a nuclear segmentation and classification model to perform nuclear segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and perform nuclear segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region. Among them, the nuclear segmentation and classification model is trained using a second dataset, and the second dataset includes multiple second samples. The second sample includes: a second sample pathological image, and the tumor cell proportion value corresponding to the second sample pathological image. By using the nuclear segmentation and classification model to perform nuclear segmentation and classification on the target RoI region and the target non-RoI region, since the nuclear segmentation and classification model is based on a convolutional neural network and can perform nuclear instance segmentation and classification simultaneously, not only can the efficiency of segmenting the target RoI region and the target non-RoI region be achieved, but also accurate segmentation can be achieved. In addition, when the nuclear segmentation and classification model performs nuclear segmentation and classification on the target RoI region and the target non-RoI region, various methods can also be adopted. When using the horizontal and vertical distances between a nuclear pixel and its centroid to separate clustered cells, since the horizontal and vertical distances between a nuclear pixel and its centroid can more accurately locate tumor type cells, the accuracy of nuclear segmentation and classification can be further improved to a certain extent. After sampling the RoI region and the non-RoI region in the pathological image of step S110 above, the sampled RoI region and non-RoI region are obtained, and the target RoI region and the target non-RoI region are obtained. Then, the trained nuclear segmentation and classification model described above is used to perform nuclear segmentation and classification on the target RoI region and the target non-RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region and the second tumor cell proportion value corresponding to the target non-RoI region.

[0057] As an alternative embodiment, when sampling the RoI region and the non-RoI region in the pathological image to obtain the target RoI region and the target non-RoI region, and performing nucleus segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and performing nucleus segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region, this can also be completed at the magnification required for this operation. For example, when sampling the RoI region and the non-RoI region in the pathological image to obtain the target RoI region and the target non-RoI region, and performing nucleus segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and performing nucleus segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region, the following processing can be adopted: magnify the pathological image including the RoI region and the non-RoI region by a second predetermined multiple to obtain a second magnified image, where the second predetermined multiple is higher than the second multiple, and the second multiple is greater than the first multiple; sample the RoI region and the non-RoI region in the second magnified image to obtain the target RoI region and the target non-RoI region, and perform nucleus segmentation and classification on the target RoI region to obtain the first tumor cell proportion value corresponding to the target RoI region, and perform nucleus segmentation and classification on the target non-RoI region to obtain the second tumor cell proportion value corresponding to the target non-RoI region.

[0058] In a specific implementation manner, when sampling the RoI region and the non-RoI region in the second magnified image, upsampling can be used for sampling to obtain the target RoI region and the target non-RoI region. Optionally, the target RoI region and the target non-RoI region can be high-resolution RoI region and non-RoI region. Then, nucleus segmentation and classification processing are performed on the obtained high-resolution target RoI region and high-resolution target non-RoI region. Among them, the above operations are performed at a second predetermined multiple higher than the second multiple for magnifying the pathological image. For example, the second multiple can be at a magnification of 40x or more. Performing pathological sampling at the second predetermined multiple can obtain a higher-quality pathological image record.

[0059] Step S114, based on the first tumor cell proportion value, the second tumor cell proportion value, and the area proportion, obtain the target tumor cell proportion value of the pathological image.

[0060] As an alternative embodiment, the first tumor cell proportion value corresponding to the target RoI region is weighted according to the area ratio of the RoI region to the non-RoI region in the pathological image to obtain the first weight corresponding to the first tumor cell proportion value corresponding to the target RoI region. The second tumor cell proportion value corresponding to the target non-RoI region is weighted according to the area ratio of the RoI region to the non-RoI region in the pathological image to obtain the second weight corresponding to the second tumor cell proportion value corresponding to the target non-RoI region. Based on the first tumor cell proportion value and the first weight, as well as the second tumor cell proportion value and the second weight, the target tumor cell proportion value of the pathological image is determined.

[0061] For example, when the area ratio of the RoI region to the non-RoI region in the pathological image is 2:5, the first tumor cell proportion value corresponding to the target RoI region is 80%, and the second tumor cell proportion value corresponding to the target non-RoI region is 10%, the weight corresponding to the first tumor cell proportion value corresponding to the target RoI region is 2:7, and the weight corresponding to the second tumor cell proportion value corresponding to the target non-RoI region is 5:7. Finally, the target tumor cell proportion value of the pathological image is calculated to be 22.93%.

[0062] In the embodiment of the present invention, the pathological image is divided to obtain the RoI region and the non-RoI region, samples are taken from the RoI region and the non-RoI region to obtain the target RoI region and the target non-RoI region, and the first tumor cell proportion value corresponding to the target RoI region and the second tumor cell proportion value corresponding to the target non-RoI region are obtained respectively. Then, based on the area ratio of the RoI region to the non-RoI region, as well as the above first tumor cell proportion value and the second tumor cell proportion value, the target tumor cell proportion value of the pathological image is determined. Since the RoI region may be the region where tumor cells are more likely to be located, based on the differentiated division of the pathological image and the sampling process after division of the regions, the data processing amount is effectively reduced. Therefore, the efficiency of determining the tumor region in the pathological image is effectively achieved. In addition, when determining the target tumor cell proportion value, it is determined based on the first tumor cell proportion value corresponding to the target RoI region, the second tumor cell proportion value corresponding to the target non-RoI region, and the area ratio of the RoI region to the non-RoI region. Therefore, when determining the target tumor cell proportion value, the final target tumor cell proportion value of the pathological image is determined proportionally according to the area ratio of the regions, effectively improving the accuracy of determining the tumor region in the pathological image. Therefore, the technical effect of effectively avoiding nuclear segmentation and classification processing of the full-pathological image, improving the processing efficiency, thus achieving accurate discrimination of categories according to the area ratio while improving the time processing efficiency, and providing a quantitative evaluation index for auxiliary diagnosis is achieved. Furthermore, the technical problem of inaccuracy in determining the tumor region in the pathological image is solved.

[0063] Based on the above embodiments and alternative embodiments, an alternative implementation manner is also provided.

[0064] In related technologies, for the interpretation of conventional pathological tissue sections, hematoxylin and eosin (H&E) staining is mainly used. Then, a pathologist makes manual judgments and annotations through observation under a microscope, that is, the pathologist observes the tissue section stained with hematoxylin and eosin (H&E) under the microscope and manually estimates the proportion of tumor cells. Due to the scarcity of pathologists and the uneven levels, and the low degree of automation of related equipment, the efficiency of pathological diagnosis is low and the consistency of image annotation is poor. When performing auxiliary diagnosis on pathological images, under digital pathology technology, tissue sections are digitized through a digital pathology scanner. The following several technical solutions for realizing the automation of auxiliary diagnosis based on digital image processing technology can be attempted:

[0065] When performing enhancement, normalization, and other basic digital image processing on H&E staining images, the processing results given by this solution have low accuracy, so the guidance for doctors' diagnosis is low, and it is more about improving the image quality and contrast.

[0066] Based on traditional solutions for image segmentation, such as filtering, moment descriptors, neighborhood analysis, etc., although such solutions give relatively clear auxiliary diagnosis suggestions, there are many misjudgments, resulting in low accuracy and poor effects. At the same time, there is also the problem of high time complexity.

[0067] Based on deep learning for image processing, this solution performs cell segmentation, region segmentation, classification, etc. on the original stained image or the preprocessed image. The existing such solutions have a certain improvement in the accuracy of auxiliary diagnosis compared with traditional methods for image segmentation, but the problems of serious time consumption and misjudgment still exist.

[0068] Therefore, the following problems and disadvantages exist in the process of performing pathological image processing by the above solutions: When performing enhancement, normalization, and other basic digital image processing on H&E staining images, there are many misjudgments during image segmentation of pathological images, the accuracy is not high, and the guidance for doctors is low. When performing image segmentation based on traditional solutions, the following problems exist: Although relatively clear auxiliary diagnosis suggestions are given, there are many misjudgments, resulting in low accuracy and poor effects. In addition, there is the problem of high time complexity in pathological image processing. Based on deep learning for image processing, this solution is for the original stained image or the preprocessed image, but the problems of serious time consumption and misjudgment still exist.

[0069] Generally speaking, the above technical solutions all have certain limitations and problems when implementing pathological image processing and analysis. Therefore, in the embodiments of the present invention, by predicting the proportion of tumor cells in pathological sections through a deep learning model, a more efficient, accurate and reliable solution is provided, overcoming some limitations existing in the above technical solutions.

[0070] Therefore, in an alternative embodiment of the present invention, an automatic calculation method for the proportion of tumor cells in pathological tissues based on a deep learning model is provided. In this method, the RoI segmentation model is combined with the nucleus segmentation and classification model to achieve accurate discrimination of categories while improving the time processing efficiency, and to provide a technical effect of a quantitative evaluation index for determining the tumor region, thereby solving the technical problem of low accuracy in determining the tumor region in pathological images in the related art.

[0071] In an alternative embodiment of the present invention, the RoI region segmentation model is used to perform RoI region of interest segmentation on a low-resolution 5x image to obtain the RoI region and the non-RoI region. By randomly sampling in the RoI and non-RoI regions, regions to be inferred are selected on a high-magnification 40x image, so as to reduce the inference region of the model, effectively reduce the data processing volume, avoid improving the time processing efficiency while accurately discriminating categories, obtain the tumor proportion in the pathological image, and achieve the goal of improving the accuracy of tumor region determination according to the tumor proportion in the pathological image.

[0072] It should be noted that, in an alternative embodiment of the present invention, the target object is described by taking the pathological image of a breast cancer sample as an example.

[0073] In an alternative embodiment of the present invention, a breast cancer pathological section image stained with H&E is used to determine the tumor region, that is, the proportion of breast cancer tumor cells (Tumor Cell Fraction, TCF) in the pathological section is predicted through a deep learning model. It is defined as an objective benchmark during the breast cancer tumor cell scoring process, and is measured by the proportion of the number of breast cancer tumor cells in the whole pathological image relative to the total number of cells.

[0074] In an alternative embodiment of the present invention, two datasets are used in the process of training the deep learning model:

[0075] Breast Cancer (BC) dataset, which contains 54 complete pathological section images from private breast cancer samples and the global tumor cell ratio of the corresponding sections is manually annotated by pathologists. The BC dataset will be used to verify the similarity between the global tumor cell ratio inferred by artificial intelligence (AI) in the complete pathological section images and the manual annotation results of pathologists. To train the region segmentation model, 10 section images are selected from it and the tumor regions are segmented and annotated by manual pathologists to construct the Breast Cancer Region of Interest (BCRoI) dataset. The BCRoI dataset will be used to train the model for predicting the tumor region in the complete pathological section images.

[0076] A scalable crowdsourcing deep learning method and dataset for nucleus classification, localization and segmentation (NuCLS) contains more than 220,000 labeled cell nuclei from breast cancer images of The Cancer Genome Atlas (TCGA). These cell nuclei are annotated through the collaborative efforts of pathologists using digital pathological section picture archives. These data can be used in various ways to develop and validate nuclear detection, classification and segmentation algorithms.

[0077] In an alternative embodiment of the present invention, two deep learning models are constructed for H&E stained breast cancer pathological section images, namely an RoI segmentation model and a network capable of simultaneously achieving nuclear segmentation and classification.

[0078] In an alternative embodiment of the present invention, the RoI segmentation model (RoI-segnet) is trained using the BCRoI dataset.

[0079] In an alternative embodiment of the present invention, 10 digital pathological images are selected from the BC dataset, and the regions of interest (RoI) are annotated by pathologists under the complete pathological section images at 40x magnification to obtain the Breast Cancer Region of Interest (BCRoI) dataset. Since the region of interest in the alternative embodiment of the present invention is the breast cancer tumor region, the RoI regions annotated by pathologists from the images are breast cancer tumor regions. Pathologists perform segmentation region annotation on each RoI region.

[0080] In an alternative embodiment of the present invention, the training and inference data of the RoI segmentation model are performed on the 5x low-magnification image. Specifically, the above-mentioned annotation data in which a pathologist annotates the region of interest (RoI) in the whole pathological section image at a magnification of 40x is downsampled by 8 times, and the image is the pathological section image at a magnification of 5x as the training and inference data. In an alternative embodiment of the present invention, 10-fold cross-validation training is performed on these 10 pieces of annotation data. Among them, the obtained training accuracy is 98%. The RoI segmentation model trained by the BCRoI dataset in the alternative embodiment of the present invention can more accurately divide the RoI region in the pathological image to verify the accuracy of the RoI segmentation model in dividing the RoI region.

[0081] In an alternative embodiment of the present invention, an encoder-decoder structure network is used to segment the RoI region. Figure 2 It is a schematic diagram of the encoder-decoder network structure provided according to an alternative embodiment of the present invention. It consists of two parts: an encoder and a decoder. The encoder is used to extract the features of the input image, and the decoder is used to map the extracted features back to the original image size and generate a pixel-level segmentation result. Figure 2 A completely symmetric encoder-decoder structure strategy is adopted. Due to the edge effect of the convolution operation, no padding operation is performed in the encoder-decoder structure, and the edges are directly discarded. Therefore, the input image of this RoI segmentation model is larger than the output image. Taking an image with an input of 1148*1148 as an example, the output result is only the RoI region prediction result corresponding to the middle 776*776 region.

[0082] Figure 3 It is a schematic diagram of the encoder-decoder structure provided according to an alternative embodiment of the present invention Figure 1 , MBConv (MobileInverted Bottleneck Convolution) is a convolutional neural network module. The downsampling of the MBConv module is achieved by using a convolution operation with a stride of 2, and it is usually used in lightweight neural network designs and is widely applied in tasks with limited computing resources executed on mobile devices. Figure 4 It is a schematic diagram of the encoder-decoder structure provided according to an alternative embodiment of the present invention Figure 2 , Fused-MBConv (Fused Mobile Inverted Bottleneck Convolution) is a structure used for neural network design and is usually used in computer vision tasks such as image classification and object detection. The upsampling operation in Fused-MBConv is achieved by adding a transposed convolution layer to the model. The transposed convolution layer is used to enlarge the size of the feature map to achieve the effect of upsampling.

[0083] In an alternative embodiment of the present invention, the inference process of the RoI region is performed on a 5x pathological section image, but the image at this magnification is still large (about tens of millions of pixels). Therefore, in an alternative embodiment of the present invention, the RoI region of a complete breast cancer pathological image is inferred and predicted by performing block-by-block inference on the breast cancer pathological image. Specifically, first, a complete pathological image is divided into multiple blocks, then the RoI region is segmented for each block one by one, and finally, the segmentation results of each block region are stitched back to obtain the RoI region and non-RoI region of the complete pathological image. Since the input size of the RoI segmentation model is larger than the size of the output pathological image including the RoI region and non-RoI region, in the inference process of the RoI region segmentation model, in an alternative embodiment of the present invention, a block-overlapping method is adopted in the breast cancer pathological image segmentation process, that is, there are overlapping regions between the segmented blocks. The size of the segmented block image is 1148*1148, and the size of the non-overlapping region is 776*776. This method not only ensures that there will be no ambiguity due to different segmentation results of multiple images, but also there will be no obvious stitching boundaries.

[0084] In an alternative embodiment of the present invention, a nucleus segmentation and classification model is utilized. The nucleus segmentation and classification model is trained with NuCLS to obtain a cell segmentation model for breast cancer images. The nucleus segmentation and classification model is a model based on a convolutional neural network and capable of simultaneously performing nuclear instance segmentation and classification. This network uses the horizontal and vertical distances between nuclear pixels and their centroids to separate clustered cells. Since the horizontal and vertical distances between nuclear pixels and their centroids can more accurately locate tumor type cells, the accuracy of nucleus segmentation and classification can be further improved to a certain extent, especially in regions with overlapping instances. Figure 5 It is a schematic diagram of the cell segmentation model provided according to an alternative embodiment of the present invention. According to the full-image segmentation result obtained by segmenting the region of interest of the pathological image using the above RoI segmentation model, random sampling is performed in the RoI region and non-RoI region of the pathological image at 40x magnification to obtain a series of sample blocks, and the sample blocks include the target RoI region and the target non-RoI region. As Figure 5 shown, the nucleus segmentation and classification model is used to perform nucleus segmentation and classification on the target RoI region and the target non-RoI region, so as to achieve precise positioning and classification of cells, calculate the tumor cell fraction TCF, and improve the accuracy of tumor region determination.

[0085] Figure 6 It is a pathological image analysis flowchart of the image processing method according to an embodiment of the present invention. In an alternative embodiment of the present invention, a pathological image analysis method process as shown in Figure 6 is constructed to improve the operation time and reduce the use of operation resources:

[0086] The region of interest of the pathological image is segmented using the RoI segmentation model. Block segmentation is performed on the 5x pathological image to obtain multiple block regions, and the region of interest is segmented for each block to obtain the RoI regions and non-RoI regions included in the multiple block regions respectively. For the pathological image at 5x magnification, segmentation is performed according to the preset size mentioned above. The size of the segmented block image is 1148*1148, and segmentation inference of the region of interest is performed for each region. The segmented mask images are merged to obtain the full-image segmentation result, and the area ratio of RoI and non-RoI in the breast cancer pathological image is obtained.

[0087] According to the full-image segmentation result obtained by segmenting the region of interest of the pathological image using the RoI segmentation model, random sampling is performed in the RoI region and non-RoI region at 40x magnification of the pathological image to obtain a series of sample blocks, and the sample blocks include the target RoI region and the target non-RoI region.

[0088] The nucleus segmentation and classification model is used to perform nucleus segmentation and detection on the sample blocks obtained by the above random sampling, namely the target RoI region and the target non-RoI region. The TCF in each sample block is statistically calculated to obtain the first tumor cell proportion value corresponding to the target RoI region and the second tumor cell proportion value corresponding to the target non-RoI region.

[0089] According to the first tumor cell proportion value corresponding to the target RoI region and the second tumor cell proportion value corresponding to the target non-RoI region obtained by the nucleus segmentation and classification model, weighted calculation is performed according to the area ratio of RoI and non-RoI obtained above:

[0090] TCF whole slide ≈w s S+w o O

[0091] S = the first tumor cell proportion value corresponding to the target RoI region;

[0092] O = the second tumor cell proportion value corresponding to the target non-RoI region;

[0093] W S = the first weight corresponding to the first tumor cell proportion value;

[0094] W o = the second weight corresponding to the second tumor cell proportion value.

[0095] Thus, the final TCF of breast cancer tumor cells in the whole image is obtained. In an alternative embodiment of the present invention, the proportion value of breast cancer tumor cells in the final breast cancer tumor pathological image is determined proportionally according to the area ratio of the region, effectively improving the accuracy of determining the tumor region in the breast tumor pathological image. While achieving accurate discrimination of categories based on the area ratio, the time processing efficiency is improved, providing a technical effect of a quantitative evaluation index for auxiliary diagnosis, and thus solving the technical problem of inaccuracy in determining the tumor region in the pathological image.

[0096] In an alternative embodiment of the present invention, the tumor region inference time can be compressed to 20 - 30 minutes. Because the constructed nucleus segmentation and classification algorithm is directly used to perform nucleus segmentation and classification of the breast cancer tumor region on the breast cancer tumor pathological image magnified 40x, and the first tumor cell proportion value corresponding to the target RoI region and the second tumor cell proportion value TCF corresponding to the target non - RoI region are statistically calculated. A whole 40x pathological image contains nearly 100,000 * 100,000 pixel points, and the processing time for one image is about 2 - 3 hours.

[0097] In an alternative embodiment of the present invention, the constructed software algorithm method was verified on 65 breast cancer samples collected from cooperative hospitals. The correlation coefficient between the output result of the algorithm and the manual annotation result of the pathologist reached above 0.75.

[0098] In summary, based on the deep - learning algorithm model to process digital pathological images, by randomly sampling and selecting the regions to be inferred in the RoI region and non - RoI region at high magnification, calculating the proportion of breast cancer tumor cells in the image, it can replace the conventional manual calculation method currently used in pathological diagnosis, achieving a significant reduction in the labor intensity of pathologists, saving the diagnosis and inference time, and improving the accuracy, objectivity, and consistency of tumor region determination.

[0099] In an embodiment of the present invention, an image - processing device is further provided. Figure 7 It is a schematic diagram of an image - processing device according to Embodiment 2 of the present invention, as Figure 7 shown. The device includes: an acquisition module 70, a first division module 71, a second division module 72, a first determination module 73, a sampling module 74, a third division module 75, and a second determination module 76. The device will be described in detail below.

[0100] An acquisition module 70 is configured to acquire a pathological image; a first partitioning module 71 is configured to partition the pathological image to obtain a plurality of sub-block regions; a second partitioning module 72 is configured to perform region of interest (RoI) segmentation on each of the plurality of sub-block regions to obtain the RoI regions and non-RoI regions included in each of the plurality of sub-block regions; a first determination module 73 is configured to determine, based on the RoI regions and non-RoI regions included in each of the plurality of sub-block regions, the area ratio between the RoI region and the non-RoI region in the pathological image; a sampling module 74 is configured to sample the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region; a third partitioning module 75 is configured to perform nucleus segmentation and classification on the target RoI region to obtain a first tumor cell proportion value corresponding to the target RoI region, and perform nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell proportion value corresponding to the target non-RoI region; a second determination module 76 is configured to obtain a target tumor cell proportion value of the pathological image based on the first tumor cell proportion value, the second tumor cell proportion value, and the area ratio.

[0101] In an embodiment of the present invention, there is also provided a computer-readable storage medium, which includes an executable program stored therein. When the executable program runs, it controls the device where the computer-readable storage medium is located to execute the image processing method described in any one of the above.

[0102] In an embodiment of the present invention, there is also provided an electronic device, including: a memory storing an executable program; a processor configured to run the program, where when the program runs, it executes the image processing method described in any one of the above.

[0103] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0104] In the above embodiments of the present invention, the descriptions of the respective embodiments have their own focuses. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the units or modules can be in an electrical or other form.

[0106] The unit described as a separating component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0107] In addition, each functional unit in various embodiments of the present invention may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0108] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0109] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire pathological images; Dividing the pathological image to obtain a plurality of block regions; Performing segmentation of the multiple block regions into regions of interest (ROIs) respectively, to obtain RoI regions and non-RoI regions respectively included in the multiple block regions; Determine the area ratio of the RoI area to the non-RoI area in the pathological image based on the RoI area and the non-RoI area respectively included in the multiple block areas; Sampling the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region; Performing cell nucleus segmentation and classification on the target RoI region to obtain a first tumor cell ratio value corresponding to the target RoI region, and performing cell nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell ratio value corresponding to the target non-RoI region; Based on the first tumor cell ratio value, the second tumor cell ratio value and the area ratio, a target tumor cell ratio value of the pathological image is obtained.

2. The method according to claim 1, characterized in that The step of obtaining a target tumor cell ratio value of the pathological image based on the first tumor cell ratio value, the second tumor cell ratio value and the area ratio includes: Based on the area ratio, determining a first weight corresponding to the first tumor cell ratio value and a second weight corresponding to the second tumor cell ratio value; Based on the first tumor cell ratio value and the first weight, and the second tumor cell ratio value and the second weight, a target tumor cell ratio value of the pathological image is determined.

3. The method according to claim 1 or 2, characterized in that: The RoI regions and non-RoI regions respectively included in the plurality of block regions are obtained by segmenting the plurality of block regions into interested RoI regions respectively. The RoI regions and non-RoI regions respectively included in the plurality of block regions include: The RoI segmentation model is used to segment the multiple block areas into RoI regions of interest, respectively, to obtain RoI regions and non-RoI regions respectively included in the multiple block areas, wherein the RoI segmentation model is trained using a first data set, the first data set includes multiple first samples, and the first samples include: a first sample pathological image, and a RoI region segmentation result corresponding to the first sample pathological image.

4. The method according to claim 3, characterized in that The method further comprises: Adjusting the size of the first sample pathology image in the first data set to obtain a first target sample pathology image, wherein the size of the first target sample pathology image is larger than the size of the RoI prediction region; The first target sample pathological image in the first data set is used for machine training to obtain the RoI segmentation model.

5. The method according to any one of claims 1 to 4, characterized in that The performing cell nucleus segmentation and classification on the target RoI region to obtain a first tumor cell ratio value corresponding to the target RoI region, and the performing cell nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell ratio value corresponding to the target non-RoI region, include: A cell nucleus segmentation and classification model is used to perform cell nucleus segmentation and classification on the target RoI area to obtain a first tumor cell ratio value corresponding to the target RoI area, and cell nucleus segmentation and classification are performed on the target non-RoI area to obtain a second tumor cell ratio value corresponding to the target non-RoI area, wherein the cell nucleus segmentation and classification model is trained using a second data set, the second data set includes multiple second samples, and the second samples include: a second sample pathological image, and a tumor cell ratio value corresponding to the second sample pathological image.

6. The method according to any one of claims 1 to 5, characterized in that The step of dividing the pathological image to obtain a plurality of block regions, and segmenting the plurality of block regions into regions of interest (ROIs) to obtain RoI regions and non-RoI regions respectively included in the plurality of block regions includes: Enlarging the pathological image by a first predetermined multiple to obtain a first enlarged image, wherein the first predetermined multiple is lower than the first multiple; The first enlarged image is divided to obtain the multiple block areas, and the multiple block areas are respectively segmented into RoI regions to obtain RoI regions and non-RoI regions respectively included in the multiple block areas.

7. The method according to claim 6, characterized in that The sampling of the RoI region and the non-RoI region in the pathological image to obtain a target RoI region and a target non-RoI region, and performing cell nucleus segmentation and classification on the target RoI region to obtain a first tumor cell ratio value corresponding to the target RoI region, and performing cell nucleus segmentation and classification on the target non-RoI region to obtain a second tumor cell ratio value corresponding to the target non-RoI region, includes: Amplify the pathological image including the RoI region and the non-RoI region by a second predetermined multiple to obtain a second magnified image, wherein the second predetermined multiple is higher than a second multiple, and the second multiple is greater than the first multiple; The RoI area and the non-RoI area in the second enlarged image are sampled to obtain a target RoI area and a target non-RoI area, and the target RoI area is segmented and classified for nuclei to obtain a first tumor cell ratio value corresponding to the target RoI area, and the target non-RoI area is segmented and classified for nuclei to obtain a second tumor cell ratio value corresponding to the target non-RoI area.

8. An image processing device, characterized in that: include: An acquisition module, used for acquiring pathological images; A first segmentation module is used to segment the pathological image to obtain a plurality of segmented regions; A second segmentation module is used to segment the multiple block areas into interested RoI regions respectively, so as to obtain RoI regions and non-RoI regions respectively included in the multiple block areas; A first determination module is used to determine the area ratio of the RoI area to the non-RoI area in the pathological image based on the RoI area and the non-RoI area respectively included in the multiple block areas; A sampling module, used for sampling the RoI area and the non-RoI area in the pathological image to obtain a target RoI area and a target non-RoI area; The third segmentation module is used to segment and classify the cell nuclei of the target RoI area to obtain a first tumor cell ratio value corresponding to the target RoI area, and segment and classify the cell nuclei of the target non-RoI area to obtain a second tumor cell ratio value corresponding to the target non-RoI area; The second determination module is used to obtain a target tumor cell ratio value of the pathological image based on the first tumor cell ratio value, the second tumor cell ratio value and the area ratio.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor executes the image processing method according to any one of claims 1 to 7.

Images