Immunohistochemistry slice image analysis method and device, electronic equipment and storage medium

Abstract

The application provides an immunohistochemical slice image analysis method and device, electronic equipment and storage medium. The method comprises the following steps: based on an immunohistochemical staining signal, extracting a strong staining signal sampling point in a tissue slice image to obtain a strong staining signal sampling point set; clustering the strong staining signal sampling points to determine a set of interest region images corresponding to the tissue slice image, wherein the set of interest region images comprises at least one interest region image; and based on the set of interest region images, determining a pathological analysis result of the tissue slice image. The method reduces the dependence on manual operation, avoids the generation of various biases and noises, and reduces the cost of immunohistochemical slice image analysis.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to immunohistochemical slide image analysis methods, apparatus, electronic devices, and storage media. Background Technology

[0002] Immunohistochemistry (IHC) is a commonly used technique and method in clinical pathological diagnosis. Immunohistochemistry applies the fundamental principle of immunology—the antigen-antibody reaction, where the binding between an antibody and an antigen is highly specific. A specific chemical substance is first extracted from tissues or cells to serve as an antigen or hapten. Then, specific antibodies are obtained by immunizing an animal, and these antibodies are used to detect similar antigenic substances in the tissues or cells. Since antigen-antibody complexes are colorless, histochemical methods (e.g., fluorescein, enzymes, metal ions, isotopes, etc.) are necessary to visualize the site of antigen-antibody binding, enabling qualitative, localization, or quantitative studies of unknown antigens in tissues or cells. Immunohistochemistry primarily targets tissue samples, which are obtained from patients (humans) or animals and are frozen or paraffin-embedded. These tissues are prepared into sections approximately 4 μm thick, mounted, and then processed.

[0003] Tissue microarrays (also known as tissue chips) involve arranging numerous individual tissue specimens in a regular array on the same substrate (most commonly a glass slide) for in situ histological studies of the same parameters. In essence, multiple tissues are placed on a single glass slide for immunohistochemical experiments. Finally, the tissues on this slide are photographed to obtain the immunohistochemical array image. Its greatest advantage lies in the fact that the experimental conditions for tissue samples on the chip are completely consistent, allowing for excellent quality control and saving time and reagents.

[0004] Based on immunohistochemical array images, each immunohistochemical slice in the array is analyzed to determine the presence, location, and intensity of tumors. Existing immunohistochemical analysis software requires manual selection of the region of interest, and manual setting of thresholds and parameters according to different experimental batches and the selected region of interest, before outputting corresponding analysis results. Researchers then rely on existing experience to determine whether the selected region contains a tumor and, if so, its intensity.

[0005] Extensive human intervention not only introduces biases but also requires highly skilled personnel and consumes significant human and material resources. This hinders large-scale experimental validation during drug development and screening, severely limiting throughput and keeping drug development costs high. Therefore, developing methods, devices, electronic equipment, and storage media for analyzing immunohistochemical slide images without relying on manual intervention is urgently needed. Summary of the Invention

[0006] This invention provides an immunohistochemical slide image analysis method, apparatus, electronic device, and storage medium, which can be used to solve the problem of reliance on manual analysis in related technologies.

[0007] In a first aspect, embodiments of the present invention provide an immunohistochemical slide image analysis method, the method comprising: extracting strong staining signal sampling points from a tissue slide image based on immunohistochemical staining signals to obtain a set of strong staining signal sampling points; clustering the strong staining signal sampling points to determine a set of regions of interest images corresponding to the tissue slide image, the set of regions of interest images including at least one region of interest image; and determining the pathological analysis result of the tissue slide image based on the set of regions of interest images.

[0008] In some optional implementations, the above-mentioned determination of the pathological analysis result of the tissue section image based on the set of region of interest images includes: for each region of interest image in the set of region of interest images, performing the following first tumor intensity value extraction operation: segmenting a strong tumor signal region image from the region of interest image; performing a clustering operation on the pixels in the strong tumor signal region image based on the position and pixel value of the pixels in the strong tumor signal region image to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixels in the strong tumor signal region image; determining the set of cell membrane region pixels based on the cluster edges of the tumor cell clusters; calculating the first tumor intensity value corresponding to the region of interest image based on the set of cell membrane region pixels; and determining the pathological analysis result of the tissue section image based on the first tumor intensity value corresponding to the region of interest image.

[0009] In some optional implementations, the method further includes: inputting each region of interest image in the region of interest image set into a pre-trained tumor intensity determination model to obtain a second tumor intensity value corresponding to the region of interest image, wherein the tumor intensity determination model is pre-trained through the following preset training steps: obtaining a training sample set, the training samples including sample images and corresponding labeled tumor intensity values ​​used to characterize the intensity of tissue tumors in the sample images; performing supervised training on the initial tumor intensity determination model based on the training sample set to obtain a tumor intensity determination model used to characterize the correspondence between images and tumor intensity values; and determining the pathological analysis results of the tissue slice image based on the second tumor intensity value corresponding to the region of interest image.

[0010] In some optional embodiments, the above-mentioned determination of the pathological analysis result of the tissue section image includes: calculating the fused tumor intensity value corresponding to the region of interest image based on the first tumor intensity value and the second tumor intensity value corresponding to the region of interest image; adding the fused tumor intensity value corresponding to each region of interest image in the set of regions of interest images corresponding to the tissue section image to the set of fused tumor intensity values ​​corresponding to the tissue section image; and calculating the pathological analysis result of the tissue section image based on the set of fused tumor intensity values ​​corresponding to the tissue section image.

[0011] In some optional implementations, the above-described extraction of strong staining signal sampling points from tissue section images based on immunohistochemical staining signals to obtain a set of strong staining signal sampling points includes: performing channel separation operations on the tissue section images to obtain a staining signal channel map corresponding to the tissue section images; performing binarization operations on the staining signal channel map to obtain a mask of tumor staining signals; performing at least one morphological operation on the mask of tumor staining signals to obtain a mask of the tumor region; performing grayscale operations on the tissue section images to obtain grayscale tissue section images; extracting a grayscale mask image of the tumor region corresponding to the mask of the tumor region from the grayscale tissue section images; and performing the following sampling point extraction operation on each pixel in the grayscale mask image of the tumor region: in response to the grayscale value of the pixel being greater than a preset extraction threshold, adding the pixel as a strong staining signal sampling point to the set of strong staining signal sampling points.

[0012] In some optional implementations, the above-mentioned clustering of strongly stained signal sampling points to determine the set of regions of interest images corresponding to the tissue slice image includes: performing a clustering operation on the strongly stained signal sampling points in the set of strongly stained signal sampling points based on a preset distance clustering algorithm to obtain a set of sampling point clusters, wherein the set of sampling point clusters includes a preset number of cluster centers for regions of interest; and cropping the tissue slice image based on the set of sampling point clusters to obtain the set of regions of interest images corresponding to the tissue slice image.

[0013] In some optional implementations, before extracting strong staining signal sampling points from tissue slice images based on immunohistochemical staining signals, the method further includes: acquiring an immunohistochemical image, wherein the immunohistochemical image is obtained by image acquisition of a tissue slice array including multiple tissue slice elements; determining the center coordinates of the tissue slices; and cropping the immunohistochemical image based on the center coordinates of the tissue slice elements to obtain a set of tissue slice images.

[0014] In some optional embodiments, the above method further includes: preprocessing the immunohistochemical image: binarizing the immunohistochemical image to obtain a binarized immunohistochemical image; performing at least one morphological operation on the binarized immunohistochemical image to obtain a tissue slice element region mask in the immunohistochemical image; obtaining the contour of each tissue slice element based on a preset edge detection algorithm for the region mask of the tissue slice elements; estimating the radius of the tissue slice corresponding to the tissue slice element based on the size and number of the tissue slice elements to obtain an estimated radius parameter; determining the set of center coordinates and the set of radii of the tissue slice elements based on the estimated radius parameter using a preset circle detection algorithm; and completing the set of center coordinates in response to the existence of undetected center coordinates of the tissue slice elements to obtain the center coordinates of the tissue slice elements.

[0015] In some optional implementations, the above-mentioned response to the existence of undetected center coordinates of tissue slice array elements, and the completion processing of the center coordinate set to obtain the center coordinates of the tissue slice array elements, includes: performing a position sorting operation on the center coordinates of the tissue slice array elements: in response to the absolute value of the difference between the first coordinate components of the center coordinates of two adjacent tissue slice array elements being less than or equal to a first preset radius, the center coordinates of the two tissue slice array elements are grouped together to obtain a set of center coordinate columns of tissue slice array elements; for each group in the set of center coordinate groups, the following completion operation is performed: in response to the distance between two adjacent center coordinates in the group or the distance between the second coordinate components of two adjacent center coordinates being greater than a second preset radius, linear interpolation is performed on the two center coordinates, and the linear interpolation result is added to the set of center coordinates as the center coordinates of the tissue slice array element; wherein, one of the first coordinate component and the second coordinate component is the abscissa, and the other is the ordinate.

[0016] In some optional implementations, the above-mentioned method of cropping the immunohistochemical image based on the center coordinates of the tissue slice array elements to obtain a set of tissue slice images includes: cropping the immunohistochemical image according to a third preset radius based on the center coordinates corresponding to the tissue slice array elements to obtain a set of tissue slice images.

[0017] In some optional implementations, the above method includes: assigning labels to tissue slide images in a tissue slide image set based on the center coordinates corresponding to the tissue slide array elements and preset grouping information; determining the pathological analysis results of the tissue slide images includes: determining the pathological analysis results corresponding to the labels.

[0018] Secondly, embodiments of the present invention provide an immunohistochemical slide image analysis device, comprising: an extraction module configured to extract strong staining signal sampling points from a tissue slide image based on immunohistochemical staining signals to obtain a set of strong staining signal sampling points; a region of interest determination module configured to cluster the strong staining signal sampling points to determine a set of region of interest images corresponding to the tissue slide image, wherein the set of region of interest images includes at least one region of interest image; and a pathological analysis module configured to determine the pathological analysis results of the tissue slide image based on the set of region of interest images.

[0019] In some optional implementations, the extraction module is further configured to: perform channel separation operation on the immunohistochemical staining signal of the tissue slice image to obtain a staining signal channel map corresponding to the tissue slice image; perform binarization operation on the staining signal channel map to obtain a mask of tumor staining signal; perform at least one morphological operation on the mask of tumor staining signal to obtain a mask of tumor region; perform grayscale operation on the tissue slice image to obtain a grayscale tissue slice image; extract the mask image of grayscale tumor region corresponding to the mask of tumor region from the grayscale tissue slice image; and perform the following sampling point extraction operation on each pixel in the mask image of grayscale tumor region: in response to the grayscale value of the pixel being greater than a preset extraction threshold, add the pixel as a strong staining signal sampling point to the set of strong staining signal sampling points.

[0020] In some optional implementations, before extracting strong staining signal sampling points from tissue slice images based on immunohistochemical staining signals, the method further includes: acquiring an immunohistochemical image, wherein the immunohistochemical image is obtained by image acquisition of a tissue slice array including multiple tissue slice elements; determining the center coordinates of the tissue slice elements; and cropping the immunohistochemical image based on the center coordinates of the tissue slice elements to obtain a set of tissue slice images.

[0021] In some optional embodiments, the above method further includes: preprocessing the immunohistochemical image: binarizing the immunohistochemical image to obtain a binarized immunohistochemical image; performing at least one morphological operation on the binarized immunohistochemical image to obtain a tissue slice element region mask in the immunohistochemical image; performing edge detection on the tissue slice element region mask to obtain the contour of each tissue slice element; estimating the tissue slice radius corresponding to the tissue slice element based on the size and number of tissue slice elements to obtain an estimated radius parameter; determining the set of center coordinates and the set of radii of the tissue slice elements based on the estimated radius parameter using a preset circular detection algorithm; and completing the set of center coordinates in response to the existence of undetected center coordinates of tissue slice elements to obtain the center coordinates of the tissue slice elements.

[0022] In some optional implementations, the above-mentioned response to the existence of undetected center coordinates of tissue slice array elements, performing a completion process on the set of center coordinates to obtain the center coordinates of the tissue slice array elements, includes: performing a position sorting operation on the center coordinates of the set of center coordinates: in response to the absolute value of the difference between the first coordinate components of the center coordinates of two adjacent tissue slice array elements being less than or equal to a first preset radius, the center coordinates of the two tissue slice array elements are grouped together to obtain a set of grouped center coordinates of tissue slice array elements; for each group in the set of grouped center coordinates, the following completion process is performed: in response to the distance between two adjacent center coordinates in the group or the distance between the second coordinate components of two adjacent center coordinates being greater than a second preset radius, linear interpolation is performed on the two center coordinates, and the result of the linear interpolation is added to the set of center coordinates as the center coordinates of the tissue slice array element; wherein, one of the first coordinate component and the second coordinate component is the abscissa, and the other is the ordinate.

[0023] In some optional implementations, the above method includes: cropping the immunohistochemical image according to a third preset radius based on the center coordinates corresponding to the tissue slice array elements to obtain a set of tissue slice images.

[0024] In some optional implementations, the above method includes: assigning labels to tissue slide images in a tissue slide image set based on the center coordinates corresponding to the tissue slide array elements and preset grouping information; determining the pathological analysis results of the tissue slide images includes: determining the pathological analysis results corresponding to the labels.

[0025] In some optional implementations, the aforementioned region of interest determination module is further configured to: perform clustering operations on the strongly stained signal sampling points in the set of strongly stained signal sampling points based on a preset distance clustering algorithm to obtain a set of sampling point clusters, wherein the set of sampling point clusters includes a preset number of cluster centers for the region of interest; based on the set of sampling point clusters, further crop the tissue slice image to obtain a set of region of interest images corresponding to the tissue slice image. In some optional implementations, the aforementioned pathological analysis module is further configured to: for each region of interest image in the set of region of interest images, perform the following first tumor intensity value extraction operation: segment the strongly stained tumor signal region image from the region of interest image; perform clustering operations on the pixels in the strongly stained tumor signal region image to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixels in the strongly stained tumor signal region image; determine the set of cell membrane region pixels based on the cluster edges of the tumor cell clusters; calculate the first tumor intensity value corresponding to the region of interest image based on the set of cell membrane region pixels; and determine the pathological analysis result of the tissue slice image based on the first tumor intensity value corresponding to the region of interest image.

[0026] In some optional implementations, the method further includes: inputting each region of interest image in the region of interest image set into a pre-trained tumor intensity determination model to obtain a second tumor intensity value corresponding to the region of interest image, wherein the tumor intensity determination model is pre-trained through the following preset training steps: obtaining a training sample set, the training samples including sample images and corresponding labeled tumor intensity values ​​used to characterize the intensity of tissue tumors in the sample images; performing supervised training on the initial tumor intensity determination model based on the training sample set to obtain a tumor intensity determination model used to characterize the correspondence between images and tumor intensity values; and determining the pathological analysis results of the tissue slice image based on the second tumor intensity value corresponding to the region of interest image.

[0027] In some optional embodiments, the above-mentioned determination of the pathological analysis result of the tissue section image includes: calculating the fused tumor intensity value corresponding to the region of interest image based on the first tumor intensity value and the second tumor intensity value corresponding to the region of interest image; adding the fused tumor intensity value corresponding to each region of interest image in the set of regions of interest images corresponding to the tissue section image to the set of fused tumor intensity values ​​corresponding to the tissue section image; and calculating the pathological analysis result of the tissue section image based on the set of fused tumor intensity values ​​corresponding to the tissue section image.

[0028] Thirdly, embodiments of the present invention provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, which, when executed by the one or more processors, cause the one or more processors to implement the method described in any implementation of the first aspect.

[0029] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by one or more processors, implements the method as described in any implementation of the first aspect.

[0030] To reduce the reliance on manual intervention in the analysis of immunohistochemical slide images and improve analytical results, the immunohistochemical slide image analysis method, apparatus, electronic device, and storage medium provided in the embodiments of the present invention extract strong staining signal sampling points from tissue slide images, cluster these sampling points to obtain cluster centers, and then crop the images around the cluster centers to obtain regions of interest (ROIs) related to tumor risk. This can automatically generate ROI images containing a large number of tumor areas with high tumor intensity, sampled at the required sampling density. This largely overcomes the reliance on manual operation, replacing the process of researchers observing and judging each region in the tissue slide to determine the ROI, avoiding various biases and noise, and reducing the cost of immunohistochemical slide image analysis. Attached Figure Description

[0031] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments, taken with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings:

[0032] Figure 1 This is a schematic diagram of an implementation environment in which an embodiment of the present invention can be applied.

[0033] Figure 2A This is a flowchart of an embodiment of the immunohistochemical slide image analysis method according to the present invention.

[0034] Figure 2B This is a breakdown flowchart of an embodiment of step 201 according to the present invention.

[0035] Figure 2C This is a breakdown flowchart of an embodiment of step 202 according to the present invention.

[0036] Figure 2D This is a breakdown flowchart of an embodiment of step 203 according to the present invention.

[0037] Figure 2E This is a breakdown flowchart of an embodiment of step 201' according to the present invention.

[0038] Figure 3 This is a schematic diagram of an embodiment of the immunohistochemical slide image analysis device according to the present invention.

[0039] Figure 4 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present invention.

[0040] Figure 5 The image shows a tumor region in a stained tissue section, which may include tumor cells and non-tumor cells. The cell membranes of tumor cells are stained brown, have a strong immunohistochemical staining signal, and the cell nuclei are swollen.

[0041] Figure 6 A step-by-step result diagram of an embodiment of step 202 of the present invention is shown.

[0042] Figure 7 The image shows the region of interest obtained after cropping in step 2022 of the present invention, wherein the brown signal is stronger in the cell membrane region.

[0043] Figure 8 A step-by-step result diagram of an embodiment of step 203 of the present invention is shown. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0045] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0046] Figure 1 A schematic diagram of an implementation environment in which the immunohistochemical slide image analysis method, apparatus, electronic device, and storage medium of the present invention can be applied is shown.

[0047] like Figure 1 As shown, the implementation environment includes an electronic device 100, and the immunohistochemical slide image analysis method in this embodiment of the invention can be executed by terminal devices 101, 102, 103, and 104. For example, the electronic device 100 may include at least one of a terminal device or a server.

[0048] Terminal devices 101, 102, 103, and 104 can be either hardware or software. When terminal devices 101, 102, 103, and 104 are hardware, they can be various electronic devices with a display screen that support information input (e.g., text input and / or voice input), including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, 103, and 104 are software, they can be installed on the terminal devices listed above. They can be implemented as multiple software programs or software modules (e.g., for providing immunohistochemical slide image analysis services) or as a single software program or software module. No specific limitations are imposed here.

[0049] It should be understood that Figure 1 The number of terminal devices shown is merely illustrative. Depending on implementation needs, any number of terminal devices can be used.

[0050] Continue to refer to Figure 2A The diagram illustrates a flowchart 200 of an embodiment of an immunohistochemical slide image analysis method according to the present invention, which includes the following steps:

[0051] Step 201: Based on the immunohistochemical staining signal, extract the strong staining signal sampling points in the tissue section image to obtain a set of strong staining signal sampling points.

[0052] Tissue sections can contain both tumor cells and non-tumor cells. Tumor cell membranes are stained, exhibiting strong immunohistochemical staining signals (e.g., ...). Figure 5 (As shown).

[0053] For example, the immunohistochemical staining signal is determined from an image, and can be the pixel value of each pixel in the image corresponding to the staining channel. The image corresponding to the staining channel can be determined based on a tissue section image, for example, by fusing the R and G channels in an RGB image to obtain the image corresponding to the staining channel.

[0054] In this embodiment, the entity executing the immunohistochemical slide image analysis method can first obtain the immunohistochemical staining signal, and then, in order to obtain the set of strong staining signal sampling points, it is necessary to extract the strong staining signal sampling points in the tissue slide image to be analyzed.

[0055] In some optional implementations, step 201 can be performed as follows: convert the tissue section image to a grayscale image, filter it using a preset threshold, and extract pixels larger than the preset threshold as sampling points for strong staining signals in the tissue section image. According to this optional implementation, sampling points with stronger staining signals in the tissue section image to be analyzed can be accurately and uniformly extracted based on the preset threshold, for use in subsequent steps to divide regions of interest. This can replace a large amount of repetitive manual work and reduce experimental noise.

[0056] In some alternative implementations, step 201 may include, for example: Figure 2B The following steps 2011 to 2016 are shown:

[0057] Step 2011: Perform channel separation operation on the immunohistochemical staining signal of the tissue section image to obtain the staining signal channel map corresponding to the tissue section image.

[0058] Step 2012: Binarize the staining signal channel map to obtain a mask of the tumor staining signal.

[0059] Step 2013: Perform at least one morphological operation on the mask of tumor staining signals to obtain the mask of the tumor region.

[0060] Step 2014: Perform a grayscale conversion operation on the tissue section image to obtain a grayscale tissue section image.

[0061] Step 2015: Extract the mask image of the gray-scale tumor region corresponding to the mask of the tumor region from the gray-scale tissue slice image.

[0062] Step 2016: Perform sampling point extraction operation on each pixel in the mask image of the grayscale tumor region.

[0063] Here, the sampling point extraction operation may specifically include: in response to the gray value of the pixel being greater than a preset extraction threshold, the pixel is used as a strong staining signal sampling point set.

[0064] According to this optional implementation, sampling points are extracted only within the mask area of ​​the tumor region, which can accurately and relatively precisely sample strong staining signal sampling points in the tissue slice image, avoiding the mistaken identification of pixels with large pixel values ​​outside the mask area as strong sampling signal sampling points.

[0065] Step 202: Cluster the sampling points of strong staining signals to determine the set of regions of interest images corresponding to the tissue slice images. The set of regions of interest images includes at least one region of interest image.

[0066] Typically, multiple regions of interest (ROIs) are selected from tissue section images to analyze the intensity of tumor cells. The goal is for each selected ROI to contain a significant amount of tumor tissue, and for multiple ROIs to include areas of high tumor intensity.

[0067] Strong staining signal sampling points are points with high tumor intensity, so the region of interest can be determined based on the distribution of strong staining signal sampling points.

[0068] In this embodiment, the region of interest image is determined based on the clustering results after clustering the strongly stained signal sampling points. This application does not specifically limit the algorithm used to perform this clustering process. For example, algorithms that perform this clustering process include, but are not limited to, K-Nearest Neighbor (KNN) algorithm, K-means Clustering Algorithm, K-means++ algorithm, bi-kmeans algorithm, DBSCAN algorithm, OPTICS algorithm, Agglomerative algorithm, etc. The clustering algorithm can be a distance-based clustering algorithm, where strongly stained signal sampling points that are close in distance are grouped into the same cluster.

[0069] In some alternative implementations, step 202 may include, for example: Figure 2C The following steps are shown from 2021 to 2022:

[0070] Step 2021: Based on a preset distance clustering algorithm, perform clustering operations on the strongly stained signal sampling points in the set of strongly stained signal sampling points to obtain a set of sampling point clusters, wherein the set of sampling point clusters includes a preset number of cluster centers for the region of interest.

[0071] Understandably, when you want to sample more regions of interest, you can set the number of preset regions of interest to be larger; when the tumor area is larger, you usually want to set the number of preset regions of interest to be larger in order to maintain the sampling density required to obtain accurate results.

[0072] Step 2022: Based on the cluster set of sampling points, the tissue slice image is cropped to obtain the set of regions of interest images corresponding to the tissue slice image.

[0073] In some optional implementations, steps 2021 to 2022 may be performed as follows: A preset number of regions of interest (e.g., 30) is established, and this number of regions of interest is used as the K-value in K-means clustering. Then, samples the strongly stained signal points (e.g., ...) from the set of strongly stained signal sampling points. Figure 6 Perform K-means clustering on (as shown in A) to obtain a preset number of interest regions (e.g., 30) of sampling points and their cluster centers (e.g., ...). Figure 6 As shown in B), that is, corresponding to sampling the same number (e.g., 30) of interest region images (such as...). Figure 6 As shown in C), the tissue slice image is then cropped using the cluster center as the center of the region of interest image, resulting in a set of region of interest images corresponding to the tissue slice image. The cropped region of interest images are shown in Figure C. Figure 7 As shown.

[0074] According to this optional implementation, clustering is performed on sampling points with strong staining signals in the tissue section image to obtain cluster centers (the number of cluster centers can be determined according to the number of regions of interest required). Then, the images of interest are cropped with the cluster centers as the center. This can automatically generate images of regions of interest that contain more tumor areas, have higher tumor intensity, and are sampled at the required sampling density. This replaces the process of researchers observing and judging each region in the tissue section to determine the regions of interest, and avoids the heavy reliance on human experience in immunohistochemical image analysis.

[0075] Step 203: Based on the image set of regions of interest, determine the pathological analysis results of the tissue slice images.

[0076] In this embodiment, the entity executing the immunohistochemical slide image analysis method has already obtained the set of region-of-interest (ROI) images corresponding to the tissue slide image to be analyzed in step 202. Generally, the RIO images can reflect the tumor signal intensity or other pathologically related indicators (e.g., the location of staining signals, organelle morphology, etc.) used to characterize the pathological condition of the tissue slide image. Therefore, the entity executing the immunohistochemical slide image analysis method can determine the pathological analysis results of the tissue slide image based on the RIO image set and the indicators reflected by the RIO images. According to this optional implementation, complex manual parameter and threshold settings are not required, and the judgment and analysis of pathological conditions such as tumor intensity of the sample can be automated and intelligent.

[0077] In this embodiment, three optional implementations A, B, and C of step 203 are provided as follows:

[0078] In an alternative implementation A, step 203 may include, for example: Figure 2D The following steps 2031A to 2032B are shown:

[0079] Step 2031A: For each region of interest image in the region of interest image set, perform the first tumor intensity value extraction operation.

[0080] Here, the first tumor intensity value extraction operation may specifically include: segmenting a strong tumor signal region image from the region of interest image; performing a clustering operation on the pixels in the strong tumor signal region image to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixels in the strong tumor signal region image; determining the set of cell membrane region pixels based on the cluster edges of each tumor cell cluster; and calculating the first tumor intensity value corresponding to the region of interest image based on the set of cell membrane region pixels.

[0081] For example, segmenting a region of strong tumor signal from the region of interest image includes: segmenting the region of interest image as follows: Figure 7 The image of the region of interest "Region of Interest 2" is converted to grayscale to obtain the corresponding grayscale image of the region of interest (e.g., Figure 8 As shown in A), morphological operations are then performed on the grayscale region of interest image to remove smaller targets and holes, as well as adhesion regions, to obtain a mask of the strong tumor signal region corresponding to the grayscale region of interest image; based on the mask of the strong tumor signal region, the strong tumor signal region image (such as...) is determined. Figure 8 (as shown in B).

[0082] For example, clustering pixels in an image of a strong tumor signal region to obtain at least one tumor cell cluster includes: based on the position and pixel value of the pixels in the image of the strong tumor signal region, using the estimated cell number as the cluster category, performing a clustering operation on the pixels in the image of the strong tumor signal region to obtain a number of tumor cell clusters with an estimated cell number. The estimated cell number can be determined by dividing the strong tumor signal region by the average cell area. The cell area can be determined based on the number of pixels in the cell image region. This embodiment of the invention does not limit the clustering algorithm for pixel clustering; for example, the K-means clustering method is used to perform this clustering operation.

[0083] For example, determining the set of pixels in the cell membrane region based on the cluster edges of the tumor cell clusters includes: dilating the cluster edges, and using the pixels corresponding to the dilated cluster edges as pixels in the set of pixels in the cell membrane region (e.g., ...). Figure 8 (as shown in C).

[0084] For example, calculating the first tumor intensity value corresponding to the region of interest image based on the set of pixels in the cell membrane region includes: using the average pixel value of the pixels in the set of pixels in the cell membrane region as the first tumor intensity value. Specifically, the tumor staining signal channel in the pixels of the cell membrane region can be extracted, the staining signal can be converted into a range of 0 to 1, summed, and then divided by the cell membrane area (i.e., the number of pixels in the cell membrane region) to obtain the first tumor intensity value.

[0085] Step 2032A: Determine the pathological analysis result of the tissue slice image based on the first tumor intensity value corresponding to the region of interest image.

[0086] For example, the mean of the first tumor intensity value corresponding to each region of interest image is used as the pathological analysis result of the tissue section image; the mean of the top N largest first tumor intensity values ​​corresponding to each region of interest image is used as the pathological analysis result of the tissue section image; and the first tumor intensity value with the highest frequency among the first tumor intensity values ​​corresponding to each region of interest image is used as the pathological analysis result of the tissue section image.

[0087] Cell membrane staining signals are an important basis for tumor identification, and their priority is higher than that of cell nucleus and cytoplasm. However, in tumor tissue sections, cell membranes and cytoplasm are often stained together, resulting in uniform staining in cytoplasm in non-tumor areas, which may be misidentified as tumor areas, leading to an increase in false positives, requiring manual removal.

[0088] According to this optional implementation, the cell membrane region can be automatically and accurately extracted from the region of interest, and the tumor intensity corresponding to the region of interest can be determined based on the staining of the cell membrane. The staining signal of the cell membrane can be accurately analyzed without the need to manually set complex parameters and thresholds, and is not affected by cytoplasmic staining. It can automatically and intelligently judge and analyze the pathological conditions such as tumor intensity of the sample.

[0089] In an alternative implementation B, step 203 may include steps 2031B and 2032B:

[0090] Step 2031B: Input each region of interest image in the region of interest image set into the pre-trained tumor intensity determination model to obtain the second tumor intensity value corresponding to the region of interest image. The tumor intensity determination model is pre-trained through a preset training step.

[0091] Here, the pre-set training steps may specifically include: obtaining a training sample set, which includes sample images and corresponding labeled tumor intensity values ​​used to characterize the intensity of tissue tumors in the sample images; and conducting supervised training on the initial tumor intensity determination model based on the training sample set to obtain a tumor intensity determination model used to characterize the correspondence between images and tumor intensity values.

[0092] Compared to the nucleus of normal cells, the nucleus of tumor cells is enlarged and can be distinguished from the nuclei of surrounding normal tissues. Therefore, nuclear morphology is also an important indicator for assessing tumor intensity. For example, the tumor intensity value is determined by comprehensively considering both cell membrane staining intensity and nuclear morphology.

[0093] For example, multiple tumor regions of interest in different tissue sections can be labeled with several levels (e.g., 6 levels), where level 0 comes from normal tissue sections, and levels 1 to 5 come from tumor regions of interest, with higher levels indicating higher tumor intensity. Correspondingly, the second tumor intensity value also includes one of these levels.

[0094] For example, the sample images include the original sample image and the enhanced sample image obtained by enhancing the original sample image.

[0095] For example, the tumor intensity determination model is a classification model based on a sliding window and hierarchical architecture using a sequential network architecture. The corresponding algorithm includes: first, inputting the image of the region of interest into a slicing module for block segmentation; then, inputting the segmentation result into a linear transformation module for channel fusion to reduce the number of channels; next, inputting the channel fusion result sequentially into multiple stage modules (e.g., four stage modules) for feature extraction and downsampling, with each stage module outputting a smaller feature map size and an increased number of channels. Within each stage module, image features are extracted using a window self-attention module and a sliding window self-attention module. The features output from the last stage module are then input into a linear transformation module to obtain the second tumor intensity value. Compared to other deep neural network architectures, this system's deep neural network, through the window self-attention module, helps analyze tumor intensity in smaller regions, while the sliding window self-attention module helps analyze tumor intensity in larger regions, enabling judgments at different scales. Furthermore, the interconnected structure of multiple stage modules facilitates the gradual abstraction of high-level semantic information.

[0096] Step 2032B: Determine the pathological analysis results of the tissue section image based on the second tumor intensity value corresponding to the region of interest image.

[0097] Analogous to step 2032A, the mean of the second tumor intensity values ​​corresponding to each region of interest image can be used as the pathological analysis result of the tissue section image; the mean of the top N largest second tumor intensity values ​​corresponding to each region of interest image can be used as the pathological analysis result of the tissue section image; and the first tumor intensity value with the highest frequency among the second tumor intensity values ​​corresponding to each region of interest image can be used as the pathological analysis result of the tissue section image.

[0098] In this embodiment, a neural network method is used to evaluate tumor intensity. The evaluation considers not only the staining intensity of the cell membrane but also the morphological factors of the cell nucleus, which can automatically and accurately determine the tumor intensity of the region of interest.

[0099] In an alternative implementation C, step 203 may include steps 2031C and 2032C:

[0100] Step 2031C: For each region of interest image in the region of interest image set, perform a first tumor intensity value extraction operation, and input each region of interest image in the region of interest image set into a pre-trained tumor intensity determination model to obtain a second tumor intensity value corresponding to the region of interest image. The tumor intensity determination model is pre-trained through a preset training step.

[0101] Here, analogous to steps 2031A and 2031B in optional implementations A and B, the first tumor intensity value extraction operation in step 2031C specifically includes: segmenting a strong tumor signal region image from the region of interest image; performing a clustering operation on the pixels in the strong tumor signal region image to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixels in the strong tumor signal region image; determining the set of cell membrane region pixels based on the cluster edges of each tumor cell cluster; and calculating the first tumor intensity value corresponding to the region of interest image based on the set of cell membrane region pixels. The preset training step in step 2031C specifically includes: obtaining a training sample set, the training samples including sample images and corresponding labeled tumor intensity values ​​used to characterize the intensity of tissue tumors in the sample images; and performing supervised training on the initial tumor intensity determination model based on the training sample set to obtain a tumor intensity determination model used to characterize the correspondence between the image and the tumor intensity value.

[0102] Other exemplary implementations of step 2031C can be compared with exemplary implementations of steps 2031A and 2031B in optional implementations A and B.

[0103] Understandably, when the executing entity performs step 2031C, there is no specific order restriction on the execution of the first tumor intensity value extraction operation and the determination of the second tumor intensity value corresponding to the region of interest image. For example, the executing entity can first determine the second tumor intensity value corresponding to the region of interest image and then perform the first tumor intensity value extraction operation, or the executing entity can simultaneously perform the first tumor intensity value extraction operation and determine the second tumor intensity value corresponding to the region of interest image.

[0104] Step 2032C: Based on the first tumor intensity value and the second tumor intensity value corresponding to the region of interest image, determine the pathological analysis result of the tissue section image.

[0105] Based on this optional implementation C, step 2032C may further include the following steps:

[0106] Step a: Based on the first tumor intensity value and the second tumor intensity value corresponding to the region of interest image, calculate the fused tumor intensity value corresponding to the region of interest image.

[0107] Step b: Add the fused tumor intensity value corresponding to each region of interest image in the set of region of interest images corresponding to the tissue slice image to the set of fused tumor intensity values ​​corresponding to the tissue slice image.

[0108] Step c: Based on the set of fused tumor intensity values ​​corresponding to the tissue slice images, the pathological analysis results of the tissue slice images are calculated.

[0109] In some optional implementations, step a above can be performed as follows: For each region of interest image, a first tumor intensity value and a second tumor intensity value are calculated using a weighted fusion method to obtain the fused tumor intensity value corresponding to that region of interest image. For example, the calculation formula is as follows:

[0110] P = αP1 + (1-α)P2

[0111] Where P represents the fusion tumor intensity value, P1 is the first tumor intensity value, P2 is the second tumor intensity value, and α is the weighting coefficient of the first tumor intensity value, which is between 0 and 1. For example, α can be 0.2.

[0112] In some optional implementations, step c can be performed as follows: In the same tissue slice image, the average fused tumor intensity value of the images of several regions of interest with the highest fused tumor intensity value can be used as the tumor intensity value corresponding to the tissue slice image to characterize the pathological analysis result of the tissue slice image.

[0113] According to this optional implementation method, the fusion of computational methods and neural network methods to evaluate tumor intensity not only considers the intensity of cell membrane staining but also the morphological changes of the cell nucleus, making the tumor intensity evaluation results more accurate and reliable. At the same time, it can generate a tumor intensity value corresponding to each tissue section image in the immunohistochemical image, which can intuitively display and compare the tumor intensity of different tissue sections, saving a lot of manual operation and experimental costs, supporting large-scale biological experimental verification, and improving drug verification throughput.

[0114] In some optional implementations, the entity performing the immunohistochemical slide image analysis method may also perform, before performing step 201, the following: Figure 2A The following steps 201'1 to 201'3 are shown:

[0115] Step 201'1: Obtain immunohistochemical images, wherein the immunohistochemical images are obtained by acquiring images of a tissue slice array including multiple tissue slice elements.

[0116] Step 201'2: Determine the center coordinates of each tissue slice element.

[0117] Step 201'3: Based on the center coordinates of each tissue slice element, crop the immunohistochemical image to obtain a set of tissue slice images.

[0118] According to this optional implementation, missing elements in an array of immunohistochemical images composed of multiple tissue slice images can be completed, which facilitates subsequent cropping of the overall immunohistochemical image based on the center coordinates of the elements and obtaining a set of tissue slice images. This saves the step of manually adjusting parameters and avoids the inability to detect samples due to missing images.

[0119] In some alternative implementations, step 201'2 may further include, for example: Figure 2E The following steps 201'21 to 201'25 are shown:

[0120] Step 201'21: Perform preprocessing operations on the immunohistochemical images.

[0121] Here, preprocessing may specifically include: binarizing the immunohistochemical image to obtain a binarized immunohistochemical image; and performing at least one morphological operation on the binarized immunohistochemical image to obtain a region mask of the tissue slice array elements in the immunohistochemical image.

[0122] Step 201'22: For the region mask of the tissue slice array element, obtain the contour of each tissue slice array element based on the preset edge detection algorithm.

[0123] Step 201'23: Based on the size and number of tissue slice array elements, estimate the radius of the tissue slice corresponding to the tissue slice array element to obtain the estimated radius parameter.

[0124] Step 201'24: Based on the estimated radius parameter, determine the set of center coordinates and radius set of tissue slice array elements through a preset circular detection algorithm.

[0125] Step 201'25: In response to the presence of undetected center coordinates of tissue slice array elements, the set of center coordinates is completed to obtain the center coordinates of the tissue slice array elements.

[0126] In response to the absence of center coordinates for undetected tissue slice elements, proceed to step 201'3.

[0127] This application does not specifically limit the algorithm used for edge detection. For example, algorithms for edge detection include, but are not limited to, the Roberts algorithm, the Prewitt algorithm, the Sobel algorithm, the Canny algorithm, and the Laplacian algorithm. Similarly, this application does not specifically limit the algorithm used for circle detection. For example, algorithms for circle detection include, but are not limited to, the Hough algorithm, the Circular Hough Transform algorithm (CHT), the Randomized Hough Transform algorithm (RHT), and the Randomized Circle detection algorithm (RCD).

[0128] In some optional implementations, steps 201'1'1 to 201'1'4 can be performed as follows: A region mask of the tissue slice array elements is applied, and the outline of each tissue slice array element is output using the Canny algorithm; based on the size and number of the tissue slice array elements, the radius of the corresponding tissue slice is estimated to obtain the estimated radius parameter; and based on the estimated radius parameter, a preset circular detection algorithm is used to determine the set of center coordinates and the set of radii of the tissue slice array elements. According to this optional implementation, the center coordinates and radius of each array element can be automatically generated through parameter estimation and parameter iteration, thereby facilitating the subsequent positioning and segmentation of each tissue slice array element.

[0129] In some alternative implementations, step 201'25 may include the following steps:

[0130] The first step is to perform a position sorting operation on the center coordinates of the center coordinate set.

[0131] Here, the position sorting operation may specifically include: in response to the absolute value of the difference between the first coordinate components of the center coordinates of two adjacent tissue slice array elements being less than or equal to a first preset radius, taking the center coordinates of the two tissue slice array elements as the same group, and obtaining a set of grouped center coordinates of the tissue slice array elements.

[0132] The second step is to perform a completion operation on each group in the center coordinate grouping set.

[0133] Here, the completion process may specifically include: in response to the distance between two adjacent center coordinates in the group or the distance between the second coordinate components of two adjacent center coordinates being greater than a second preset radius, performing linear interpolation on the two center coordinates, and adding the result of the linear interpolation as the center coordinate of the tissue slice array element to the center coordinate set; wherein, one of the first coordinate component and the second coordinate component is the abscissa and the other is the ordinate.

[0134] In some optional embodiments, the first to second steps described above can be performed as follows: The center coordinates of the tissue slice array elements are sorted from left to right. When the difference in the abscissa of the center coordinates of adjacent array elements is less than or equal to a first preset radius (e.g., 1.5 times the average radius), the two adjacent tissue slice array elements are considered to be in the same column. In each column, the ordinate distance between the center coordinates of two adjacent tissue slice array elements is calculated. When the ordinate distance between the center coordinates of two adjacent center coordinates is greater than a second preset radius (e.g., 3 times the average radius), it is considered that there is a gap between the two adjacent tissue slice array elements. Linear interpolation is then performed on the two adjacent center coordinates to obtain the completed center coordinates. According to this optional embodiment, based on the distance between the tissue slice array elements in the array, the existence of a gap is automatically determined, and the gap position is located and filled, eliminating reliance on manual judgment and avoiding the inability to detect samples due to image gaps.

[0135] In some alternative implementations, step 201'3 may also be performed as follows:

[0136] Based on the center coordinates corresponding to the tissue slice array elements, the immunohistochemical image is cropped according to the third preset radius to obtain a set of tissue slice images.

[0137] In some alternative implementations, the aforementioned executing entity may also perform the following step 201'4 after completing step 201'3 and before completing step 203:

[0138] Step 201'4: Based on the center coordinates of the tissue slice array elements and the preset grouping information, assign labels to the tissue slice images in the tissue slice image set.

[0139] For example, the preset grouping information mentioned above may include grouping by type of tissue slice, such as normal tissue grouping, tumor tissue grouping, or grouping by source of tissue slice, such as adjacent normal tissue grouping.

[0140] It should be noted that the aforementioned executing entity may execute step 201'4 after executing step 201'3 and before executing step 203. This disclosure does not specifically limit the timing of the execution of step 201'4. For example, it may be executed after step 201'3 and before step 201, or it may be executed after step 201 and before step 202.

[0141] Based on this optional implementation, since step 201'4 has already assigned labels to the tissue slice images involved in step 203, step 203 can also be performed as follows: determine the pathological analysis results corresponding to the labels.

[0142] Accordingly, determining the pathological analysis results of tissue section images includes: determining the pathological analysis results corresponding to the labels. For example, the labels may include at least one of the following: drug type, drug concentration, drug action time, and drug target. This establishes the correspondence between the labels and the pathological analysis results, facilitating subsequent statistical analysis of the drug's effects.

[0143] In some optional implementations, steps 201'3 to 201'4 can be performed as follows: after obtaining the center coordinates corresponding to the tissue slice elements, the immunohistochemical image after completing the missing tissue slice elements is cropped with a third preset radius (e.g., 1.1 times the average radius); after cropping, labels are assigned to the tissue slice images in the tissue slice image set based on the center coordinates corresponding to the tissue slice elements and preset grouping information. According to this optional implementation, tissue slice element images can be automatically segmented from the tissue array image through parameter estimation and parameter iteration, corresponding to a single sample in a biological scene, and grouping labels can be assigned to the tissue slice element images corresponding to a single sample, without manual intervention, which facilitates subsequent analysis by experimental personnel.

[0144] Further reference Figure 3 As an implementation of the methods shown in the above figures, the present invention provides an embodiment of an immunohistochemical slide image analysis device, which is similar to... Figure 2A The method embodiment shown corresponds to a device that can be applied to various electronic devices.

[0145] like Figure 3 As shown, the immunohistochemical slide image analysis device 300 of this embodiment includes: an extraction module 301, a region of interest determination module 302, and a pathological analysis module 303. The extraction module 301 is configured to extract strong staining signal sampling points from the tissue slide image based on the immunohistochemical staining signal, obtaining a set of strong staining signal sampling points; the region of interest determination module 302 is configured to cluster the strong staining signal sampling points to determine a set of region of interest images corresponding to the tissue slide image, the set of region of interest images including at least one region of interest image; the pathological analysis module 303 is configured to determine the pathological analysis results of the tissue slide image based on the set of region of interest images.

[0146] In this embodiment, the specific processing of the extraction module 301, the region of interest determination module 302, and the pathological analysis module 303 of the immunohistochemical slide image analysis device 300, and the resulting technical effects, can be referred to respectively. Figure 2A The relevant descriptions of steps 201, 202 and 203 in the corresponding embodiments will not be repeated here.

[0147] In some optional embodiments, the extraction module 301 is further configured to: perform channel separation operation on the immunohistochemical staining signal of the tissue slice image to obtain a staining signal channel map corresponding to the tissue slice image; perform binarization operation on the staining signal channel map to obtain a mask of tumor staining signal; perform at least one morphological operation on the mask of tumor staining signal to obtain a mask of tumor region; perform grayscale operation on the tissue slice image to obtain a grayscale tissue slice image; extract the mask image of grayscale tumor region corresponding to the mask of tumor region from the grayscale tissue slice image; and perform the following sampling point extraction operation on each pixel in the mask image of grayscale tumor region: in response to the grayscale value of the pixel being greater than a preset extraction threshold, add the pixel as a strong staining signal sampling point to the set of strong staining signal sampling points.

[0148] In some optional implementations, before extracting strong staining signal sampling points from tissue slice images based on immunohistochemical staining signals, the method further includes: acquiring immunohistochemical images, wherein the immunohistochemical images are obtained by image acquisition of a tissue slice array including multiple tissue slice elements; determining the center coordinates of each tissue slice element; and cropping the immunohistochemical images based on the center coordinates of each tissue slice image element to obtain a set of tissue slice images.

[0149] In some optional embodiments, the method further includes performing the following preprocessing operations on the immunohistochemical image: binarizing the immunohistochemical image to obtain a binarized immunohistochemical image; performing at least one morphological operation on the binarized immunohistochemical image to obtain a region mask of tissue slice elements in the immunohistochemical image; obtaining the contour of each tissue slice element based on a preset edge detection algorithm for the region mask of the tissue slice elements; estimating the radius of the tissue slice corresponding to the tissue slice element based on the size and number of the tissue slice elements to obtain an estimated radius parameter; determining the set of center coordinates and the set of radii of the tissue slice elements based on the estimated radius parameter using a preset circle detection algorithm; and completing the set of center coordinates in response to the existence of undetected center coordinates of tissue slice elements to obtain the center coordinates of the tissue slice elements.

[0150] In some optional implementations, the above-mentioned response to the existence of undetected center coordinates of tissue slice array elements, performing a completion process on the set of center coordinates to obtain the center coordinates of the tissue slice array elements, includes: performing a position sorting operation on the center coordinates of the set of center coordinates: in response to the absolute value of the difference between the first coordinate components of the center coordinates of two adjacent tissue slice array elements being less than or equal to a first preset radius, the center coordinates of the two tissue slice array elements are grouped together to obtain a set of grouped center coordinates of tissue slice array elements; for each group in the set of grouped center coordinates, the following completion process is performed: in response to the distance between two adjacent center coordinates in the group being greater than a second preset radius, the two center coordinates are linearly interpolated, and the result of the linear interpolation is added to the set of center coordinates as the center coordinates of the tissue slice array elements to obtain the center coordinates of the tissue slice array elements; wherein, one of the first coordinate component and the second coordinate component is the abscissa, and the other is the ordinate.

[0151] In some optional implementations, the above-mentioned method of cropping the immunohistochemical image based on the center coordinates of each tissue slice element to obtain a set of tissue slice images includes: cropping and supplementing the missing tissue slice elements with the immunohistochemical image according to a third preset radius based on the center coordinates corresponding to the tissue slice elements to obtain a set of tissue slice images.

[0152] In some optional implementations, the above-mentioned method of cropping the immunohistochemical image based on the center coordinates of each tissue slice array element to obtain a tissue slice image set further includes: assigning labels to the tissue slice images in the tissue slice image set based on the center coordinates corresponding to the tissue slice array elements and preset grouping information.

[0153] In some optional implementations, the above-mentioned region of interest determination module 302 is further configured to: perform clustering operation on the strong staining signal sampling points in the set of strong staining signal sampling points based on a preset distance clustering algorithm to obtain a set of sampling point clusters, wherein the set of sampling point clusters includes a preset number of cluster centers for the region of interest; and based on the set of sampling point clusters, crop the tissue slice image to obtain a set of region of interest images corresponding to the tissue slice image.

[0154] In some optional implementations, the pathological analysis module 303 is further configured to: for each region of interest image in the region of interest image set, perform the following first tumor intensity value extraction operation: segment a strong tumor signal region image from the region of interest image; based on the position and pixel value of the pixels in the strong tumor signal region image, perform a clustering operation on the pixels in the strong tumor signal region image to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixels in the strong tumor signal region image; determine the set of cell membrane region pixels based on the clustering edges of the tumor cell clusters; calculate the first tumor intensity value corresponding to the region of interest image based on the set of cell membrane region pixels; and determine the pathological analysis result of the tissue section image based on the first tumor intensity value corresponding to the region of interest image.

[0155] In some optional implementations, the method further includes: inputting each region of interest image in the region of interest image set into a pre-trained tumor intensity determination model to obtain a second tumor intensity value corresponding to the region of interest image, wherein the tumor intensity determination model is pre-trained through the following preset training steps: obtaining a training sample set, the training samples including sample images and corresponding labeled tumor intensity values ​​used to characterize the intensity of tissue tumors in the sample images; performing supervised training on the initial tumor intensity determination model based on the training sample set to obtain a tumor intensity determination model used to characterize the correspondence between images and tumor intensity values; and determining the pathological analysis results of the tissue slice image based on the second tumor intensity value corresponding to the region of interest image.

[0156] In some optional embodiments, the above-mentioned determination of the pathological analysis result of the tissue section image includes: calculating the fused tumor intensity value corresponding to the region of interest image based on the first tumor intensity value and the second tumor intensity value corresponding to the region of interest image; adding the fused tumor intensity value corresponding to each region of interest image in the set of regions of interest images corresponding to the tissue section image to the set of fused tumor intensity values ​​corresponding to the tissue section image; and calculating the pathological analysis result of the tissue section image based on the set of fused tumor intensity values ​​corresponding to the tissue section image.

[0157] like Figure 4 As shown, it illustrates a structural schematic of a computer system 400 suitable for implementing the electronic device of the present invention. Figure 4 The computer system 400 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0158] like Figure 4As shown, the computer system 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the computer system 400. The processing device 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0159] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows computer system 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 A computer system 400 with various electronic devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0160] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a storage device 408, or installed from a ROM 402. When the computer program is executed by a processing device 401, it performs the functions defined in the methods of the embodiments of the present invention.

[0161] It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0162] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0163] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the following functions: Figure 2A The embodiments shown and their alternative implementations illustrate an immunohistochemical slide image analysis method.

[0164] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0165] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0166] The units described in the embodiments of the present invention can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0167] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. An immunohistochemical slice image analysis method, the method comprising: extracting strong staining signal sampling points in a tissue slice image based on immunohistochemical staining signals, specifically comprising: performing a channel separation operation of immunohistochemical staining signals on the tissue slice image to obtain a staining signal channel image corresponding to the tissue slice image; performing a binarization operation on the staining signal channel image to obtain a tumor staining signal mask; performing at least one morphological operation on the tumor staining signal mask to obtain a tumor region mask; performing a grayscale operation on the tissue slice image to obtain a grayscale tissue slice image; extracting a grayscale tumor region mask image corresponding to the tumor region mask from the grayscale tissue slice image; performing the following sampling point extraction operation on each pixel point in the grayscale tumor region mask image: in response to the gray value of the pixel point being greater than a preset extraction threshold, adding the pixel point as a strong staining signal sampling point to a strong staining signal sampling point set; performing a clustering operation on the strong staining signal sampling points in the strong staining signal sampling point set based on a preset distance clustering algorithm to obtain a sampling point cluster set, wherein the sampling point cluster set includes a preset number of interest region cluster centers; cropping the tissue slice image based on the sampling point cluster set to obtain a set of interest region images corresponding to the tissue slice image, the set of interest region images including at least one interest region image; determining a pathological analysis result of the tissue slice image based on the set of interest region images.

2. The method of claim 1, wherein determining a pathological analysis result of the tissue slice image based on the set of interest region images comprises: for each interest region image in the set of interest region images, performing the following first tumor intensity value extraction operation: segmenting a strong tumor signal region image from the interest region image; performing a clustering operation on the pixel points in the strong tumor signal region image based on the positions and pixel values of the pixel points to obtain at least one tumor cell cluster, each tumor cell cluster being composed of different pixel points in the strong tumor signal region image; determining a cell membrane region pixel point set according to the cluster edges of the tumor cell clusters; calculating a first tumor intensity value corresponding to the interest region image based on the cell membrane region pixel point set; determining a pathological analysis result of the tissue slice image based on the first tumor intensity values corresponding to each interest region image.

3. The method of claim 1 or 2, further comprising: inputting each of the interest region images in the interest region image set into a pre-trained tumor intensity determination model to obtain a second tumor intensity value corresponding to the interest region image, wherein the tumor intensity determination model is pre-trained through a preset training step as follows: obtaining a training sample set, the training sample including a sample image and a corresponding labeled tumor intensity value used to represent the tumor intensity of the tissue in the sample image; based on the training sample set, performing supervised training on an initial tumor intensity determination model to obtain a tumor intensity determination model used to represent the corresponding relationship between an image and a tumor intensity value; and determining a pathological analysis result of the tissue section image based on the second tumor intensity value corresponding to the interest region image.

4. The method of claim 3, wherein the determining the pathological analysis result of the tissue section image comprises: calculating a fusion tumor intensity value corresponding to the interest region image based on the first tumor intensity value and / or the second tumor intensity value corresponding to the interest region image; adding the fusion tumor intensity value corresponding to each of the interest region images in the interest region image set corresponding to the tissue section image into a fusion tumor intensity value set corresponding to the tissue section image; and calculating the pathological analysis result of the tissue section image based on the fusion tumor intensity value set corresponding to the tissue section image.

5. The method of claim 1, wherein, Before the extracting the strong staining signal sample points in the tissue section image based on the immunohistochemical staining signals, the method further comprises: obtaining an immunohistochemical image, wherein the immunohistochemical image is obtained by image acquisition on a tissue section array including a plurality of tissue section elements; determining the center coordinates of each of the tissue section elements; based on the center coordinates of each of the tissue section elements, cutting the immunohistochemical image to obtain the set of tissue section images.

6. The method of claim 5, wherein, The determining the center coordinates of each of the tissue section elements comprises: performing the following preprocessing operations on the immunohistochemical image: performing binaryzation processing on the immunohistochemical image to obtain a binaryzated immunohistochemical image; performing at least one morphological operation on the binaryzated immunohistochemical image to obtain a tissue section element region mask of the tissue section elements in the immunohistochemical image; performing edge detection on the tissue section element region mask to obtain a set of contours of each of the tissue section elements; based on the size and number of the tissue section elements, estimating the tissue section radius corresponding to the tissue section elements to obtain an estimated radius parameter; based on the estimated radius parameter, determining a center coordinate set and a radius set of the tissue section elements through a preset circle detection algorithm; in response to the existence of the center coordinates of the undetected tissue section elements, performing a completion processing on the center coordinate set to obtain the center coordinates of the tissue section elements.

7. The method of claim 6, wherein, The response to the existence of the center coordinates of the undetected tissue section elements, performing a completion processing on the center coordinate set to obtain the center coordinates of the tissue section elements comprises: performing the following position ordering operation on the center coordinates of the center coordinate set: in response to an absolute value of a first coordinate component difference of center coordinates of two adjacent tissue slice array elements being less than or equal to a first preset radius, regarding the center coordinates of the two tissue slice array elements as being in the same group to obtain a center coordinate group set of the tissue slice array elements; performing the following completion processing operation on each group in the center coordinate group set: in response to a distance between two adjacent center coordinates in the group or a distance between second coordinate components of the two adjacent center coordinates being greater than a second preset radius, performing linear interpolation on the two center coordinates, and adding a result of the linear interpolation to the center coordinate set as a center coordinate of the tissue slice array element; wherein one of the first coordinate component and the second coordinate component is a horizontal coordinate, and the other is a vertical coordinate.

8. The method of claim 5, wherein, The cutting of the immunohistochemical image based on the center coordinates of the tissue slice array elements to obtain the set of tissue slice images includes: cutting the immunohistochemical image based on the center coordinates corresponding to the tissue slice array elements according to a third preset radius to obtain the set of tissue slice images.

9. The method of claim 5, wherein, The method further includes: specifying a label for a tissue slice image in the set of tissue slice images based on the center coordinates corresponding to the tissue slice array elements and preset grouping information; and determining the pathological analysis result of the tissue slice image includes determining a pathological analysis result corresponding to the label. 10.An immunohistochemical slice image analysis apparatus, comprising: an extraction module configured to: perform a channel separation operation on an immunohistochemical staining signal of a tissue slice image to obtain a staining signal channel image corresponding to the tissue slice image; and perform a binarization operation on the staining signal channel image to obtain a mask of tumor staining signals; performing at least one morphological operation on the mask of tumor staining signals to obtain a mask of tumor regions; performing a grayscale operation on the tissue slice image to obtain a grayscale tissue slice image; and extracting a mask image of a grayscale tumor region corresponding to the mask of tumor regions from the grayscale tissue slice image; performing the following sampling point extraction operation on each pixel point in the mask image of the grayscale tumor region: in response to a grayscale value of the pixel point being greater than a preset extraction threshold, adding the pixel point to a strong staining signal sampling point set as a strong staining signal sampling point; an interest region determination module configured to: perform a clustering operation on strong staining signal sampling points in the strong staining signal sampling point set based on a preset distance clustering algorithm to obtain a sampling point cluster set, wherein the sampling point cluster set includes a preset number of cluster centers; and based on the sampling point cluster set, cut the tissue slice image to obtain a set of interest region images corresponding to the tissue slice image, the set of interest region images including at least one interest region image; a pathological analysis module configured to determine a pathological analysis result of the tissue slice image based on the set of interest region images. 11.An electronic device, comprising: one or more processors; a storage device having stored thereon one or more programs, the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1-9.

12. A computer readable storage medium having stored thereon a computer program, wherein, the computer program, when executed by one or more processors, performs the method of any one of claims 1-9.

Images