Plaque typing system and device and storage medium

Abstract

The invention discloses a plaque typing system and device and a storage medium. The system includes a processor configured to perform a plaque typing method, the method including: determining a region of interest in an image of a target tissue sample, the image including cell marker information and plaque marker information; determining position data of at least two extracellular plaques included in the region of interest; according to the position data of the at least two extracellular plaques, a compactness index corresponding to the region of interest is determined, and the compactness index is determined based on the coverage proportion of the extracellular plaques and the distance between every two extracellular plaques; the coverage proportion of the extracellular plaques is the ratio of the area of the corresponding extracellular plaques to the sum of the areas of all the extracellular plaques included in the region of interest; and determining a plaque typing result corresponding to the region of interest according to the compactness index corresponding to the region of interest, wherein the plaque typing result is a dispersion type, a compact type or an intermediate type. The problem that existing plaque morphological description is low in accuracy is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a plaque typing system, device and storage medium. Background Art

[0002] Alzheimer's disease (AD) is the most common cause of dementia. The pathological features of AD in the brain are amyloid-β (Aβ) plaques and neurofibrillary tangles caused by abnormal phosphorylated Tau protein (p-Tau).

[0003] Accurate description of plaque morphology can help improve the accuracy of pathological microenvironment research, but existing morphological descriptions of plaques, such as plaque density and plaque size, have low accuracy. Summary of the invention

[0004] The present invention provides a plaque typing system, a device and a storage medium to solve the problem of low accuracy in existing plaque morphology description.

[0005] According to one aspect of the present invention, a plaque typing system is provided, comprising a processor, wherein the processor is configured to execute a plaque typing method, the method comprising:

[0006] Determine a region of interest in the target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information;

[0007] Determining position data of at least two extracellular plaques included in the region of interest;

[0008] Determining a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest;

[0009] The plaque classification result corresponding to the region of interest is determined according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

[0010] In one embodiment, determining the location data of at least two extracellular plaques included in the region of interest comprises:

[0011] Performing cell segmentation on the region of interest to obtain a cell segmentation result image, and determining position data of each cell in the cell segmentation result image;

[0012] Performing plaque segmentation on the region of interest to obtain a plaque segmentation result image; determining position data of each plaque in the plaque segmentation result image, wherein the plaque image includes at least two plaques;

[0013] According to the position data of each of the cells and the position data of each of the plaques, at least two extracellular plaques are screened out from the at least two plaques to obtain the position data of at least two extracellular plaques.

[0014] In one embodiment, the determining of a region of interest in a target tissue sample image, wherein the region of interest includes cell labeling information and plaque labeling information, includes:

[0015] Determine regions of interest in a first image and a second image of a target tissue sample, respectively, wherein the region of interest in the first image and the region of interest in the second image include cell marking information and plaque marking information, respectively;

[0016] The determining of the location data of at least two extracellular plaques included in the region of interest comprises:

[0017] Determine the position data of each cell in the region of interest of the first image, and determine the position data of at least two plaques included in the region of interest of the second image;

[0018] According to the position data of each of the cells and the position data of the at least two plaques, at least two extracellular plaques are screened out from the at least two plaques to obtain the position data of the at least two extracellular plaques.

[0019] In one embodiment, the extracellular plaque screening method comprises:

[0020] For each of the at least two patches, determining whether the current patch has no intersection with each cell in the region of interest;

[0021] If yes, the current plaque is regarded as an extracellular plaque.

[0022] In one embodiment, determining the position data of each cell in the region of interest of the first image and determining the position data of at least two plaques included in the region of interest of the second image include:

[0023] Respectively converting the region of interest of the first image and the region of interest of the second image into grayscale images to obtain a first grayscale image and a second grayscale image;

[0024] Binarizing the first grayscale image and the second grayscale image to obtain a first binary image and a second binary image respectively;

[0025] Determine the position data of each cell in the region of interest of the first image according to the first binary image;

[0026] The position data of at least two patches included in the region of interest of the second image are determined according to the second binary image.

[0027] In one embodiment, binarizing the first grayscale image and the second grayscale image to obtain a first binary image and a second binary image respectively includes:

[0028] Determine the average brightness and brightness standard deviation of the region of interest of the first image, and use the sum of the average brightness and twice the brightness standard deviation as a first threshold;

[0029] Determine the average brightness and brightness standard deviation of the region of interest of the second image, and use the sum of the average brightness and twice the brightness standard deviation as a second threshold;

[0030] Binarizing the first grayscale image using the first threshold to obtain a first binary image;

[0031] The second grayscale image is binarized using the second threshold to obtain a second binary image.

[0032] In one embodiment, the density index is the sum of the correlation coefficients of all extracellular plaque combinations corresponding to the region of interest;

[0033] All the extracellular plaque combinations include any two combinations of the at least two extracellular plaques;

[0034] The correlation coefficient is the product of the coverage ratio of each extracellular plaque included in the extracellular plaque combination and the Euclidean distance between the corresponding two extracellular plaques.

[0035] In one embodiment, determining the plaque typing result corresponding to the region of interest according to the density index corresponding to the region of interest includes:

[0036] If the density index is greater than a first set threshold, the plaque in the region of interest is determined to be a dense plaque;

[0037] If the density index is less than a second set threshold, the plaque in the region of interest is determined to be a diffuse plaque;

[0038] If the density index is greater than or equal to the second set threshold and less than or equal to the first set threshold, the plaque in the region of interest is determined to be an intermediate plaque;

[0039] The first set threshold is greater than the second set threshold.

[0040] In one embodiment, the target tissue sample is brain tissue; and the extracellular plaque is amyloid-β plaque.

[0041] According to another aspect of the present invention, there is provided a plaque typing device, comprising:

[0042] An acquisition module, used to determine a region of interest in a target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information;

[0043] A position data module, used to determine the position data of at least two extracellular plaques included in the region of interest;

[0044] An index module, used to determine a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest;

[0045] The classification module is used to determine the plaque classification result corresponding to the region of interest according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

[0046] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the plaque typing method described in any embodiment of the present invention when executed by a processor.

[0047] The plaque typing method provided by the embodiment of the present invention determines the position data of at least two extracellular plaques included in the region of interest; determines the density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, and determines the plaque typing result corresponding to the region of interest according to the density index corresponding to the region of interest. Since the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, it can accurately describe the distribution state of the extracellular plaques included in the region of interest, so the plaque typing result determined by the density index has a higher accuracy.

[0048] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 is a schematic diagram of the structure of a plaque typing system provided according to an embodiment of the present invention;

[0051] Figure 2 is a flow chart of a plaque typing method provided according to an embodiment of the present invention;

[0052] Figure 3A is a target tissue sample image including a plaque staining result provided according to an embodiment of the present invention;

[0053] Figure 3B is a target tissue sample image including a cell staining result provided according to an embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of the distribution of intracellular plaques and extracellular plaques provided according to an embodiment of the present invention;

[0055] Figure 5 is another flow chart of a plaque typing method provided according to an embodiment of the present invention;

[0056] Figure 6 is a cell segmentation result image provided by an embodiment of the present invention;

[0057] Figure 7 is a plaque segmentation result image provided by an embodiment of the present invention;

[0058] Figure 8 is another flow chart of a plaque typing method provided according to an embodiment of the present invention;

[0059] Fig. 9 is a schematic structural diagram of a plaque typing device provided according to an embodiment of the present invention;

[0060] Fig.10 is another structural schematic diagram of a plaque typing device provided according to an embodiment of the present invention;

[0061] Fig.11 It is a schematic diagram of the structure of a first position data unit provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0063] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] Figure 1 Schematic diagram of the structure of the plaque typing system provided by the embodiment of the present invention. Figure 1 As shown, the system can be various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0065] like Figure 1 As shown, the plaque typing system 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the plaque identification system 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0066] Multiple components in the plaque typing system 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the plaque typing system 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0067] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described below, such as a plaque typing method.

[0068] Figure 2 This is a flowchart of a plaque typing method provided by an embodiment of the present invention. This embodiment is applicable to the case of determining the plaque typing result corresponding to the region of interest of the target tissue sample image. The method can be executed by a plaque typing device, which can be implemented in the form of hardware and / or software. The plaque typing device can be configured in a processor of a plaque typing system. Figure 1 As shown, the method includes:

[0069] S110 , determining a region of interest in the target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information.

[0070] The target tissue sample may be a brain tissue sample, a retina, or the like.

[0071] In one embodiment, the target sample tissue is brain tissue; and the extracellular plaque is amyloid-β plaque (Aβ plaque).

[0072] In one embodiment, the cell marking information is cell nucleus staining information, and the plaque marking information is plaque staining information.

[0073] In one embodiment, the target tissue is brain tissue. Abeta1-16 (β-amyloid antibody 1-16) and NeuN (neuronal marker antibody) are used to perform multi-channel immunofluorescence staining on brain tissue slices of patients with Alzheimer's disease to obtain matching neuronal nucleus staining results and Aβ plaque staining results. The laser is controlled to simultaneously output the laser used to excite the fluorescent material in the neuronal nucleus staining results and the laser used to excite the fluorescent material in the Aβ plaque staining results; the high-power microscope is controlled to image the field of view. The resolution of the microscope is 0.346m, and the size of the entire brain tissue slice is about 3cm×3cm. During the imaging process, the lens is controlled to move in the horizontal and vertical directions, and multiple high-definition local tissue images are taken, and then spliced ​​into a complete imaging image of a brain tissue slice. The target tissue sample image includes one or more brain tissue slice images. It can be understood that if the target tissue is cut into at least two brain tissue slices in the vertical direction, the target tissue sample image includes at least two brain tissue slice images. Among them, the horizontal direction is the x-axis direction, the vertical direction is the y-axis direction, and the vertical direction is the z-axis direction.

[0074] Brain tissue slice images include fluorescent marker information of plaques (see Figure 3A ) or fluorescent labeling information of neuronal cells (see Figure 3B ).

[0075] The region of interest is a region of interest to the user, which may be a two-dimensional image or a three-dimensional image. It is understood that if the region of interest corresponds to one brain tissue slice image, then the region of interest is a two-dimensional image; if the region of interest corresponds to at least two brain tissue slice images, then the region of interest is a three-dimensional image.

[0076] S120, determining position data of at least two extracellular plaques included in the region of interest.

[0077] Among them, extracellular plaques refer to plaques located outside cells. In diseased brain tissue, there may also be plaques located inside cells, namely intracellular plaques. Figure 3A and Figure 3B It can be seen that the color development position of some plaques overlaps with the color development position of cells, and the overlapping part is the intracellular plaque.

[0078] The distribution information of plaques and cells included in the region of interest is determined according to the cell marking information and plaque marking information included in the region of interest; the distribution information of each extracellular plaque is determined according to the distribution information of each plaque and cell; when the number of extracellular plaques is greater than or equal to 2, the position data of the at least two extracellular plaques are determined. Figure 4 The distribution diagram of intracellular plaques and extracellular plaques provided by the embodiment of the present invention is shown in FIG. The position data of each extracellular plaque can be determined by the figure.

[0079] This step aims to identify cells and plaques in the region of interest, identify all extracellular plaques based on the positional relationship between the plaques and surrounding cells, and determine the position data of each extracellular plaque.

[0080] S130. Determine a density index corresponding to the region of interest based on the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between each pair of extracellular plaques, and the coverage ratio of the extracellular plaques is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest.

[0081] Among them, the coverage ratio of extracellular plaques can be expressed as:

[0082]

[0083] Among them, S(i) is the number of pixels included in the extracellular plaque marked as i. Therefore, s(i) (the coverage ratio of the extracellular plaque marked as i) represents the ratio of the number of pixels included in the extracellular plaque marked as i to the sum of the number of pixels included in all extracellular plaques.

[0084] In one embodiment, the density index is the sum of the correlation coefficients of all extracellular plaque combinations corresponding to the region of interest; all extracellular plaque combinations include pairwise combinations of at least two extracellular plaques; the correlation coefficient is the product of the coverage ratio of each extracellular plaque included in the extracellular plaque combination and the Euclidean distance between the corresponding two extracellular plaques.

[0085] Among them, the density index can be expressed as:

[0086]

[0087] Among them, δ ij It refers to the Euclidean distance between the centroid of the patch labeled i and the centroid of the patch labeled j.

[0088] S140. Determine a plaque classification result corresponding to the region of interest according to a density index corresponding to the region of interest, wherein the plaque classification result is diffuse type, dense type, or intermediate type.

[0089] Among them, the dense type refers to a relatively compact distribution of plaques. The diffuse type refers to a relatively loose distribution of plaques, and the distance between plaques is farther than that of the dense type and the intermediate type. The density of plaques corresponding to the intermediate type is between the diffuse type and the dense type.

[0090] If the density index is greater than the first set threshold, the plaque in the region of interest is determined to be a dense plaque; if the density index is less than the second set threshold, the plaque in the region of interest is determined to be a diffuse plaque; if the density index is greater than or equal to the second set threshold and less than or equal to the first set threshold, the plaque in the region of interest is determined to be an intermediate plaque; the first set threshold is greater than the second set threshold.

[0091] Among them, the first set threshold and the second set threshold are empirical values. In one embodiment, the target tissue sample is a brain tissue sample, and the extracellular plaque is an extracellular Aβ plaque. At least two brain tissue samples including extracellular Aβ plaques are distributed to at least two doctors with rich clinical experience, and the doctors mark each brain tissue sample as diffuse type, dense type or intermediate type based on clinical experience. Determine the density index corresponding to each target tissue sample, and determine the density index range corresponding to the diffuse type, dense type and intermediate type respectively according to the density index corresponding to each target tissue sample and the plaque classification results marked by the doctor. Since there is no overlapping part or interval part between the density index ranges corresponding to the diffuse type, dense type and intermediate type, it is possible to determine the first set threshold for segmenting the dense type and the intermediate type, and the second set threshold for segmenting the diffuse type and the intermediate type.

[0092] In one embodiment, after the plaque typing result corresponding to the region of interest is determined, it is output to a visual interactive interface so that the user can intuitively obtain the corresponding typing result of the region of interest.

[0093] The plaque typing method provided by the embodiment of the present invention determines the position data of at least two extracellular plaques included in the region of interest; determines the density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, and determines the plaque typing result corresponding to the region of interest according to the density index corresponding to the region of interest. Since the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, it can accurately describe the distribution state of the extracellular plaques included in the region of interest, so the plaque typing result determined by the density index has a higher accuracy.

[0094] Figure 5 Flow chart of the plaque typing method provided in an embodiment of the present invention. This embodiment is used to refine the method for determining the extracellular plaque location data in the above-mentioned embodiment. Figure 5 As shown, the method includes:

[0095] S210 : Determine a region of interest in the target tissue sample image, where the region of interest includes cell marking information and plaque marking information.

[0096] S2201, performing cell segmentation on the region of interest to obtain a cell segmentation result image, and determining position data of each cell in the cell segmentation result image.

[0097] The location data includes the center of mass and the radius.

[0098] In one embodiment, the staining object is a cell nucleus rather than a cell, and therefore the position data of the corresponding cell needs to be determined based on the position data of the cell nucleus.

[0099] Figure 6 The cell segmentation result image provided by the embodiment of the present invention. The region of interest may be a two-dimensional image or a three-dimensional image. When the region of interest is a three-dimensional image, the cell segmentation result image includes at least two two-dimensional cell segmentation result images. Regardless of whether the region of interest is a two-dimensional image or a three-dimensional image, when the region of interest includes zero cells, the plaques in the region of interest are all extracellular plaques. Therefore, when it is detected that the region of interest includes zero cells, the position data of each plaque in the region of interest is directly determined, and the position data of each plaque is used as the position data of at least two extracellular plaques included in the region of interest.

[0100] The cell segmentation threshold can be determined based on an existing threshold determination method.

[0101] S2202, performing plaque segmentation on the region of interest to obtain a plaque segmentation result image; determining position data of each plaque in the plaque segmentation result image, wherein the plaque segmentation result image includes at least two plaques.

[0102] Based on the plaque segmentation threshold, the region of interest is segmented to obtain a plaque segmentation result image (see Figure 7 ), and then determine the position data of each patch in the patch segmentation result.

[0103] When the region of interest is a three-dimensional image, the plaque segmentation result image includes at least two two-dimensional plaque segmentation result images. If the region of interest includes zero plaques or one plaque, a prompt message indicating that the density index cannot be calculated is output.

[0104] The plaque segmentation threshold can be determined based on the existing threshold determination method.

[0105] S2203. Filter out at least two extracellular plaques from the at least two plaques according to the position data of each of the cells and the position data of each of the plaques, so as to obtain the position data of at least two extracellular plaques.

[0106] In one embodiment, for each of the at least two plaques, it is determined whether the current plaque has no intersection with each cell in the region of interest; if yes, the current plaque is regarded as an extracellular plaque; if no, the current plaque is regarded as an intracellular plaque. This embodiment identifies the extracellular plaque by whether there is an intersection between the cell and the plaque, and can accurately complete the screening operation of the extracellular plaque.

[0107] In one embodiment, the screening method of extracellular plaques includes: determining whether the distance between the center of the current plaque and the center of each cell is greater than the sum of the radius of the corresponding cell and the radius of the current plaque; if so, the current plaque is regarded as an extracellular plaque. The radius of the current plaque is the radius of the current plaque in the direction in which its center of mass points to the center of the corresponding cell. This embodiment identifies extracellular plaques by the distance between the plaque and the cell, and the relationship between the sum of the plaque radius and the cell radius and the distance, and can quickly and efficiently complete the identification of extracellular plaques.

[0108] In one embodiment, the screening method of extracellular plaques includes: determining the position range of the at least two plaques according to the position data of the at least two plaques, and determining the position range of each cell according to the position data of each cell; for each plaque in the at least two plaques, if there is no cell in the set neighborhood range of the current plaque, then the current plaque is determined to be an extracellular plaque; if the set neighborhood range of the current plaque includes at least one cell, then the current plaque may be an intracellular plaque, and further judgment is required: determining the overlapping part of the position range of the current plaque and the position range of each cell in the neighborhood range, and the ratio of the overlapping part to the position range of the corresponding cell; if all ratios are less than the set overlapping threshold, then the current plaque is regarded as an extracellular plaque. This embodiment identifies extracellular plaques by determining the overlapping part of the plaque and the cell, and the ratio of the overlapping part to the cell position range, and can quickly and efficiently complete the identification of extracellular plaques.

[0109] S230. Determine a density index corresponding to the region of interest based on the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between each pair of extracellular plaques, and the coverage ratio of the extracellular plaques is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest.

[0110] S240. Determine a plaque classification result corresponding to the region of interest according to a density index corresponding to the region of interest, wherein the plaque classification result is diffuse type, dense type, or intermediate type.

[0111] The embodiment of the present invention selects at least two extracellular plaques from at least two plaques based on the position data of at least two cells and the position data of at least two plaques to obtain the position data of at least two extracellular plaques, thereby achieving the purpose of simply, quickly and accurately determining the position data of the extracellular plaques.

[0112] Figure 8 This is another flow chart of the plaque typing method provided in an embodiment of the present invention. This embodiment refines the method for determining the extracellular plaque location data in the previous embodiment. Figure 8 As shown, the method includes:

[0113] S310 , respectively determining regions of interest in a first image and a second image of a target tissue sample, wherein the regions of interest in the first image and the regions of interest in the second image include cell marking information and plaque marking information, respectively.

[0114] In one embodiment, after the sample tissue section is stained, the laser is controlled to output a first laser for exciting the fluorescent material in the cell nucleus staining result, and the image of the field of view is obtained through the camera module to obtain the first image; within the set time, the laser mode is switched to output a second laser for exciting the fluorescent material in the Aβ plaque staining result, and the image of the field of view is obtained through the camera module to obtain the second image. It can be understood that when the set time is short enough, the first image and the second image can be considered to be taken at the same time, and the combination of the two is equivalent to the target tissue sample image in the aforementioned embodiment.

[0115] S3201. Determine the position data of each cell in the region of interest of the first image, and determine the position data of at least two plaques included in the region of interest of the second image.

[0116] In one embodiment, the position data of each cell and the position data of at least two plaques are determined by the following steps, including:

[0117] Step b1: converting the region of interest of the first image and the region of interest of the second image into grayscale images to obtain a first grayscale image and a second grayscale image respectively.

[0118] Step b2: binarize the first grayscale image and the second grayscale image respectively to obtain a first binary image and a second binary image.

[0119] In one embodiment, the average brightness and brightness standard deviation of the first image are determined, and the sum of the average brightness and twice the brightness standard deviation is used as the first threshold; the average brightness and brightness standard deviation of the second image are determined, and the sum of the average brightness and twice the brightness standard deviation is used as the second threshold; the second threshold is used to binarize the second grayscale image to obtain a second binary image. Specifically, the pixel values ​​of pixels in the first grayscale image whose pixel values ​​are greater than the first threshold are modified to 255, and the pixel values ​​of pixels in the first grayscale image whose pixel values ​​are less than or equal to the first threshold are modified to 0. The pixel values ​​of pixels in the second grayscale image whose pixel values ​​are greater than the second threshold are modified to 255; the pixel values ​​of pixels in the second grayscale image whose pixel values ​​are less than or equal to the second threshold are modified to 0. This embodiment can improve the accuracy of determining the binary images of the first image and the second image.

[0120] In one embodiment, after the first binary map and the second binary map are determined, the white connected area in the first binary map is identified as a cell area, and the white area in the second binary map is identified as a plaque area. The cell area in the first binary map that is less than the first noise threshold is deleted to update the first binary map; the plaque area in the second binary map that is less than the second noise threshold is deleted to update the second binary map. This embodiment performs noise reduction processing on the first binary map and the second binary map by corresponding noise thresholds to improve the quality of the first binary map and the second binary map, thereby improving the accuracy of the typing results corresponding to the region of interest. Among them, the first noise threshold is the number of pixels included in the minimum acceptable cell area; the second noise threshold is the number of pixels included in the minimum acceptable plaque area. The first noise threshold and the second noise threshold can be set according to actual conditions.

[0121] Step b3: determining the position data of each cell in the region of interest of the first image according to the first binary image.

[0122] In one embodiment, for each cell in the first binary image, when the current cell is identified, the cell identification and position data of the current cell are determined; and the combination of the cell identification and position data of the current cell is used as the position data of the current cell. Exemplarily, [001, (x1, y1, r1)] means that the centroid coordinates of the cell with the cell identification 001 are (x1, y1) and the radius is r1.

[0123] Step b4: determining position data of at least two patches included in the region of interest of the second image according to the second binary image.

[0124] In one embodiment, for each patch in the first binary image, when the current patch is identified, the patch identification and position data of the current patch are determined; the combination of the patch identification and position data of the current patch is used as the position data of the current patch. Exemplarily, [001, (x1, y1, r1)] indicates that the centroid coordinates of the patch with the patch identification of 001 are (x1, y1) and the radius is r1.

[0125] S3202. Filter out at least two extracellular plaques from the at least two plaques according to the position data of each of the cells and the position data of the at least two plaques, so as to obtain the position data of the at least two extracellular plaques.

[0126] S330. Determine a density index corresponding to the region of interest based on the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between each pair of extracellular plaques, and the coverage ratio of the extracellular plaques is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest.

[0127] S340. Determine a plaque classification result corresponding to the region of interest according to a density index corresponding to the region of interest, wherein the plaque classification result is diffuse type, dense type, or intermediate type.

[0128] The embodiment of the present invention determines the position data of at least two extracellular plaques based on a first image including cell marking information and a second image including plaque marking information, thereby improving the details and accuracy of single information by simplifying the image content, thereby improving the accuracy of the position data of the extracellular plaques in the region of interest and the accuracy of the plaque typing results corresponding to the region of interest.

[0129] The following is an embodiment of a plaque typing device provided in an embodiment of the present invention. The device and the plaque typing method of the above-mentioned embodiment belong to the same inventive concept. For details not described in detail in the embodiment of the plaque typing device, reference can be made to the contents of the above-mentioned embodiments.

[0130] Fig. 9 Schematic diagram of the structure of the plaque typing device provided in the embodiment of the present invention. Fig. 9 As shown, the device comprises:

[0131] An acquisition module 41 is used to determine a region of interest in a target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information;

[0132] A position data module 42, used to determine position data of at least two extracellular plaques included in the region of interest;

[0133] An index module 43, configured to determine a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between any two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest;

[0134] The classification module 44 is used to determine the plaque classification result corresponding to the region of interest according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

[0135] In one embodiment, the location data module 42 includes:

[0136] Performing cell segmentation on the region of interest to obtain a cell segmentation result image, and determining position data of each cell in the cell segmentation result image;

[0137] Performing plaque segmentation on the region of interest to obtain a plaque segmentation result image; determining position data of each plaque in the plaque segmentation result image, wherein the plaque segmentation result image includes at least two plaques;

[0138] According to the position data of each of the cells and the position data of the at least two plaques, at least two extracellular plaques are screened out from the at least two plaques to obtain the position data of at least two extracellular plaques.

[0139] In one embodiment, the acquisition module 41 is specifically used for:

[0140] Determine regions of interest in a first image and a second image of a target tissue sample, respectively, wherein the region of interest in the first image and the region of interest in the second image include cell marking information and plaque marking information, respectively;

[0141] Position data module 42 (see Fig.10 )include:

[0142] A first position data unit 421, used to determine position data of each cell included in the region of interest of the first image, and determine position data of at least two plaques included in the region of interest of the second image;

[0143] The second position data unit 422 is used to screen out at least two extracellular plaques from the at least two plaques according to the position data of each of the cells and the position data of the at least two plaques, so as to obtain the position data of the at least two extracellular plaques.

[0144] In one embodiment, the second position data unit performs extracellular plaque screening by an extracellular plaque screening unit, and the extracellular plaque screening unit is used to:

[0145] For each of the at least two plaques, determining whether the current plaque has no intersection with each of the cells;

[0146] If yes, the current plaque is regarded as an extracellular plaque.

[0147] In one embodiment, Fig.11 As shown, the first position data unit 421 includes:

[0148] A grayscale image subunit 4211 is used to convert the region of interest of the first image and the region of interest of the second image into grayscale images to obtain a first grayscale image and a second grayscale image respectively;

[0149] A binarization subunit 4212, used for binarizing the first grayscale image and the second grayscale image to obtain a first binary image and a second binary image respectively;

[0150] A first position data subunit 4213 is used to determine the position data of cells included in the region of interest of the first image according to the first binary image;

[0151] The second position data subunit 4214 is used to determine the position data of at least two patches included in the region of interest of the second image according to the second binary image.

[0152] In one embodiment, the binarization subunit is specifically used for:

[0153] Determine the average brightness and brightness standard deviation of the region of interest of the first image, and use the sum of the average brightness and twice the brightness standard deviation as a first threshold;

[0154] Determine the average brightness and brightness standard deviation of the region of interest of the second image, and use the sum of the average brightness and twice the brightness standard deviation as a second threshold;

[0155] Binarizing the first grayscale image using the first threshold to obtain a first binary image;

[0156] The second grayscale image is binarized using the second threshold to obtain a second binary image.

[0157] In one embodiment, the density index is the sum of the correlation coefficients of all extracellular plaque combinations corresponding to the region of interest; all extracellular plaque combinations include pairwise combinations of at least two extracellular plaques; the correlation coefficient is the product of the coverage ratio of each extracellular plaque included in the extracellular plaque combination and the Euclidean distance between the corresponding two extracellular plaques.

[0158] In one embodiment, the typing module 44 is specifically used for:

[0159] If the density index is greater than a first set threshold, the plaque in the region of interest is determined to be a dense plaque;

[0160] If the density index is less than a second set threshold, the plaque in the region of interest is determined to be a diffuse plaque;

[0161] If the density index is greater than or equal to the second set threshold and less than or equal to the first set threshold, the plaque in the region of interest is determined to be an intermediate plaque;

[0162] The first set threshold is greater than the second set threshold.

[0163] In one embodiment, the target tissue sample is brain tissue; and the extracellular plaque is amyloid-β plaque.

[0164] The plaque typing device provided by the embodiment of the present invention determines the position data of at least two extracellular plaques included in the region of interest; determines the density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, and determines the plaque typing result corresponding to the region of interest according to the density index corresponding to the region of interest. Since the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, it can accurately describe the distribution state of the extracellular plaques included in the region of interest, so the plaque typing result determined by the density index has a higher accuracy.

[0165] The plaque typing device provided in the embodiment of the present invention can execute the plaque typing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0166] In some embodiments, the plaque typing method can be implemented as a computer program, which is tangibly contained in a computer readable storage medium, such as storage unit 18, see Figure 1 In some embodiments, part or all of the computer program may be loaded and / or installed on the plaque identification system 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the plaque typing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the plaque typing method in any other appropriate manner (e.g., by means of firmware).

[0167] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0169] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0171] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0172] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0173] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0174] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A plaque typing system, characterized in that: comprising a processor configured to perform a plaque typing method comprising: Determine a region of interest in the target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information; Determining position data of at least two extracellular plaques included in the region of interest; Determining a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest; The plaque classification result corresponding to the region of interest is determined according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

2. The system according to claim 1, characterized in that The determining of the location data of at least two extracellular plaques included in the region of interest comprises: Performing cell segmentation on the region of interest to obtain a cell segmentation result image, and determining position data of each cell in the cell segmentation result image; Performing plaque segmentation on the region of interest to obtain a plaque segmentation result image; determining position data of each plaque in the plaque segmentation result image, wherein the plaque segmentation result image includes at least two plaques; According to the position data of each of the cells and the position data of each of the plaques, at least two extracellular plaques are screened out from the at least two plaques to obtain the position data of at least two extracellular plaques.

3. The system according to claim 1, characterized in that The determining of a region of interest in the target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information, comprises: Determine regions of interest in a first image and a second image of a target tissue sample, respectively, wherein the region of interest in the first image and the region of interest in the second image include cell marking information and plaque marking information, respectively; The determining of the location data of at least two extracellular plaques included in the region of interest comprises: Determine the position data of each cell in the region of interest of the first image, and determine the position data of at least two plaques included in the region of interest of the second image; According to the position data of each of the cells and the position data of the at least two plaques, at least two extracellular plaques are screened out from the at least two plaques to obtain the position data of the at least two extracellular plaques.

4. The system according to claim 2 or 3, characterized in that: Extracellular plaque screening methods, including: For each of the at least two patches, determining whether the current patch has no intersection with each cell in the region of interest; If yes, the current plaque is regarded as an extracellular plaque.

5. The system according to claim 3, characterized in that The step of determining the position data of each cell in the region of interest of the first image and determining the position data of at least two plaques included in the region of interest of the second image comprises: Respectively converting the region of interest of the first image and the region of interest of the second image into grayscale images to obtain a first grayscale image and a second grayscale image; Binarizing the first grayscale image and the second grayscale image to obtain a first binary image and a second binary image respectively; Determine the position data of each cell in the region of interest of the first image according to the first binary image; The position data of at least two patches included in the region of interest of the second image are determined according to the second binary image.

6. The system according to claim 5, characterized in that The step of binarizing the first grayscale image and the second grayscale image to obtain a first binary image and a second binary image comprises: Determine the average brightness and brightness standard deviation of the region of interest of the first image, and use the sum of the average brightness and twice the brightness standard deviation as a first threshold; Determine the average brightness and brightness standard deviation of the region of interest of the second image, and use the sum of the average brightness and twice the brightness standard deviation as a second threshold; Binarizing the first grayscale image using the first threshold to obtain a first binary image; The second grayscale image is binarized using the second threshold to obtain a second binary image.

7. The system according to claim 1, characterized in that The density index is the sum of the correlation coefficients of all extracellular plaque combinations corresponding to the region of interest; All the extracellular plaque combinations include any two combinations of the at least two extracellular plaques; The correlation coefficient is the product of the coverage ratio of each extracellular plaque included in the extracellular plaque combination and the Euclidean distance between the corresponding two extracellular plaques.

8. The system according to claim 1, characterized in that Determining the plaque classification result corresponding to the region of interest according to the density index corresponding to the region of interest includes: If the density index is greater than a first set threshold, the plaque in the region of interest is determined to be a dense plaque; If the density index is less than a second set threshold, the plaque in the region of interest is determined to be a diffuse plaque; If the density index is greater than or equal to the second set threshold and less than or equal to the first set threshold, the plaque in the region of interest is determined to be an intermediate plaque; The first set threshold is greater than the second set threshold.

9. The system according to claim 1, characterized in that The target tissue sample is brain tissue; the extracellular plaque is amyloid-β plaque.

10. A plaque typing device, characterized in that: include: An acquisition module, used to determine a region of interest in a target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information; A position data module, used to determine the position data of at least two extracellular plaques included in the region of interest; An index module, used to determine a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest; The classification module is used to determine the plaque classification result corresponding to the region of interest according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

11. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the following method when executed, including: Determine a region of interest in the target tissue sample image, wherein the region of interest includes cell marking information and plaque marking information; Determining position data of at least two extracellular plaques included in the region of interest; Determining a density index corresponding to the region of interest according to the position data of the at least two extracellular plaques, wherein the density index is determined based on the coverage ratio of each extracellular plaque and the distance between two extracellular plaques, and the coverage ratio of the extracellular plaque is the ratio of the area of ​​the corresponding extracellular plaque to the sum of the areas of all extracellular plaques included in the region of interest; The plaque classification result corresponding to the region of interest is determined according to the density index corresponding to the region of interest, and the plaque classification result is diffuse type, dense type or intermediate type.

Images