Image processing method, device, computer equipment and storage medium
The staining value threshold is determined by histogram analysis of cell images, and the problem of changes in acquisition conditions in the prior art affecting the recognition accuracy is solved, and the robustness of cell state recognition is improved.
Patent Information
- Application Number
- CN202210021939.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-10
AI Technical Summary
When using network models to identify cell states, the prior art is less robust due to changes in the collection conditions and is difficult to accurately identify cell states under conditions.
By performing histogram analysis on the stained cell images, the staining value threshold of the cells was determined, and the cell status was judged based on the threshold, which improved the recognition robustness.
It can accurately identify the cell state under changes in acquisition conditions, and improve the robustness and stability of image processing.
Smart Images

Figure CN114359247B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular to an image processing method, apparatus, computer equipment, and storage medium. Background Art
[0002] Usually when using IHC (Immunohistochemistry) to detect cell status, the cell tissue to be detected is first stained with a dye. Cells in different states will be stained in different colors. Then, the pathological image corresponding to the cell tissue is taken. By calling the trained network model and processing the pathological image, the status of each cell can be obtained.
[0003] However, this method of identifying cell status by calling the network model requires that the acquisition conditions of the processed pathological images are the same as those of the sample pathological images used when training the network model in order to accurately identify the cell status. If the acquisition conditions change, the accuracy of recognition will be affected. Therefore, the robustness of this technology is poor. Summary of the invention
[0004] The embodiments of the present application provide an image processing method, apparatus, computer device and storage medium, which improve the robustness. The technical solution is as follows:
[0005] In one aspect, an image processing method is provided, the method comprising:
[0006] Determining a histogram of a first pathological image based on staining values of a plurality of cell centers in a first pathological image, wherein the first pathological image is obtained by photographing a plurality of stained cells, the staining values of the cell centers representing staining conditions of cells where the cell centers are located, and the histogram representing distribution conditions of staining values corresponding to the plurality of cells;
[0007] Determining a staining value threshold corresponding to the first pathological image based on a first staining value in the histogram, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values;
[0008] The staining value of each cell center is compared with the staining value threshold to obtain the state of the cell where each cell center is located.
[0009] In another aspect, an image processing device is provided, the device comprising:
[0010] a histogram determination module, configured to determine a histogram of a first pathological image based on staining values of a plurality of cell centers in the first pathological image, wherein the first pathological image is obtained by photographing a plurality of stained cells, the staining values of the cell centers representing staining conditions of cells where the cell centers are located, and the histogram representing distribution conditions of staining values corresponding to the plurality of cells;
[0011] a threshold determination module, configured to determine a staining value threshold corresponding to the first pathological image based on a first staining value in the histogram, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values;
[0012] The state determination module is used to compare the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located.
[0013] Optionally, the threshold determination module includes:
[0014] a number determination unit, configured to reduce the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells;
[0015] A first determining unit, configured to determine a maximum staining value among a plurality of staining values corresponding to a cell number not greater than the reference cell number and less than the first staining value as a second staining value;
[0016] a second determining unit, configured to determine a difference between the first coloring value and the second coloring value and a sum of the first coloring value as a third coloring value;
[0017] A threshold determination unit is used to determine the third coloring value as the coloring value threshold.
[0018] Optionally, the threshold determination module includes:
[0019] a number determination unit, configured to reduce the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells;
[0020] A first determining unit, configured to determine a maximum staining value among a plurality of staining values corresponding to a cell number not greater than the reference cell number and less than the first staining value as a second staining value;
[0021] a second determining unit, configured to determine a difference between the first coloring value and the second coloring value and a sum of the first coloring value as a third coloring value;
[0022] a third determining unit, configured to determine the number of cells corresponding to adjacent target number of staining values in the histogram each time, and when the number of cells corresponding to the middle staining value among the target number of staining values is the smallest, determine the middle staining value as the fourth staining value, and the target number is an odd number;
[0023] The threshold determination unit is further used to determine the minimum coloring value greater than the third coloring value among the determined multiple fourth coloring values as the coloring value threshold.
[0024] Optionally, the device further comprises:
[0025] An image acquisition module, used for acquiring a second pathological image, wherein the second pathological image is obtained by photographing the plurality of cells after staining;
[0026] The image extraction module is used to extract the first pathological image corresponding to a target channel from the second pathological image, where the target channel is a channel reflecting the staining conditions of the multiple cells in the second pathological image.
[0027] Optionally, the image extraction module is used to:
[0028] Determine a reference matrix, where the reference matrix represents color information of multiple channels included in the second pathological image, where the multiple channels include the target channel;
[0029] Based on the reference matrix, deconvolution processing is performed on the second pathological image to obtain pathological images corresponding to the multiple channels, wherein the obtained multiple pathological images include the first pathological image corresponding to the target channel.
[0030] Optionally, the device further comprises:
[0031] A center prediction module, used for performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells;
[0032] The dyeing value determination module is used to determine the dyeing value corresponding to the coordinates of each cell center as the dyeing value of each cell center.
[0033] Optionally, the device further comprises:
[0034] A center prediction module, used for performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells;
[0035] A filtering module is used to perform median filtering on the first pathological image based on the coordinates of the center of each cell.
[0036] Optionally, each cell in the first pathological image includes a plurality of position points, each position point has a corresponding staining value, and the center prediction module is used to:
[0037] For each of the cells:
[0038] Obtaining staining values of multiple position points contained in the cell;
[0039] Determine the maximum and identical multiple coloring values from the obtained multiple coloring values;
[0040] Connecting the position points corresponding to the determined multiple coloring values to form a connected domain;
[0041] The coordinates corresponding to the center point of the connected domain are determined as the coordinates of the cell center.
[0042] Optionally, the center prediction module is used to obtain, for each of the cells, a maximum staining value among staining values of a plurality of position points contained in the cell, and determine the coordinates of the position point corresponding to the maximum staining value as the coordinates of the center of the cell.
[0043] Optionally, the center prediction module is used to call the center prediction model to perform center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells.
[0044] Optionally, the state determination module is used to:
[0045] For any cell center in the first pathological image:
[0046] When the staining value of the cell center is greater than the staining value threshold, determining that the cell where the cell center is located is in the first state;
[0047] When the staining value of the cell center is not greater than the staining value threshold, it is determined that the cell where the cell center is located is in the second state.
[0048] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method described in the above aspects.
[0049] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method described in the above aspects.
[0050] On the other hand, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the operations performed by the image processing method described in the above aspects are implemented.
[0051] In the technical solution provided in the embodiment of the present application, since cells in different states present different colors after staining, the histogram can reflect the staining condition corresponding to each cell in the first pathological image, so that based on the histogram, the staining value threshold applicable to the cells in the first pathological image can be determined, and the cell state is determined according to the staining value threshold. This method of determining the cell state is not affected by the acquisition conditions of the first pathological image. Even if the acquisition conditions of the first pathological image change, the corresponding staining value threshold can be determined according to the histogram, thereby determining the state of each cell, and has high robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0053] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;
[0054] Figure 2 is a flow chart of an image processing method provided by an embodiment of the present application;
[0055] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present application;
[0056] Figure 4 is a schematic diagram of a histogram provided in an embodiment of the present application;
[0057] Figure 5 is a schematic diagram of a pathological image provided in an embodiment of the present application;
[0058] Figure 6 is a flowchart of another image processing method provided by an embodiment of the present application;
[0059] Figure 7 is a flowchart of another image processing method provided by an embodiment of the present application;
[0060] Figure 8 is a flowchart of another image processing method provided by an embodiment of the present application;
[0061] Fig. 9is a structural schematic diagram of an image processing device provided in an embodiment of the present application;
[0062] Fig.10 is a structural schematic diagram of another image processing device provided in an embodiment of the present application;
[0063] Fig.11 is a schematic diagram of the structure of a terminal provided in an embodiment of the present application;
[0064] Fig.12 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0065] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.
[0066] It is understood that the terms "first", "second", etc. used in this application can be used in this article to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, the first arrangement order can be called the second arrangement order, and the second arrangement order can be called the first arrangement order.
[0067] The terms "at least one", "a plurality of", "each", "any" and the like used in this application include one, two or more than two, a plurality of includes two or more than two, each refers to each of the corresponding plurality, and any refers to any one of the plurality. For example, the plurality of virtual object identifiers include three virtual object identifiers, and each virtual object identifier refers to each of the three virtual object identifiers, and any refers to any one of the three virtual object identifiers, which may be the first, the second, or the third.
[0068] In order to facilitate understanding of the embodiments of the present application, the key words involved in the embodiments of the present application are first explained:
[0069] IHC: Immunohistochemistry, abbreviated as IHC, is a method of using specific primary antibodies in pathology to label proteins in cell tissues and use detection reagents to visualize targets. IHC can be used to evaluate protein expression through colorimetric detection or fluorescent detection.
[0070] DAB (3,3'-Diaminobenzidine): a chromogenic substrate used for specific staining in pathological immunohistochemistry, which usually makes cells or substances in the first state appear brown.
[0071] AEC (3-amino-9-ethylcarbazole): a chromogenic substrate used for specific staining in pathological immunohistochemistry, which usually makes cells or substances in the first state appear dark red.
[0072] HE staining (Hematoxylin-Eosin Staining): It is usually used to stain tissues in the process of making pathological sections. Hematoxylin staining solution is alkaline, which mainly stains the chromatin in the cell nucleus and the nucleic acid in the cytoplasm purple-blue; eosin is an acidic dye, which mainly stains the components in the cytoplasm and extracellular matrix red. In particular, hematoxylin is also used in immunohistochemistry, which stains the cell nucleus blue or blue-purple.
[0073] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0074] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.
[0075] Computer Vision (CV) is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify and measure targets, and further processing graphics to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and mapping, automatic driving, smart transportation and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.
[0076] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.
[0077] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, Internet of Vehicles, automatic driving, smart transportation, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.
[0078] The solution provided in the embodiments of the present application involves technologies such as artificial intelligence computer vision, which is specifically illustrated by the following embodiments.
[0079] The image processing method provided in the embodiment of the present application is executed by a computer device, which is a terminal or a server. Optionally, the terminal 101 is a computer, a mobile phone, a tablet computer, an AI microscope or other terminals. Optionally, the server 102 is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0080] For example, the computer device is an AI microscope, which includes a microscope, a camera and a processing device. The microscope is used to observe pathological sections, which are sections containing multiple stained cells. The camera is used to photograph the pathological sections, and the processing device is used to process the photographed pathological images.
[0081] In one possible implementation, the image processing method provided in the embodiment of the present application is executed interactively by a terminal and a server. Figure 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application. Figure 1 , the implementation environment includes a terminal 101 and a server 102. The terminal 101 and the server 102 are connected via a wireless or wired network.
[0082] The target application provided by the server 102 is installed on the terminal 101, and the terminal 101 can realize functions such as data transmission and message interaction through the target application. Optionally, the target application is a target application in the operating system of the terminal 101, or a target application provided by a third party. For example, the target application is an image processing application, which has the function of image processing. Of course, the image processing application can also have other functions. Optionally, the server 102 is the background server of the target application or a cloud server that provides services such as cloud computing and cloud storage.
[0083] In a possible implementation, the terminal 101 photographs the pathological slice to obtain a pathological image, and sends the pathological image to the server 102 . The server 102 processes the pathological image to obtain the state of each cell in the pathological image.
[0084] Figure 2 is a flowchart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 2 , the method comprises the following steps:
[0085] 201. A computer device determines a histogram of a first pathological image based on staining values of multiple cell centers in the first pathological image.
[0086] The first pathological image is obtained by photographing a plurality of cells after staining, and the first pathological image includes a plurality of cells, each of which has a cell center. Since the cell nucleus can be stained when the cell is stained, the cell nucleus of cells in different states is stained with different colors, therefore, when determining the cell state, the staining value of the cell nucleus can be considered, that is, the staining value of the cell center represents the staining condition of the cell where the cell center is located.
[0087] The histogram can represent the relationship between each staining value and the number of cells corresponding to the staining value in the first pathological image, that is, the histogram represents the distribution of staining values corresponding to a plurality of cells.
[0088] 202. The computer device determines a staining value threshold corresponding to the first pathological image based on the first staining value in the histogram.
[0089] In the embodiment of the present application, the staining value threshold for classifying different states is determined by the first staining value, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values.
[0090] 203. The computer device compares the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located.
[0091] For each cell in the first pathological image, the state of the cell at the center of the cell is determined according to the staining value of the center of the cell and the staining value threshold.
[0092] The method provided in the embodiment of the present application, since cells in different states present different colors after staining, the histogram can reflect the staining status of each cell in the first pathological image, so that based on the histogram, a staining value threshold applicable to the cells in the first pathological image can be determined, and the cell state is determined according to the staining value threshold. This method of determining the cell state is not affected by the acquisition conditions of the first pathological image. Even if the acquisition conditions of the first pathological image change, the corresponding staining value threshold can be determined according to the histogram, thereby determining the state of each cell, and has high robustness.
[0093] Figure 2 The embodiment shown briefly introduces the process of determining the state of each cell in the first pathological image based on the distribution of staining values corresponding to multiple cells in the histogram. Figure 3 The illustrated embodiment provides a detailed description of the implementation method for determining the staining value threshold.
[0094] Figure 3 is a flowchart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 3 , the method comprises the following steps:
[0095] 301. A computer device determines a histogram of a first pathological image based on staining values of multiple cell centers in the first pathological image.
[0096] The first pathological image is obtained by photographing a plurality of cells after staining, and the first pathological image includes a plurality of cells, each of which has a cell center. Since the cell nucleus can be stained when the cell is stained, the cell nucleus of cells in different states is stained with different colors. Therefore, when determining the cell state, the staining value of the cell nucleus can be considered, that is, the staining value of the cell center represents the staining condition of the cell where the cell center is located, and the pixel value of each position point in the first pathological image is the staining value.
[0097] In the embodiment of the present application, the range of the staining value is [0, 255], and the staining value is an integer. For each staining value, the computer device counts the number of cell centers corresponding to the staining value in the first pathological image to obtain the number of cells corresponding to each staining value, thereby determining the histogram of the first pathological image according to the number of cells corresponding to each staining value. Among them, the histogram takes the staining value as the ordinate and the number of cells as the abscissa; or takes the number of cells as the ordinate and the staining value as the abscissa. For example, k∈[0,255] represents the staining value, that is, k takes an integer from 0 to 255. According to the staining value of each cell center in the first pathological image, the independent variable k is mapped to h(k), and h(k) represents the mapping relationship between the staining value and the number of cells. According to the mapping relationship, a histogram is established.
[0098] For example, see Figure 4 Schematic diagram of histograms of multiple pathological images shown, Figure 4 The histogram shown uses the staining value as the ordinate and the number of cells as the abscissa. The ordinate ranges from 0 to 255 from bottom to top in different histograms. However, in different histograms, the abscissa range is adjusted according to the number of cells in the corresponding first pathological image. When the number of cells in the first pathological image is large, the abscissa range is set to 0 to 500, 0 to 600 or other ranges, and when the number of cells in the first pathological image is small, the abscissa range is set to 0 to 300, 0 to 200 or other ranges. Figure 4 It can be seen that different histograms have the same characteristics, that is, there is a peak number of cells in the histogram, and the peak number of cells corresponds to a smaller staining value, and a larger staining value corresponds to a smaller number of cells.
[0099] 302. The computer device reduces the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells.
[0100] Since cells in different states have different staining conditions, in the first pathological image, the staining value of the cell center corresponding to the cell in the first state is larger, and the span of the staining value interval formed by the staining values of the cell centers corresponding to multiple cells in the first state is larger, and the distribution in the histogram is more discrete, while the staining value of the cell center corresponding to the cell in the second state is smaller, and since the staining is more uniform, the span of the staining value interval formed by the staining values of the cell centers corresponding to the cells in the second state is smaller, and the distribution in the histogram is more concentrated. The embodiment of the present application uses the characteristics of the stained cells to determine the staining value threshold to determine the state of the cells.
[0101] According to the characteristics of the cells after the above staining, it can be determined that a large number of cells in the second state are likely to correspond to the same staining value, resulting in the number of cells corresponding to a certain staining value in the histogram being greater than the number of cells corresponding to other staining values. In this case, this staining value is determined as the first staining value, and the cells corresponding to the first staining value are the cells in the second state.
[0102] For example, the number of cells corresponding to the first staining value is expressed as:
[0103]
[0104] Wherein, V1 represents the first staining value, h(V1) represents the number of cells corresponding to the first staining value, It represents the maximum value of h(k) obtained after traversing all k values.
[0105] Furthermore, considering that the first pathological image includes a large number of cells, for the cells in the second state, the mapping relationship between the staining values corresponding to these cells and the number of cells presents a normal distribution on the histogram. For example, see Figure 4 , the maximum peak in each histogram corresponds to the first staining value and the maximum number of cells, and the number of cells corresponding to the staining values on both sides of the first staining value gradually decreases. Based on this feature, a target ratio is set, and the number of cells corresponding to the first staining value is reduced according to the target ratio. The reference cell number can be determined so that the reference cell number is smaller than the number of cells corresponding to the first staining value, and the staining values corresponding to the reference cell number can be located on both sides of the first staining value. Among them, the target ratio is a smaller value, for example, the target ratio is 0.05, 0.06, etc.
[0106] 303. The computer device determines the maximum staining value among multiple staining values whose corresponding cell numbers are not greater than the reference cell number and less than the first staining value as the second staining value, and determines the difference between the first staining value and the second staining value and the sum of the first staining value as the third staining value.
[0107] In an embodiment of the present application, after determining the reference cell number, a second staining value is determined from the histogram based on the distribution of staining values in the histogram, the cell number corresponding to the second staining value is not greater than the reference cell number, and the second staining value is the maximum staining value among multiple staining values that are less than the first staining value, thereby ensuring that the second staining value satisfies the requirement that the corresponding cell number is not greater than the reference cell number.
[0108] For example, the second coloring value is determined using the following formula:
[0109] V0=max{k|h(k)≤h(V1)×T and k <V1}
[0110] Wherein, V0 represents the second staining value, V1 represents the first staining value, T represents the target ratio, h(V1) represents the number of cells corresponding to the first staining value, h(V1)×T represents the reference number of cells, and max{x|y} represents the maximum value of multiple x that satisfy condition y.
[0111] When the first coloring value and the second coloring value in the histogram are known, a coloring value width can be obtained according to the difference between the first coloring value and the second coloring value, and the third coloring value can be obtained by calculating the sum of the first coloring value and the coloring value width. For example, the third coloring value is expressed as 2V1-V0.
[0112] It should be noted that, in one possible implementation, the obtained third staining value can be directly determined as the staining value threshold. In another possible implementation, when calculating the third staining value, the target ratio used is preset, and there may be inaccuracies. For example, if the cells corresponding to the staining values adjacent to the third staining value and greater than the third staining value in the histogram are cells in the second state, it means that the target ratio is too small, and the number of cells corresponding to the staining values adjacent to the third staining value and greater than the third staining value in the histogram is less than the number of cells corresponding to the third staining value. Therefore, in order to further improve the accuracy of the determined staining value threshold, it is necessary to perform the following steps 304-305.
[0113] 304. The computer device determines the number of cells corresponding to adjacent target number of staining values in the histogram each time, and when the number of cells corresponding to the middle staining value among the target number of staining values is the smallest, determines the middle staining value as the fourth staining value.
[0114] In the embodiment of the present application, the histogram is subjected to minimum value filtering, with the target number as the filter mask width, starting from the staining value 0, with the adjacent target number of staining values as an interval, the number of cells corresponding to the staining values in each interval is determined, and then it is determined whether the number of cells corresponding to the middle staining value in the target number of staining values is the minimum number of cells in the number of cells corresponding to each staining value in the interval. If so, the middle staining value is determined as the fourth staining value. If not, the staining value is ignored, and the above process is repeated, and the histogram is polled according to the target number each time, and finally multiple fourth staining values will be obtained. Among them, in order to ensure that the middle staining value in the interval can be determined, the target number needs to be set to an odd number greater than 1.
[0115] For example, the target number is 3. Starting from the staining value 0, the staining values 0, 1 and 2 form an interval. For the interval, the cell numbers corresponding to the staining values 0, 1 and 2 are obtained respectively. Among these three cell numbers, when the cell number corresponding to the staining value 1 is the smallest, the cell number corresponding to the staining value 1 is determined as the fourth staining value; then, polling is continued according to the target number, and the staining values 3, 4 and 5 form an interval. The cell numbers corresponding to the staining values 3, 4 and 5 are obtained respectively. Among these three cell numbers, when the cell number corresponding to the staining value 4 is the smallest, the cell number corresponding to the staining value 4 is determined as the fourth staining value, and then other fourth staining values in the subsequent staining values are determined, until all staining values are traversed to obtain multiple fourth staining values.
[0116] For example, the following formula is used to perform minimum value filtering on the histogram:
[0117] g(k)=minFilter S (h(k))
[0118] Among them, g(k) represents the mapping relationship between the fourth staining value and the corresponding number of cells, h(k) represents the histogram, minFilter S (·) indicates minimum filtering with S as the filter mask width.
[0119] 305. The computer device determines the minimum coloring value that is greater than the third coloring value among the determined multiple fourth coloring values as a coloring value threshold.
[0120] The multiple fourth staining values determined in the above steps are all staining values corresponding to the local minimum number of cells in the histogram. In order to ensure the accuracy of the staining value threshold, the minimum staining value greater than the third staining value is selected from the multiple fourth staining values as the staining value threshold. The staining value threshold can be used as a dividing line for different states.
[0121] For example, the following formula is used to determine the staining value threshold:
[0122]
[0123] Wherein, V2 represents the coloring value threshold, Q represents a coloring value set including a plurality of fourth coloring values, and Q={k|h(k)=g(k)}.
[0124] 306. The computer device compares the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located.
[0125] When the staining value of the cell center is greater than the staining value threshold, the cell at the cell center is determined to be in the first state; when the staining value of the cell center is not greater than the staining value threshold, the cell at the cell center is determined to be in the second state. The first state refers to an abnormal cell state, for example, a cell in the first state indicates that the cell has a disease, and the second state refers to a normal cell state, for example, a cell in the second state indicates that the cell has no disease.
[0126] In a possible implementation, after the computer device determines the states of different cells, it can mark the cells in the first pathological image, for example, see Figure 5 In the three pathological images shown, darker dots represent cells in a first state, and lighter dots represent cells in a second state.
[0127] In another embodiment, after the computer device determines the state of each cell, it can further determine the aggregation area of cells in the first state based on the state of each cell. Figure 5 The area framed by lines is the aggregation area of cells in the first state.
[0128] It should be noted that the embodiment of the present application is only described by taking the above-mentioned first pathological image as an example. In another embodiment, for other pathological images, a similar implementation method to the above-mentioned embodiment can be adopted to determine the state of each cell in the pathological image.
[0129] The method provided in the embodiment of the present application, since cells in different states present different colors after staining, the histogram can reflect the staining status of each cell in the first pathological image, so that based on the histogram, a staining value threshold applicable to the cells in the first pathological image can be determined, and the cell state is determined according to the staining value threshold. This method of determining the cell state is not affected by the acquisition conditions of the first pathological image. Even if the acquisition conditions of the first pathological image change, the corresponding staining value threshold can be determined according to the histogram, thereby determining the state of each cell, and has high robustness.
[0130] Furthermore, in an embodiment of the present application, in order to avoid the influence of the target ratio on the accuracy of the staining value threshold, when determining the staining value threshold, after preliminarily determining the range of staining values corresponding to cells in the second state, that is, the range determined based on the second staining value and the third staining value, the histogram will also be subjected to minimum value filtering to obtain multiple fourth staining values in the histogram, thereby determining the staining value threshold based on the multiple fourth staining values and the third staining value, thereby improving the accuracy of the staining value threshold.
[0131] In some embodiments, before determining the histogram of the first pathological image based on the staining values of the multiple cell centers in the first pathological image, the computer device also needs to process the second pathological image initially taken to obtain the first pathological image and each cell center in the first pathological image. For a detailed processing process, see Figure 6 .
[0132] Figure 6 is a flowchart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device. Figure 6 , the method comprises the following steps:
[0133] 601. The computer device acquires a second pathological image.
[0134] In an embodiment of the present application, the second pathological image is an RGB (Red Green Blue) image, and the second pathological image is an image obtained by directly photographing the pathological section under a microscope by a computer device through a camera, for example, the second pathological image is an image photographed by an AI microscope; or it is an image photographed by other devices and sent to the computer device, for example, the computer device is an AI pathology cloud server, then the second pathological image is uploaded to the AI pathology cloud server by other devices, and the embodiment of the present application does not limit this.
[0135] Regarding the cell staining method, in one possible implementation, DAB can be used for staining, and the DAB can dye the cells in the first state into brown. Of course, since different cells have different staining degrees, different cells in the first state can be dyed into any one of light brown, medium brown or dark brown.
[0136] In another possible implementation, AEC can be used for staining, and the AEC can dye the cells in the first state red. Similarly, due to different degrees of staining of different cells, different cells in the first state can be dyed any one of light red, medium red or dark red.
[0137] In another embodiment, other stains can also be used to stain the cells, and the embodiment of the present application does not limit the stains used for staining.
[0138] 602. The computer device extracts the first pathological image corresponding to the target channel from the second pathological image.
[0139] The target channel is a channel that reflects the staining conditions of multiple cells in the second pathological image. The target channel is predetermined. For example, the target channel is a DAB channel, an AEC channel or other channels.
[0140] In a possible implementation, the computer device performs color deconvolution on the second pathological image to obtain the first pathological image. That is, the computer device determines a reference matrix, the reference matrix represents color information of multiple channels contained in the second pathological image, and the multiple channels include a target channel; based on the reference matrix, the second pathological image is deconvolved to obtain pathological images corresponding to the multiple channels, wherein the obtained multiple pathological images include the first pathological image corresponding to the target channel.
[0141] For example, DAB is used to stain cells. Since DAB can dye cells in the first state brown, the cell state needs to be determined based on the information of the DAB channel in the subsequent processing process, so the DAB channel is determined as the target channel. Then, in order to extract the first pathological image corresponding to the DAB channel, HE-DAB (Hematoxylin-Eosin-3,3'-Diaminobenzidine) color deconvolution is performed on the second pathological image, that is, the pathological image corresponding to the HE channel and the pathological image corresponding to the DAB channel are extracted from the second pathological image, and the pathological image corresponding to the DAB channel is determined as the first pathological image.
[0142] Another point that needs to be explained is that the embodiments of the present application only use the above-mentioned methods of extracting the first pathological image as examples for explanation. In another embodiment, other methods can also be used to extract the first pathological image. For example, the second pathological image is converted to the HSV (Hue Saturation Value) color space, and then the first pathological image corresponding to the hue channel is extracted, and subsequent operations are performed on the first pathological image; or the second pathological image is directly converted into a single-channel grayscale image, the grayscale image is determined as the first pathological image, and subsequent operations are performed on the first pathological image.
[0143] 603. The computer device obtains a staining value of the cell center of each cell in the first pathological image.
[0144] The computer device first needs to determine the cell center of each cell in the first pathological image, that is, to predict the center of each cell in the first pathological image to obtain the coordinates of each cell center; then the staining value corresponding to the coordinates of each cell center is determined as the staining value of each cell center.
[0145] Regarding the determination of the cell center, since the cell center is usually stained darker when the cell is stained, the cell center needs to be determined based on the position point corresponding to the maximum staining value. In one possible implementation, for each cell, the computer device obtains the staining values of multiple position points contained in the cell, determines the maximum and identical multiple staining values from the obtained multiple staining values, connects the position points corresponding to the determined multiple staining values to form a connected domain, and determines the coordinates corresponding to the center point of the connected domain as the coordinates of the cell center.
[0146] In another possible implementation, for each cell, the computer device obtains the maximum staining value among the staining values of multiple position points contained in the cell, and determines the coordinates of the position point corresponding to the maximum staining value as the coordinates of the cell center.
[0147] In another possible implementation, the computer device calls a center prediction model to predict the center of each cell in the first pathological image to obtain the coordinates of the center of each cell. The center prediction model is a model for predicting the center of a cell.
[0148] It should be noted that the embodiment of the present application is only described by taking the example of performing center prediction on each cell in the first pathological image to obtain the coordinates of the center of each cell. Since the cells in the second pathological image and the first pathological image are the same, in another embodiment, the computer device can perform center prediction on each cell in the second pathological image to obtain the coordinates of the center of each cell, and determine the coordinates of the center of each cell as the coordinates of the center of each cell in the first pathological image. In this case, the computer device can simultaneously execute the process of extracting the first pathological image corresponding to the target channel and the process of determining the coordinates of the center of each cell.
[0149] 604. The computer device determines a histogram of the first pathological image based on staining values of multiple cell centers in the first pathological image.
[0150] 605. The computer device determines a staining value threshold corresponding to the first pathological image based on the first staining value in the histogram.
[0151] 606. The computer device compares the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located.
[0152] The implementation of step 604 to step 606 is the same as the implementation of step 301 to step 306 described above, and will not be repeated here.
[0153] It should be noted that if it is possible to ensure that other pathological sections are photographed under the same acquisition conditions as those corresponding to the second pathological image to obtain a third pathological image, since the acquisition conditions are the same, the staining value threshold will not be affected by the acquisition conditions, so the staining value threshold determined for the second pathological image can also be determined as the staining value threshold corresponding to the third pathological image, thereby determining the state of cells in the third pathological image based on the staining value threshold, thereby achieving the staining value threshold. When the acquisition conditions change, after photographing other pathological sections to obtain the third pathological image, the staining value threshold can still be re-determined, thereby determining the state of cells in the third pathological image based on the re-determined staining value threshold. Among them, the acquisition conditions include slice preparation, the dye used, the shooting equipment, the shooting environment, etc.
[0154] As for the method of re-determining the staining value threshold, optionally, the method of determining the staining value threshold for the third pathological image is similar to step 601 to step 606 in the above embodiment. Alternatively, in the case where multiple staining value thresholds have been obtained, a moving average method is used to obtain a new staining value threshold, for example, directly averaging multiple staining value thresholds to obtain a new staining value threshold, or performing weighted averaging on multiple staining value thresholds according to the weight of each staining value threshold to obtain a new staining value threshold, the weight of the staining value threshold is determined according to the time between the acquisition time of each staining value threshold and the current time interval, the longer the time between the acquisition time of the staining value threshold and the current time interval, the smaller the weight of the staining value threshold, and the shorter the time between the acquisition time of the staining value threshold and the current time interval, the greater the weight of the staining value threshold.
[0155] In the embodiment of the present application, since the cell center is easy to stain when staining cells, the staining condition of the cell can be represented by obtaining the staining value of the cell center of each cell. During the processing, there is no need to use all the staining values corresponding to each cell, which improves the processing efficiency while ensuring accurate identification of the cell status.
[0156] In another embodiment, since there may be uneven staining when staining cells, in order to prevent uneven staining from affecting the subsequent determination of the cell state, the second pathological image is first filtered before determining the staining value of each cell center in the second pathological image. For a detailed processing process, see Figure 7 .
[0157] Figure 7is a flowchart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a computer device, see Figure 7 , the method comprises the following steps:
[0158] 701. A computer device predicts the center of each cell in the first pathological image to obtain the coordinates of the center of each cell.
[0159] The implementation of step 701 is similar to the method of determining the coordinates of each cell center in step 603, and will not be repeated here.
[0160] 702. The computer device performs median filtering on the first pathological image based on the coordinates of the center of each cell.
[0161] In the embodiment of the present application, the coordinates of the center of each cell are taken as the center point, and each cell in the first pathological image is subjected to median filtering to remove the staining values with large differences due to uneven staining in the first pathological image. Taking 3*3 median filtering as an example, for any cell, the coordinates of the cell center of the cell are taken as the middle coordinates, and then the surrounding 8 coordinates closest to the coordinates of the cell center are determined, and then the staining values corresponding to these 9 coordinates are sorted, and the staining value arranged in the middle is used as the new staining value corresponding to these 9 coordinates.
[0162] It should be noted that the embodiment of the present application is only described by taking median filtering of the first pathological image as an example. In another embodiment, the computer device can also use other filtering methods to filter the first pathological image, such as mean filtering.
[0163] 703. The computer device determines a histogram of the first pathological image based on staining values of multiple cell centers in the first pathological image.
[0164] 704. The computer device determines a staining value threshold corresponding to the first pathological image based on the first staining value in the histogram.
[0165] 705. The computer device compares the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located.
[0166] The implementation of steps 703 to 705 is the same as the implementation of steps 301 to 306 described above, and will not be repeated here.
[0167] In an embodiment of the present application, before determining the staining value of each cell center in the second pathological image, the second pathological image is first filtered, and the filtered second pathological image is subsequently processed to avoid the influence of uneven staining, thereby improving the accuracy of the determined staining value threshold.
[0168] Also, for example, see Figure 8 The operation process of the embodiment of the present application may include: acquiring a second pathological image, performing color deconvolution on the second pathological image, obtaining a first pathological image corresponding to the target channel, and at the same time, performing center prediction on the cells in the second pathological image to obtain the coordinates of each cell center, determining the staining value corresponding to each cell center according to the coordinates of each cell center and the first pathological image, determining a histogram based on the staining values corresponding to multiple cell centers, and then determining the first staining value in the histogram, and determining the staining value threshold based on the first staining value and the histogram. The specific implementation method for determining the staining value threshold is detailed in the above Figure 3 After that, after a plurality of staining value thresholds are determined in the above manner, for a new second pathological image, a staining value threshold corresponding to the second pathological image can be determined by moving average.
[0169] Fig. 9 is a structural diagram of an image processing device provided in an embodiment of the present application. Fig. 9 , the device comprises:
[0170] The histogram determination module 901 is used to determine the histogram of the first pathological image based on the staining values of multiple cell centers in the first pathological image, wherein the first pathological image is obtained by photographing multiple stained cells, the staining value of the cell center indicates the staining condition of the cell where the cell center is located, and the histogram indicates the distribution of the staining values corresponding to the multiple cells;
[0171] A threshold determination module 902 is used to determine a staining value threshold corresponding to the first pathological image based on a first staining value in the histogram, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values;
[0172] The state determination module 903 is used to compare the staining value of each cell center with the staining value threshold to obtain the state of each cell where the cell center is located.
[0173] The device provided in the embodiment of the present application can reflect the staining status of each cell in the first pathological image through a histogram, because cells in different states present different colors after staining. Therefore, based on the histogram, a staining value threshold suitable for the cells in the first pathological image can be determined, and the cell status can be determined according to the staining value threshold. This method of determining the cell status is not affected by the acquisition conditions of the first pathological image. Even if the acquisition conditions of the first pathological image change, the corresponding staining value threshold can be determined according to the histogram, thereby determining the status of each cell, and has high robustness.
[0174] Alternatively, see Fig.10 , the threshold determination module 902 comprises:
[0175] The number determination unit 9021 is used to reduce the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells;
[0176] A first determining unit 9022 is used to determine the maximum staining value among a plurality of staining values corresponding to a cell number that is not greater than the reference cell number and less than the first staining value as a second staining value;
[0177] A second determining unit 9023 is used to determine the sum of the difference between the first coloring value and the second coloring value and the first coloring value as a third coloring value;
[0178] The threshold determination unit 9024 is used to determine the third coloring value as the coloring value threshold.
[0179] Alternatively, see Fig.10 , the threshold determination module 902 comprises:
[0180] The number determination unit 9021 is used to reduce the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells;
[0181] A first determining unit 9022 is used to determine the maximum staining value among a plurality of staining values corresponding to a cell number that is not greater than the reference cell number and less than the first staining value as a second staining value;
[0182] A second determining unit 9023 is used to determine the sum of the difference between the first coloring value and the second coloring value and the first coloring value as a third coloring value;
[0183] The third determining unit 9025 is used to determine the number of cells corresponding to adjacent target number of staining values in the histogram each time, and when the number of cells corresponding to the middle staining value among the target number of staining values is the smallest, determine the middle staining value as the fourth staining value, and the target number is an odd number;
[0184] The threshold determination unit 9024 is further configured to determine the minimum coloring value, among the determined plurality of fourth coloring values, which is greater than the third coloring value, as the coloring value threshold.
[0185] Alternatively, see Fig.10 , the device further comprises:
[0186] An image acquisition module 904 is used to acquire a second pathological image, where the second pathological image is obtained by photographing the plurality of cells after staining;
[0187] The image extraction module 905 is used to extract the first pathological image corresponding to a target channel from the second pathological image, where the target channel is a channel reflecting the staining conditions of the multiple cells in the second pathological image.
[0188] Optionally, the image extraction module 901 is used to:
[0189] Determine a reference matrix, where the reference matrix represents color information of multiple channels included in the second pathological image, where the multiple channels include the target channel;
[0190] Based on the reference matrix, deconvolution processing is performed on the second pathological image to obtain pathological images corresponding to the multiple channels, wherein the obtained multiple pathological images include the first pathological image corresponding to the target channel.
[0191] Alternatively, see Fig.10 , the device further comprises:
[0192] A center prediction module 906 is used to perform center prediction on each cell in the first pathological image to obtain the coordinates of the center of each cell;
[0193] The dyeing value determination module 907 is used to determine the dyeing value corresponding to the coordinates of each cell center as the dyeing value of each cell center.
[0194] Alternatively, see Fig.10 , the device further comprises:
[0195] A center prediction module 905 is used to perform center prediction on each cell in the first pathological image to obtain the coordinates of the center of each cell;
[0196] The filtering module 908 is used to perform median filtering on the first pathological image based on the coordinates of the center of each cell.
[0197] Optionally, each cell in the first pathological image includes a plurality of position points, each position point having a corresponding staining value, see Fig.10 , the center prediction module 906 is used to:
[0198] For each of these cells:
[0199] Obtain the staining values of multiple position points contained in the cell;
[0200] Determine the maximum and identical multiple coloring values from the obtained multiple coloring values;
[0201] Connecting the position points corresponding to the determined multiple coloring values to form a connected domain;
[0202] The coordinates corresponding to the center point of the connected domain are determined as the coordinates of the cell center.
[0203] Alternatively, see Fig.10 The center prediction module 906 is used to obtain, for each cell, the maximum staining value among the staining values of multiple position points contained in the cell, and determine the coordinates of the position point corresponding to the maximum staining value as the coordinates of the center of the cell.
[0204] Alternatively, see Fig.10 The center prediction module 906 is used to call the center prediction model to perform center prediction on each cell in the first pathological image to obtain the coordinates of the center of each cell.
[0205] Optionally, the state determination module 903 is used to:
[0206] For any cell center in the first pathological image:
[0207] When the staining value of the cell center is greater than the staining value threshold, determining that the cell where the cell center is located is in the first state;
[0208] When the staining value of the cell center is not greater than the staining value threshold, it is determined that the cell where the cell center is located is in the second state.
[0209] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0210] It should be noted that: when the image processing device provided in the above embodiment processes an image, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the image processing device provided in the above embodiment and the image processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0211] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method of the above embodiment.
[0212] Optionally, the computer device is provided as a terminal. Fig.11 1 is a schematic diagram of a terminal 1100 provided in an embodiment of the present application. The terminal 1100 includes: a processor 1101 and a memory 1102 .
[0213] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.
[0214] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one computer program, which is used to be executed by the processor 1101 to implement the image processing method provided in the method embodiment of the present application.
[0215] In some embodiments, the terminal 1100 may further optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, the memory 1102 and the peripheral device interface 1103 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1103 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107 and a power supply 1108.
[0216] The peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.
[0217] The radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1104 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1104 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.
[0218] The display screen 1105 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 1105 is a touch display screen, the display screen 1105 also has the ability to collect touch signals on the surface of the display screen 1105 or above the surface. The touch signal can be input to the processor 1101 as a control signal for processing. At this time, the display screen 1105 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1105 can be one, set on the front panel of the terminal 1100; in other embodiments, the display screen 1105 can be at least two, respectively set on different surfaces of the terminal 1100 or in a folding design; in other embodiments, the display screen 1105 can be a flexible display screen, set on the curved surface or folding surface of the terminal 1100.
[0219] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. The front camera is set on the front panel of the terminal, and the rear camera is set on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions.
[0220] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1101 for processing, or input them into the radio frequency circuit 1104 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 1100. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1107 may also include a headphone jack.
[0221] The power supply 1108 is used to power various components in the terminal 1100. The power supply 1108 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1108 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.
[0222] Those skilled in the art will understand that Fig.11 The structure shown in the figure does not constitute a limitation on the terminal 1100, and the terminal 1100 may include more or fewer components than those shown in the figure, or combine some components, or adopt a different component arrangement.
[0223] Optionally, the computer device is provided as a server. Fig.12This is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 1200 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 1201 and one or more memories 1202, wherein the memory 1202 stores at least one computer program, and the at least one computer program is loaded and executed by the processor 1201 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the server may also include other components for implementing device functions, which will not be described in detail here.
[0224] An embodiment of the present application further provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method of the above embodiment.
[0225] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the operations performed by the image processing method of the above embodiment are implemented.
[0226] In some embodiments, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.
[0227] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0228] The above are only optional embodiments of the embodiments of the present application and are not intended to limit the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.
Claims
1. An image processing method, characterized in that: The method comprises: Determining a histogram of a first pathological image based on staining values of a plurality of cell centers in a first pathological image, wherein the first pathological image is obtained by photographing a plurality of stained cells, the staining values of the cell centers representing staining conditions of cells where the cell centers are located, and the histogram representing distribution conditions of staining values corresponding to the plurality of cells; Determining a staining value threshold corresponding to the first pathological image based on a first staining value in the histogram, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values; Comparing the staining value of each cell center with the staining value threshold, to obtain the state of the cell where each cell center is located; The determining, based on the first staining value in the histogram, a staining value threshold corresponding to the first pathological image includes: Reduce the number of cells corresponding to the first staining value according to the target ratio to obtain a reference number of cells; determine the maximum staining value among multiple staining values whose corresponding cell numbers are not greater than the reference number of cells and less than the first staining value as the second staining value; determine the difference between the first staining value and the second staining value and the sum of the first staining value as the third staining value; The third staining value is determined as the staining value threshold; or, the number of cells corresponding to adjacent target number staining values in the histogram is determined each time, and when the number of cells corresponding to the middle staining value among the target number staining values is the smallest, the middle staining value is determined as the fourth staining value, and the target number is an odd number; the minimum staining value among the determined multiple fourth staining values that is greater than the third staining value is determined as the staining value threshold.
2. The method according to claim 1, characterized in that Before determining the histogram of the first pathological image based on the staining values of the multiple cell centers in the first pathological image, the method further includes: Acquire a second pathological image, where the second pathological image is obtained by photographing the plurality of cells after staining; The first pathological image corresponding to a target channel is extracted from the second pathological image, where the target channel is a channel reflecting staining conditions of the plurality of cells in the second pathological image.
3. The method according to claim 2, characterized in that The step of extracting the first pathological image corresponding to the target channel from the second pathological image includes: Determine a reference matrix, where the reference matrix represents color information of multiple channels included in the second pathological image, where the multiple channels include the target channel; Based on the reference matrix, deconvolution processing is performed on the second pathological image to obtain pathological images corresponding to the multiple channels, wherein the obtained multiple pathological images include the first pathological image corresponding to the target channel.
4. The method according to claim 1, characterized in that: Before determining the histogram of the first pathological image based on the staining values of the multiple cell centers in the first pathological image, the method further includes: Performing center prediction on each of the cells in the first pathological image to obtain coordinates of the center of each of the cells; The staining value corresponding to the coordinates of each of the cell centers is respectively determined as the staining value of each of the cell centers.
5. The method according to claim 1, characterized in that Before determining the histogram of the first pathological image based on the staining values of the multiple cell centers in the first pathological image, the method further includes: Performing center prediction on each of the cells in the first pathological image to obtain coordinates of the center of each of the cells; Based on the coordinates of the center of each cell, median filtering is performed on the first pathological image.
6. The method according to claim 4 or 5, characterized in that: Each cell in the first pathological image includes a plurality of position points, each position point has a corresponding staining value, and performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells includes: For each of the cells: Obtaining staining values of multiple position points contained in the cell; Determine the maximum and identical multiple coloring values from the obtained multiple coloring values; Connecting the position points corresponding to the determined multiple coloring values to form a connected domain; The coordinates corresponding to the center point of the connected domain are determined as the coordinates of the cell center.
7. The method according to claim 4 or 5, characterized in that: The performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells includes: For each of the cells, the maximum staining value among the staining values of the multiple position points contained in the cell is obtained, and the coordinates of the position point corresponding to the maximum staining value are determined as the coordinates of the center of the cell.
8. The method according to claim 4 or 5, characterized in that: The performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells includes: The center prediction model is called to perform center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells.
9. The method according to any one of claims 1 to 5, characterized in that: The comparing the staining value of each cell center with the staining value threshold to obtain the state of the cell where each cell center is located includes: For any cell center in the first pathological image: When the staining value of the cell center is greater than the staining value threshold, determining that the cell where the cell center is located is in the first state; When the staining value of the cell center is not greater than the staining value threshold, it is determined that the cell where the cell center is located is in the second state.
10. An image processing device, characterized in that: The device comprises: a histogram determination module, configured to determine a histogram of a first pathological image based on staining values of a plurality of cell centers in the first pathological image, wherein the first pathological image is obtained by photographing a plurality of stained cells, the staining values of the cell centers representing staining conditions of cells where the cell centers are located, and the histogram representing distribution conditions of staining values corresponding to the plurality of cells; a threshold determination module, configured to determine a staining value threshold corresponding to the first pathological image based on a first staining value in the histogram, wherein the number of cells corresponding to the first staining value in the histogram is greater than the number of cells corresponding to other staining values; A state determination module, used for comparing the staining value of each cell center with a staining value threshold, to obtain the state of the cell where each cell center is located; The threshold determination module comprises: a number determination unit, configured to reduce the number of cells corresponding to the first staining value according to a target ratio to obtain a reference number of cells; A first determining unit, configured to determine a maximum staining value among a plurality of staining values corresponding to a cell number not greater than the reference cell number and less than the first staining value as a second staining value; a second determining unit, configured to determine a difference between the first coloring value and the second coloring value and a sum of the first coloring value as a third coloring value; a threshold determination unit, configured to determine the third coloring value as the coloring value threshold; or, The third determination unit is used to determine the number of cells corresponding to adjacent target number staining values in the histogram each time, and when the number of cells corresponding to the middle staining value among the target number staining values is the smallest, the middle staining value is determined as the fourth staining value, and the target number is an odd number; the threshold determination unit is used to determine the minimum staining value among the determined multiple fourth staining values that is greater than the third staining value as the staining value threshold.
11. The device according to claim 10, characterized in that The device also includes: An image acquisition module, used for acquiring a second pathological image, wherein the second pathological image is obtained by photographing the plurality of cells after staining; The image extraction module is used to extract the first pathological image corresponding to a target channel from the second pathological image, where the target channel is a channel reflecting the staining conditions of the multiple cells in the second pathological image.
12. The device according to claim 11, characterized in that The image extraction module is used for: Determine a reference matrix, where the reference matrix represents color information of multiple channels included in the second pathological image, where the multiple channels include the target channel; Based on the reference matrix, deconvolution processing is performed on the second pathological image to obtain pathological images corresponding to the multiple channels, wherein the obtained multiple pathological images include the first pathological image corresponding to the target channel.
13. The device according to claim 10, characterized in that The device also includes: A center prediction module, used for performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells; The dyeing value determination module is used to determine the dyeing value corresponding to the coordinates of each cell center as the dyeing value of each cell center.
14. The device according to claim 10, characterized in that The device also includes: A center prediction module, used for performing center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each of the cells; A filtering module is used to perform median filtering on the first pathological image based on the coordinates of the center of each cell.
15. The device according to claim 13 or 14, characterized in that Each cell in the first pathological image includes a plurality of position points, each position point has a corresponding staining value, and the center prediction module is used to: For each of the cells: Obtaining staining values of multiple position points contained in the cell; Determine the maximum and identical multiple coloring values from the obtained multiple coloring values; Connecting the position points corresponding to the determined multiple coloring values to form a connected domain; The coordinates corresponding to the center point of the connected domain are determined as the coordinates of the cell center.
16. The device according to claim 13 or 14, characterized in that The center prediction module is used to obtain, for each of the cells, a maximum staining value among the staining values of a plurality of position points contained in the cell, and determine the coordinates of the position point corresponding to the maximum staining value as the coordinates of the center of the cell.
17. The device according to claim 13 or 14, characterized in that The center prediction module is used to call the center prediction model to perform center prediction on each of the cells in the first pathological image to obtain the coordinates of the center of each cell.
18. The device according to any one of claims 10 to 14, characterized in that The state determination module is used to: For any cell center in the first pathological image: When the staining value of the cell center is greater than the staining value threshold, determining that the cell where the cell center is located is in the first state; When the staining value of the cell center is not greater than the staining value threshold, it is determined that the cell where the cell center is located is in the second state.
19. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the operations performed by the image processing method according to any one of claims 1 to 9.
20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the operations performed by the image processing method according to any one of claims 1 to 9.
21. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the operations performed by the image processing method according to any one of claims 1 to 9 are implemented.
Citation Information
Patent Citations
Quantitative analysis method aiming at apoptotic cell morphology of fluorescence microscopic image
CN103994964A
Image processing method and device, equipment and medium
CN110458835A
Micropathology image cell sufficiency detection method, system, equipment, device and medium
CN111429440A
Cell segmentation method and device based on microscopic image, equipment and storage medium
CN112330690A
Cancer diagnosis device, cancer diagnosis method and program
JP2017198486A
