A method, apparatus, and storage medium for removing handwriting from pathological slide images.

Abstract

This invention discloses a method, apparatus, and storage medium for removing handwriting from pathological slide images. The method specifically includes: S1: scanning the pathological slide to obtain a WSI image; S2: dividing the WSI image into multiple small image blocks; S3: calculating the average color value of each small image block to obtain an average color feature matrix (MAP); S4: performing pixel clustering on the MAP; S5: based on prior knowledge, identifying the class containing handwriting based on the classification results; updating the RGB values ​​of the MAP for the handwriting class to obtain a new color feature map (newMap); adjusting the color of the identified small image blocks of the handwriting class according to the new color feature map (newMap) to remove handwriting while preserving original image details; S6: merging all small image blocks into a WSI image of the original size. This invention uses a multi-feature joint judgment method to find the handwriting region to be removed and uses color adjustment to remove handwriting while preserving image details.

Description

Technical Field

[0001] This invention belongs to the field of computer medical image information processing technology, specifically relating to a method, device and storage medium for removing handwriting from pathological slide images. Background Technology

[0002] With the advancement of science and technology, the traditional method of diagnosis relying on pathologists under a microscope faces several problems, such as being time-consuming, labor-intensive, and heavily influenced by the doctor's subjectivity. This has led to the large-scale development of automated pathology diagnosis. In recent years, the use of digital scanners to create slide images of pathological tissues (whole-slide image, WSI) has freed pathologists from the limitations of microscopes, greatly improving the efficiency of pathology teaching and case discussions. It has also made remote consultations and the development of artificial intelligence in pathology possible, resulting in the increasingly widespread use of whole-slide image (WSI). WSI images have a very large data volume, typically containing hundreds of millions of pixels.

[0003] Obtaining clear sample images through rapid pathological image acquisition equipment is crucial. The digitization of pathological slides has accumulated a vast amount of data, a vital prerequisite for AI-assisted pathological diagnosis, with a large number of samples being scanned daily. However, some problems have arisen during its use. For example, during slide preparation, many slides are marked by doctors under the microscope using markers, and these markings are also scanned into the WSI (Wide Sample Injection System), causing significant difficulties in its use and hindering its clinical pathological applications.

[0004] Slides discarded due to handwriting issues significantly hamper clinical pathology workflows, as these slides require re-fabrication or re-scanning, prolonging pathological diagnosis time and wasting human and material resources. Using these images directly negatively impacts the development and validation of classifiers for tasks such as disease detection, diagnosis, and prognosis, particularly for deep learning and machine learning-based methods. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method, apparatus, and storage medium for removing handwriting from pathological slide images.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] On one hand, this invention discloses a method for removing handwriting from pathological slide images, specifically including the following steps:

[0008] S1: Scan the pathological slides to obtain WSI images;

[0009] S2: Divide the WSI image into multiple small image blocks;

[0010] S3: Calculate the average color value of each small image patch and obtain the average color feature matrix MAP;

[0011] S4: Perform pixel clustering on the average color feature matrix MAP;

[0012] S5: Based on prior knowledge, find the class containing handwriting after classification.

[0013] For the average color feature matrix Map of handwriting, update its RGB values ​​to obtain a new color feature map newMap;

[0014] For small image blocks of the identified handwriting type, adjust the color ratio according to the new color feature map newMap, remove the handwriting, and preserve the original image details;

[0015] S6: Merge all small image patches into a WSI image of the original size.

[0016] Based on the above technical solution, the following improvements can be made:

[0017] As a preferred solution, S3 specifically obtains the average color feature matrix MAP using the following formula;

[0018] M(i,j)=mean(block) (i,j) );

[0019] Where M(i,j) is the average color feature matrix of the small image patch in the i-th row and j-th column;

[0020] block (i,j) This is the small image patch in the i-th row and j-th column.

[0021] As a preferred approach, the classified results include: background and fat, handwriting, and normal tissue.

[0022] As a preferred solution, S5's "updates the RGB values ​​of the average color feature matrix Map for handwriting classes to obtain a new color feature map newMap" specifically includes the following:

[0023] For each pixel in the average color feature matrix Map of the handwriting class, replace its original value with the median of the RGB fluxes of its neighborhood class to obtain a new color feature map newMap.

[0024] As a preferred solution, S5's "removing handwriting and preserving original image details by adjusting the color ratio of the determined small image blocks of handwriting type according to the new color feature map newMap" specifically includes the following:

[0025] For small image blocks of handwriting (k,t) Find the color parameters in the corresponding color feature map newMap and calculate the color tone factor for each channel;

[0026] Kr = R(newMap(k, t)) / mr;

[0027] Kg = G(newMap(k,t)) / mg;

[0028] Kb = B(newMap(k, t)) / mb;

[0029] Where: mr, mg, and mb are the mean values ​​of RGB, respectively;

[0030] Using the above modulation factor, small image blocks of handwriting type... (k,t) Adjust the color ratio of all pixels in the array;

[0031] For any point P, its RGB three-channel pixel values ​​are as follows:

[0032] R(P) = R(P) * Kr;

[0033] G(P) = G(P) * Kg;

[0034] B(P) = B(P) * Kb.

[0035] As a preferred embodiment, S6 specifically includes the following: sequentially merging all small image blocks into a WSI image of the original size.

[0036] Furthermore, the present invention also discloses an apparatus for removing handwriting from pathological slide images, comprising:

[0037] One or more processors;

[0038] Memory;

[0039] And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, and the one or more programs include a method for removing handwriting from any of the above-described pathological slide images.

[0040] Furthermore, the present invention also discloses a storage medium storing one or more programs, the one or more programs including instructions adapted to be loaded by a memory and executed as described above for the method of removing handwriting from a pathological slide image.

[0041] This invention discloses a method, apparatus, and storage medium for removing handwriting from pathological slide images. By using a multi-feature joint judgment method, the area requiring handwriting removal is located, and by adjusting color ratios, the handwriting is removed while preserving image details. It has the following beneficial effects:

[0042] First, the present invention employs a combination of multiple features to extract features from different granularities.

[0043] Second, this invention uses an unsupervised method, which requires no additional annotation.

[0044] Third, this invention uses macroscopic and microscopic features for judgment. First, small image blocks are classified using macroscopic features to reduce the amount of computation. Then, detailed features are used to determine whether they are handwriting, and finally, they are removed.

[0045] In summary, this invention greatly improves the efficiency of using pathological images, making it possible for pathological images to be widely used, and has extremely high clinical significance. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a method for removing handwriting from pathological slide images provided in an embodiment of the present invention.

[0048] Figure 2 A WSI image with handwriting provided for an embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram illustrating the division of a WSI image into multiple small image blocks, as provided in an embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram illustrating the calculation of the average color feature matrix (MAP) from small image patches, as provided in an embodiment of the present invention.

[0051] Figure 5 This is a schematic diagram of pixel clustering of the average color feature matrix (MAP) provided in an embodiment of the present invention.

[0052] Figure 6 This is a comparison diagram of color toning ratios before and after, provided for an embodiment of the present invention.

[0053] Figure 6 (a) is a schematic diagram before color adjustment;

[0054] Figure 6 (b) is a schematic diagram after color adjustment.

[0055] Figure 7 The image provided in this embodiment of the invention is a WSI image after removing handwriting. Detailed Implementation

[0056] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Using ordinal numbers such as “first,” “second,” “third,” etc. to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other way.

[0059] Furthermore, the expression "includes" is an "open-ended" expression, which means only that there is a corresponding component or step, and should not be interpreted as excluding additional components or steps.

[0060] To achieve the objectives of this invention, in some embodiments of a method, apparatus, and storage medium for removing handwriting from pathological slide images, the method for removing handwriting from pathological slide images specifically includes the following steps, such as... Figure 1 As shown:

[0061] S1: Scan the pathological slides to obtain WSI images;

[0062] S2: Divide the WSI image into multiple small image blocks;

[0063] S3: Calculate the average color value of each small image patch and obtain the average color feature matrix MAP;

[0064] S4: Perform pixel clustering on the average color feature matrix MAP;

[0065] S5: Based on prior knowledge, find the class containing handwriting after classification.

[0066] For the average color feature matrix Map of handwriting, update its RGB values ​​to obtain a new color feature map newMap;

[0067] For small image blocks of the identified handwriting type, adjust the color ratio according to the new color feature map newMap, remove the handwriting, and preserve the original image details;

[0068] S6: Merge all small image patches into a WSI image of the original size.

[0069] Each step will be explained in detail below.

[0070] S1: Scan the pathological slides to obtain WSI images, such as... Figure 2 As shown.

[0071] Pathological slides are scanned using a scanner to obtain target pathological images. WSI images have a very large data volume, typically containing hundreds of millions of pixels. Therefore, they are divided into multiple smaller image blocks for easier processing.

[0072] S2: Divide the WSI image into multiple small image blocks, such as Figure 3 As shown.

[0073] like Figure 2 The WSI image shown has a resolution of 115072*62765 and is divided into 461*262 small image blocks with a step size of 240 pixels.

[0074] S3: Calculate the average color value of each small image patch and obtain the average color feature matrix MAP.

[0075] S3 specifically obtains the average color feature matrix MAP using the following formula;

[0076] M(i,j)=mean(block) (i,j) );

[0077] Where M(i,j) is the average color feature matrix of the small image patch in the i-th row and j-th column;

[0078] block (i,j) This is the small image patch in the i-th row and j-th column.

[0079] For all the small image blocks obtained in the previous step, calculate the mean of their RGB three color channels, and assign this value to the corresponding value in the average color feature map. For example... Figure 4 As shown, an average color feature matrix (MAP) of 461*261*3 can be obtained.

[0080] S4: Perform pixel clustering on the average color feature matrix MAP.

[0081] The pixel clustering method is used to divide the pixels in the average color feature matrix Map into three unsupervised categories, denoted as cluster1, cluster2, and cluster3.

[0082] S5: Based on the classification results, find the class containing handwriting based on prior knowledge, such as... Figure 5 As shown;

[0083] For the average color feature matrix Map of handwriting, update its RGB values ​​to obtain a new color feature map newMap;

[0084] For small image patches identified as having handwriting patterns, color adjustments are made based on a new color feature map (newMap) to remove handwriting while preserving original image details. Figure 6 As shown.

[0085] The classification results include: background and fat, handwriting, and normal tissue.

[0086] Specifically, since the background and fat have relatively little substance, they are generally close to white. Therefore, for these three categories, the category with the largest sum of RGB flux is the background and adipose tissue.

[0087] For the remaining two clusters, we determine their performance based on detailed features. We randomly select 10 boloks from each cluster, calculate the gradient matrix of the two sets of images, and then calculate the variance.

[0088] Because small image blocks containing handwriting are less blurred compared to normal tissue, their variance is smaller. Therefore, this category containing handwriting can be selected and denoted as `cluster_pen`.

[0089] For each pixel in the average color feature matrix Map of the handwriting class, replace its original value with the median of the RGB fluxes of its neighborhood class to obtain a new color feature map newMap.

[0090] For small image blocks of handwriting (k,t) Find the color parameters in the corresponding color feature map newMap and calculate the color tone factor for each channel;

[0091] Kr = R(newMap(k, t)) / mr;

[0092] Kg = G(newMap(k,t)) / mg;

[0093] Kb = B(newMap(k, t)) / mb;

[0094] Where: mr, mg, and mb are the mean values ​​of RGB, respectively;

[0095] Using the above modulation factor, small image blocks of handwriting type... (k,t) Adjust the color ratio of all pixels in the array;

[0096] For any point P, its RGB three-channel pixel values ​​are as follows:

[0097] R(P) = R(P) * Kr;

[0098] G(P) = G(P) * Kg;

[0099] B(P) = B(P) * Kb.

[0100] S6: Merge all small image patches into a WSI image of the original size, such as... Figure 7 As shown.

[0101] S6 specifically includes the following: Based on the numbering of the small image patches, merge all the small image patches in sequence into a WSI image of the original size.

[0102] Furthermore, embodiments of the present invention also disclose an apparatus for removing handwriting from pathological slide images, comprising:

[0103] One or more processors;

[0104] Memory;

[0105] And one or more programs, wherein the one or more programs are stored in memory and configured to be executed by one or more processors, the one or more programs including a method for removing handwriting from pathological slide images disclosed in any of the above embodiments.

[0106] Furthermore, embodiments of the present invention also disclose a storage medium storing one or more programs, the one or more programs including instructions adapted to be loaded by a memory and executed as a method for removing handwriting from pathological slide images disclosed in any of the above embodiments.

[0107] This invention discloses a method, apparatus, and storage medium for removing handwriting from pathological slide images. By using a multi-feature joint judgment method, the area requiring handwriting removal is located, and by adjusting color ratios, the handwriting is removed while preserving image details. It has the following beneficial effects:

[0108] First, the present invention employs a combination of multiple features to extract features from different granularities.

[0109] Second, this invention uses an unsupervised method, which requires no additional annotation.

[0110] Third, this invention uses macroscopic and microscopic features for judgment. First, small image blocks are classified using macroscopic features to reduce the amount of computation. Then, detailed features are used to determine whether they are handwriting, and finally, they are removed.

[0111] In summary, this invention greatly improves the efficiency of using pathological images, making it possible for pathological images to be widely used, and has extremely high clinical significance.

[0112] The above-mentioned implementation methods can be carried out in parallel or in a cross-cutting manner.

[0113] It should be understood that the various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a floppy disk, CD-ROM, hard disk, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, that machine becomes an apparatus for practicing the present invention.

[0114] In the description of this invention, it should be understood that the terms "coaxial," "bottom," "one end," "top," "middle," "other end," "upper," "side," "top," "inner," "front," "center," "both ends," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0115] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "screw connection," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0116] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.

Claims

1. A method of pathological section image de-inked, characterized in that, Specifically comprising the following steps: S1: scanning the pathological section to obtain a WSI image; S2: dividing the WSI image into a plurality of small image blocks; S3: calculating the color mean value of each small image block to obtain an average color feature matrix MAP; S4: performing pixel clustering on the average color feature matrix MAP to obtain three classification results of background and fat class, handwriting class and normal tissue class; S5: finding a handwriting-containing class according to prior knowledge based on the classified results; For each pixel in the average color feature matrix Map of the handwriting class, the median values of the RGB three fluxes of its neighborhood class are used to replace its original values to obtain a new color feature map newMap; The small image block of the determined handwriting class is subjected to color tone ratio according to the new color feature map newMap, the handwriting is removed, and the original image details are retained; The color tone ratio specifically comprises: For the small image block block of handwriting (k,t) , find the color parameters in the corresponding color feature map newMap, and calculate the color tone ratio factor of each channel; Kr=R(newMap(k,t)) / mr; Kg=G(newMap(k,t)) / mg; Kb=B(newMap(k,t)) / mb; Wherein: mr, mg, mb are the mean values of RGB respectively; Using the above adjustment factor, all pixels in the small image block (k,t) of the handwriting class are color adjusted. For any point P, the RGB three channel pixel values are as follows: R(P)=R(P)*Kr; G(P)=G(P)*Kg; B(P)=B(P)*Kb; S6: combining all the small image blocks into a WSI image of the original size.

2. The method of claim 1, wherein, S3 specifically obtains the average color feature matrix MAP by the following formula: M(i, j) = mean(block (i,j) ); Wherein: M(i,j) is the average color feature matrix of the i-th row and j-th column small image block; block (i,j) A small image block for the i-th row, j-th column.

3. The method of claim 1, wherein, S6 specifically comprises the following contents: combining all the small image blocks in sequence into a WSI image of the original size.

4. An apparatus for de-inking of pathological section images, characterized by Comprise: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, one or more programs comprising a method for removing handwriting from a pathological section image according to any one of the above claims 1-3.

5. A storage medium, characterized by The storage medium stores one or more programs, and one or more programs include instructions suitable for being loaded by the memory and executing the method for removing handwriting from a pathological section image according to any one of the above claims 1-3.

Images