A pathological section scanning method and system

Abstract

The present invention relates to the field of medical microscopic imaging technology, and more specifically, to a pathology section scanning method and system, comprising: step S1: acquiring a pathology section and scanning the section to generate a preview image; step S2: performing image recognition on the preview image to obtain a tissue region; and step S3: rescanning the tissue region in the section to output a scan result. The present invention has the beneficial effect of accurately identifying masked regions in tissue by setting a target detection model, avoiding the problem in the prior art of being unable to accurately extract tissue regions from pathology section images formed by specific staining, achieving more accurate detection of pathology sections, and thereby generating more accurate section images.

Description

Technical Field

[0001] The present invention relates to the technical field of medical microscopic imaging, and in particular to a pathological section scanning method and system. Background Art

[0002] Pathological section refers to a type of specimen used in pathological examination. Normally, pathological section examination is a detection technology that takes a certain size of diseased tissue, makes a pathological section using histopathological methods, and then further examines the characteristics of the diseased tissue under optical microscopy to explore the medical principles of the diseased tissue. For example, the diseased tissue is embedded in a paraffin block, cut into thin slices with a microtome, and then stained with hematoxylin-eosin (HE) to make the specific diseased tissue clearly imaged under a microscope. Pathological section scanner refers to a section inspection device based on digital imaging technology, which is used to collect and store microscopic images of pathological sections for medical diagnosis or scientific research. Normally, this type of scanner integrates an optical microscope and a digital optical imaging device, which is used to optically magnify pathological sections and take digital images of them.

[0003] In the prior art, to achieve optimal digital image acquisition, pathology sections are typically pre-scanned to determine the tissue's position in the image, followed by a subsequent scan of the actual tissue region. However, during actual implementation, the inventors discovered that existing tissue identification methods, such as background difference, tissue region color characteristics, and tissue region shape characteristics, are easily affected by factors such as impurities and changes in lighting conditions. Background difference requires timely updating when the blank image undergoes significant changes, and cannot avoid the influence of impurities. IHC sections are generally light in color, easily affected by impurities, resulting in ineffective extraction during image binarization. Summary of the Invention

[0004] In view of the above problems existing in the prior art, a method and system for scanning pathological sections are provided.

[0005] The specific technical solutions are as follows:

[0006] A method for scanning a pathological section, comprising:

[0007] Step S1: obtaining a pathological section, and scanning the pathological section to generate a preview image;

[0008] Step S2: performing image recognition on the preview image to obtain a tissue region;

[0009] Step S3: re-scanning the tissue region in the pathological section to output a scanning result.

[0010] Preferably, step S2 includes:

[0011] Step S21: using a pre-established target detection model to identify the preview image and generate at least one mask area;

[0012] Step S22: performing connected domain screening on the mask area to output the tissue area.

[0013] Preferably, the target detection model includes:

[0014] a residual layer, the residual layer receiving the preview image and preprocessing the preview image;

[0015] A feature extraction layer, the feature extraction layer is connected to the residual layer, and the feature extraction layer extracts image features from the preprocessed preview image;

[0016] a classification layer, the classification layer being connected to the feature extraction layer, and the classification layer generating a region category for each of the mask regions according to the image features;

[0017] A prediction layer is connected to the feature extraction layer, and the prediction layer generates the mask area according to the image features.

[0018] Preferably, the training method of the target detection model includes:

[0019] Step A1: obtaining a plurality of training slices corresponding to the region category, and labeling the training slices to generate labeled slices;

[0020] Step A2: performing image processing on the labeled slices to generate a training set;

[0021] Step A3: Convert the format of the training set and input it into the target detection model to complete the training process.

[0022] Preferably, the region category includes at least one of a hematoxylin-eosin staining region, an immunohistochemical staining region, and a liquid-based cell staining region.

[0023] Preferably, the step S21 includes:

[0024] Step S211: inputting the preview image into the target detection model, the target detection model recognizes the preview image and outputs the mask area and the area category corresponding to the mask area respectively;

[0025] Step S212: merging the mask regions according to the region categories to generate new mask regions.

[0026] Preferably, the training method further comprises, after step A3:

[0027] Step A4: using the target detection model to identify the training slice, outputting the region category, and judging whether the region category meets the requirements based on the labeled slice;

[0028] If so, it indicates that the target detection model is trained successfully, and the target detection model is output;

[0029] If not, it indicates that the target detection model training is unsuccessful, and the process returns to step A1.

[0030] A pathology section scanning system, comprising:

[0031] a scanner, wherein the scanner scans the pathological section to generate a scanned image;

[0032] a memory storing pre-generated computer instructions;

[0033] A processor is connected to the memory and the scanner, and runs the computer instructions to perform the above scanning method.

[0034] The above technical solution has the following advantages or beneficial effects: by setting up a target detection model, accurate identification of the mask area in the tissue is achieved, avoiding the problem in the existing technology that the pathological section image formed by specific staining cannot accurately extract its tissue area, and achieving more accurate detection of pathological sections, thereby generating more accurate section images. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0036] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the sub-steps of step S2 in an embodiment of the present invention;

[0038] Figure 3 Schematic diagram of a target detection model in an embodiment of the present invention;

[0039] Figure 4 Schematic diagram of a target detection model training method according to an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of sub-steps of step S21 in an embodiment of the present invention;

[0041] Figure 6 A schematic diagram of a target detection model training method according to another embodiment of the present invention;

[0042] Figure 7 This is a functional block diagram of a scanning system according to an embodiment of the present invention;

[0043] Figure 8 This is the recognition result of the hematoxylin-eosin stained section by the scanning system in an embodiment of the present invention;

[0044] Figure 9 The result of the scanning system identifying the immunohistochemically stained sections in the embodiment of the present invention is shown in FIG.

[0045] Figure 10 This is the recognition result of the liquid-based cell-stained slice by the scanning system in the embodiment of the present invention.

[0046] Figure 11 This is the recognition result of the stained slice using the existing technology.

[0047] Figure 12 This is the recognition result of light-stained slices using existing technology. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0049] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0050] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0051] The present invention comprises:

[0052] A scanning method for pathological sections, such as Figure 1 Shown include:

[0053] Step S1: obtaining a pathological section and scanning the pathological section to generate a preview image;

[0054] Step S2: performing image recognition on the preview image to obtain a tissue region;

[0055] Step S3: Scan the tissue area in the pathological section again to output a scan result.

[0056] Specifically, in order to address the problem in the prior art that pathological sections formed by specific staining methods cannot accurately identify tissue regions, the present invention provides a target detection model to achieve feature extraction of tissue region parts in the image and predict the position of the specific tissue region in the image, thereby facilitating re-scanning based on the position of the tissue region to obtain a more accurate scanned image.

[0057] In a preferred embodiment, Figure 2 As shown, step S2 includes:

[0058] Step S21: using a pre-established target detection model to identify the preview image and generate at least one mask area;

[0059] Step S22: screening the connected domains of the mask area to output the tissue area.

[0060] Specifically, in order to address the problem in the prior art that the technology is easily interfered by impurities and cannot accurately identify specific tissue areas, this embodiment eliminates interference factors through connected domain detection, and then accurately determines the tissue area that needs to be scanned again to obtain a more accurate scan image.

[0061] During implementation, the mask region is the target selection area separated from the preview image by the target detection model. A connected domain is defined as the connected domain formed by the foreground pixels marked by the mask region. Connected domain analysis determines the size of the mask region and, based on a preset threshold corresponding to the tissue type, filters out irrelevant, smaller objects while retaining the tissue area actually required for scanning.

[0062] In a preferred embodiment, Figure 3 As shown, the target detection model includes:

[0063] Residual layer 1, which receives the preview image and preprocesses the preview image;

[0064] Feature extraction layer 2, which is connected to residual layer 1, extracts image features from the preprocessed preview image;

[0065] Classification layer 3, which is connected to feature extraction layer 2, generates a region category for each mask region based on image features;

[0066] Prediction layer 4: The prediction layer 4 is connected to the feature extraction layer 2, and the prediction layer 4 generates a mask area according to the image features.

[0067] Specifically, in response to the problem that tissue areas cannot be accurately identified in the existing technology, this embodiment achieves effective extraction of feature information in the image by setting up a target detection model, and combines the position features in the feature information to achieve better detection effect of tissue areas.

[0068] During implementation, the residual layer 1 includes a deep residual network in which a plurality of residual units connected in series are provided. Each residual unit includes a pair of parallel convolutional networks and shortcuts. In each residual unit, a preliminary feature extraction of the preview image is performed through the convolutional network, and the shallow features of the shortcut input, that is, the input data of the residual unit at this level are added and then input into the residual unit at the next level, thereby achieving partial retention of the upper-level data and avoiding the problem of network degradation. In one embodiment, the residual layer 1 is set to a ResNet34 network.

[0069] Feature extraction layer 2 includes a feature pyramid network, which sequentially extracts features, upsampling, and fusion features from the image to extract image features corresponding to tissue regions from the preview image. These image features also include location information, enabling direct extraction of the tissue region's location within the image, shortening recognition time. In one embodiment, feature extraction layer 2 is configured as an FPN feature pyramid network.

[0070] In a preferred embodiment, Figure 4 As shown in Figure 2, the training method of the target detection model includes:

[0071] Step A1: obtaining multiple training slices corresponding to region categories, and labeling the training slices to generate labeled slices;

[0072] Step A2: Perform image processing on the labeled slices to generate a training set;

[0073] Step A3: Convert the training set format and input it into the object detection model to complete the training process.

[0074] During the implementation process, the training slices are slice images corresponding to the pathological slices to be identified, which generate the tissue regions in each slice image and the regional categories of the tissue regions by manual annotation. In order to achieve better training results, the number of training slices for each regional category should be as large as possible. For example, in one embodiment, the number is 1000. At the same time, in order to effectively expand the training set, in step A2, one or more of a series of operations such as translation, rotation, scaling, mirroring, and cropping are performed on each annotated slice to effectively expand the training set. Furthermore, in order to achieve faster training speed, in step A3, the training set uses the COCO dataset format and is packaged into a JSON file to facilitate model training.

[0075] In a preferred embodiment, the region category includes at least one of a hematoxylin-eosin staining region, an immunohistochemical staining region, and a liquid-based cell staining region.

[0076] Specifically, in response to the problem in the prior art that it is difficult to identify tissue regions formed by the above-mentioned staining methods, this embodiment trains the target detection model by selecting the above-mentioned regions as training sets, thereby achieving better recognition effects on pathological sections formed by the above-mentioned types of staining.

[0077] In a preferred embodiment, Figure 5 As shown, step S21 includes:

[0078] Step S211: inputting the preview image into the target detection model, which recognizes the preview image and outputs the mask area and the area category corresponding to the mask area;

[0079] Step S212: merging the mask regions according to the region categories to generate a new mask region.

[0080] Specifically, in order to achieve better recognition effect of the mask area for subsequent re-scanning, this embodiment chooses to merge the mask areas of different area categories, thereby achieving effective screening of possible tissue areas and avoiding the problem of incomplete tissue area scanning due to possible area category recognition errors.

[0081] In a preferred embodiment, Figure 6 As shown, the training method further includes, after step A3:

[0082] Step A4: Use the target detection model to identify the training slices, output the region category, and determine whether the region category meets the requirements based on the labeled slices;

[0083] If so, it indicates that the target detection model training is successful, and the target detection model is output;

[0084] If not, it indicates that the target detection model training is unsuccessful, and the process returns to step A1.

[0085] Specifically, in order to achieve a better recognition effect of the target detection model, the present embodiment selects the regional category to detect the trained target detection model, thereby improving the accuracy of the target detection model in actual use.

[0086] A scanning system for pathological sections, such as Figure 7 Shown, including:

[0087] Scanner X1, the scanner X1 scans the pathological section to generate a scanned image;

[0088] Memory X2, where pre-generated computer instructions are stored;

[0089] The processor X3 is connected to the memory X2 and the scanner X1, and the processor X3 runs computer instructions to execute the above scanning method.

[0090] Specifically, in order to solve the problem that the scanning system in the prior art cannot accurately identify a specific tissue area, this embodiment achieves a better recognition accuracy by executing the above scanning method in the scanner. Figure 8 Hematoxylin-eosin stained sections shown, Figure 9 Immunohistochemically stained sections as shown and Figure 10 The liquid-based cytologically stained sections shown in the figure all achieve effective tissue detection within the target detection frame X4. Multiple experiments have shown that the scanning system in this embodiment achieves a classification accuracy of 92% for hematoxylin-eosin-stained sections, immunohistochemically stained sections, and liquid-based cytologically stained sections, and a masked area identification accuracy of 96%. Liquid-based cytologically stained sections achieve particularly accurate recognition.

[0091] In contrast, the recognition results of the stained slices based on the FasterRCNN detection model in the prior art are as follows: Figure 11 As shown in the figure, it can be seen that the coverage of the first target detection box Y1 output by the FasterRCNN detection model is significantly larger than the tissue area, which may be Figure 11 Furthermore, the second target detection frame Y2 does not accurately select the right boundary of the tissue region on the right side. This shows that the tissue region detection based on the FasterRCNN detection model is less accurate than the target detection model established in the present invention.

[0092] Likewise, Figure 12 This is the recognition result of the liquid-based cell-stained slide based on the YOLOv5 model. As can be seen, because the liquid-based cell-stained slide stains the tissue area lightly, the YOLOv5 model does not accurately cover the tissue area, causing the target detection box Z1 to miss the lower half of the tissue area.

[0093] The beneficial effects of the present invention are as follows: by setting up a target detection model, accurate identification of the mask area in the tissue is achieved, avoiding the problem in the prior art that the pathological section image formed by specific staining cannot accurately extract its tissue area, achieving more accurate detection of pathological sections, and thus generating more accurate section images.

[0094] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for scanning a pathological section, characterized in that: include: Step S1: obtaining a pathological section, and scanning the pathological section to generate a preview image; The plurality of groups of pathological sections include different staining types; Step S2: performing image recognition on the preview image to obtain a tissue region; Step S3: re-scanning the tissue region in the pathological section to output a scan result; The step S2 comprises: Step S21: using a pre-established target detection model to identify the preview image and generate at least one mask area; In the step S21, the mask regions with different region categories are merged in advance; Step S22: performing connected domain screening on the mask area to output the tissue area; The target detection model includes: a residual layer, the residual layer receiving the preview image and preprocessing the preview image; A feature extraction layer, the feature extraction layer is connected to the residual layer, and the feature extraction layer extracts image features from the preprocessed preview image; a classification layer, the classification layer being connected to the feature extraction layer, and the classification layer generating the region category of each mask region according to the image features; The region categories include: at least one of a hematoxylin-eosin staining region, an immunohistochemical staining region, and a liquid-based cell staining region; A prediction layer, the prediction layer being connected to the feature extraction layer, and the prediction layer generating the mask region according to the image features; The residual layer comprises a deep residual network in which a plurality of residual units connected in series are provided; Each of the remaining units includes a pair of parallel convolutional networks and shortcuts; In each of the remaining units, preliminary feature extraction is performed on the preview image through the convolutional network, and the shallow features of the shortcut input are added together, and then input into the remaining units of the next level.

2. The scanning method according to claim 1, wherein: The training method of the target detection model includes: Step A1: obtaining a plurality of training slices corresponding to the region category, and labeling the training slices to generate labeled slices; Step A2: performing image processing on the labeled slices to generate a training set; Step A3: Convert the format of the training set and input it into the target detection model to complete the training process.

3. The scanning method according to claim 1, wherein: The step S21 includes: Step S211: inputting the preview image into the target detection model, the target detection model recognizes the preview image and outputs the mask area and the area category corresponding to the mask area respectively; Step S212: merging the mask regions according to the region categories to generate new mask regions.

4. The scanning method according to claim 2, wherein: The training method further comprises, after step A3: Step A4: using the target detection model to identify the training slice, outputting the region category, and judging whether the region category meets the requirements based on the labeled slice; If so, it indicates that the target detection model is trained successfully, and the target detection model is output; If not, it indicates that the target detection model training is unsuccessful, and the process returns to step A1.

5. A pathology section scanning system, characterized in that: include: a scanner, wherein the scanner scans the pathological section to generate a scanned image; a memory storing pre-generated computer instructions; A processor is connected to the memory and the scanner, and the processor runs the computer instructions to perform the scanning method according to any one of claims 1 to 4.

Images