Scraping guidance system and method for specimen slides

Abstract

The present invention provides a scraping guidance system and method for specimen slides. The system displays an electronic image of the specimen through an interactive interface. Users can adjust parameter values ​​to instantly change the analysis results marked on the electronic image. The system then divides the electronic image into multiple sub-regions, selects and marks at least some of the sub-regions based on the analysis results, and generates and outputs scraping guidance. By superimposing the scraping guidance output by the present invention on the physical slide, the physical slide can be marked, assisting the user in efficiently performing specimen scraping.

Description

Technical Field

[0001] The present invention relates to guiding scraping operations, and in particular to a scraping guiding system and method for specimen slides. Background Art

[0002] In the existing scraping method of the specimen, professional clinical personnel usually read the physical slide of the specimen, manually determine the area to be scraped, and then perform the scraping operation.

[0003] The accuracy and efficiency of the aforementioned manual method of determining the scraping area completely rely on the professionalism of the clinical staff. However, this approach cannot ensure that the interpretation standards or selection methods of different clinical staff are consistent, which in turn leads to uneven specimen quality.

[0004] Therefore, the existing scraping specimen collection method has the above-mentioned problems, and a more effective solution is urgently needed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a scraping system and method for guiding the scraping of specimen slides, which can generate and output scraping instructions for assisting in scraping specimens.

[0006] In one embodiment, a method for scraping guidance for a specimen slide includes: a display step, comprising displaying an electronic image of a specimen on an interactive interface, wherein a first analysis result of a first parameter value is marked on the electronic image; a change step, comprising, upon accepting a parameter value change operation, displaying the electronic image marked with a second analysis result of the changed second parameter value, wherein the first analysis result and the second analysis result represent the distribution of target cells with different smoothness; a segmentation step, comprising superimposing the electronic image with a region segmentation map to segment the electronic image into a plurality of sub-regions, wherein an output size of the region segmentation map corresponds to the size of a physical slide of the specimen; a selection step, comprising selecting at least a portion of the sub-regions based on the analysis result; and a guidance step, comprising marking the selected sub-regions during the region segmentation to generate a scraping guidance, and outputting the scraping guidance based on the output size, wherein after the output scraping guidance is superimposed on the physical slide, the markings in the scraping guidance serve as a marking on the physical slide.

[0007] In one embodiment, a scraping guidance system for a specimen slide comprises: an image library for storing an electronic image of a specimen; and a platform end connected to the image library, the platform end being used to establish a network connection with a user end via a network and provide an interactive interface to the user end, the platform end being configured to display the electronic image marked with a first analysis result of a first parameter value on the interactive interface, and when a parameter value change operation is accepted via the interactive interface, display the electronic image marked with a second analysis result of a changed second parameter value on the interactive interface. The platform end is configured to superimpose the electronic image and a region segmentation map on the interactive interface to divide the electronic image into multiple sub-regions. An output size of the region segmentation map is the size of a physical glass slide corresponding to the specimen. The platform end is configured to select at least a portion of the sub-region based on the analysis result, mark the selected sub-region in the region segmentation map to generate a scraping guide, and output the scraping guide based on the output size, wherein after the output scraping guide is superimposed on the physical glass slide, the mark in the scraping guide has a marking effect on the physical glass slide.

[0008] By superimposing the scraping instructions output by the present invention on the physical glass slide, a marking effect can be achieved on the physical glass slide, which can assist the user in efficiently performing specimen scraping and improve the consistency of specimen scraping standards. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 The current procedures for scraping and testing specimens;

[0010] Figure 2 is an architectural diagram of a scraping guidance system according to an embodiment of the present invention;

[0011] Figure 3 This is an architectural diagram of a processing module on a platform side according to an embodiment of the present invention;

[0012] Figure 4 A flowchart of a scraping guidance method according to an embodiment of the present invention;

[0013] Figure 5 A flowchart of changing parameter values ​​according to an embodiment of the present invention;

[0014] Figure 6 A flowchart of selecting a sub-region according to an embodiment of the present invention;

[0015] Figure 7 A flowchart of calibrating output size according to an embodiment of the present invention;

[0016] Figure 8 A flowchart of generating analysis results according to an embodiment of the present invention;

[0017] Figure 9 A flowchart of target identification and analysis according to an embodiment of the present invention;

[0018] Figure 10 A flowchart of cell identification and analysis according to an embodiment of the present invention;

[0019] Figure 11 A flowchart of an analysis result of generating parameter values ​​according to an embodiment of the present invention;

[0020] Figure 12 A schematic diagram of an interface for selecting a sub-region according to an embodiment of the present invention;

[0021] Figure 13 This is a schematic diagram of an interface for marking sub-regions according to an embodiment of the present invention;

[0022] Figure 14 A schematic diagram illustrating the use of scraping instructions according to an embodiment of the present invention;

[0023] Figure 15 is a schematic diagram of marking an electronic image based on a first parameter value according to an embodiment of the present invention;

[0024] Figure 16 is a schematic diagram of marking an electronic image based on a second parameter value according to an embodiment of the present invention;

[0025] Figure 17 is a schematic diagram of scale measurement according to an embodiment of the present invention;

[0026] Figure 18 Based on Figure 17 Schematic diagram of the calibration output size;

[0027] Figure 19 Based on Figure 18 Schematic diagram of the generated scraping instructions.

[0028] Explanation of Figure Numbers

[0029] 1: Platform side

[0030] 10: Processing module

[0031] 100: web module

[0032] 101: Interactive Control Module

[0033] 102: Output control module

[0034] 11: Storage module

[0035] 110: Storage array

[0036] 12: Communication module

[0037] 13: Human-Machine Interface

[0038] 2: User side

[0039] 20: Processor

[0040] 21: Input device

[0041] 22: Display

[0042] 220: Electronic Scraping Guide

[0043] 23: Network devices

[0044] 24: Printing device

[0045] 25: Storage device

[0046] 3: Identification and analysis module

[0047] 30: Learning Model

[0048] 31: Algorithm module

[0049] 4: Database

[0050] 40: Image Library

[0051] 41: Result Library

[0052] 50: Entity Scraping Guide

[0053] 51: Physical slide

[0054] 60: Parameter value transformation module

[0055] 61: Segmentation module

[0056] 62: Manually select module

[0057] 63: Automatically select module

[0058] 630: Purity calculation module

[0059] 631: External computing module

[0060] 632: Recommended Region Decision Module

[0061] 633: Area Restriction Module

[0062] 64: Rendering module

[0063] 65: Guide generation module

[0064] 66: Image Adjustment Module

[0065] 70-71, 80-81: Cluster

[0066] 72, 82: Grid area

[0067] S10-S15: Generate scraping guidance steps

[0068] S20-S24: Steps for changing parameter values

[0069] S30-S32: Manual selection steps

[0070] S40-S47: Automatic selection step

[0071] S50-S54: Calibrate output size steps

[0072] S60-S64: Generate analysis results step

[0073] S70-S72: Target identification and analysis steps

[0074] S80-S73: Cell identification and analysis steps

[0075] S90-93: Analysis and filtering steps DETAILED DESCRIPTION

[0076] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0077] See also Figure 1 , which is the current specimen scraping and testing procedure. Currently, there is a computer-assisted specimen scraping method that can assist clinical staff in identifying the specimen location that needs to be scraped.

[0078] Specifically, after obtaining a specimen (such as a lesion or other tissue to be examined after surgery), it is cut into multiple thin slices and placed on multiple physical slides. Since the thin slices of the specimen on these physical slides are taken from the same tissue location, they usually have the same or similar appearance and cellular composition.

[0079] One of these slides is then scanned into an electronic image, i.e., an electronic image of the thin section of the specimen. A computer then performs feature analysis on the electronic image to generate analysis results, such as the location of the lesion in the electronic image.

[0080] Then, clinical staff can look at the analysis results displayed on the computer screen and manually mark the target cells (such as tumor cells or other diseased cells) on the remaining physical glass slide to mark the location to be scraped.

[0081] Finally, clinical staff scrape the manually marked locations and send the scraped tissue to a specialized testing unit for testing and analysis, such as Next Generation Sequencing (NGS).

[0082] However, when clinical staff manually mark the physical slides while looking at the computer screen, it is difficult for them to accurately circle the correct location due to the gap between the virtual and the real, which can easily lead to marking errors.

[0083] Furthermore, the aforementioned computer-assisted specimen scraping method can only display analysis results for a set of preset criteria (amplifying fixed tumor boundaries) and cannot provide analysis results for different preset criteria for clinical staff to compare or select, thereby failing to meet changing clinical needs.

[0084] In addition, since the production of physical slides is easily affected by cutting or slicing (knife marks, tissue cracks or positional offset, etc.), even two adjacent tissue slides may not be identical in shape and characteristics. This makes it possible that the analysis results displayed on the computer screen may not completely correspond to the physical slides, making it impossible to accurately compare them. In other words, the specimens on the physical slides used for feature analysis may have morphological differences with those on the physical slides used for scraping.

[0085] To address these issues, the present invention proposes a scraping guidance system and method for specimen slides. These systems automatically suggest scraping areas and generate scraping instructions. Furthermore, when a user (e.g., a clinical staff member) overlays the generated scraping instructions on a physical slide, the scraping instructions clearly indicate the appropriate scraping location.

[0086] In addition, the scraping guidance system and scraping guidance method of the present invention can also allow clinical personnel to quickly switch between different analysis results of different rigor (parameter values) to select a scraping area suitable for the current analysis result.

[0087] Please also refer to Figure 2 , is a schematic diagram of a scraping guidance system according to an embodiment of the present invention. The scraping guidance system of the present invention mainly includes a platform terminal 1 and a user terminal 2 connected via a network (such as the Internet or a local area network).

[0088] The platform end 1 (such as a computer device that can provide network services such as a server or a cloud service platform) is used to set up a website (web service) and provide services such as electronic images and scraping instructions through the website.

[0089] The user terminal 2 (such as a desktop computer, laptop computer, tablet computer, etc., a computer device that can be operated by the user) is used to connect to the platform terminal 1 through the network, so that the user can generate and edit the scraping instructions (electronic scraping instructions 220) for the electronic image of the specified specimen by operating the web page.

[0090] Furthermore, the user can operate the user terminal 2 to output the electronic scraping guide 220 to obtain a physical scraping guide 50 (such as a paper), and superimpose the physical scraping guide 50 with the physical glass slide 51 of the specimen.

[0091] Thereby, the physical scraping guide 50 can serve as a mark on the physical glass slide 51 , so that the user can scrape the specimen from the physical glass slide 51 according to the mark.

[0092] It is worth mentioning that the electronic scratching guide 220 corresponds to the physical scratching guide 50, that is, the physical scratching guide 50 is the physicalized electronic scratching guide 220. By physicalizing the scratching guide, the present invention can eliminate the gap between the virtual and the real, thereby improving the efficiency and accuracy of marking.

[0093] In one embodiment, the platform 1 may include a storage module 11 , a communication module 12 , a human-machine interface 13 , and a processing module 10 electrically connected to the above components.

[0094] The storage module 11 (such as RAM, EEPROM, solid-state hard disk, magnetic disk, flash memory and other storage devices or any combination thereof) is used to store data. The communication module 12 (such as a network interface card, NIC) is used to connect to a network (such as the Internet) and communicate with external devices (such as the user terminal 2, the database 4 and / or the identification and analysis module 3) through the network. The human-computer interface 13 (including input interface and output interface, such as a mouse, keyboard, various buttons, touch panel, display, touch screen, projection module, etc.) is used for interaction by the operator of the platform terminal 1 (such as a network administrator). The processing module 10 (which can be a processor such as a CPU, GPU, TPU, MCU or any combination thereof) is used to control the platform terminal 1 and realize the functions proposed by the present invention.

[0095] In one embodiment, the processing module 10 may include a web module 100 , an interactive control module 101 and / or an output control module 102 .

[0096] The web module 100 (eg, a website server module) is configured to provide web services to connect with the client 2 via a web protocol (eg, HTTP or HTTPS).

[0097] In one embodiment, the web module 100 can authenticate the client 2 (eg, account and password authentication, one-time password authentication, hardware binding authentication, etc.), and only allow the client 2 to use the service after passing the authentication.

[0098] The interactive control module 101 is configured to generate an interactive interface (which may be a graphical user interface (GUI), such as a web page) displayed on the client 2. Figures 12 to 19 The interface shown in FIG2 is displayed), and the operation of the user terminal 2 is received and information is displayed to the user through the interactive interface.

[0099] The output control module 102 is configured to transmit the generated (edited) scraping instructions to the user terminal 2 .

[0100] In one embodiment, the client 2 may include an input device 21 , a display 22 , a network device 23 , a printing device 24 , a storage device 25 , and a processor 20 electrically connected to the aforementioned devices.

[0101] The input device 21 (such as a mouse, keyboard, touchpad, etc.) is used for the user to input data or perform operations. The display 22 (such as an LCD, projector, touch screen, etc.) is used to display information (such as displaying a web page and the generated electronic scraping instructions 220). The network device 23 (such as a network interface card, NIC) is used to connect to a network (such as the Internet) and communicate with an external device (such as the platform end 1) through the network. The printing device 24 (such as a printer) is used to print the electronic scraping instructions 220 as a physical scraping instructions 50, such as printing on paper or a thin transparent plate, or directly printing on a physical glass slide 51, etc., without limitation. The storage device 25 (such as a RAM, EEPROM, solid-state hard drive, magnetic hard drive, flash memory, etc. storage device or any combination thereof) is used to store data. The processor 20 (such as a CPU, GPU, TPU, MCU, etc. processor or any combination thereof) is used to control the user end 2.

[0102] In one embodiment, the platform 1 can be connected (e.g., via a network connection or a local line connection) to the recognition and analysis module 3. The recognition and analysis module 3 can be implemented on a server or cloud service platform (e.g., Amazon Web Services, Google Cloud Platform, or Microsoft Azure) and used to perform target recognition and analysis on electronic images (described in detail later).

[0103] In one embodiment, the identification and analysis module 3 can be directly built into the platform 1 .

[0104] In one embodiment, the recognition and analysis module 3 may perform automatic image recognition and analysis based on artificial intelligence (AI), and includes a learning model 30 and an algorithm module 31 .

[0105] The learning model 30 (eg, a classifier) ​​may be configured to perform image classification (eg, image classification of cells) based on machine learning.

[0106] The algorithm module 31 is configured to use the learning model 30 to identify cells (eg, non-target cells and target cells) and to calculate the number of cells at each target cell location.

[0107] In one embodiment, the platform 1 can be connected (eg, via a network) to a database 4 (eg, a network database, a local database, a relational database, or a combination thereof) for storing a large amount of data.

[0108] In one embodiment, the platform 1 may include an image library 40 , which may store a plurality of electronic images of a plurality of specimens.

[0109] In one embodiment, the platform 1 may include a result library 41. The result library 41 may store analysis results of electronic images of various specimens (such as the result storage array 110 described later).

[0110] Please also refer to Figure 3 , is an architecture diagram of a processing module on a platform side according to an embodiment of the present invention. The processing module 10 may include modules 60-66, and the automatic selection module 63 may include modules 630-633.

[0111] These modules 60-66, 630-633, Figure 2 The modules 100-102 and the identification and analysis module 3 are respectively configured to perform different functions (to be described in detail later).

[0112] The aforementioned modules are interconnected (which may be electrical connections and information connections) and may be hardware modules (such as electronic circuit modules, integrated circuit modules, SoCs, etc.), software modules (such as firmware, operating systems or applications) or a mix of hardware and software modules, without limitation.

[0113] It is worth mentioning that when the aforementioned module is a software module (such as firmware, operating system or application), the storage module 11 of the platform end 1 may include a non-transitory computer-readable recording medium, and the aforementioned non-transitory computer-readable recording medium stores a computer program, and the computer program records a computer-executable program code. When the processing module 10 executes the aforementioned program code, the function of the corresponding module can be realized.

[0114] Please also refer to Figure 4 , is a flow chart of a scraping guidance method according to an embodiment of the present invention. The scraping guidance method according to each embodiment of the present invention can be Figure 2-Figure 3 This is achieved by the scraping guidance system shown.

[0115] In the scraping guidance method of this embodiment, the platform 1 can provide web services to the client 2 via the network and the web module 100. The client 2 can send a service request to the platform 1 and perform identity verification. After passing the verification, the client can log in to the webpage (or not) and display the interactive interface provided by the platform 1 through the interactive control module 101.

[0116] Step S10: The platform 1 may display an electronic image of the specimen on an interactive interface. In one embodiment, the user 2 may select an electronic image of a specific specimen through the interactive interface, and the platform 1 may load the specified electronic image (e.g., a hematoxylin and eosin (H&E) stained slide image) from the image library 40.

[0117] Step S11 : the platform end 1 may set parameter values ​​(which may be preset by the system or set by the user end 2 ), obtain analysis results of the parameter values, and mark the analysis results on the electronic image through the rendering module 64 .

[0118] In one embodiment, the result library 41 may pre-store different analysis results of the same electronic image (the same specimen) using different parameter values, and read the corresponding analysis results according to the currently set parameter values.

[0119] In one embodiment, different parameter values ​​represent different stringencies or smoothnesses (described in detail later), and different analysis results with different parameter values ​​represent the distribution status of target cells with different smoothnesses.

[0120] In one embodiment, different analysis results with different parameter values ​​are labeled with different rigor levels.

[0121] In one embodiment, the platform 1 can instantly generate different analysis results for different parameter values ​​after loading the electronic image, and present the corresponding analysis results on the electronic image according to the parameter values ​​set by the user.

[0122] In one embodiment, the user terminal 2 can instantly switch different parameter values ​​(such as switching from a first parameter value to a second parameter value) to enable the platform terminal 2 to instantly present different analysis results (such as changing from a first analysis result to a second analysis result) on the electronic image.

[0123] Step S12: The platform 1 can segment the electronic image into multiple sub-regions by superimposing the electronic image and the region segmentation map through the segmentation module 61. The output size of the region segmentation map corresponds to the size of the physical slide of the specimen (eg, the same or a fixed ratio).

[0124] Step S13: The platform end 1 can automatically select at least a portion of the sub-region (such as selecting a portion or selecting all, without limitation) based on the analysis results through the automatic selection module 63, such as the sub-region where the target cells are located or the sub-region that is easy to scrape.

[0125] In one embodiment, the platform 1 may allow the user terminal 2 to manually select a sub-region through the manual selection module 62 .

[0126] In one embodiment, the platform 1 may use the rendering module 64 to instantly present the selected sub-region in the interactive interface.

[0127] Step S14 : The platform 1 may generate a scraping guide (electronic scraping guide 220 ) by marking the selected sub-region in the region segmentation map through the guide generating module 65 .

[0128] In one embodiment, the platform 1 may use the rendering module 64 to instantly present the generated electronic scraping guide 220 in the interactive interface.

[0129] Step S15 : the platform end 1 outputs a scraping instruction with a specified output size to the user end 2 through the output control module 102 .

[0130] In one embodiment, the platform 1 may request the user 2 to allow the user 2 to control the printing device 24 to print the generated scratching instructions on paper.

[0131] In one embodiment, the user terminal 2 can proactively print out scraping instructions (physical scraping instructions 50 ) with a specified output size. Furthermore, after the printed scraping instructions are superimposed on the physical glass slide 51 , the markings in the scraping instructions serve as markers for the recommended scraping positions on the physical glass slide 51 .

[0132] Thus, by superimposing the scraping instructions output by the present invention on the physical glass slide, a marking effect can be achieved on the physical glass slide, thereby assisting the user to efficiently perform specimen scraping.

[0133] Please also refer to Figures 12 to 14 , Figure 12 FIG. 1 is a schematic diagram of an interface for selecting a sub-region according to an embodiment of the present invention. Figure 13 This is a schematic diagram of the interface for marking sub-regions according to an embodiment of the present invention. Figure 14 Schematic diagram of the use of scraping instructions according to an embodiment of the present invention. Figures 12 to 14 It is used to illustrate a specific embodiment of the present invention.

[0134] In this example, if Figure 12 、 13 As shown, the region segmentation map includes a plurality of grid lines, and the plurality of grid lines overlapping the electronic image segment the specimen image of the electronic image into a plurality of grid regions as a plurality of sub-regions.

[0135] The platform 1 can present the analysis results of the current parameter values ​​on the electronic image in the interactive interface, so as to display the target cell selection mark on the electronic image (such as circling the location of the target cell). The smoothness of the aforementioned selection mark changes with the parameter value.

[0136] In addition, the platform end 1 can display an electronic image divided into multiple sub-areas in the interactive interface and automatically select a sub-area recommended for scraping (eg, with higher purity of target cells), such as clusters 70 and 71 of multiple grid areas.

[0137] In one embodiment, the platform 1 may further perform marking with different colors or patterns according to the purity of target cells (eg, tumor cell purity, TCP) in each grid area. For example, the higher the purity, the darker the color of the grid area.

[0138] In one embodiment, the platform 1 may further display sub-regions where scraping is not recommended (eg, the purity of target cells is too low), such as the grid region 72 , on the interactive interface.

[0139] In addition, the user terminal 2 can manually select or deselect the grid area to change the range of the clusters 70 and 71.

[0140] like Figure 13 As shown, after the selection is completed, the user terminal 2 can request to generate an electronic scraping guide. The scraping guide in this example is a line mark (or as shown in FIG. Figure 19 (As shown, marked with a color block) The position where the specimen is to be scraped.

[0141] Then, if Figure 14 As shown, after the electronic scraping guide is printed as a physical scraping guide, it can be used to overlap with the physical glass slide, so that the marks on the physical scraping guide (such as the range marks of clusters 70 and 71 and the range marks of grid area 72) can clearly mark the specimen slices in the physical glass slide, allowing the user to perform the scraping operation correctly and efficiently.

[0142] Please also refer to Figure 5 , is a flow chart of changing parameter values ​​according to an embodiment of the present invention. Figure 4 The scraping guidance method of this embodiment further provides a parameter value changing function implemented by steps S20-S24.

[0143] Step S20: The platform 1 switches to the parameter setting mode. In one embodiment, the platform 1 can switch to the parameter setting mode after loading the electronic image (step S10), so that the user terminal 2 can set appropriate parameter values.

[0144] Step S21: the platform end 1 identifies the currently set parameter value (such as the first parameter value) through the parameter value conversion module 60, and obtains the analysis result of the current parameter value (such as the first analysis result).

[0145] In one embodiment, the platform 1 can obtain the number or purity of target cells (e.g., tumor cells or other pathological cells) at each image location of the electronic image of the specimen. This information can be pre-analyzed and stored in the result library 41 (e.g., via steps S60-S64 described below). Next, the platform 1 calculates the target cell distribution locations at different rigor levels based on all parameter values ​​and adds these calculated locations to the analysis results for the corresponding parameter values ​​for subsequent user selection.

[0146] Furthermore, the aforementioned different parameter values ​​correspond to different degrees of precision. The present invention primarily calculates target cell distribution markers with varying degrees of smoothness based on different parameter values. A higher smoothness indicates a wider range of target cell identification, i.e., a lower degree of precision; a lower smoothness indicates a narrower range of target cell identification, i.e., a higher degree of precision. Thus, the present invention allows users to obtain target cell distributions identified at varying degrees of precision.

[0147] Step S22: The platform 1 marks the analysis result of the current parameter value on the electronic image through the rendering module 64, such as presenting the analysis result in the electronic image to circle the image position of the target cell distribution position using a mark with a corresponding smoothness (such as Figure 12 Image location of target cells as shown).

[0148] Step S23: The platform end 1 detects via the parameter value conversion module 60 whether the parameter value change operation of the user end 2 is accepted via the interactive interface.

[0149] If the user terminal 2 changes the parameter value, it identifies the new parameter value (such as the second parameter value) after the transformation, executes steps S21-S22 again to obtain the analysis result of the new parameter value (such as the second analysis result), and uses the new analysis result to mark the electronic image.

[0150] If the user terminal 2 has not changed the parameter value or has determined the final parameter value, step S24 is executed: the platform terminal 1 determines whether to leave the parameter value setting mode, such as whether the user is sure not to change the parameter value.

[0151] If the parameter value setting mode is not left, step S23 is executed again to detect again. Otherwise, the other steps of the method are continued.

[0152] Thus, the present invention can provide analysis results with different rigor, that is, markers with different smoothness, and can meet different clinical rigor requirements.

[0153] Please also refer to Figure 11 , is a flowchart of the analysis results for generating parameter values. Figure 4 and Figure 5 The scraping guidance method of this embodiment further provides analysis results of target cells with different rigor achieved by steps S90-S93.

[0154] In this embodiment, different parameter values ​​represent different degrees of rigor, and the degree of image detail retained by filtering is controlled according to the different parameter values.

[0155] Step S90: The platform 1 reads all possible parameter values. For example, multiple rigor levels are pre-specified and mapped to multiple parameter values. For example, a parameter value of 50 corresponds to a lower rigor level, and a parameter value of 70 corresponds to a higher rigor level.

[0156] Step S91: The platform 1 calculates corresponding filter masks according to different parameter values. Different filter masks corresponding to different parameter values ​​have different filtering capabilities, such as being used to filter out signals in different frequency ranges.

[0157] Step S92: The platform end 1 uses different filter masks with different parameter values ​​to perform filtering processing on the image frequency domain respectively to obtain different filtering results with different filter masks.

[0158] In one embodiment, the platform 1 first converts the electronic image from the spatial domain to the frequency domain (eg, by Fourier transform, fast Fourier transform, etc.), and then performs the aforementioned filtering process.

[0159] In one embodiment, the filter mask is a low-pass filter mask. Each filter mask is used to filter high-frequency signals within a different frequency range during the filtering process. For example, a wider low-pass filter mask will filter more high-frequency signals (less stringent), while a narrower low-pass filter mask will retain more high-frequency signals (more stringent).

[0160] Step S93 : After filtering, the platform 1 can obtain the image position of the target cell under different parameter values ​​(different filter masks) as the analysis result of the parameter value and store it in the database 4 .

[0161] In this way, the present invention can produce analysis results of different stringencies.

[0162] Please also refer to Figures 15 and 16 , Figure 15 is a schematic diagram of marking an electronic image based on a first parameter value according to an embodiment of the present invention. Figure 16 FIG. 1 is a schematic diagram of marking an electronic image based on a second parameter value according to an embodiment of the present invention.

[0163] In this example, the target cells are living tumor cells (Variable Cancer Cells, VCCs). In this example, the parameter value represents the stringency of the VCC representation, and the parameter value is proportional to the stringency. That is, higher parameter values ​​indicate higher stringency (finer marking boundaries, making selection difficult); lower parameter values ​​indicate lower stringency (smoother marking boundaries, making selection easier), but this is not a limitation.

[0164] In another example, the parameter value can be set to be inversely proportional to the stringency, that is, the higher the parameter value, the lower the stringency; and the lower the parameter value, the higher the stringency.

[0165] It is worth mentioning that the present invention can control the filtering process based on different parameter values ​​(such as Figure 11 The degree of retention of marking details in the steps shown) to provide different levels of rigor.

[0166] Figure 15 In [1], the parameter value is 70, which sets a higher level of accuracy. With this setting, the filtering process retains more high-frequency signals (i.e., details), producing a more detailed indication of the VCC's image location, thereby providing a more accurate VCC location detection result.

[0167] Figure 16 In [1], the parameter value is 50, which sets a lower precision. Under this setting, the filtering process retains less high-frequency signals, resulting in a coarser indication of the VCC image position, thus providing a less precise VCC position detection result.

[0168] Thus, the present invention allows users to set different parameter values ​​according to different rigor requirements and view the corresponding analysis results in real time, and can provide analysis results of different standards for clinical staff to compare or select, so as to avoid failure to meet changing clinical needs.

[0169] Figure 6 FIG. 1 is a flowchart of selecting a sub-region according to an embodiment of the present invention. Figure 4 The scraping guidance method of this embodiment further provides a manual selection mode implemented by steps S30-S32 and an automatic selection mode implemented by steps S40-S47 in step S13.

[0170] Manual selection mode includes the following steps:

[0171] Step S30: The platform 1 accepts the sub-region selection operation of the user terminal 2 through the manual selection module 62 and the interactive interface. The sub-region selection operation is to select or deselect at least one sub-region.

[0172] Step S31: The platform end 1 changes the status of the sub-area selected in step S30 to selected, unselected or not scraped through the manual selection module 62.

[0173] In one embodiment, when the sub-region is selected for the first time, its status changes from unselected to selected; when the sub-region is selected for the second time, its status changes from selected to unselected.

[0174] In one embodiment, the user terminal 2 can directly set the status of some sub-regions to not be scraped (such as regions with low purity of target cells or regions that are difficult to scrape), so as to avoid scraping these regions by mistake in the future.

[0175] Step S32: The platform 1 uses the rendering module 64 to instantly present the changed status of all sub-areas on the interactive interface.

[0176] The automatic selection mode includes the following steps.

[0177] Step S40: The platform end 1 obtains the purity of each sub-region of the electronic image through the purity calculation module 630, and selects the sub-regions whose purity is greater than a preset purity (such as 30%, 50% or 70%).

[0178] Step S41 : The platform 1 uses the external calculation module 631 to cluster multiple adjacent sub-regions among the selected sub-regions into a candidate region with a wider range.

[0179] In one embodiment, the platform 1 may generate a minimum bounding rectangle of multiple sub-regions as candidate regions.

[0180] It is worth mentioning that since a single sub-region is too small to be easily scraped, the present invention can effectively improve the accuracy and efficiency of the scraping operation by clustering multiple adjacent sub-regions suitable for scraping into a larger recommended range.

[0181] Step S42: The platform 1 determines whether each candidate area meets the preset conditions one by one through the recommended area decision module 632.

[0182] In one embodiment, the aforementioned preset condition may include that the purity of the candidate region is not less than a preset purity (eg, 30%, 50%, or 70%, etc.).

[0183] In one embodiment, the aforementioned preset condition may include that the number of target cells in the candidate region is not less than a preset number (eg, 5,000, 10,000, 15,000, 20,000, etc.).

[0184] If any candidate region meets the preset conditions, step S43 is executed for the candidate region that meets the conditions: the platform end 1 sets the candidate region that meets the conditions as the recommended region through the recommended region decision module 632 .

[0185] If any candidate area does not meet the preset conditions, step S44 is executed for the non-compliant candidate area: the platform end 1 reduces the scope of the candidate area (such as excluding one or more sub-areas) through the area limitation module 633, so that the reduced candidate area meets the preset conditions and is set as the recommended area.

[0186] In one embodiment, the platform 1 may calculate the purity of target cells on each side of the non-compliant candidate region through the region narrowing module 633 and exclude the sub-region on the side with the lowest purity, thereby improving the overall purity of the narrowed candidate region.

[0187] In one embodiment, the platform 1 may directly set the unqualified candidate areas as non-recommended areas, or set the unqualified candidate areas as non-recommended areas after the unqualified candidate areas have been reduced to blank areas.

[0188] Step S45: The platform 1 determines whether any recommended area is generated.

[0189] If any recommended area is generated, step S46 is executed: the platform 1 presents the recommended area in the electronic image of the interactive interface through the rendering module 64 .

[0190] If no recommended region is generated, a warning indicating no recommended region may be issued directly, or step S47 may be executed: the platform 1 presents the sub-region covering the target cell in the electronic image of the interactive interface through the rendering module 64 as a reference for manual selection by the user 2 .

[0191] In one embodiment, the platform 1 may further display all sub-regions in the electronic image of the interactive interface through the rendering module 64 , and provide the number of target cells and the number of non-target cells in each sub-region.

[0192] Please also refer to Figure 7 、 Figure 17-Figure 19 , Figure 7 This is a flow chart of calibrating the output size according to an embodiment of the present invention. Figure 17 is a schematic diagram of scale measurement according to an embodiment of the present invention, Figure 18 Based on Figure 17 Schematic diagram of the calibration output size, Figure 19 Based on Figure 18 Schematic diagram of the generated scraping instructions.

[0193] Compared to Figure 4 The scraping guidance method of this embodiment further provides a calibrated output size implemented by steps S50-S54 before outputting the scraping guidance.

[0194] Step S50: The user can first align the specimen in the physical slide with the physical region segmentation map ( Figure 17 (for physical grid paper) and confirm the corresponding scale (at Figure 17 The length and width scale in the figure is (12,8).

[0195] The aforementioned physical region segmentation map and the aforementioned region segmentation map used to segment the electronic image have corresponding sizes.

[0196] Step S51 : The platform end 1 receives a scale input operation through the interactive interface via the image adjustment module 66 to obtain an input scale.

[0197] like Figure 18As shown in the figure above, the user terminal 2 can modify the current scale (X, Y) = (12, 9) to the actual scale (X, Y) = (12, 8) measured in step S51, that is, the input scale.

[0198] Step S52: the platform end 1 adjusts the output size of the electronic image through the image adjustment module 66 so that the current scale of the electronic image conforms to the input scale.

[0199] In one embodiment, the platform 1 scales the electronic image and the scratching guide in equal proportions, so that the output sizes of the scaled electronic image and the scratching guide conform to the input scale.

[0200] like Figure 18 As shown in the figure below, since the electronic image is zoomed, the range and position of the cluster 70-71 and the grid area 72 of the scraping guide are also zoomed and moved accordingly, and changed to cluster 80-81 and grid area 82.

[0201] Step S53 : The platform 1 outputs the adjusted scraping instruction through the instruction generating module 65 . The output size of the scraping instruction has been adjusted in step S52 .

[0202] Then, the user terminal 2 can print the adjusted scraping instructions based on the adjusted output size. In this way, the printed scraping instructions can conform to the size of the specimen on the physical slide.

[0203] like Figure 19 As shown, due to proportional scaling, the clusters 80 - 81 and the grid area 82 of the adjusted scraping guide (physical grid paper with color block markings) may not align with the grid lines, but will better fit the size of the physical slide.

[0204] Step S54: the platform end 1 can align the printed scraping guide with the mark with the physical glass slide and the physical glass slide to be scraped, and perform scraping according to the mark.

[0205] Thus, the present invention can make up for the difference in shape between the specimen on the physical glass slide for characteristic analysis and the specimen on the physical glass slide for scraping, thereby improving the accuracy of the scraping operation.

[0206] Please also refer to Figure 8 , is a flow chart of generating analysis results according to an embodiment of the present invention. Figure 4 The scraping guidance method of this embodiment further provides an analysis result calculation function implemented by steps S60-S64. The aforementioned analysis result calculation function can calculate the detailed analysis results of the specified electronic image to be used as the selected sub-region to provide the data acquisition source of step S21.

[0207] Step S60 : The platform 1 creates a result storage array 110 of the currently selected electronic image in the storage module 11 through the recognition and analysis module 3 .

[0208] Step S61: the platform end 1 reads each sub-image (regional image) of the electronic image one by one through the recognition and analysis module 3, and executes steps S62-63 on each sub-image until the entire electronic image is processed.

[0209] It is worth mentioning that due to the very high resolution of electronic images, extremely high-specification hardware equipment is required to perform recognition and analysis processing on the entire electronic image at the same time. The present invention can effectively reduce the requirements for hardware specifications by performing processing on sub-images in batches.

[0210] Step S62: the platform end 1 performs target recognition and analysis on each sub-image through the recognition and analysis module 3 to identify the number of target cells in each sub-image as the analysis result of the sub-image.

[0211] In one embodiment, the recognition and analysis module 3 may use a classifier (such as the learning model 30 ) based on machine learning (such as trained based on image features of target cells) to recognize each target cell in the sub-image.

[0212] In one embodiment, the identification and analysis module 3 may use the learning model 30 to perform model prediction to calculate the number of target cells at each location, and may combine the target cell positions predicted by the prediction algorithm (such as a convolutional network) of the algorithm module 31 with the cell nucleus positions generated by the FRST algorithm to calculate the target cell value in each sub-region.

[0213] Step S63: The platform 1 stores the analysis results of each sub-image in the corresponding memory location of the result storage array 110. When all sub-images are analyzed, the result storage array 110 of the electronic image is obtained. The result storage array 110 records the target cell-related information at each image location.

[0214] In one embodiment, the completed result storage array 110 may be stored in the result library 41 .

[0215] Step S64 : the platform end 1 obtains the positions of target cells in all electronic images based on the result storage array 110 and stores them, such as in the result library 41 .

[0216] Thereby, the present invention can generate detailed analysis information of the electronic image, which can be used as a reference for subsequent selection of the scraping area.

[0217] Please also refer to Figure 9, which is a flow chart of target recognition and analysis according to an embodiment of the present invention. Compared to the aforementioned scraping guidance method, the scraping guidance method of this embodiment further provides a target cell analysis function implemented by steps S70-S72.

[0218] Step S70: The platform end 1 obtains the positions of all cells through the recognition and analysis module 3, and obtains the positions of all target cells, such as through automatic AI recognition.

[0219] Step S71: the platform end 1 calculates the number of cells at each target cell location through the recognition and analysis module 3 to calculate the purity of each image location.

[0220] Step S72: the platform end 1 calculates the number of target cells and non-target cells in each sub-region.

[0221] In this way, the present invention can calculate the purity and quantity of target cells at each image position.

[0222] Please also refer to Figure 10 , is a flow chart of cell identification and analysis according to an embodiment of the present invention. Compared to the aforementioned scraping guidance method, the scraping guidance method of this embodiment further provides a target cell positioning function implemented by steps S80-S83.

[0223] Step S80: The platform 1 reads the color conversion matrix. The color conversion matrix is ​​determined based on the type of target cells to be identified and the staining method used. For example, a hematoxylin and eosin (H&E) staining conversion matrix can be used for tumor cells.

[0224] Step S81: the platform end 1 uses a color conversion matrix to decompose each sub-image into multiple dyed image channels of different colors.

[0225] In one embodiment, when the target cells are tumor cells and the electronic image is a hematoxylin-eosin stained pathological image, a color conversion matrix may be used to decompose the RGB image (electronic image) into a hematoxylin-stained channel image and an eosin-stained channel image.

[0226] Step S82: the platform end 1 performs recognition processing on each staining image channel image to identify the cell position image of the cell type corresponding to each staining image channel image.

[0227] In one embodiment, the platform 1 can detect the cell position image using a FRST algorithm in the hematoxylin staining channel image.

[0228] It is worth mentioning that the dye hematoxylin can stain basophilic structures into blue-purple. Basophilic structures usually include parts containing nucleic acids, and the staining mainly stains the cell nucleus into blue-purple; while eosin can stain eosinophilic structures into pink. Eosinophilic structures are usually composed of intracellular and intercellular proteins.

[0229] Step S83 : the platform end 1 converts each cell position image into cell coordinates (such as image coordinates), and stores the cell coordinates in the aforementioned result storage array 110 .

[0230] Thereby, the present invention can effectively identify the specific location of each cell.

[0231] The above description is only a preferred embodiment of the present invention, and does not limit the scope of the claims of the present invention. Therefore, all equivalent changes made by applying the content of the present invention are similarly included in the scope of the present invention and are hereby stated.

Claims

1. A method for scraping and guiding a specimen slide, comprising: The display step includes displaying an electronic image of the specimen on an interactive interface, wherein the electronic image is marked with a first analysis result of a first parameter value; The changing step includes displaying the electronic image marked with a second analysis result using the changed second parameter value when accepting the parameter value changing operation, wherein the first analysis result and the second analysis result represent the distribution of target cells with different smoothness; a segmentation step, comprising superimposing the electronic image with a region segmentation map to segment the electronic image into a plurality of sub-regions, wherein an output size of the region segmentation map corresponds to a size of a physical slide of the specimen; A selecting step comprising selecting at least a portion of the sub-region based on the analysis result; and The guidance step includes marking the selected sub-area in the area segmentation to generate scraping guidance, and outputting the scraping guidance based on the output size, wherein after the output scraping guidance is superimposed on the physical glass slide, the mark in the scraping guidance has a marking effect on the physical glass slide.

2. The method for guiding scraping of a specimen slide according to claim 1, wherein the region segmentation diagram comprises a plurality of grid lines, and the plurality of grid lines overlapping the electronic image segment the specimen image of the electronic image into a plurality of grid regions as the plurality of sub-regions; in, Outputting the scratching instructions includes controlling a printing device to print the scratching instructions on paper; Wherein, the scraping guide includes marking the position where the specimen is to be scraped with a line or a color block; Wherein, the interactive interface is a graphical user interface.

3. The method for guiding scraping of a specimen slide according to claim 1, further comprising, before the displaying step: Provide web services to the user end through the network on the platform end; and After the user terminal passes the verification, the interactive interface is displayed through a web page.

4. The scraping guidance method for a specimen slide according to claim 1, wherein the changing step comprises: When accepting the parameter value change operation through the interactive interface, identifying the second parameter value selected by the parameter value change operation; Obtaining the second analysis result of the second parameter value from a database; and The second analysis result is presented on the electronic image to mark the distribution of the target cells in the electronic image.

5. The method for guiding scraping of a specimen slide according to claim 4, wherein before obtaining the analysis result, the method comprises: Get multiple different parameter values; Calculating a filter mask corresponding to each parameter value, wherein each filter mask is used to filter out signals in different frequency ranges; performing filtering processing on the frequency domain of the electronic image based on each of the filter masks to obtain analysis results of each of the parameter values, wherein each of the analysis results marks the image position of the target cell with a different degree of smoothness; and storing all the analysis results in the database; Wherein, the target cells are diseased cells.

6. The scraping guidance method for a specimen slide according to claim 1, wherein the selecting step comprises: When a sub-region selection operation is accepted through the interactive interface, changing the status of the selected sub-region to selected, unselected, or not to be scraped based on the sub-region selection operation; and The changed state of the sub-area is presented on the interactive interface.

7. The scraping guidance method for a specimen slide according to claim 1, wherein the selecting step comprises: obtaining the purity of each sub-region of the electronic image; selecting the sub-regions whose purity is greater than a preset purity, and setting a cluster of the selected adjacent sub-regions as a candidate region; and When any of the candidate regions meets a preset condition, the candidate region is set as a recommended region.

8. The method for guiding scraping of a specimen slide according to claim 7, wherein the selecting step further comprises: When any of the candidate regions does not meet the preset condition, the candidate region is reduced to meet the preset condition or is set as a non-recommended region.

9. The method for scraping guides for specimen slides according to claim 1, wherein before outputting the scraping guides, the method further comprises: Accepting a scale input operation through the interactive interface to obtain an input scale; adjusting the output size of the electronic image so that the current scale of the electronic image conforms to the input scale; The guiding step is to output the scraping guidance based on the adjusted output size, so that the output scraping guidance conforms to the size of the specimen on the physical slide.

10. The method for guiding scraping of a specimen slide according to claim 1, further comprising, before the displaying step: establishing a result storage array of the electronic image; Reading each sub-image of the electronic image, performing target recognition and analysis on each sub-image to identify the number of target cells in each sub-image as an analysis result of the sub-image; storing the analysis results of each of the sub-images in the result storage array; and Based on the result, a storage array stores the positions of all the target cells. 11 . The scraping guidance method for a specimen slide according to claim 10 , wherein the target recognition and analysis comprises using a machine learning-based classifier to recognize each target cell in the sub-image.

12. The scraping guidance method for a specimen slide according to claim 11, wherein the target recognition and analysis comprises: Calculating the number of cells at each location of the target cells; and The number of the target cells and non-target cells in each of the sub-regions is calculated.

13. The scraping guidance method for a specimen slide according to claim 11, wherein: Identifying the target cells comprises: Read the color conversion matrix; Decomposing each of the sub-images into a plurality of dyed image channels of different colors using the color conversion matrix; performing recognition processing on each of the staining image channels to identify a cell position image of the cell type corresponding to the staining image channel; and Each of the cell position images is converted into cell coordinates, and the cell coordinates are stored.

14. A scraping guidance system for a specimen slide, comprising: Image library, used to store electronic images of specimens; and A platform end, connected to the image library, for establishing a network connection with a user end via a network and providing an interactive interface to the user end; The platform is configured to display the electronic image marked with a first analysis result of a first parameter value on the interactive interface, and to display the electronic image marked with a second analysis result of a changed second parameter value on the interactive interface when a parameter value change operation is accepted via the interactive interface; The platform is configured to superimpose the electronic image and the region segmentation map on the interactive interface to segment the electronic image into a plurality of sub-regions, and the output size of the region segmentation map corresponds to the size of the physical slide of the specimen; The platform is configured to select at least a portion of the sub-region based on the analysis result, mark the selected sub-region in the region segmentation map to generate a scraping guide, and output the scraping guide based on the output size; After the output scraping instructions are superimposed on the physical glass slide, the mark in the scraping instructions has a marking effect on the physical glass slide; The first analysis result and the second analysis result represent the distribution of target cells with different smoothness.

15. The scraping guidance system for specimen slides according to claim 14, wherein the platform end comprises A web module is configured to provide a web service to connect to the client through a web protocol and to authenticate the client; an interactive control module for generating the interactive interface displayed on the user terminal and receiving operation and display information through the interactive interface, wherein the interactive interface is a graphical user interface; and The output control module is used for transmitting the scraping instruction to the user terminal so as to enable a printing device of the user terminal to print the scraping instruction as a paper of the output size.

16. The scraping guidance system for specimen slides according to claim 14, wherein the interactive interface comprises a parameter value change interface; The system includes a database for storing a plurality of analysis results of a plurality of parameter values; in, The platform is connected to the database and includes: a parameter value conversion module configured to, when the parameter value change operation is detected through the parameter value change interface, identify the second parameter value selected by the parameter value change operation, obtain the second analysis result of the second parameter value from the database, and present the second analysis result on the electronic image to mark the distribution of the target cells in the electronic image; a segmentation module configured to segment the specimen image of the electronic image into a plurality of grid areas as the plurality of sub-areas based on a plurality of grid lines of the area segmentation map overlapping the electronic image; and The guideline generating module is configured to generate the scraping guideline, wherein the scraping guideline includes marking a position where the specimen is to be scraped with a line or a color block.

17. The scraping guidance system for specimen slides according to claim 14, wherein the platform end comprises: a manual selection module configured to, when a sub-region selection operation is detected through the interactive interface, change the state of the selected sub-region to selected, unselected, or not to be scraped based on the sub-region selection operation; a purity calculation module configured to obtain the purity of each of the sub-regions of the electronic image; an external computing module configured to select the sub-regions having a purity greater than a preset purity, and set a cluster of the selected adjacent sub-regions as a candidate region; a region reduction module configured to remove low-purity regions from the candidate regions to improve the purity of the reduced candidate regions; a recommended area decision module configured to set any candidate area as a recommended area when the candidate area meets a preset condition, and to execute the area narrowing module on the candidate area to make it meet the preset condition or set it as a non-recommended area when the candidate area does not meet the preset condition; and The rendering module is configured to present the status of each sub-area on the interactive interface in real time.

18. The scraping guidance system for specimen slides according to claim 14, wherein the interactive interface comprises a scale input interface; The platform side includes: an image adjustment module configured to obtain an input scale of a scale input operation through the scale input interface and adjust the output size of the electronic image so that the current scale of the electronic image conforms to the input scale; and The instruction generating module is configured to output the scraping instruction based on the adjusted output size, so that the output scraping instruction conforms to the size of the specimen on the physical slide.

19. The scraping guidance system for a specimen slide according to claim 14, further comprising: A storage module, connected to the platform end, for storing the result storage array of the electronic image; and An identification and analysis module is informationally connected to the platform end and is configured to read each sub-image of the electronic image, perform target identification and analysis on each sub-image to identify the number of target cells in each sub-image, store the analysis results of each sub-image in the result storage array as the analysis results of the sub-image, and store the positions of all the target cells based on the result storage array.

20. The scraping guidance system for specimen slides according to claim 19, wherein the identification and analysis module comprises: a learning model configured to perform cell classification based on machine learning to identify each of the target cells in the sub-image; and an algorithm module configured to calculate the number of cells at each location of the target cells and the number of the target cells and non-target cells in each sub-region; Among them, the recognition and analysis module is also configured to read a color conversion matrix, use the color conversion matrix to decompose each of the sub-images into multiple staining image channels of different colors, perform recognition processing on each of the staining image channels through the learning model to identify the cell position image of the cell type corresponding to the staining image channel, convert each of the cell position images into cell coordinates, and store the cell coordinates.

Images