A digital pathology system based on low-resolution imaging

Abstract

The present invention discloses a digital pathology system based on low-resolution imaging. By scanning pathological sections, low-resolution pathological images can be quickly acquired, and then the pathological images are stored in a low-resolution manner. When in use, super-resolution image processing technology is used to construct high-resolution images. This can reduce resource consumption caused by image storage while ensuring image quality. Images with different usage frequencies are classified and stored, and frequently used high-resolution images are stored separately, which can greatly improve retrieval speed.

Description

Technical Field

[0001] The present invention relates to the field of digital pathology management, and in particular to a digital pathology system based on low-resolution imaging. Background Art

[0002] Pathological diagnosis is the gold standard for disease diagnosis. Conventional pathological diagnosis is to observe the slices under a microscope. Digital pathology is a revolutionary change in modern pathology. It can significantly improve the efficiency and quality of film reading. Its basis is the digitization of slices. Figure 1 The high-resolution imaging technology shown, such as using a high-magnification lens (such as 20 times, 40 times, 100 times) to image the slice, obtain and store high-resolution pathology whole-slice images, and then use the high-resolution images for diagnosis.

[0003] However, the cost of scanning and storing high-resolution images is huge. There are about 50 million pathology examinations per year. Even if one slice is stored per person and the imaging time is 2 minutes, it will take about 250,000,000GB of storage capacity and 200 years of imaging time. It will take about 280 years to achieve a network copy (based on the network transmission of a single 40X full-film image, which takes 3 minutes). Therefore, the application cost of digital pathology technology based on high-resolution imaging and its system is extremely high, and it cannot be widely used in daily diagnosis of digital pathology. At present, pathology diagnosis is still in the era of slices, which is seriously disproportionate to the health needs of the people. Summary of the invention

[0004] The present invention provides a digital pathology system based on low-resolution imaging, which is used to solve the technical problem of high application cost of existing digital pathology systems.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] A digital pathology system based on low-resolution imaging, comprising: a digital pathology imaging system, a data storage system, a computing system and a pathology diagnosis system, wherein the data storage system establishes communication with the digital pathology imaging system, the computing system and the pathology diagnosis system respectively;

[0007] The digital pathology imaging system is used to obtain a low-resolution pathology image of a slice and its slice number, and send the low-resolution pathology image and its slice number to the data storage system;

[0008] The data storage system comprises a temporary storage module and a permanent storage module, which is used to store the low-resolution pathological image in the permanent storage module;

[0009] The calculation module is used to call the low-resolution pathology image from the permanent storage module, convert the low-resolution pathology image into a high-resolution pathology image, and send the high-resolution pathology image to the temporary storage module of the data storage system for storage;

[0010] The pathological diagnosis system calls a low-resolution pathological image from the permanent storage module for display or calls a high-resolution pathological image from the temporary storage module for display.

[0011] Preferably, the digital pathology imaging system comprises: a low-resolution pathology scanner and a quality review module;

[0012] The low-resolution pathology scanner is used to scan the low-resolution pathology image of the pathology section and send the low-resolution pathology image to the quality review module; the quality review module is used to detect the color, fuzziness, bubbles and tissue folding of the low-resolution pathology image, and compare the color, fuzziness, bubbles and tissue folding of the low-resolution pathology image with the color, fuzziness, bubbles and tissue folding of the standard image to determine whether the low-resolution pathology image is a standard image. When the low-resolution pathology image is determined to be a standard image, the low-resolution pathology image is sent to the data storage system for storage.

[0013] Preferably, it also includes a data management module, the data management module is used to construct and store a mapping table between the patient's ID number and other identity information, the data management module is also used to construct and store a mapping table between the patient ID, pathology ID and other clinical information, and the data management module is also used to construct a mapping table between the pathology image slice number, pathology ID, status, storage location information and image acquisition time;

[0014] The permanent storage module includes multiple different storage units; the data management module is used to call the mapping table according to the slice number of the low-resolution pathology image to query the patient information corresponding to the low-resolution pathology image, and estimate the usage frequency of the low-resolution pathology image according to the patient information, and schedule or migrate different low-resolution pathology images to different storage units according to the frequency of use.

[0015] Preferably, different storage units use different storage media. The storage unit storing low-resolution pathology images with low usage frequency uses a low-cost, slow, large-capacity storage medium; the storage unit storing low-resolution pathology images with high usage frequency uses a high-cost, fast-access medium; the temporary storage module uses the fastest storage medium.

[0016] Preferably, the multiple different storage units include a recent storage unit, a mid-term storage unit and a long-term storage unit. The recent storage unit is used to store low-resolution pathology images with a high usage frequency, specifically low-resolution pathology images of patients currently in the hospital; the mid-term storage unit is used to store low-resolution pathology images with a medium usage frequency, specifically low-resolution pathology images of patients examined within one year; and the long-term storage unit is used to store low-resolution pathology images with a low usage frequency, specifically patients examined one year ago.

[0017] Preferably, the data management module further includes: a data storage module, a data retrieval module, a data generation module, a data browsing module and a data sorting module;

[0018] The data storage module is used to store the low-resolution images sent by the digital pathology imaging system in a recent storage unit;

[0019] The data scheduling module is used to query the patient information of each low-resolution pathological image in the recent storage unit, and determine whether the usage frequency of each low-resolution pathological image is high based on the patient information of each low-resolution pathological image. When it is found that the usage frequency of a low-resolution pathological image is medium or low, the low-resolution pathological image is migrated from the recent storage unit to the medium-term storage unit or the long-term storage unit; The data scheduling module is used to query the patient information of each low-resolution pathological image in the medium-term storage unit, and determine whether the usage frequency of each low-resolution pathological image is low based on the patient information of each low-resolution pathological image. When it is found that the usage frequency of a low-resolution pathological image is low, the low-resolution pathological image is migrated from the medium-term storage unit to the long-term storage unit;

[0020] The data retrieval module is used to call the mapping table according to the query instruction to retrieve the specified low-resolution pathological image from the multiple different storage units;

[0021] The data generation module is used to send the corresponding low-resolution pathological image to the computing system according to the resolution conversion instruction, receive the high-resolution image fed back by the computing system, and store the high-resolution image in the temporary storage module;

[0022] The data browsing module is used to call the corresponding high-resolution image from the temporary storage module according to the browsing instruction, and send it to the pathology diagnosis system for the user of the pathology diagnosis system to perform pathology diagnosis or image reading;

[0023] The data sorting module is used to extract all pathological images of the same patient from the permanent storage module and the clinical storage module according to the patient ID, and put them into a designated path.

[0024] Preferably, the computing system is also connected to the pathological diagnosis system; the pathological diagnosis system is also used to send low-resolution pathological images to the computing system, and the computing system includes: a priority judgment module and a high-resolution image processing module;

[0025] The priority judgment module is used to judge the processing priority of the low-resolution pathology image sent, and control the super-resolution image processing module to perform low-resolution to high-resolution image processing on the low-resolution pathology image sent in sequence according to the priority from high to low.

[0026] Preferably, the super-resolution image processing module is based on the framework of a generative adversarial network, receives a low-resolution pathological image, and outputs a high-resolution pathological image. The generator of the super-resolution image processing module has no fully connected layer, and includes three parts: an image encoding module, an image decoding module, and an upsampling layer, which are used to convert the input image into an enlarged high-resolution output image; the image encoding module and the image decoding module both use a fully convolutional neural network, and the upsampling layer uses pixel shuffling or transposed convolution to achieve image enlargement; the discriminator of the generator of the super-resolution image processing module uses perceptual cost, pixel cost, and the authenticity cost of the discriminator to evaluate the difference between the generated high-resolution image and the true high-resolution image.

[0027] Preferably, the pathological diagnosis system is composed of an image browsing window and an image reading module;

[0028] The image browsing window is a human-computer interaction interface for users to input different reading instructions and send the reading instructions to the image reading module; and display the pathological image corresponding to the reading instruction fed back by the image reading module; wherein the reading instructions include low-resolution reading instructions and high-resolution reading instructions;

[0029] The image reading module is used to parse the film reading instruction, extract the image status and the corresponding slice number or patient ID or pathological examination ID from the film reading instruction, and encapsulate the image status and the slice number or patient ID or pathological examination ID into a frame and send it to the data retrieval module; the image status includes a low-resolution pathological image and a high-resolution pathological image;

[0030] The data retrieval module is used to parse the image state packaged into frames and the slice number or patient ID or pathological examination ID. When it is determined that the retrieved image is a high-resolution image, it is used to search for the corresponding high-resolution image from the temporary storage module according to the slice number or patient ID or pathological examination ID. If the corresponding high-resolution image is found, the high-resolution image is sent to the image reading module, and the image reading module sends the high-resolution image to the image browsing window for display; if the corresponding high-resolution image is not found, the corresponding low-resolution image is searched from the permanent storage module, and the low-resolution image is fed back to the image reading module; when it is determined that the retrieved image is a low-resolution image, a call is made to search for the corresponding low-resolution image from the permanent storage module, and the low-resolution image is fed back to the image reading module;

[0031] The image reading module is also used to send the low-resolution image that needs to be converted into high-resolution and is fed back by the data retrieval module to the super-resolution image processing module, control the super-resolution image processing module to convert the low-resolution image into a high-resolution image, receive the high-resolution image fed back by the super-resolution image processing module, and send the high-resolution image to the image browsing window for display; the image reading module is also used to send the low-resolution image that needs to be directly displayed and is sent by the data retrieval module directly to the image browsing window for display.

[0032] Preferably, when the pathology diagnosis system is in the high-resolution film reading mode, the high-resolution images that need to be read are stored locally in the terminal, and the high-resolution images are moved locally by using advance download, background download, or data distribution methods of the data storage system, and the pathology ID that needs to be read is allocated to each terminal of the pathology diagnosis system, and the high-resolution images are sent to the film reading terminal in advance;

[0033] When in low-resolution film reading mode, low-resolution images are obtained from the permanent storage of the data storage system; the image blocks in the current field of view are calculated as high-resolution image blocks and displayed in the image browsing window; and the high-resolution image blocks adjacent to the current browsing field of view are calculated by background calculation while reading the film, so as to improve the real-time browsing.

[0034] The present invention has the following beneficial effects:

[0035] 1. The digital pathology system based on low-resolution imaging in the present invention stores pathological images in a low-resolution manner and then uses super-resolution image processing technology to construct high-resolution images when in use. This can reduce the resource consumption caused by image storage while ensuring image quality, and classifies and stores images with different usage frequencies, and stores frequently used super-resolution images separately, which can greatly improve the retrieval speed.

[0036] 2. In the preferred embodiment, the present invention adopts low-resolution imaging technology. Through low-resolution imaging, low-resolution images of pathological sections can be obtained faster, saving the scanning time and storage capacity of the sections. In addition, through three-level storage, permanent and temporary storage, storage costs are further saved. By calculating and obtaining high-resolution images, the needs of low-cost digitization and digital pathology reading are realized, the problem of high cost of current digital pathology is solved, and a preferred solution is provided for digital pathology.

[0037] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0039] Figure 1 A schematic diagram of the structure of a digital pathology technology and its system with high resolution imaging in the prior art;

[0040] Figure 2 A first structural diagram of a digital pathology system based on low-resolution imaging provided by an embodiment of the present invention;

[0041] Figure 3 A second structural diagram of a digital pathology system based on low-resolution imaging provided by an embodiment of the present invention;

[0042] Figure 4 A main flow chart of a digital pathology system based on low-resolution imaging provided by an embodiment of the present invention;

[0043] Figure 5 A simplified structural diagram of a digital pathology imaging system provided by an embodiment of the present invention;

[0044] Figure 6 A simplified structural diagram of a data storage system provided by an embodiment of the present invention;

[0045] Figure 7 A schematic diagram of a management hierarchy table of a data storage system provided by an embodiment of the present invention;

[0046] Figure 8 A simplified structural diagram of a computing system provided by an embodiment of the present invention;

[0047] Fig. 9 A simplified structural diagram of a pathology diagnosis system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0049] Embodiment 1:

[0050] The main technical bottleneck of existing digital pathology technology is high cost. The present invention uses a low-resolution scanner to image the slices and store low-resolution images, such as 5-fold magnified images. When in use, a super-resolution image generation algorithm is used to obtain a high-resolution image (such as 40 times). Since the pixels of the pathology image are reduced to one-sixty-fourth, the storage capacity is reduced to one-sixty-fourth, and the imaging speed is increased by sixty-four times. Therefore, the cost of digital pathology based on low-resolution images is reduced to one-sixty-fourth. In order to realize digital pathology based on low-resolution imaging, the present invention proposes information processing technology for data storage, management and scheduling, reduces the cost of slice digitization, and promotes the construction of digital pathology departments in hospitals.

[0051] Specifically, Figure 2 and Figure 3 As shown, this embodiment proposes a digital pathology system based on low-resolution imaging, including a digital pathology imaging system, a data storage system, a computing system and a pathology diagnosis system, and the digital pathology imaging system, the data storage system, the computing system and the pathology diagnosis system are connected together using a communication network.

[0052] Among them, Figure 4 As shown, the digital pathology imaging system utilizes a low-resolution pathology scanner to obtain low-resolution pathology images of slices, and stores the images after quality control.

[0053] The data storage system is divided into two levels: permanent storage and temporary storage. Permanent storage saves all low-resolution images of the slices and saves them permanently. Temporary storage saves the high-resolution images that need to be used and converted from low-resolution images.

[0054] The computing system extracts the low-resolution image from the storage system, runs the super-resolution image generation method, converts the low-resolution image into a high-resolution image, and saves it to temporary storage for use.

[0055] The pathology diagnosis system can obtain images from permanent storage or temporary storage, and has two functional modes: high-resolution image reading and low-resolution image reading.

[0056] 1. Digital Pathology Imaging System

[0057] like Figure 5As shown, the digital pathology imaging system includes a low-resolution scanner and an image quality control computer. The workflow of the digital pathology imaging system is that the pathology slice is input into the low-resolution pathology scanner. The low-resolution pathology scanner is an imaging device for pathology slices, which is characterized by using a low-magnification lens to scan the pathology slices. Then, a low-resolution photoelectric conversion device is used for photoelectric conversion, and a low-resolution image is obtained after digital sampling and amplitude quantization processing.

[0058] As a preferred solution, the magnification of the low magnification lens is 5 times, so as to be a power of 2 multiple of the commonly used 40 times high resolution image. Multiple low resolution pathology scanners can also be used to increase the image acquisition speed.

[0059] As an alternative to the above approach, the slides can be scanned with a high-resolution scanner and then downsampled to a low-resolution image for storage.

[0060] The low-resolution image is then input into an image quality control computer. The computer runs a super-resolution image processing method to convert it into a high-resolution image. Image quality review uses artificial vision or image analysis technology to automatically detect image quality.

[0061] Image quality review checks the color, blurriness, bubbles, and tissue folding of the image. The automatic review method compares the difference between the color distribution of the standard image and the color distribution of the collected image, such as calculating the mutual information and conditional entropy of the color histograms of the two images. If the threshold is exceeded, the image is considered to be color cast. Operators such as gradient and spatial frequency that characterize the edge of the image are used to check the blurriness of the image. If the threshold is exceeded, the image is considered blurry. Images with bubbles and folds are collected, the positions are manually marked, and a neural network for spatial positioning and recognition is trained. A neural network is used to check whether the image contains bubbles and tissue folds.

[0062] Preferably, the image quality control computer manages the low resolution images using a queue of low resolution images.

[0063] As a preferred solution, the review module can sample a portion of the images for quality inspection to save time. The low-resolution images that pass the inspection are input into the data storage system.

[0064] As a preferred option, the image quality control computer can be replaced by a computing system to simplify the imaging system.

[0065] As a preferred solution, the generated high-resolution image is placed in temporary storage in a data storage system to avoid repeated calculations.

[0066] As a preferred solution, if high-resolution scanning is used, the high-resolution image is placed in the temporary storage of the data storage system, and the downsampled low-resolution image is stored in the near-term storage of the permanent image.

[0067] As a preferred solution, multiple computers can be used as quality control computers to speed up the image quality inspection.

[0068] As a preferred solution, quality correction can be performed while quality control is performed. The generated images can be migrated to the standard image domain using a generative adversarial network model. Optional model structures include the cycleGAN network.

[0069] As a preferred solution, qualified images and unqualified images are used to train a generative adversarial network, and a discriminator is used to identify the images, and images that do not meet the quality requirements are identified as false.

[0070] As a preferred solution, Faster RCNN, CenterNet, YOLO and other positioning networks are used to identify the locations of bubbles and tissue folds.

[0071] As a preferred solution, a neural network based on contrastive learning is used to remove mixed distortion of color and blur at the same time. The advantage of this method is that it does not require paired qualified and unqualified images, and can remove multiple image distortions at the same time.

[0072] 2. Data storage system

[0073] like Figure 6 As shown, the data storage system implements data storage and management. Data storage is divided into permanent storage media and temporary storage media. Permanent storage is divided into three levels of storage: short-term, medium-term and long-term images. Long-term images use low-cost slow and large-capacity access media, while short-term images use high-cost fast access media. Temporary storage media is the fastest storage medium.

[0074] Data storage management is the core function of the system, responsible for data management, including six functional modules: data storage, data scheduling, data retrieval, data generation, data browsing, and data sorting. The data storage module saves images to permanent storage, such as receiving low-resolution images from the digital pathology imaging system and saving them to recent images①. The data scheduling module gradually migrates low-resolution images from recent storage media to long-term storage media according to the time of image acquisition②. The data retrieval module retrieves specified images from storage③. The data generation module inputs low-resolution images into the computing system, retrieves the generated high-resolution images, and puts them into temporary storage④. Data browsing transmits high-resolution images to the pathology diagnosis system to achieve pathology diagnosis or film reading⑤. The data sorting module extracts all pathology images of the same patient from permanent storage and clinical storage based on the patient ID and puts them into the specified path⑥.

[0075] Among them, each data management module is built as a network service to make network calls for the digital pathology imaging system, computing system and diagnostic system.

[0076] The execution steps of the data saving module are: check the patient ID, pathology ID, number and other information of the image; check whether it is a qualified low-resolution image; insert the patient information into the database table; copy the image to the recent storage; save the path of the stored image to the database table.

[0077] The data scheduling module runs when the system is idle. It checks the acquisition time of the images in recent storage, moves the images of discharged patients to medium-term storage, and deletes the high-resolution images of the patient from temporary storage; it checks the acquisition time of the images in medium-term storage, and moves them to long-term storage if the time limit is exceeded.

[0078] The execution steps of the data retrieval module are as follows: check the patient ID, pathology ID, number and other information contained in the retrieval conditions, and the resolution status (high or low). (1) Requirements for high-resolution retrieval: check temporary storage, return the image if it exists; if not, retrieve the low-resolution image from permanent storage; pass the low-resolution image to the computing system to obtain the high-resolution image, save it in temporary storage, and return the high-resolution image. (2) Requirements for low-resolution image retrieval: check permanent storage, obtain the low-resolution image and return the image.

[0079] The execution steps of the data generation module are: extract low-resolution images from permanent storage based on patient ID, pathology ID, number and other information; send them to the computing system to obtain high-resolution images and return the images.

[0080] As a preferred solution, the patient ID can be represented by an identity card.

[0081] The execution steps of the data browsing module are: based on the patient ID, pathology ID, number and other information, obtain the high-resolution image from the temporary storage and return it; if the high-resolution image does not exist, take out the low-resolution image from the permanent storage, calculate and generate the high-resolution image and return the image.

[0082] The data sorting module sorts images according to the patient ID. The execution steps are as follows: obtain all high-resolution images of the ID from the temporary storage and put them into the specified path; retrieve low-resolution images from the permanent storage, input them into the computing system to calculate the high-resolution images, and copy them to the specified path of the temporary storage.

[0083] All pathological images with the same pathological ID obtained by data collation are high-resolution images and are placed in the same path to facilitate management and migration.

[0084] A data distribution module can be added. The execution steps are: for a newly generated pathological diagnosis, if a high-resolution image exists, it is sent from temporary storage to the terminal of the designated pathological diagnosis system in advance; if it is a low-resolution image, it is sent to the computing system to calculate the high-resolution image, and then sent to the terminal of the designated pathological diagnosis system.

[0085] The distribution strategy is predefined, including specifying the number of images and body parts that each terminal is responsible for reading.

[0086] Terminals for pathological diagnosis are classified according to tasks to receive different types of images.

[0087] As a preferred solution, a data mirror is created for permanent storage, and data is saved synchronously in the permanent storage and the mirror.

[0088] As a preferred solution, the high-resolution images generated during the image review phase are stored in temporary storage to avoid repeated calculations.

[0089] As a preferred solution, recent images are images of patients currently in the hospital; mid-term images are images of patients examined within one year; and long-term images are images of patients examined one year ago.

[0090] As a preferred solution, the temporarily stored high-resolution images are deleted when the patient leaves the hospital to reduce storage costs.

[0091] Data storage is managed with patients as the core and is divided into: Figure 7 The three levels shown correspond to three sets of database tables. The patient level includes the patient's ID number and other identity information. The examination level is divided into patient ID, pathology ID and other clinical information. The image level is pathology ID, number, status, storage location path information and image acquisition time. The patient ID distinguishes patients, the pathology ID distinguishes the same pathology examination, and the number records the numbers of multiple slices of a pathology examination. The status indicates whether it is a low-resolution (0) or high-resolution image (1). The type refers to the type of storage medium where the image is saved (permanent or temporary, short-term, medium-term, long-term). The path refers to the path in the medium where the file is saved. The time refers to the time when the image is acquired. Through the three levels of information, the patient ID, pathology ID, and number are linked together to provide a basis for the above six functional modules to process data.

[0092] As a preferred solution, one patient ID corresponds to multiple pathology IDs, and one pathology ID corresponds to multiple serial numbers, which are managed using referential integrity constraints.

[0093] As a preferred solution, temporary storage and permanent storage respectively establish information database tables for the patient layer, examination layer and image layer to facilitate the separate management of the two stored images.

[0094] As a preferred solution, corresponding to the image that is about to enter the data storage, the above three levels of information are first put into the table of the input image queue to wait for processing, so as to support the parallel operation of the data storage system.

[0095] 3. Computing system

[0096] Specifically, Figure 8 As shown, the computing system uses a super-resolution image processing algorithm to convert a low-resolution image into a high-resolution image.

[0097] The data storage system and the pathology diagnosis system input the low-resolution images to be processed into the low-resolution image queues (1 and 2) of the image storage area. The low-resolution image queue records the pathology ID, number, image location and other information of the image. The priority represents the priority of processing, and images of the same priority are processed in the order (serial number) of entering the queue. The generation requirement of the pathology diagnosis system has the highest priority to ensure the real-time generation of the image for the pathologist to browse. The image location is the path where the image is stored in the computing system.

[0098] As a preferred solution, when a task with the highest priority appears, the current computing task should be interrupted to calculate a high-resolution image for the highest priority task.

[0099] As a preferred solution, the image computing task required by the low-resolution film reading mode of the pathology diagnosis system is set as the highest priority task.

[0100] As a preferred solution, the computing system reserves computing power, such as reserving a certain number of parallel computing cards, for the highest priority tasks.

[0101] As a preferred solution, for low-resolution images for daily diagnosis, high-resolution images are calculated in advance and put into temporary storage of the data storage system.

[0102] The super-resolution image processing algorithm is based on the framework of generative adversarial networks, which receives low-resolution pathological images and outputs high-resolution pathological images.

[0103] As a preferred solution, the generator includes three parts: image encoding module, image decoding module and upsampling layer, which converts the input image into an enlarged high-resolution output image. Both the image encoding and decoder use a fully convolutional neural network, and the upsampling layer uses pixel shuffling or transposed convolution to achieve image enlargement.

[0104] As a preferred solution, the main feature of the generator is that it has no fully connected layers, so it can accept low-resolution pathological images of any size, such as full-slice images or image blocks.

[0105] As a preferred solution, the algorithm is trained with a dataset of low-resolution and high-resolution images, and a discriminator is used to judge the authenticity of the generated image. The difference between the generated high-resolution image and the real high-resolution image is evaluated together with the perceptual cost, pixel cost, and authenticity cost of the discriminator. After the generator training is completed, it is used as a super-resolution image processing algorithm to generate high-resolution images from low-resolution pathological images.

[0106] As a preferred solution, a computing system cluster is formed by using multiple computing cards or multiple servers to process the input high-resolution image in parallel, generate high-resolution image blocks and splice them into high-resolution images to improve the image processing speed.

[0107] The super-resolution image processing algorithm reads images from the low-resolution image queue according to priority and sequence number, and deletes the image information in the low-resolution image queue. Then it generates a high-resolution image and saves it in the high-resolution image queue for reading by the data storage system or pathology diagnosis system.

[0108] The data storage system reads the high-resolution image queue, deletes the image information from the queue, and saves the image in temporary storage for standby. The pathology diagnosis system reads the high-resolution image queue, deletes the image information from the queue, and displays the image on the display screen of the pathology diagnosis system.

[0109] 4. Pathological diagnosis system

[0110] Specifically, Fig. 9 As shown, the pathology diagnosis system consists of an image browsing window and an image reading module. Among them, the image browsing window displays high-resolution pathology images, and can zoom in and out magnification, move the observation position and other routine pathology browsing and fill in diagnostic reports and other functions. The image reading module is responsible for generating the image required by the browsing window①. When the browsing window needs an image, the image reading module calls the data retrieval module② of the data storage management. The data retrieval module searches for temporary storage③ based on information such as the pathology ID and number, and if a high-resolution image is found, it is transmitted to the image reading module. If not found, a low-resolution image is obtained from permanent storage④. The retrieved image is sent to the image reading module⑤. After the image reading module obtains the image, if it is a high-resolution image, it is sent to the image browsing window⑥. If it is a low-resolution image, it is input to the computing system to generate a high-resolution image (high priority)⑦. The generated high-resolution image is sent to the image browsing window⑧.

[0111] As a preferred solution, the pathology diagnosis system has two modes: high-resolution film reading and low-resolution film reading.

[0112] As a preferred solution, the high-resolution film reading mode is mainly used for daily diagnosis, and the high-resolution images that need to be read are saved locally on the terminal.

[0113] As a preferred solution, in high-resolution film reading mode, high-resolution images are moved locally by advance downloading, background downloading, or data distribution functions of the data storage system, and are deleted after the film reading is completed.

[0114] As a preferred solution, in high-resolution film reading mode, a pathology ID that needs to be read is allocated to each terminal of the pathology diagnosis system, and high-resolution images are sent to the film reading terminal in advance.

[0115] As a preferred option, the low-resolution film reading mode is suitable for scenarios where high-resolution images cannot be prepared in advance, such as retrospective film reading and scientific research.

[0116] As a preferred solution, in the low-resolution film reading mode, the low-resolution image is obtained from the permanent storage of the data storage system, and the image block of the current field of view is calculated as a high-resolution image block and displayed in the image browsing window.

[0117] As a preferred solution, in the low-resolution film reading mode, a background calculation method is used to calculate the high-resolution image blocks adjacent to the current browsing field of view while reading the film, so as to improve the real-time browsing.

[0118] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A digital pathology system based on low-resolution imaging, characterized in that: include: A digital pathology imaging system, a data storage system, a computing system and a pathology diagnosis system, wherein the data storage system establishes communication with the digital pathology imaging system, the computing system and the pathology diagnosis system respectively; The digital pathology imaging system is used to obtain a low-resolution pathology image of a slice and its slice number, and send the low-resolution pathology image and its slice number to the data storage system; The data storage system comprises a temporary storage module and a permanent storage module, which is used to store the low-resolution pathological image in the permanent storage module; The computing system is used to call the low-resolution pathology image from the permanent storage module, convert the low-resolution pathology image into a high-resolution pathology image, and send the high-resolution pathology image to the temporary storage module of the data storage system for storage; The pathological diagnosis system is used to call the low-resolution pathological image from the permanent storage module for display or call the high-resolution pathological image from the temporary storage module for display; Also includes a data management module, the data management module is used to construct and store a mapping table between the patient's ID number and other identity information, the data management module is also used to construct and store a mapping table between the patient ID, pathology ID and other clinical information, the data management module is also used to construct a mapping table between the pathology image slice number, pathology ID, status, storage location information and image acquisition time; The permanent storage module includes a plurality of different storage units; the data management module is used to call the mapping table according to the slice number of the low-resolution pathology image to query the patient information corresponding to the low-resolution pathology image, and estimate the usage frequency of the low-resolution pathology image according to the patient information, and schedule or migrate different low-resolution pathology images to different storage units according to the usage frequency; The computing system is also connected to the pathological diagnosis system; the pathological diagnosis system is also used to send low-resolution pathological images to the computing system, and the computing system includes: a priority judgment module and a super-resolution image processing module; The priority judgment module is used to judge the processing priority of the low-resolution pathology image sent, and control the super-resolution image processing module to perform low-resolution to high-resolution image processing on the low-resolution pathology image sent in sequence according to the priority from high to low. The super-resolution image processing module is based on the framework of the generative adversarial network, receives the low-resolution pathology image, and outputs the high-resolution pathology image.

2. The digital pathology system based on low-resolution imaging according to claim 1, characterized in that: The digital pathology imaging system comprises: a low-resolution pathology scanner and a quality review module; The low-resolution pathology scanner is used to scan the low-resolution pathology image of the pathology section and send the low-resolution pathology image to the quality review module; the quality review module is used to detect the color, fuzziness, bubbles and tissue folding of the low-resolution pathology image, and compare the color, fuzziness, bubbles and tissue folding of the low-resolution pathology image with the color, fuzziness, bubbles and tissue folding of the standard image to determine whether the low-resolution pathology image is a standard image. When the low-resolution pathology image is determined to be a standard image, the low-resolution pathology image is sent to the data storage system for storage.

3. The digital pathology system based on low-resolution imaging according to claim 1, characterized in that: The multiple different storage units include a recent storage unit, a mid-term storage unit and a long-term storage unit. The recent storage unit is used to store low-resolution pathology images with a high usage frequency, specifically low-resolution pathology images of patients currently in the hospital; the mid-term storage unit is used to store low-resolution pathology images with a medium usage frequency, specifically low-resolution pathology images of patients examined within one year; and the long-term storage unit is used to store low-resolution pathology images with a low usage frequency, specifically patients examined one year ago.

4. The digital pathology system based on low-resolution imaging according to claim 3, characterized in that: The data management module also includes: a data storage module, a data scheduling module, a data retrieval module, a data generation module, a data browsing module and a data sorting module; The data storage module is used to store the low-resolution images sent by the digital pathology imaging system in a recent storage unit; The data scheduling module is used to query the patient information of each low-resolution pathological image in the recent storage unit, and determine whether the usage frequency of each low-resolution pathological image is high based on the patient information of each low-resolution pathological image. When it is found that the usage frequency of a low-resolution pathological image is medium or low, the low-resolution pathological image is migrated from the recent storage unit to the medium-term storage unit or the long-term storage unit; The data scheduling module is used to query the patient information of each low-resolution pathological image in the medium-term storage unit, and determine whether the usage frequency of each low-resolution pathological image is low based on the patient information of each low-resolution pathological image. When it is found that the usage frequency of a low-resolution pathological image is low, the low-resolution pathological image is migrated from the medium-term storage unit to the long-term storage unit; The data retrieval module is used to call the mapping table according to the query instruction to retrieve the specified low-resolution pathological image from the multiple different storage units; The data generation module is used to send the corresponding low-resolution pathological image to the computing system according to the resolution conversion instruction, receive the high-resolution image fed back by the computing system, and store the high-resolution image in the temporary storage module; The data browsing module is used to call the corresponding high-resolution image from the temporary storage module according to the browsing instruction, and send it to the pathology diagnosis system for the user of the pathology diagnosis system to perform pathology diagnosis or image reading; The data sorting module is used to extract all pathological images of the same patient from the permanent storage module and the clinical storage module according to the patient ID, and put them into a designated path.

5. The digital pathology system based on low-resolution imaging according to claim 1, characterized in that: The generator of the super-resolution image processing module has no fully connected layer, and includes three parts: an image encoding module, an image decoding module and an upsampling layer, which are used to convert an input image into an enlarged high-resolution output image; the image encoding module and the image decoding module both use a fully convolutional neural network, and the upsampling layer uses pixel shuffling or transposed convolution to achieve image enlargement; the generator of the super-resolution image processing module uses perceptual cost, pixel cost and the authenticity cost of the discriminator to evaluate the difference between the generated high-resolution image and the real high-resolution image.

6. The digital pathology system based on low-resolution imaging according to claim 5, characterized in that: The pathological diagnosis system is composed of an image browsing window and an image reading module; The image browsing window is a human-computer interaction interface for users to input different reading instructions and send the reading instructions to the image reading module; and display the pathological image corresponding to the reading instruction fed back by the image reading module; wherein the reading instructions include low-resolution reading instructions and high-resolution reading instructions; The image reading module is used to parse the film reading instruction, extract the image status and the corresponding slice number or patient ID or pathological examination ID from the film reading instruction, and encapsulate the image status and the slice number or patient ID or pathological examination ID into a frame and send it to the data retrieval module; the image status includes a low-resolution pathological image and a high-resolution pathological image; The data retrieval module is used to parse the image state packaged into frames and the slice number or patient ID or pathological examination ID. When it is determined that the retrieved image is a high-resolution image, it is used to search for the corresponding high-resolution image from the temporary storage module according to the slice number or patient ID or pathological examination ID. If the corresponding high-resolution image is found, the high-resolution image is sent to the image reading module, and the image reading module sends the high-resolution image to the image browsing window for display; if the corresponding high-resolution image is not found, the corresponding low-resolution image is searched from the permanent storage module, and the low-resolution image is fed back to the image reading module; when it is determined that the retrieved image is a low-resolution image, a call is made to search for the corresponding low-resolution image from the permanent storage module, and the low-resolution image is fed back to the image reading module; The image reading module is also used to send the low-resolution image that needs to be converted into high-resolution and is fed back by the data retrieval module to the super-resolution image processing module, control the super-resolution image processing module to convert the low-resolution image into a high-resolution image, receive the high-resolution image fed back by the super-resolution image processing module, and send the high-resolution image to the image browsing window for display; the image reading module is also used to send the low-resolution image that needs to be directly displayed and is sent by the data retrieval module directly to the image browsing window for display.

7. The digital pathology system based on low-resolution imaging according to claim 6, characterized in that: When the pathology diagnosis system is in the high-resolution film reading mode, the high-resolution images that need to be read are stored locally in the terminal, and the high-resolution images are moved locally by using advance download, background download, or data distribution methods of the data storage system, and the pathology ID that needs to be read is allocated to each terminal of the pathology diagnosis system, and the high-resolution images are sent to the film reading terminal in advance; When in low-resolution film reading mode, low-resolution images are obtained from the permanent storage of the data storage system; the image blocks in the current field of view are calculated as high-resolution image blocks and displayed in the image browsing window; and the high-resolution image blocks adjacent to the current browsing field of view are calculated by background calculation while reading the film, so as to improve the real-time browsing.

Images