Decentralized deployment method and system for digital pathological section scanner
By deploying digital pathology slide scanners in a decentralized manner and utilizing shared storage and virtual IP technology, resource load balancing is achieved, solving the problems of high bandwidth, heavy data management pressure, security risks, and uneven resource allocation caused by centralized deployment. This improves data transmission efficiency and security and supports flexible expansion of data storage within the hospital.
Patent Information
- Application Number
- CN202411188747.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-22
AI Technical Summary
The centralized deployment of existing digital pathology slide scanners leads to high bandwidth requirements, heavy data management pressure, numerous security risks, and uneven resource allocation, affecting the efficiency of remote diagnosis and data security.
A decentralized deployment approach is adopted, using shared storage and virtual IP technology to achieve dynamic load balancing of the scanner cluster. Multiple hosts are used to collaboratively process tasks, reducing data uploads, ensuring local data storage and point-to-point preview, and preventing data leakage.
It improves the efficiency and quality of uploading digital pathology slides, reduces bandwidth requirements and storage costs, ensures data security and resource balance, supports flexible expansion, and adapts to the data storage needs within hospitals.
Smart Images

Figure CN119007948A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of digital pathology slices, and in particular to a decentralized deployment method and system for a digital pathology slice scanner. Background Art
[0002] There are multiple digital pathology slide scanners with different configurations in the hospital. Currently, all digital pathology slide scanners are deployed in a centralized manner: the digital pathology slide scanners in the hospital are placed in a specific location and connected to multiple terminals or workstations through the network. Although this method can achieve effective resource utilization and centralized data management, there are many problems in actual work:
[0003] 1. High bandwidth requirements: When remote diagnosis is required or in-hospital slices need to be called, digital pathology slices need to be uploaded to the central storage server. However, due to the large amount of data in digital pathology slices, the capacity occupied by a single slice ranges from several hundred MB to more than ten GB. In an ordinary tertiary hospital, the number of slices per day is about 2,000-3,000. If all the data is uploaded to the central storage server, the daily upstream traffic of the uploaded data alone is about 1TB, which puts high demands on broadband requirements. In some cases, the scanned slices need to be uploaded immediately. For example, when remote frozen diagnosis (report within 30 minutes) is required, it is also necessary to prevent other slice uploads from affecting the efficiency of the remote real-time diagnosis, which makes the bandwidth requirements even more stringent.
[0004] 2. High pressure on data management: The centralized deployment method requires that all digital pathology slide data be stored in a single location. This means that when users need to access the slide data, the central storage server needs to spend a lot of energy to locate the slide data, resulting in a certain delay in the user's query work, and the need to add additional business servers to manage the slide data and decode the digital pathology slides.
[0005] 3. Security risks exist: In some diagnostic scenarios, such as remote diagnosis scenarios, pathology slice data may need to be uploaded to a third-party platform. However, the patient's digital pathology slices are the patient's private data, and uploading them to a third-party platform poses a risk of data leakage.
[0006] 4. Uneven resource allocation: Since the usage frequency of different scanners in the hospital varies according to actual conditions, the matching scanner host uses uneven resources. Scanners with low usage frequency and their related host resources will be wasted, but scanners with high usage frequency may be overloaded and have to wait in line.
[0007] In summary, the current centralized deployment of digital pathology slices in digital pathology slice scanners has many of the practical problems mentioned above. Summary of the invention
[0008] The embodiment of the present application provides a decentralized deployment method and system for a digital pathology slice scanner, which adopts a decentralized storage deployment method to replace the traditional centralized deployment method, and dynamically balances the load resources of different scanners. It adopts scanner clusters, resource load dynamic balancing algorithms and virtual IP technology to improve the efficiency and quality of uploading digital pathology slices.
[0009] In a first aspect, an embodiment of the present application provides a method, which includes:
[0010] In a second aspect, an embodiment of the present application provides a device, including:
[0011] In a third aspect, an embodiment of the present application provides a kind of
[0012] In a fourth aspect, an embodiment of the present application provides a kind of.
[0013] The main contributions and innovations of the present invention are as follows:
[0014] 1. Decentralized deployment: This solution uses a decentralized cluster deployment of the digital pathology slide scanner storage hard disk or a mounted single file storage system, and unified management and access through shared storage and virtual IP, making full use of the hardware resources of different scanners and saving the cost of central storage servers, business servers, and traffic and reducing bandwidth requirements, achieving instant storage and instant remote diagnosis, while avoiding the risk of single point failure and effectively avoiding the bandwidth requirements of a single center management server transmission.
[0015] 2. Dynamic balancing of resource load: Through heartbeat detection, the current resource usage of different scanners is automatically obtained, and priority scheduling algorithms and load balancing algorithms are defined to dynamically allocate resources based on actual conditions, thereby achieving dynamic resource balancing while making good use of spare resources.
[0016] 3. Improve transmission efficiency and security: The scanner of this solution only needs to store the slice data locally and preview it remotely in a point-to-point manner, which saves the time of data upload and reduces the network bandwidth requirements. When diagnosing remotely on the external network, there is no need to upload slices and forward data, and point-to-point instant preview is achieved, saving traffic and storage costs. In addition, the hospital does not need to upload data to a third-party platform to ensure that there is no risk of data leakage.
[0017] 4. Improve expansion flexibility: Scanners can be dynamically connected or deleted. The decentralized deployment solution can be dynamically expanded according to demand, supporting a single scanner to independently access file storage and tape library archiving, seamlessly increase capacity and linearly improve performance, thereby adapting to the growing data storage needs within the hospital.
[0018] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0020] Figure 1 is a flow chart of a decentralized deployment method of a digital pathology slide scanner according to an embodiment of the present application;
[0021] Figure 2 is a structural block diagram of a decentralized deployment system of a digital pathology slide scanner according to an embodiment of the present application;
[0022] Figure 3 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.
[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.
[0025] Embodiment 1
[0026] like Figure 1As shown, this solution provides a decentralized deployment method for a digital pathology slide scanner, aiming to achieve efficient and secure data processing and management of multiple digital pathology slide scanners through a decentralized architecture, including the following steps:
[0027] Deploy at least two hosts connected to the same switch, and deploy each digital pathology slide scanner connected to multiple hosts and each host also connected to multiple scanners;
[0028] Deploy a shared service layer in a containerized manner on each host, where the shared service layer is used to monitor the host status of each host;
[0029] Deploy a shared buffer pool, where each host accesses the shared buffer pool in the form of a shared directory, and the shared buffer pool caches image data;
[0030] Whenever any digital pathology slice scanner scans a digital pathology slice, the scanning task is divided into subtasks according to the task process and stored in the task scheduling queue, and the image data of the digital pathology slice is stored in the shared buffer pool. The shared service layer sets a label for the image data in the shared buffer pool based on the host status and task scheduling queue of each host and notifies the corresponding host, wherein the label records the subtask of the current image data and the assigned host, and the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask. It should be noted that this scheme proposes a decentralized deployment scheme specifically for digital pathology slice scanners. Different from the traditional centralized deployment of each digital pathology slice scanner, and also different from the method of connecting each digital pathology slice scanner to a single host, this scheme deploys a decentralized architecture in which multiple hosts are cross-connected with multiple digital pathology slice scanners. The advantage of this setting is that each digital pathology slice scanner can be connected to multiple hosts, and each host can also process the digital pathology slice data uploaded by multiple digital pathology slice scanners, thereby supporting task allocation and resource sharing across hosts. Each time the digital pathology slice scanner scans a digital pathology slice, only one image data is generated. However, the scanning task is divided into multiple subtasks including camera acquisition, distortion correction, white balance, light intensity correction, image stitching, image fusion, sharpening, encoding and other processes. Each subtask is placed in the task scheduling queue. The advantage of this is that the significance of cooperating with the shared service layer is that the resources of each host can be fully scheduled. For example, when a host processes a subtask, the first step of the task uses the CPU, and the second step of the task uses the GPU. When the second step of the task is executed, the CPU of the current host is in an idle state. At this time, the design of the shared service layer can be used to reasonably schedule the CPU of the current host.
[0031] In the step of "deploying at least two hosts connected to the same switch", each host is connected to the same switch through multiple network cards, which can implement link aggregation or similar load balancing technologies to increase the total bandwidth capacity, thereby significantly improving data processing speed and efficiency. Moreover, even if a network card or link fails, the system can automatically switch to another available network card or link to ensure communication continuity, improving the reliability and stability of the system.
[0032] In the step of "deploying each digital pathology slide scanner to connect to multiple hosts and each host to connect to multiple scanners", this solution designs a way for multiple digital pathology slide scanners and multiple hosts to work together, so that multiple hosts can collaboratively process the image data and scanning tasks of the digital pathology slide scanners, while avoiding the situation where a single digital pathology slide scanner is overloaded or too idle.
[0033] In the step of "Deploy the shared service layer in a containerized manner on each host", the shared service layer is deployed and the software and operating environment of the shared service layer are packaged into containers, and the containers are deployed on each host. Specifically, a Dockerfile or other container build file can be written to create a service layer image of the shared service layer, and the container can be installed on each host and the service layer image can be pulled to realize the sharing of the shared service layer. The advantage of this is that it is easy to manage and update the shared service layer, and it can provide isolation and resource control.
[0034] In some embodiments, the host status monitored by the shared service layer includes the host online status and the host resource usage, wherein the host resource usage includes one or any combination of the host CPU, host memory, and host storage capacity.
[0035] The shared service layer of this solution periodically sends heartbeat signals to the host and obtains the online status of the host based on the received heartbeat signals, where the online status is online or offline, in order to ensure the stability and reliability of the cluster. Specifically, the shared service layer of this solution periodically sends heartbeat signals to each host. After receiving the heartbeat signal, the host needs to respond to the confirmation signal immediately. If the shared service layer obtains the confirmation signal, the online status of the current host is judged to be "online". If not, the online status of the current host is judged to be "offline". When the shared service layer detects that a host is offline, it will reallocate the tasks originally assigned to the host to other online hosts, thereby ensuring the continuity of data processing tasks and the overall performance of the cluster.
[0036] In some embodiments, the shared service layer monitors the online status of each host and the usage of host resources through the Socket communication protocol.
[0037] In addition, in some embodiments, the shared service layer provided by the present solution sets the same virtual IP for all digital pathology slicers, which has the advantage that the internal IP of each digital pathology scanner can be accurately known.
[0038] In the "deploy shared buffer pool" step, each host deploys the shared buffer pool by sharing the disk, and each host can access the shared buffer pool by sharing the directory. The shared buffer pool provided by this solution can store the original image data of digital pathology slices collected by the digital pathology slicer, and the shared disk method can provide efficient read and write performance and data consistency, ensuring that each host can access and process the image data in the shared buffer pool in a timely manner, and also ensuring the immediate availability of image data.
[0039] Since this solution is a decentralized deployment solution, it can configure the digital pathology slicer with erasure coding, multiple copies or shared directory according to different actual usage scenarios. In other words, this solution supports the dynamic addition or deletion of digital pathology slicers deployed in the current decentralized architecture. Each digital pathology slicer can be flexibly configured and scheduled as an independent service unit according to demand. Of course, this method can also improve data security and reliability.
[0040] It should be noted that the shared disk mentioned in this solution refers to one or more physical disks or logical disks that can be accessed by multiple hosts. It can be provided by a dedicated storage device or built through software-defined storage. A shared directory refers to a folder or directory located on a shared disk, which can be accessed by multiple hosts in the network through a standard file system protocol.
[0041] The digital pathology slicer of this solution can cache the image data of the collected digital pathology slices into a shared buffer pool when scanning digital pathology slices. Since each digital pathology slice is composed of multiple tiles, this solution caches the image data of different tiles into the shared buffer pool, and each image data corresponds to a tile image.
[0042] In addition, each scanning task can be divided into multiple functional processes, including camera acquisition, distortion correction, white balance, light intensity correction, image stitching, image fusion, sharpening, and encoding. This solution divides the functional process of each scanning task into a subtask and stores it in the task scheduling queue. Correspondingly, in the step of "dividing the scanning task into subtasks according to the task process and storing them in the task scheduling queue", for each scanning task of scanning digital pathology sections, the scanning task is divided into: camera acquisition of images, light intensity correction, distortion correction, image cropping, image stitching, image fusion, image sharpening, and multiple subtasks of AI analysis, and the subtasks are stored in the task scheduling queue.
[0043] In the step of "the shared service layer sets labels for the image data in the shared buffer pool based on the host status and task scheduling queue of each host and notifies the corresponding host", the shared service layer obtains the "online" host according to the host online status, and sets labels for the image data in the shared buffer pool based on the host resource usage of the "online" host and the unfinished subtasks in the task scheduling queue and notifies the corresponding host. The advantage of this is that the scanning tasks of the same digital pathology slicer can be assigned to different hosts for execution, and the resource usage of each host can be taken into account, the task allocation strategy can be dynamically adjusted, resource usage can be optimized, and overload or idle resources of certain hosts can be avoided.
[0044] In some embodiments, the image data in the shared buffer pool is tagged based on the host resource utilization weight of the "online" hosts. Generally speaking, the host with the lowest host resource utilization is selected based on the host resource utilization of the "online" hosts, and the unfinished subtask is assigned to the current host.
[0045] For example: a scanner is connected to multiple hosts, and the entire scanning task process of the scanner is: camera acquisition of images, light intensity correction, distortion correction, image cropping, image stitching, image fusion, image sharpening, and AI analysis and other functional processes. If only a single host is used to handle the scanning task, if each functional process is serial, then each line of data scanned must wait until these data are completely processed before entering the next line of data scanning. However, this solution adopts the method of multiple hosts corresponding to one scanner, and the subtasks of each functional process can be split to run on each host, and the scanner scans without waiting for the processing to be completed before entering the next line. For example, the label set for image data by the shared service layer of this solution is: "The subtask of the current image data is image fusion, and the assigned host is: Host A", then notify Host A to read the image data from the shared buffer pool and execute the subtask of image fusion.
[0046] In order to facilitate the management of task execution status of different hosts, this solution includes the following steps: when the host completes the assigned subtask, it will feedback the subtask processing status to the shared service layer, and update the task processing status of the subtask in the task scheduling queue. In other words, when the host completes the corresponding subtask, it will send a task completion notification to the shared service layer and update the task status in the task scheduling queue. It should be noted that the task processing status of the subtask recorded in the task scheduling queue is "processed" and "unprocessed". The shared service layer selects the subtask with a task processing status of "unprocessed" and sets a label for the image data based on the subtask and host status.
[0047] In addition, in order to manage the digital pathology slices scanned by different digital pathology slice scanners, whenever any digital pathology slice scanner scans a digital pathology slice, the database of the shared service layer records the slice information of the current digital pathology slice and performs an MD5 check on the current digital pathology slice, wherein the slice information includes the file information, file path, and file status of the digital pathology slice. The file information of the digital pathology slice includes the creation time, file name, file format, and size of the digital pathology slice. The file path is the storage path of the image data of the digital pathology slice in the shared buffer pool. The file status marks the processing status of the current digital pathology slice, and the processing status includes one of unprocessed, in process, and processed. It should be noted that the file status recorded in the database of the shared service layer will be updated according to the actual processing situation. In some embodiments, the shared service layer can track and record the status of the current digital pathology slice in the task scheduling queue.
[0048] In some embodiments, the shared service layer generates a unique MD5 checksum corresponding to the current digital pathology slice to implement MD5 verification of the current digital pathology slice. To ensure the consistency of the digital pathology slice and prevent repeated uploading, the shared service layer generates an MD5 verification code for the digital pathology slice and stores it in the database. If the same file is uploaded again in the future, the comparison of the MD5 verification code can quickly identify duplicate files and avoid redundant data. The MD5 checksum is a string that uniquely identifies the digital pathology slice. The MD5 checksum is used to avoid repeated uploading of the same digital pathology slice.
[0049] In the step of "the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask", when the host needs to perform decoding services for digital pathology slices scanned by different digital pathology slice scanners, the decentralized deployment method of the digital pathology slice scanner includes the following steps:
[0050] Setting up a temporary memory for storing image data and a hard disk for buffering decoded image data;
[0051] When a decoding request is obtained, the tile of the digital pathology slice requested to be decoded in the deployment shared buffer pool is located based on the decoding request, the image data of the current tile is retrieved and loaded into the temporary memory and locked, the image data in the temporary memory is decoded to obtain the decoded data and the decoded data is buffered to the hard disk;
[0052] The host reads the corresponding decoded data of the digital pathology slice image data from the hard disk and executes the subtask.
[0053] In some embodiments, when the image data in the temporary memory is released, the decoded data in the hard disk is returned to the host, and the host reads the decoded data of the corresponding digital pathology slice image data from the hard disk and executes the subtask.
[0054] This solution sets up a multi-level buffering method for temporary memory and hard disk to ensure that the image data only needs to be decoded once and then cached in temporary memory. After locating the digital pathology slice requested for decoding again, it will no longer be decoded by the CPU or GPU to save resources. In addition, since the amount of slice data of digital pathology slices is large, it does not need to upload the digital pathology slices to the central management server for storage and other users can remotely and instantly access the digital pathology slices. Exemplarily, if the image data in the shared buffer pool is a jpeg image, subtasks such as decoding, cropping, and light intensity correction need to be performed for the jpeg image. These subtasks all need to decode the jpeg image data into RGB decoded data. When different subtasks need to use this original data, they only need to read the RGB decoded data in the hard disk, without having to go through the jpeg decoding process multiple times.
[0055] It should be noted that when the user is previewing the digital pathology slice, since each digital pathology slice includes many tiles, each tile needs to be decoded and color corrected and adjusted; and when the user previews the digital pathology screen, the user often drags the mouse on the same digital pathology slice or the mouse shakes unintentionally, which leads to continuous sending of decoding requests for the same digital pathology slice to the system, resulting in a waste of decoding resources. This solution avoids the problem of repeated decoding by storing tiles in temporary memory and locking them when obtaining the user's decoding request for a certain digital pathology slice. The "decoding request" referred to in this solution refers to the corresponding subtask that needs to process the decoded image data. In other words, when the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask, if the current subtask is a subtask that needs to decode the decoded image data, a corresponding decoding request is generated.
[0056] Specifically, in the "get decoding request" step, when the user's decoding instruction stays in the area of the digital pathology slice, a decoding request is generated, wherein the decoding request includes a request instruction and a decoding instruction staying in the area of the current digital pathology slice. According to the stop position of the current digital pathology slice, the digital pathology slice that needs to be processed can be known. The shared service layer sets a label for the image data of the current digital pathology slice in the shared buffer pool based on the host status and task scheduling queue of each host and notifies the corresponding host. The host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask. However, in order to further reduce resource utilization during the execution process, this solution sets temporary memory and hard disk to cache the decoding data of the current digital pathology slice image data, so that the host can directly read the decoding data from the hard disk and execute the subtask.
[0057] In some embodiments, the user's decoding instruction refers to the user's mouse. In this solution, when the user's mouse stays on the digital pathology slice, a corresponding decoding request is generated. In the step of "locating the tile of the digital pathology slice requested to be decoded based on the decoding request", the tile information of the tile is marked according to the area of the current digital pathology slice where it is stopped, wherein the tile information records the digital pathology slice to which the current tile belongs, the level and position in the digital pathology slice, and the MD5 checksum of the digital pathology slice to which it belongs, and the tile of the digital pathology slice requested to be decoded is located in the shared buffer pool according to the tile information. In other words, after obtaining the decoding request, this solution will mark the tile and record the tile information of the current tile to ensure that the background service can accurately locate and process the tile requested by the user.
[0058] In the step of "calling the image data of the current tile to load into the temporary memory and lock it", the image data of the current tile is loaded into the temporary memory and locked. The advantage of this is to ensure that the image data of the tile will not be frequently decoded during user browsing.
[0059] In the step of "decoding the image data in the temporary memory to obtain decoded data and buffering the decoded data to the hard disk", the decoded data obtained by decoding the image data is cached in the hard disk at the same time, and the hard disk serves as a backup storage for the temporary memory. It should be noted that if multiple decoding requests for the same tile are generated in a short period of time, since the image data of the tile has been locked, the image data of the current tile can be directly obtained from the temporary memory and decoded to avoid repeated decoding.
[0060] In some embodiments, the present solution selects an NVMe SSD as a hard disk for buffering decoded image data.
[0061] In the step of "decoding the image data in the temporary memory to obtain decoded data", the image data is decoded to obtain decoded data that can be processed by the subtask.
[0062] Since the decentralized deployment method of the digital pathology slide scanner provided by this solution can greatly alleviate the pressure of data storage and decoding, this solution has very good applications in scenarios such as remote frozen diagnosis (reports need to be issued within 30 minutes) or other scenarios that require rapid preview of digital pathology slides. The following is an example of a remote frozen diagnosis scenario:
[0063] (1) Hospital A establishes signaling with remote user B, obtains the IP and port of the public network of both parties, issues signaling information to the other party and establishes a connection between the two parties; (2) After B obtains the slice name of the digital pathology slice to be accessed, it initiates a request to hospital A to obtain the digital pathology slice. (3) When previewing the digital pathology slice, B immediately sends the tile of the digital pathology slice to be previewed to hospital A. After receiving the information, hospital A uses the task scheduling service to call a single or multiple digital pathology slice scanners with low resource usage to decode the slice, and encrypts the tile and returns it to B, so that A and B can transmit data slice data point-to-point and preview it in real time, avoiding the upload and download delay problems of the central storage server.
[0064] Embodiment 2
[0065] Based on the same idea, refer to Figure 2 , the application also proposes a decentralized deployment system for a digital pathology slide scanner, including:
[0066] A decentralized deployment architecture, wherein at least two hosts connected to the same switch are deployed in the decentralized deployment architecture, and each digital pathology slide scanner is deployed to connect multiple hosts and each host is also connected to multiple scanners; a shared service layer is deployed on each host in a containerized manner, wherein the shared service layer is used to monitor the host status of each host; a shared buffer pool is deployed, wherein each host accesses the shared buffer pool in a shared directory manner, and the shared buffer pool caches image data;
[0067] The data processing system is used for dividing the scanning task into subtasks according to the task process and storing them in the task scheduling queue whenever any digital pathology slice scanner scans the digital pathology slice, storing the image data of the digital pathology slice in the shared buffer pool, and the shared service layer sets a label for the image data in the shared buffer pool based on the host status of each host and the task scheduling queue and notifies the corresponding host, wherein the label records the subtask of the current image data and the assigned host, and the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask.
[0068] Of course, in some embodiments, the decentralized deployment system of the digital pathology slice scanner includes: a decoding system, which is used to set a temporary memory for storing image data and a hard disk for buffering decoded image data; when a decoding request is obtained, the tile of the digital pathology slice requested to be decoded in the deployment shared buffer pool is located based on the decoding request, the image data of the current tile is retrieved and loaded into the temporary memory and locked, the image data in the temporary memory is decoded to obtain the decoded data and the decoded data is buffered to the hard disk; the host reads the decoded data of the corresponding digital pathology slice image data from the hard disk and executes the subtask. .
[0069] The contents of the second embodiment that are the same as those of the first embodiment will not be described again.
[0070] Embodiment 3
[0071] This embodiment also provides an electronic device, referring to Figure 3 , including a memory 304 and a processor 302, wherein the memory 304 stores a computer program, and the processor 302 is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the decentralized deployment method of the digital pathology slide scanner.
[0072] Specifically, the processor 302 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0073] Among them, the memory 304 may include a large capacity memory 304 for data or instructions. By way of example and not limitation, the memory 304 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 304 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 304 may be inside or outside the data processing device. In a specific embodiment, the memory 304 is a non-volatile memory. In a specific embodiment, the memory 304 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (Static Random-Access Memory, abbreviated as SRAM) or a dynamic random access memory (Dynamic Random Access Memory, abbreviated as DRAM), wherein the DRAM can be a fast page mode dynamic random access memory 304 (Fast Page Mode Dynamic Random Access Memory, abbreviated as FPMDRAM), an extended data output dynamic random access memory (Extended Date Out Dynamic Random Access Memory, abbreviated as EDODRAM), a synchronous dynamic random access memory (Synchronous Dynamic Random-Access Memory, abbreviated as SDRAM), etc.
[0074] The memory 304 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 302 .
[0075] The processor 302 reads and executes the computer program instructions stored in the memory 304 to implement any one of the decentralized deployment methods of the digital pathology slide scanner in the above embodiments.
[0076] Optionally, the electronic device may further include a transmission device 306 and an input / output device 308 , wherein the transmission device 306 is connected to the processor 302 , and the input / output device 308 is connected to the processor 302 .
[0077] The transmission device 306 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 306 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0078] The input / output device 308 is used to input or output information. In this embodiment, the input information may be a decoding request, etc., and the output information may be a digital pathology slice, etc.
[0079] Optionally, in this embodiment, the processor 302 may be configured to perform the following steps through a computer program:
[0080] Deploy at least two hosts connected to the same switch, and deploy each digital pathology slide scanner connected to multiple hosts and each host also connected to multiple scanners;
[0081] Deploy a shared service layer in a containerized manner on each host, where the shared service layer is used to monitor the host status of each host;
[0082] A shared buffer pool is deployed, in which each host accesses the shared buffer pool in the form of a shared directory, and the shared buffer pool caches image data; whenever any digital pathology slice scanner scans a digital pathology slice, the scanning task is divided into subtasks according to the task process and stored in the task scheduling queue, and the image data of the digital pathology slice is stored in the shared buffer pool. The shared service layer sets a label for the image data in the shared buffer pool based on the host status of each host and the task scheduling queue and notifies the corresponding host, wherein the label records the subtask of the current image data and the assigned host, and the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask.
[0083] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.
[0084] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.
[0085] Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.
[0086] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0087] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A decentralized deployment method for a digital pathology slide scanner, characterized in that: include: Deploy at least two hosts connected to the same switch, and deploy each digital pathology slide scanner connected to multiple hosts and each host also connected to multiple scanners; Deploy a shared service layer in a containerized manner on each host, where the shared service layer is used to monitor the host status of each host; Deploy a shared buffer pool, where each host accesses the shared buffer pool in the form of a shared directory, and the shared buffer pool caches image data; Whenever any digital pathology slice scanner scans a digital pathology slice, the scanning task is divided into subtasks according to the task process and stored in the task scheduling queue, and the image data of the digital pathology slice is stored in a shared buffer pool. The shared service layer sets a label for the image data in the shared buffer pool based on the host status and task scheduling queue of each host and notifies the corresponding host, where the label records the subtask of the current image data and the assigned host. The host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask.
2. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: The host status monitored by the shared service layer includes the host online status and the host resource usage, wherein the host resource usage includes one or any combination of the host CPU, host memory and host storage capacity.
3. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: For each scanning task of scanning digital pathology sections, the scanning task is divided into multiple subtasks: camera image acquisition, light intensity correction, distortion correction, image cropping, image stitching, image fusion, image sharpening and AI analysis, and the subtasks are stored in the task scheduling queue.
4. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: The shared service layer obtains the "online" host according to the host online status, and sets a label for the image data in the shared buffer pool based on the host resource usage of the "online" host and the unfinished subtasks in the task scheduling queue and notifies the corresponding host.
5. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: When the host completes the assigned subtask, it feeds back the subtask processing status to the shared service layer and updates the task processing status of the subtask in the task scheduling queue.
6. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: Whenever any digital pathology slice scanner scans a digital pathology slice, the database of the shared service layer records the slice information of the current digital pathology slice and performs an MD5 check on the current digital pathology slice, where the slice information includes the file information, file path, and file status of the digital pathology slice. The file information of the digital pathology slice includes the creation time, file name, file format, and size of the digital pathology slice. The file path is the storage path of the image data of the digital pathology slice in the shared buffer pool. The file status marks the processing status of the current digital pathology slice, and the processing status includes one of unprocessed, processing, and processed.
7. The decentralized deployment method of a digital pathology slide scanner according to claim 1, characterized in that: A temporary memory for storing image data and a hard disk for buffering decoded image data are set; when a decoding request is obtained, the tile of the digital pathology slice requested to be decoded in the deployment shared buffer pool is located based on the decoding request, the image data of the current tile is retrieved and loaded into the temporary memory and locked, the image data in the temporary memory is decoded to obtain decoded data and the decoded data is buffered to the hard disk, the host reads the decoded data of the corresponding digital pathology slice image data from the hard disk and executes the subtask.
8. The decentralized deployment method of a digital pathology slide scanner according to claim 7, characterized in that: The tile information of the tile is marked according to the area of the current digital pathology slice where the tile is stopped, wherein the tile information records the digital pathology slice to which the current tile belongs, the level and position in the digital pathology slice, and the MD5 checksum of the digital pathology slice to which the tile belongs, and the tile of the digital pathology slice requested to be decoded is located according to the tile information.
9. A decentralized deployment system for a digital pathology slide scanner, characterized in that: include: A decentralized deployment architecture, wherein at least two hosts connected to the same switch are deployed in the decentralized deployment architecture, and each digital pathology slide scanner is deployed to connect multiple hosts and each host is also connected to multiple scanners; a shared service layer is deployed on each host in a containerized manner, wherein the shared service layer is used to monitor the host status of each host; a shared buffer pool is deployed, wherein each host accesses the shared buffer pool in a shared directory manner, and the shared buffer pool caches image data; The data processing system is used for dividing the scanning task into subtasks according to the task process and storing them in the task scheduling queue whenever any digital pathology slice scanner scans the digital pathology slice, storing the image data of the digital pathology slice in the shared buffer pool, and the shared service layer sets a label for the image data in the shared buffer pool based on the host status of each host and the task scheduling queue and notifies the corresponding host, wherein the label records the subtask of the current image data and the assigned host, and the host reads the corresponding digital pathology slice image data from the shared buffer pool and executes the subtask.
10. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the decentralized deployment method of the digital pathology slide scanner according to any one of claims 1 to 8.
