Data stream processing methods for digital pathology images

Abstract

This disclosure describes a data stream processing method for digital pathology images, providing a data stream of multiple digital pathology images to a computing system. The method includes preparing a storage environment; acquiring digital pathology images and storing them in a first storage unit of an image storage unit; the first storage unit reads the metadata of the digital pathology images; based on the metadata and in byte order, the digital pathology images are divided into byte-level blocks to obtain multiple sub-blocks of multiple image sub-slices including adjacent regions; then, the multiple sub-blocks are stored, and sub-slice information for each image sub-slice is generated; the metadata and sub-slice information are stored in a metadata unit; upon receiving a data request to acquire the data stream, at least one data stream corresponding to an image sub-slice is acquired based on the metadata, the byte offset of the image sub-slice, and the byte size. Therefore, digital pathology images can be stored quickly and at low cost, and shared based on the data stream.

Description

[0001] This application is a divisional application of the application filed on November 8, 2021, with application number 2021113158009, entitled "Data Stream Processing System for Digital Pathological Images". Technical Field

[0002] This disclosure generally relates to a data stream processing method for digital pathological images. Background Technology

[0003] Currently, the use of digital pathology image analysis is becoming increasingly scenario-based and professional. Based on ultra-high-resolution digital pathology images and deep learning technologies such as artificial intelligence, pathology analysis systems are gradually becoming tools to assist clinicians in pathology analysis. Digital pathology images are obtained by scanning traditional pathology slides with a digital scanner to acquire high-resolution digital pathology images, and then stitching together the obtained localized images. Compared with traditional pathology slides, digital pathology images have many advantages in terms of storage and management, teaching, remote diagnosis, and image reproducibility, effectively solving the problems of traditional pathology slides being easily damaged, faded, lost, difficult to copy, and difficult to retrieve.

[0004] Currently, digital pathology images are often stored in hospital storage or scanner control workstations in specific formats and viewed with specific client software. Alternatively, image sub-slices of digital pathology images are stored hierarchically based on open-source distributed storage systems, and caches are designed to accelerate their access.

[0005] However, digital pathology images contain a large amount of information and have large file sizes. Storing them in hospital storage or on scanner control workstations often hinders data sharing and makes expansion inconvenient. Another approach, storing tens of thousands to hundreds of thousands of image subslices hierarchically, often consumes significant hardware resources and time, and reading and writing small files places high demands on hard drive performance, typically requiring expensive solid-state drives (SSDs). Furthermore, online viewing of digital pathology images is often highly random, and AI analysis of digital pathology images often requires analysis based on the entire image data, resulting in low cache hit rates. Therefore, the storage and sharing of digital pathology images still present significant challenges. Summary of the Invention

[0006] This disclosure is made in view of the above-mentioned situation, and its purpose is to provide a data stream processing system for digital pathology images that can quickly and at low cost store and share digital pathology images based on data streams.

[0007] To this end, this disclosure provides a data stream processing system for digital pathological images. This system provides a data stream of multiple digital pathological images to a computing system. The computing system has multiple computing tasks. The computing system caches the data stream corresponding to each digital pathological image so that the multiple computing tasks can perform parallel computational analysis on each digital pathological image by reusing the data stream. After completing the computational analysis of each digital pathological image, the data stream is deleted from the cache. The processing system includes: an acquisition unit for acquiring the digital pathological image and storing it in an image storage unit; the image storage unit includes a first storage unit for reading metadata of the digital pathological image, acquiring the byte size of an image sub-slice of the digital pathological image based on the metadata, and performing byte-level processing on the digital pathological image based on the byte size of the image sub-slice and in byte order. Other blocks are used to obtain multiple sub-blocks, store the multiple sub-blocks, and generate sub-slice information for each image sub-slice. Each sub-block includes multiple image sub-slices in adjacent regions. The byte size of the sub-block is larger than the byte size of the data stream. The sub-slice information includes the sub-block in which the image sub-slice is located and its byte offset relative to the sub-block. A metadata unit is used to record the metadata and the sub-slice information. A parsing unit is used to obtain the sub-slice information of each image sub-slice in the at least one image sub-slice corresponding to the data stream from the metadata unit. Then, based on the byte offset in the sub-slice information and the byte size of the image sub-slice, the data stream corresponding to the at least one image sub-slice is read from the first storage unit one at a time from the sub-block in a manner that shares the handle opened by the sub-block and returned to the computing system. In this scenario, segmenting digital pathology images byte-by-byte in sequence avoids image encoding and decoding operations, reducing hardware resource consumption and preventing the generation of massive amounts of small files. This simplifies file addressing and lowers the performance requirements of the storage medium. Furthermore, by obtaining the data stream corresponding to the image sub-slice based on byte offsets, data blocks of the corresponding byte size for the image sub-slice at the specified byte offset can be retrieved without opening the entire sub-block, further reducing the workload. This enables fast and low-cost storage and data stream-based sharing of digital pathology images. Additionally, it allows image sub-slices with similar pixel regions to be stored in the same sub-block, enabling a localized transition from random to sequential reading and reducing the number of times sub-blocks need to be opened. Thus, a reasonable balance between data capacity, performance, and cost is achieved.

[0008] Furthermore, in the processing system disclosed herein, optionally, each sub-block has the same byte size. When the first storage unit divides the digital pathology image into blocks, it sequentially divides the data stream corresponding to the image sub-slice into the plurality of sub-blocks in byte order. When the remaining space of a sub-block is insufficient to store the data stream corresponding to an image sub-slice, blank data is used to fill the remaining space. In this case, subsequent individual image sub-slices can be obtained from one sub-block, and the block division process can be simplified.

[0009] Additionally, in the processing system disclosed herein, optionally, the image storage unit further includes a second storage unit for storing the entire digital pathology image, and the sub-slice information further includes a byte offset relative to the entire digital pathology image. This allows for convenient subsequent reading of image sub-slices from the entire digital pathology image based on the byte offset.

[0010] Additionally, in the processing system disclosed herein, optionally, the parsing unit determines the at least one image sub-slice to be read based on the data request; if the digital pathology image is stored in the first storage unit, the storage path of the sub-block where the image sub-slice is located and the byte offset relative to the sub-block are obtained based on the sub-slice information of each image sub-slice, and then a data block of the byte size of the image sub-slice at the byte offset is read from the storage path as the data stream corresponding to the image sub-slice; if the digital pathology image is stored in the second storage unit, the storage path of the entire digital pathology image and the byte offset relative to the entire digital pathology image are obtained, and then a data block of the byte size of the image sub-slice at the byte offset is read from the storage path as the data stream corresponding to the image sub-slice; and the data stream corresponding to the at least one image sub-slice is used as a response to the data request. Thus, the data stream corresponding to the image sub-slice can be obtained based on the byte offset.

[0011] In addition, in the processing system disclosed herein, optionally, the metadata unit is also used to record a partitioning strategy and partitioning information, wherein the partitioning information includes the partitioning number and storage path of each sub-partition. This allows for convenient adjustment of the partitioning strategy and management of the partitioning information.

[0012] Additionally, in the processing system disclosed herein, optionally, the processing system further includes a service registration unit. This service registration unit receives service registration requests and registers and manages services. When the image storage unit and the metadata unit are started, they send the service registration request to the service registration unit to register with it. This allows for convenient adjustment of the segmentation strategy and management of segmentation information.

[0013] Additionally, in the processing system disclosed herein, optionally, the metadata includes the color channels of the digital pathology image, the image level, and the pixel width and height of each image sub-slice at the image level. The byte size of each image sub-slice is the pixel width and height of the image sub-slice multiplied by the number of color channels. The sub-slice information also includes the byte size of the image sub-slice and the image level to which it belongs. In this case, the information of the image sub-slices corresponding to each image level can be obtained quickly, thereby enabling the acquisition of the image sub-slices corresponding to each image level.

[0014] Additionally, in the processing system disclosed herein, optionally, the first storage unit is further configured to acquire and store the macroscopic image, label image, and thumbnail of the digital pathology image. The processing system also includes a data interface for receiving the data request. This data interface includes at least one of the following interfaces: a DeepZoom interface for acquiring an image layer matching the field of view of a display device; a metadata interface for acquiring the metadata; a macroscopic image interface for acquiring the macroscopic image; a thumbnail interface for acquiring the thumbnail; a label image interface for acquiring the label image; a tile interface for acquiring the image sub-slices; and a target region interface for acquiring at least one image sub-slice or the data stream corresponding to at least one image sub-slice within a target region. In this configuration, relevant data for digital pathology images in different usage scenarios can be easily acquired.

[0015] Additionally, in the processing system disclosed herein, optionally, the processing system further includes a protocol adaptation unit. This unit adapts reusable or non-reusable request methods based on the usage scenario of the data interface. Reusable request methods include requests based on the HTTP / 2 gRPC protocol and requests based on the socket protocol. Non-reusable request methods include HTTP requests and RESTful API requests. In this case, the serialization requirements for computational analysis can be met, and the overhead of connection establishment can be reduced. Furthermore, it is also well-suited for random access such as retrieval, browsing, zooming, and changes in field of view.

[0016] Additionally, in the processing system disclosed herein, optionally, when the parsing unit reads the digital pathology image, it shares the handle opened by the digital pathology image and reads the data stream corresponding to at least one image sub-slice at a time. In this case, the data stream corresponding to at least one image sub-slice can be read quickly, and the number of times the entire image needs to be opened can be reduced.

[0017] According to this disclosure, a data stream processing system for digital pathology images is provided, which enables rapid and low-cost storage and data stream-based sharing of digital pathology images. Attached Figure Description

[0018] This disclosure will now be explained in further detail by way of example only with reference to the accompanying drawings, in which:

[0019] Figure 1 This is a schematic diagram illustrating the digital pathology images involved in the examples of this disclosure.

[0020] Figure 2 This is a schematic diagram illustrating the environment of a data stream processing system for digital pathological images involved in the examples of this disclosure.

[0021] Figure 3 This is a block diagram illustrating one example of a processing system as described in this disclosure.

[0022] Figure 4 This is a block diagram illustrating an image storage unit involved in an example of this disclosure.

[0023] Figure 5 This is a block diagram illustrating another example of the processing system involved in the examples of this disclosure.

[0024] Figure 6 This is a flowchart illustrating an example of a data stream processing method for digital pathological images as described in this disclosure. Detailed Implementation

[0025] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals are used for the same components, and repeated descriptions are omitted. Furthermore, the drawings are merely schematic diagrams, and the proportions of the components or the shapes of the components may differ from actual figures.

[0026] It should be noted that the terms "comprising" and "having," and any variations thereof, meaning a process, method, system, product, or apparatus that includes or has a series of steps or units, are not necessarily limited to those steps or units explicitly listed, but may include or have other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. All methods described in this disclosure may be performed in any suitable order unless otherwise indicated herein or clearly contradicted by the context.

[0027] The digital pathology image data stream processing system disclosed herein enables rapid and low-cost storage and data stream-based sharing of digital pathology images. This system may also be referred to as a processing system, data stream server, image processing system, or image sharing system. The processing system disclosed herein can be stateless, can operate as a standalone machine, and can be expanded to adapt to changes in storage or data retrieval needs. Data retrieval can involve obtaining full or partial images of digital pathology images from remote or local sources for viewing or processing (e.g., computational analysis using the digital pathology image data stream). The processing system disclosed herein is particularly suitable for batch or streaming data stream invocation during artificial intelligence processing and / or computational analysis of digital pathology images.

[0028] The processing system disclosed herein can provide a data stream of multiple digital pathological images to a computing system. The computing system can be an artificial intelligence-based system capable of performing computational analysis on the digital pathological images to obtain lesion regions and / or classification results. In some examples, the computing system can have multiple computational tasks. The computing system can cache the data stream corresponding to each digital pathological image so that multiple computational tasks can perform parallel computational analysis on each digital pathological image by reusing the data stream. After completing the computational analysis of each digital pathological image, the data stream can be deleted from the cache. Generally, digital pathological images are very large, and the computing system involves training a large number of digital pathological image samples or simultaneously recognizing multiple digital pathological images. In particular, it often involves multiple computational tasks performing parallel computational analysis on the entire or partial images of each digital pathological image. In this case, providing the computing system with a data stream of digital pathological images, which can be easily cached to avoid repeatedly opening handles to digital pathological images or sub-blocks, can reduce the overhead of the data provider (e.g., the processing system). This enables support for high-concurrency data stream requests.

[0029] Figure 1 This is a schematic diagram illustrating the digital pathology images involved in the examples of this disclosure.

[0030] The digital pathological images involved in this disclosure can be pathological images obtained by scanning pathological slides using a digital scanner. In some examples, the digital pathological images can be at least one of black and white images, grayscale images, and color images (e.g., RGB images).

[0031] Furthermore, digital pathology images can be pyramid images with different resolutions (i.e., digital pathology images can include images of multiple resolutions). As an illustration of a digital pathology image, Figure 1 A schematic diagram of a digital pathology image is shown. (For example...) Figure 1As shown, digital pathology images can have multiple image levels. For example, the number of image levels can be n, and the n image levels can include image level L1, image level L2, image level L3, ..., image level Ln-2, ​​image level Ln-1, and image level Ln.

[0032] Furthermore, the resolution of images at different image levels can be different. In practical applications, images at appropriate image levels can be obtained for processing and / or display based on the usage scenario. In some examples, a digital pathology image can also be an image at a single image level. The pathology analysis system disclosed herein can be applied to digital pathology images with multiple image levels, as well as digital pathology images with a single image level.

[0033] In some examples, images at each image level have multiple image sub-slices (also called tiles). That is, each image level can include multiple image sub-slices. The size of an image sub-slice is typically tens of kilobytes (KB) or a few KB. In some examples, the size of an image sub-slice in a digital pathology image can be obtained by extracting metadata from the digital pathology image. In other words, the size of an image sub-slice in a digital pathology image can be obtained using metadata. In some examples, metadata can be obtained by parsing the file header of the digital pathology image using a toolkit provided by the manufacturer of the digital pathology imaging equipment. Metadata can be used to describe the attributes of the digital pathology image, thereby facilitating the location of image sub-slices within the digital pathology image.

[0034] In some examples, metadata may include at least one of the following: color channels of a digital pathology image, image level, and the pixel width and height of each image sub-slice within that image level. In some examples, the image level of the digital pathology image and the size of each image sub-slice within that image level can be obtained using metadata. In some examples, the pixel width and height of the image sub-slice in the metadata can be multiplied by the number of color channels to obtain the byte size of the image sub-slice. Thus, the byte size of the image sub-slice can be obtained based on the metadata.

[0035] In some examples, metadata may also include at least one of the following: filename, file size, and scan time (i.e., imaging time) of the digital pathology image. In some examples, metadata may also include at least one of the following: scanner manufacturer, scan magnification, total number of image sub-slices, area per pixel, image compression ratio, image quality, color encoding, pixel width and height of the entire image, pixel width and height of non-empty areas of the entire image, overlapping pixels between image sub-slices, and number of image sub-slices at each image level. Additionally, scan magnification may represent a numerical magnification. For example, scan magnification could be 40x. Furthermore, area per pixel could be the area of ​​the body part corresponding to that pixel. Additionally, color encoding can correspond to color channels. For example, if there are three color channels: R, G, and B, then color encoding could be RGB.

[0036] Furthermore, digital pathology images are generally very large; for example, whole-slide images (WSI) can range in size from 600 Mb to 10 Gb. Therefore, traditional image storage and sharing methods are generally not suitable for digital pathology images. For instance, storing digital pathology images often requires high-performance storage media, and sharing them is easily limited by the efficiency of reading and transmitting them.

[0037] Figure 2 This is a schematic diagram illustrating the environment of a data stream processing system for digital pathological images involved in the examples of this disclosure. Furthermore, the environment described in the examples of this disclosure is for the purpose of more clearly illustrating the technical solutions of this disclosure and does not constitute a limitation on the technical solutions provided in this disclosure.

[0038] In some examples, the processing system disclosed herein can be applied to, for example... Figure 2 The environment 100 shown may include a client device 102, a storage device 104, a metadata device 106, and an interface device 108 that are communicatively coupled to each other via a network. Computing devices that can be used to implement the client device 102, storage device 104, metadata device 106, and interface device 108 can be configured in various ways. In some examples, the computing device may be, but is not limited to, a mobile device, a laptop computer, and a desktop computer. In some examples, the computing device may be a server. For example, the computing device may be a cloud server. Additionally, the computing device may represent multiple different devices. For example, the computing device may represent multiple cloud servers.

[0039] In environment 100, the acquisition unit 110 of client device 102 can be used to acquire digital pathology images (see [link]). Figure 2The digital pathology image is stored in storage device 104. The image storage unit 120 of storage device 104 reads metadata when the digital pathology image is stored, and can divide the digital pathology image into multiple sub-blocks based on the metadata and in byte order (see [link to storage device 104]). Figure 2 Each sub-block includes multiple image sub-slices of adjacent regions. Then, the multiple sub-blocks are stored and sub-slice information of each image sub-slice is generated. The metadata and sub-slice information are stored in the metadata unit 130 of the metadata device 106.

[0040] Additionally, within environment 100, stored digital pathology images can be accessed. Specifically, after interface device 108 receives a data request, parsing unit 140 can obtain sub-slice information of at least one image sub-slice from metadata device 106 based on the data request, and obtain at least one image sub-slice and / or the corresponding data stream from storage device 104 based on the sub-slice information.

[0041] The processing system disclosed herein takes into account the application scenarios of digital pathology images, considering that the reading of digital pathology images is usually ordered when the computing system performs computational analysis. For example, starting from a specific scaling factor (i.e., a specific image level) of the digital pathology image, the corresponding parts of the image are read one by one in a certain size until all parts are read. Furthermore, it also considers scenarios in the medical field, such as image reading or consultations where data reading is random, placing very high demands on the performance of the underlying storage. Therefore, the processing system disclosed herein, when storing digital pathology images, divides and stores the images in byte-level blocks based on metadata, ensuring that image sub-slices with similar pixel regions (i.e., multiple image sub-slices in adjacent regions) are stored in the same sub-block (i.e., located on the same sub-block as much as possible). In this case, a local conversion from random to sequential reading is achieved. Thus, a relative balance between capacity, performance, and cost is achieved.

[0042] Figure 3 This is a block diagram illustrating one example of a processing system as described in this disclosure.

[0043] like Figure 3As shown, in some examples, the processing system may include an acquisition unit 110, an image storage unit 120, a metadata unit 130, and a parsing unit 140. In some examples, the number of image storage units 120 may be one or more. Multiple image storage units 120 can form a distributed file storage system. In some examples, the number of metadata units 130 may be one or more. Multiple metadata units 130 can form a distributed storage system. In some examples, the number of parsing units 140 may be one or more. Multiple parsing units 140 can support load-balanced deployment. In this case, data requests can be distributed across the parsing units 140. This improves load capacity.

[0044] In some examples, such as Figure 3 As shown, the processing system may include an acquisition unit 110. The acquisition unit 110 may be configured to acquire digital pathology images and store them.

[0045] In some examples, the acquisition unit 110 may acquire digital pathology images in at least one of the following ways: client program file listening upload, upload via a shared path, upload via a visual interface (e.g., via a browser or client), file copy upload (e.g., via USB flash drive or external hard drive), and scanner upload. This allows for the support of multiple methods for acquiring digital pathology images. Furthermore, client program file listening upload can acquire digital pathology images by monitoring changes in the file system through file I / O operations. Specifically, file upload can be achieved by recursively monitoring changes in file events under a specific storage path, where both file changes and changes to file directories can be monitored. In some examples, changes to digital pathology images can be monitored only by limiting the file extension (also known as the file suffix). This improves efficiency. Additionally, scanner upload can involve the scanner communicating with the processing system to upload the digital pathology images acquired by the scanner.

[0046] In some examples, the acquisition unit 110 can acquire digital pathological images of different diseases. In some examples, the diseases may include, but are not limited to, cervical liquid-based cytology, thyroid cytology, urothelial cytology, histopathology, and pleural and peritoneal fluid cytology.

[0047] In some examples, the acquisition unit 110 may store digital pathology images to the image storage unit 120 (described later). The image storage unit 120 may include a first storage unit 121 and / or a second storage unit 122. In some examples, the acquisition unit 110 may store digital pathology images to the first storage unit 121 and / or the second storage unit 122 of the image storage unit 120.

[0048] In some examples, acquisition unit 110 can store digital pathology images to a shared path of image storage unit 120. In some examples, the shared path can be implemented via shared mounting. For example, the shared path can be mounted to the corresponding directory of image storage unit 120. In some examples, shared mounting can be implemented through, but is not limited to, SAMBA software (SAMBA is a free software that implements the SMB protocol on Linux and UNIX systems; SMB stands for Server Messages Block), NFS (Network File System) protocol, and iSCSI (Internet Small Computer System Interface) protocol. In some examples, the shared path can be a shared access path for institutional storage, such as within a hospital, and / or a mount directory for distributed file storage.

[0049] In some examples, such as Figure 3 As shown, the processing system may include an image storage unit 120. The image storage unit 120 can be used to store digital pathology images. In some examples, the image storage unit 120 can be used to store the entire digital pathology image and / or sub-blocks. In some examples, the image storage unit 120 can be used to perform block storage and / or direct storage of the digital pathology image. In this case, multiple storage methods can be supported, and while both storage methods are compatible with third-party storage systems (e.g., existing storage systems within the hospital), block storage of digital pathology images is also possible. Therefore, the storage costs of third-party storage systems can be considered, while also meeting the increasing storage demands.

[0050] In some examples, the storage location of the digital pathology image can be maintained by the metadata unit 130 (described later). Specifically, the metadata unit 130 can record the storage medium of the digital pathology image (e.g., the first storage unit 121 and / or the second storage unit 122) and the storage path.

[0051] Figure 4 This is a block diagram illustrating the image storage unit 120 involved in the example of this disclosure.

[0052] In some examples, such as Figure 4As shown, the image storage unit 120 may include a first storage unit 121 and / or a second storage unit 122. In some examples, the image storage unit 120 may support distributed deployment. That is, the first storage unit 121 and / or the second storage unit 122 in the image storage unit 120 may have multiple storage systems, providing unified storage services. It should be noted that, unless there is a contradiction, the following description of the image storage unit 120 also applies to the first storage unit 121 and the second storage unit 122.

[0053] In some examples, image storage unit 120 can store digital pathology images based on their metadata. In some examples, the metadata can be read by image storage unit 120. In some examples, the metadata can be read by image storage unit 120 when the digital pathology image is written. In some examples, image storage unit 120 can generate an image number to uniquely identify the digital pathology image when storage is complete. In some examples, the image number can be calculated by combining the file name, file size (i.e., byte capacity), and imaging time of the digital pathology image from the metadata. In some examples, the image number is irreversible. For example, the file name, file size, and imaging time of the digital pathology image cannot be deduced from the image number.

[0054] In some examples, the first storage unit 121 can be a native distributed file storage system, and the second storage unit 122 can be a storage system mounted based on the NFS, SAMBA, or iSCSI protocols (also known as a third-party storage system). In this case, it can adapt to the storage requirements of digital pathology images in different environments and meet the storage needs at different stages.

[0055] As described above, in some examples, image storage unit 120 may include a first storage unit 121. In some examples, the first storage unit 121 may be used to store digital pathology images in blocks. Furthermore, in block storage, the digital pathology image may be divided into blocks to obtain multiple sub-blocks, and then the multiple sub-blocks may be stored.

[0056] In some examples, the first storage unit 121 can store digital pathology images in blocks according to a block-segmentation strategy. In some examples, the block-segmentation strategy can include the byte size of the sub-blocks. In this case, the digital pathology image can be blocked according to a certain byte size. In some examples, the byte size of the sub-blocks can be set according to the byte size of the digital pathology image. For example, the byte size of the sub-blocks can be positively correlated with the byte size of the digital pathology image. This further reduces the number of sub-blocks. In some examples, the block-segmentation strategy can also include a block order. For example, the block order can include at least one of ascending byte order, descending byte order, image level, or a custom order. In some examples, the block-segmentation strategy can also include a block range. The block range can specify the data blocks in the digital pathology image that can be used for block-segmentation.

[0057] As described above, metadata can be used to obtain the image layer of a digital pathology image and the byte size of each image sub-slice within each image layer. In some examples, the first storage unit 121 can, during the writing of a digital pathology image, divide the image into byte-level blocks based on the byte size of the image sub-slices to obtain multiple sub-blocks, and then store these multiple sub-blocks. In this case, dividing the digital pathology image into blocks based on bytes avoids image encoding and decoding operations, reducing the consumption of hardware resources (e.g., CPU). Furthermore, it avoids generating a massive number of small files, reducing the difficulty of file addressing, and thus lowering the performance requirements of the storage medium, achieving high-performance data storage and retrieval even on ordinary mechanical storage media.

[0058] In some examples, the first storage unit 121 can segment a digital pathology image by dividing corresponding image sub-slices into corresponding sub-blocks. In some examples, during segmentation, the byte offset of each image sub-slice relative to the entire digital pathology image can be obtained based on the file size of the digital pathology image, image level, the number of image sub-slices at each image level, and the byte size of each image sub-slice. Then, based on the byte size of the image sub-slice and its byte offset relative to the entire digital pathology image, the data segment corresponding to the image sub-slice can be obtained, and this data segment (i.e., the data stream corresponding to the image sub-slice) can be divided into sub-blocks. Thus, the digital pathology image can be segmented.

[0059] In some examples, the first storage unit 121 can divide the digital pathology image into byte-level blocks in byte order. Specifically, the data stream corresponding to the image sub-slices can be divided into multiple sub-blocks sequentially in byte order. In some examples, the byte order can be either ascending or descending.

[0060] In some examples, when the first storage unit 121 divides the digital pathology image into blocks, it can divide the data stream corresponding to the image sub-slice into multiple sub-blocks sequentially in byte order. When the remaining space in a sub-block is insufficient to store the data stream corresponding to an image sub-slice, blank data can be used to fill the remaining space. That is, the image sub-slices in each sub-block can be complete. In this case, subsequent individual image sub-slices can be obtained from one sub-block, simplifying the block division process. In other examples, when the remaining space in a sub-block is insufficient to store an image sub-slice, the remaining part of the image sub-slice can be stored in another sub-block. In this case, a single image sub-slice can correspond to one or more sub-blocks. This saves storage space.

[0061] In other examples, chunking can be performed out of byte order. Byte streams corresponding to multiple image sub-slices at a specific location, such as the same location across different image layers, can be merged to generate multiple sub-chunks. For example, image sub-slices from the 1st to the 50th of each image layer can be merged. It should be noted that there are no specific restrictions on the specific location; it depends on the specific chunking strategy. In this case, adjusting the chunking strategy according to the data retrieval requirements can generate sub-chunks that better match the data retrieval needs.

[0062] In some examples, after the first storage unit 121 has stored multiple sub-blocks, it can generate sub-slice information for each image sub-slice. In some examples, the sub-slice information can be recorded in the metadata unit 130 (described later). This allows for convenient subsequent retrieval of image sub-slices based on the sub-slice information.

[0063] Specifically, the first storage unit 121 can read metadata when writing a digital pathology image. Based on the metadata, it multiplies the pixel width and height of the image sub-slice in the metadata by the number of color channels to obtain the byte size of the image sub-slice. Then, based on the byte size of the image sub-slice and in byte order, it divides the digital pathology image into byte-level blocks to obtain multiple sub-blocks. Finally, it stores the multiple sub-blocks and generates sub-slice information for each image sub-slice. Thus, it is possible to divide the digital pathology image into byte-level blocks and generate sub-slice information based on the metadata.

[0064] In other examples, sub-slice information may not be generated at storage time; instead, it can be obtained based on metadata and tiling strategies when requesting image sub-slices.

[0065] As described above, digital pathology images can be segmented to obtain multiple sub-blocks. In some examples, each sub-block may include multiple image sub-slices. In this case, the number of blocks and segmentation time can be reduced, and the storage path of sub-blocks can be simplified, improving the efficiency of sub-block storage and retrieval. In addition, the performance requirements of the storage medium are lower, which can reduce storage costs.

[0066] In some examples, each sub-block may include multiple image sub-slices from adjacent regions, and the byte size of the sub-block may be larger than the byte size of the data stream provided to the data consuming system (e.g., a computing system). In this case, image sub-slices with similar pixel regions can be stored in the same sub-block as much as possible, thereby locally enabling the conversion from random to sequential reading. This reduces the number of times sub-blocks are opened.

[0067] In some examples, the size of each sub-block can be the same in bytes. This simplifies the blocking process. In other examples, the size of each sub-block can be greater than or equal to a preset size. For example, the preset size could be 16MB, 32MB, 64MB, or 128MB. In this case, the size of each sub-block is much larger than the size of the data stream requested by the computing system. This allows the data stream corresponding to a single request to be read within one sub-block as much as possible.

[0068] In some examples, each sub-block has corresponding block information. In some examples, the block information can be generated after multiple sub-blocks have been stored in the first storage unit 121. In some examples, the block information may include the block number and storage path of each sub-block. In this case, after determining the sub-block where the image sub-slice belongs, the corresponding sub-block can be read according to the storage path.

[0069] In some examples, the first storage unit 121 can also be used to acquire and store at least one of a macroscopic image, a label image, and a thumbnail image of a digital pathology image. The macroscopic image can be an image reflecting the actual appearance of the pathology slide (i.e., the image seen by the human eye). The label image can be an image with a label for identifying the pathology number, such as a QR code. Furthermore, the macroscopic image and label image can be acquired from specific locations within the entire digital pathology image or can be directly acquired from separately stored locations. The method of acquiring the macroscopic image and label image depends on the manufacturer of the scanner.

[0070] As described above, in some examples, image storage unit 120 may include a second storage unit 122. In some examples, the second storage unit 122 may be used for direct storage of digital pathology images. Additionally, in direct storage, the entire digital pathology image may be stored. In some examples, subslice information may be generated after the entire digital pathology image is stored. In other examples, subslice information may not be generated during storage; instead, it may be obtained based on metadata when requesting image subslices.

[0071] As described above, in some examples, sub-slice information can be generated when storing digital pathology images. In some examples, if the digital pathology image is stored in blocks through the first storage unit 121, the sub-slice information may include the sub-block where the image sub-slice is located and the byte offset relative to the sub-block. Therefore, it is convenient to subsequently read the image sub-slice and / or the data stream corresponding to the image sub-slice from the sub-block based on the byte offset. In some examples, if the digital pathology image is directly stored through the second storage unit 122, the sub-slice information may include the byte offset relative to the entire digital pathology image. Therefore, it is convenient to subsequently read the image sub-slice and / or the data stream corresponding to the image sub-slice from the entire digital pathology image based on the byte offset. In other examples, the sub-slice information may include the sub-block where the image sub-slice is located, the byte offset relative to the sub-block, and the byte offset relative to the entire digital pathology image. Therefore, it is possible to support obtaining the image sub-slice and / or the data stream corresponding to the image sub-slice from the sub-block and the entire digital pathology image.

[0072] In some examples, the sub-slice information may also include the byte size of the image sub-slice. This allows for convenient subsequent retrieval of the image sub-slice based on its byte size and byte offset.

[0073] In some examples, the sub-slice information may also include the image layer to which the image sub-slice belongs. In this case, it is possible to quickly obtain the information of the image sub-slices corresponding to each image layer, and thus obtain the image sub-slices corresponding to each image layer.

[0074] In some examples, such as Figure 3 As shown, the processing system may include a metadata unit 130. The metadata unit 130 can be used to record data other than the file stream of the digital pathology image. In some examples, the metadata unit 130 can support distributed deployment. In some examples, the metadata unit 130 can be used to record metadata read by the image storage unit 120 (i.e., the first storage unit 121 and / or the second storage unit 122). In this case, subsequent image sub-slices can be obtained based on the recorded metadata, thereby reducing the number of reads of the digital pathology image or sub-blocks.

[0075] In some examples, metadata unit 130 can also be used to record sub-slice information. In this case, it facilitates subsequent reading of image sub-slices and can adapt to adjustments in the block partitioning strategy. In some examples, for the sub-slice information corresponding to the first storage unit 121, metadata unit 130 can record the primary key, secondary key, the sub-block number where the image sub-slice is located, and the offset relative to the sub-block. The primary key can be the image number of the digital pathology image, and the secondary key can consist of the image level and the number corresponding to the image sub-slice. Thus, the sub-slice information of an image sub-slice can be uniquely determined based on the primary key and secondary key.

[0076] As described above, in some examples, the sub-slice information may also include the byte size of the image sub-slice. The metadata unit 130 may also be used to record the byte size of the image sub-slice.

[0077] In some examples, the metadata unit 130 can also be used to record node information of the image storage unit 120. In some examples, the node information may include an access address, access protocol, and copy information. In this case, data stored in the image storage unit 120 can be obtained based on the access protocol of the image storage unit 120. Additionally, the access address may include an IP address or domain name, and a port. Additionally, the copy information may include the location of copies of the data stored in the image storage unit 120. In some examples, the node information may also include the type of storage medium (i.e., the first storage unit 121 or the second storage unit 122). This allows for appropriate settings based on the performance of the storage medium. In some examples, the metadata unit 130 can also be used to record a partitioning strategy and partitioning information. This allows for convenient adjustment of the partitioning strategy and management of the partitioning information. In some examples, the metadata unit 130 can also be used to record the storage path of the entire digital pathology image.

[0078] In some examples, such as Figure 3 As shown, the processing system may include a parsing unit 140. The parsing unit 140 can be used to parse the received data request to determine the data requested by the data request.

[0079] In some examples, the data requested in a data request may be related data of a digital pathology image. Related data of a digital pathology image may be data obtained from a file stream of the digital pathology image (e.g., a data stream of the digital pathology image).

[0080] In some examples, the relevant data of a digital pathology image may include at least one of the following: metadata, macroscopic image, thumbnail, label image, tile image (i.e., image sub-slice), and target region data. The format of the relevant data may include, but is not limited to, numbers, text, images, or data streams. Furthermore, when storing a digital pathology image in image storage unit 120 (e.g., first storage unit 121), macroscopic images, label images, and thumbnails can be extracted and stored. Additionally, the target region data may be data composed of at least one image sub-slice within a target region. The target region can be a region within the digital pathology image and can be determined according to the actual usage scenario. For example, when browsing a digital pathology image, the target region may be a region within the field of view; during computational analysis, the target region may be a region within the digital pathology image used for computational analysis. In some examples, the target region may be a region of interest in the digital pathology image or a region cropped from the digital pathology image.

[0081] In some examples, a data request may include an identifier for a digital pathology image (e.g., image number or storage path of the digital pathology image). This allows it to be determined that the requested data is an image sub-slice corresponding to a digital pathology image. In some examples, a data request may include an identifier for a digital pathology image and an image layer. This allows it to be determined that the requested data is at least one image sub-slice at the corresponding image layer. In some examples, a data request may include an identifier for a digital pathology image, an image layer, and the location of the image sub-slice. This allows it to be determined that the requested data is at least one image sub-slice at the corresponding location. In some examples, a data request may include an identifier for a digital pathology image, an image layer, and information about a target region. The information about the target region may include the starting coordinates, width, and height of the target region. This allows it to be determined that the requested data is at least one image sub-slice corresponding to the target region.

[0082] In some examples, if the data requested by the data request includes at least one image sub-slice (that is, the data request is a request to obtain at least one image sub-slice), the parsing unit 140 can obtain at least one image sub-slice from the image storage unit 120.

[0083] In some examples, at least one image sub-slice can be retrieved from a sub-block stored in the first storage unit 121. In some examples, at least one image sub-slice can be retrieved from the full image of a digital pathology image stored in the second storage unit 122. In some examples, at least one image sub-slice can be retrieved from the first storage unit 121 via a distributed file storage client. In some examples, at least one image sub-slice can be retrieved from the second storage unit 122 via NFS, SAMBA, or iSCSI protocols.

[0084] In some examples, the parsing unit 140 can obtain a data stream corresponding to at least one image sub-slice from the image storage unit 120. In some examples, the parsing unit 140 can read the entire digital pathology image or a sub-block from the image storage unit 120 to obtain a data stream corresponding to at least one image sub-slice. In some examples, when reading the entire digital pathology image or a sub-block from the image storage unit 120, the handle to the entire digital pathology image or a sub-block can be shared, and the data stream corresponding to at least one image sub-slice can be read at once. In this case, the data stream corresponding to at least one image sub-slice can be read quickly, and the number of times the entire image or sub-block needs to be opened can be reduced.

[0085] In some examples, the parsing unit 140 can obtain sub-slice information of at least one image sub-slice from the metadata unit 130, and obtain the data stream corresponding to at least one image sub-slice from the first storage unit 121 and / or the second storage unit 122 based on the sub-slice information. This allows for convenient acquisition of the data stream corresponding to the image sub-slice.

[0086] In some examples, the parsing unit 140 can obtain sub-slice information of at least one image sub-slice from the metadata unit 130, and then obtain the data stream corresponding to at least one image sub-slice from the digital pathology image stored in the image storage unit 120 based on the byte offset in the sub-slice information and the byte size of the image sub-slice. In this case, a data block of the byte size of the image sub-slice at the corresponding byte offset can be obtained without needing to open the entire digital pathology image or its sub-blocks. This reduces the workload. In some examples, the byte size of the image sub-slice can be obtained based on metadata or recorded in the metadata unit 130 through sub-slice information when storing the digital pathology image.

[0087] In some examples, the parsing unit 140 can open a handle to the full image or a sub-block of the digital pathology image to obtain a pointer to a file stream, move the file stream pointer to the position corresponding to the byte offset, and read the image sub-slice of the corresponding byte size starting from that position. Thus, the data stream corresponding to the image sub-slice can be read. In some examples, the total byte offset corresponding to at least one image sub-slice can be calculated, and the data stream corresponding to at least one image sub-slice can be obtained based on the total byte offset.

[0088] In some examples, when providing a data stream of multiple digital pathology images to the computing system, the parsing unit 140 obtains information about each sub-slice in at least one image sub-slice corresponding to the data stream from the metadata unit 130. Then, based on the byte offsets and byte sizes of the image sub-slices in the information of each sub-slice, it reads the data stream corresponding to at least one image sub-slice from the full image or sub-block of the digital pathology image at a time from the first storage unit 121 or the second storage unit 122, sharing a handle for opening a sub-block or the entire digital pathology image, and returns it to the computing system. In this case, without needing to open the entire digital pathology image or sub-block as a whole, it can obtain a data block of the byte size of the image sub-slice at the corresponding byte offset. This reduces the load. In some examples, the data stream can correspond to an image region. That is, after receiving a data request from the computing system to obtain the data stream, the parsing unit 140 can obtain information about each sub-slice in at least one image sub-slice corresponding to the target region from the metadata unit 130, and then obtain the data stream corresponding to at least one image sub-slice of the target region based on the information of each sub-slice and return it to the computing system.

[0089] In some examples, for the first storage unit 121, the parsing unit 140 can obtain sub-slice information of at least one image sub-slice from the metadata unit 130, and then obtain the data stream corresponding to at least one image sub-slice from the sub-blocks stored in the first storage unit 121 based on the byte offset of the relative sub-block in the sub-slice information and the byte size of the image sub-slice.

[0090] In some examples, for the second storage unit 122, the parsing unit 140 can obtain sub-slice information of at least one image sub-slice from the metadata unit 130, and then obtain the data stream corresponding to at least one image sub-slice from the full image of the digital pathology image stored in the second storage unit 122 based on the byte offset relative to the full image of the digital pathology image in the sub-slice information and the byte size of the image sub-slice. In some examples, the byte offset relative to the full image of the digital pathology image can be obtained based on metadata or recorded in the metadata unit 130 through the sub-slice information of the stored digital pathology image.

[0091] In other examples, when storing digital pathology images, only metadata may be stored, without generating sub-slice information. Byte offsets (i.e., the byte offsets of the image sub-slice relative to the full image and / or sub-blocks of the digital pathology image) and the byte size of the image sub-slice can be obtained based on the metadata. Then, based on the byte offsets and the byte size of the image sub-slice, a data stream corresponding to at least one image sub-slice can be obtained from the digital pathology image stored in image storage unit 120.

[0092] In some examples, the parsing unit 140 can use the data stream corresponding to at least one image sub-slice as a response to a data request. In this case, computational analysis can be performed directly based on the data stream, thereby improving the efficiency of computational analysis. In other examples, the data stream corresponding to at least one image sub-slice can be converted into an image. For example, the image can be a JPEG image or a PNG image. This facilitates easy viewing.

[0093] In some examples, if the digital pathology image is stored in the first storage unit 121, the parsing unit 140 can obtain the storage path of the sub-block where the image sub-slice is located and the byte offset relative to that sub-block based on the sub-slice information of each image sub-slice. Then, it reads a data block of the size of the image sub-slice at the byte offset from the storage path as the data stream corresponding to the image sub-slice. If the digital pathology image is stored in the second storage unit 122, the parsing unit 140 can obtain the storage path of the entire digital pathology image and the byte offset relative to the entire digital pathology image. Then, it reads a data block of the size of the image sub-slice at the byte offset from the storage path as the data stream corresponding to the image sub-slice. The parsing unit 140 uses at least one data stream corresponding to an image sub-slice as a response to a data request. Thus, the data stream corresponding to an image sub-slice can be obtained based on the byte offset.

[0094] Figure 5 This is a block diagram illustrating another example of the processing system involved in the examples of this disclosure.

[0095] As described above, the parsing unit 140 can be used to parse data requests. Figure 5 As shown, in some examples, the processing system may also include a data interface 150. The data interface 150 can be used to receive data requests. In some examples, the data interface 150 can send the received data request to the parsing unit 140, which then parses the data request. In this case, the data interface 150 reduces coupling, thereby facilitating expansion to support load balancing. This enables support for large data volumes and high concurrency access.

[0096] In some examples, data interface 150 may include at least one of the following interfaces: DeepZoom interface, metadata interface, macroscopic map interface, thumbnail interface, tag map interface, tile interface, and target region interface. In this case, relevant data from digital pathology images in different use cases can be easily obtained.

[0097] Additionally, the DeepZoom interface can be used to obtain image layers that match the field of view of the display device. In this case, it is possible to obtain image layers that match the display device, and thus obtain appropriate image sub-slices of the image layers for display. This improves the display effect. In some examples, the image layer that matches the field of view of the display device can be obtained based on the image size of the image layer with the highest resolution and the screen size of the current display device. Specifically, the larger of the width and height values ​​of the image size can be repeatedly divided by 2 until the result is closest to the larger of the width and height values ​​of the screen size, and then the number of times it is divided by 2 is taken as the image layer.

[0098] Additionally, the metadata interface can be used to retrieve metadata. The macroscopic map interface can be used to retrieve macroscopic maps. The thumbnail interface can be used to retrieve thumbnails. The tag map interface can be used to retrieve tag maps. The tile interface can be used to retrieve at least one image sub-tile or the data stream corresponding to at least one image sub-tile. The target region interface can be used to retrieve at least one image sub-tile within a target region or the data stream corresponding to at least one image sub-tile.

[0099] like Figure 5 As shown, in some examples, the processing system may also include a protocol adaptation unit 160. The protocol adaptation unit 160 can adapt reusable or non-reusable request methods based on the usage scenario of the data interface 150. For example, interfaces involved in calculation and analysis can use reusable request methods, while interfaces involved in data retrieval can use non-reusable request methods. In this case, the data interface 150, which can provide multiple scenario-specific request methods, can meet the data retrieval needs of different usage scenarios and reduce the connection overhead to the data interface 150.

[0100] In some examples, the protocol adaptation unit 160 can be used to distinguish whether the data retrieval is for browsing-based or computational applications. In some examples, browsing-based applications may include online image reading and diagnosis, consultation, and annotation, while computational applications may include data analysis, training, and inference.

[0101] In some examples, after distinguishing the use cases for data retrieval, the protocol adaptation unit 160 can convert the request method for data requests in browsing-type services into a non-reusable request method, and the request method for data requests in computing-type services into a reusable request method. This reduces the connection overhead to the data interface 150. In some examples, the use cases for data retrieval can be distinguished based on the user-agent information agreed upon in the request header. In some examples, data can be requested using either a reusable or non-reusable request method by modifying the user-agent information.

[0102] In some examples, reusable request methods may include, but are not limited to, requests based on the gRPC protocol and requests based on the socket protocol (i.e., reusable request methods may include, but are not limited to, requests establishing a connection via the gRPC protocol and requests establishing a connection via the socket protocol). gRPC is designed based on the HTTP / 2 protocol standard. In some examples, reusable request methods can be supported to obtain data streams for computational analysis. In this case, the serialization requirements during computational analysis can be met, and the overhead of connection establishment can be reduced. In some examples, the target area interface can support reusable request methods. This can meet the needs of computational analysis of digital pathology images. In some examples, non-reusable request methods may include HTTP requests and RESTful API requests. In this case, it is more suitable for random access such as retrieval, browsing, zooming, and changes in field of view. In some examples, the DeepZoom interface, metadata interface, macroscopic map interface, thumbnail interface, tag map interface, and tile map interface can support non-reusable request methods. In this case, supporting non-reusable request methods for interfaces with low access frequency or those more often used for random access can reduce the connection overhead to the data interface.

[0103] like Figure 5 As shown, in some examples, the processing system may further include a service registration unit 170. The service registration unit 170 can be used to receive service registration requests and register and manage services. In some examples, after the image storage unit 120 and metadata unit 130 are started, a service registration request can be sent to the service registration unit 170 to register with it. In this case, the image storage unit 120 can be perceived by the acquisition unit 110 (e.g., the storage client) and the metadata unit 130. Additionally, the metadata unit 130 can be perceived by the image storage unit 120. In some examples, the service registration unit 170 can handle service registration in a manner that supports at least one of the following: primary / backup + VIP, Raft, and Paxos.

[0104] In some examples, the processing system may also include a security module (not shown). The security module can be used for secure management of access to digital pathology images. This improves the security of sharing digital pathology images. In some examples, the security module can be used to authenticate and authorize data requests based on token authentication. In this case, an authorized source can request and obtain the relevant data for the digital pathology images. In some examples, the security module can also be used to log access. This facilitates subsequent security audits and improves security.

[0105] The following describes, with reference to the accompanying drawings, a data stream processing method for digital pathological images as described in this disclosure. Figure 6 This is a flowchart illustrating one example of a data stream processing method for digital pathological images as described in this disclosure. In some examples, such as... Figure 6 As shown, the data stream processing method may include preparing a storage environment (step S110), acquiring digital pathological images and storing them in blocks or directly (step S120), and receiving data requests and returning relevant data of the digital pathological images (step S130).

[0106] In some examples, the storage environment can be prepared in step S110. Specifically, the node information of the image storage unit 120 storing digital pathological images can be managed by the metadata unit 130, and the metadata unit 130 and the image storage unit 120 can be registered with the server registration unit. Thus, the image storage unit 120 can be perceived by the acquisition unit 110, and the metadata unit 130 can be perceived by the image storage unit 120.

[0107] In some examples, in step S120, a digital pathological image can be acquired and stored in blocks or directly. Specifically, the acquisition unit 110 can acquire the digital pathological image and store it in the perceived image storage unit 120. When writing the digital pathological image, the image storage unit 120 can extract metadata, generate an image number based on the metadata, and then store the digital pathological image based on the metadata. The digital pathological image can be stored in the first storage unit 121 and / or the second storage unit 122 of the image storage unit 120. In some examples, the digital pathological image can be stored in either the first storage unit 121 or the second storage unit 122 according to actual storage needs. See the relevant description of the image storage unit 120 for details.

[0108] In some examples, the first storage unit 121 can divide the digital pathology image into blocks according to a block-segmentation strategy and generate block information and sub-slice information, and then store the metadata, block information, and sub-slice information in the metadata unit 130. In some examples, the first storage unit 121 can divide the digital pathology image into byte-level blocks based on the byte size of the image sub-slices when writing the digital pathology image to obtain multiple sub-blocks, and then store the multiple sub-blocks. For details, please refer to the relevant descriptions of the first storage unit 121 and the metadata unit 130.

[0109] In some examples, the second storage unit 122 can directly store digital pathology images and store metadata and the storage path of the full-image digital pathology image in the metadata unit 130. In some examples, when storing digital pathology images, the second storage unit 122 can also generate sub-slice information of image sub-slices and store the sub-slice information in the metadata unit 130. See the relevant descriptions of the second storage unit 122 and the metadata unit 130 for details.

[0110] In some examples, in step S130, a data request can be received and relevant data of the digital pathology image can be returned. As described above, in some examples, the relevant data of the digital pathology image may include at least one of the following: metadata, macroscopic image, thumbnail, label image, tile image (i.e., image sub-slice), and target region data. Taking a tile image as an example, a data request can be received through data interface 150 and sent to parsing unit 140. After parsing unit 140 determines that the requested data is at least one image sub-slice, it queries the metadata unit 130 for the storage path of the full image or sub-block of the digital pathology image and the node information of the corresponding storage unit. Then, it reads the full image or sub-block of the digital pathology image from the storage path using the access address and access protocol in the node information, and then obtains at least one image sub-slice or the data stream corresponding to at least one image sub-slice from the full image or sub-block of the digital pathology image. In some examples, parsing unit 140 can obtain sub-slice information of at least one image sub-slice from metadata unit 130, and then obtain the data stream corresponding to at least one image sub-slice from the digital pathology image stored in image storage unit 120 based on the byte offset in the sub-slice information and the byte size of the image sub-slice. The digital pathology image is stored in either the first storage unit 121 or the second storage unit 122, and the process for obtaining the data stream corresponding to the image sub-slice differs. See the relevant description of parsing unit 140 for details.

[0111] The processing system disclosed herein provides a computing system with data streams of multiple digital pathology images. It acquires the digital pathology images and extracts metadata. Based on the byte size of the image sub-slices obtained from the metadata, the system divides the digital pathology images into byte-level blocks in byte order to obtain multiple sub-blocks containing multiple adjacent regions. These sub-blocks are then stored, and the metadata and the byte offset of each image sub-slice relative to its respective sub-block are recorded. When a data request to acquire a data stream is received, the system reads the data stream corresponding to at least one image sub-slice from the sub-block at a time, using the recorded byte offset relative to the sub-block and the byte size of the image sub-slice, sharing the handle to open the sub-block. In this case, dividing the digital pathology image into byte-level blocks in byte order avoids image encoding and decoding operations, reducing hardware resource consumption and preventing the generation of massive small files, thus reducing file addressing difficulty and lowering the performance requirements of the storage medium. Furthermore, by acquiring the data stream corresponding to the image sub-slice based on the byte offset, the system can acquire data blocks of the image sub-slice size at the corresponding byte offset without opening the entire sub-block, thereby reducing the workload. This enables rapid and low-cost storage and data stream-based sharing of digital pathology images. Furthermore, it allows image sub-slices with similar pixel regions to be stored in the same sub-block, thus enabling a localized transition from random to sequential reading and reducing the number of times sub-blocks are opened. This achieves a reasonable balance between data capacity, performance, and cost. Additionally, the processing system supports storing the entire digital pathology image and retrieving the data stream corresponding to at least one image sub-slice using an offset relative to the entire image. In this case, even when storing the entire digital pathology image, the data stream corresponding to the image sub-slice can be read quickly.

[0112] While the present disclosure has been specifically described above in conjunction with the accompanying drawings and examples, it is to be understood that the foregoing description does not limit the present disclosure in any way. Those skilled in the art can make modifications and variations to the present disclosure as needed without departing from its essential spirit and scope, and all such modifications and variations shall fall within the scope of the present disclosure.

Claims

1. A data stream processing method for digital pathological images, characterized in that, A processing system for providing a data stream of multiple digital pathological images to a computing system with multiple computational tasks, wherein the multiple computational tasks perform parallel computational analysis on each digital pathological image by multipleiating the data stream, the method comprising: The metadata unit of the processing system manages the node information of the image storage unit of the processing system, and registers the metadata unit and the image storage unit to the service registration unit of the processing system. The digital pathological image is acquired by the acquisition unit of the processing system and stored in the first storage unit of the image storage unit sensed by the acquisition unit; The first storage unit reads the metadata of the digital pathology image, obtains the byte size of the image sub-slices of the digital pathology image based on the metadata, divides the digital pathology image into byte-level blocks based on the byte size of the image sub-slices and in byte order to obtain multiple sub-blocks, stores the multiple sub-blocks and generates sub-slice information for each image sub-slice, and stores the metadata and the sub-slice information in the metadata unit. The metadata includes the color channels of the digital pathology image, the image level, and the pixel width and height of each image sub-slice at the image level. The byte size of each image sub-slice is the pixel width and height of the image sub-slice multiplied by the number of color channels. The sub-slice information also includes the byte size of the image sub-slice and its corresponding image level. Each sub-block includes multiple image sub-slices in adjacent regions. The sub-slice information includes the sub-block in which the image sub-slice is located and its byte offset relative to the sub-block. The system receives a data request to acquire the data stream and sends it to the parsing unit of the processing system. The parsing unit obtains information about each sub-slice in at least one image sub-slice corresponding to the data stream from the metadata unit. Then, based on the byte offset in the information of each sub-slice in the at least one image sub-slice and the byte size of the image sub-slice, it reads the data stream corresponding to the at least one image sub-slice from the first storage unit and returns it to the computing system.

2. The method according to claim 1, characterized in that: If the digital pathological image is stored in the first storage unit, the parsing unit obtains the storage path of the sub-block where the image sub-slice is located and the byte offset relative to the sub-block based on the sub-slice information of each image sub-slice, and then reads the data block of the image sub-slice size at the byte offset from the storage path as the data stream corresponding to the image sub-slice; If the digital pathology image is stored in a second storage unit for storing the full image of the digital pathology image, the parsing unit obtains the storage path of the full image of the digital pathology image and the byte offset relative to the full image of the digital pathology image, and then reads a data block of the size of the image sub-slice at the byte offset from the storage path as the data stream corresponding to the image sub-slice.

3. The method according to claim 2, characterized in that: When the parsing unit reads the digital pathology image, it shares the handle opened by the digital pathology image and reads the data stream corresponding to at least one image sub-slice at a time. and / or When the parsing unit reads the sub-block, it shares the handle opened by the sub-block and reads the data stream corresponding to the at least one image sub-slice from the sub-block one at a time.

4. The method according to claim 1, characterized in that: The byte order is either ascending or descending.

5. The method according to claim 1, characterized in that: Each sub-block has the same byte size. When the first storage unit divides the digital pathological image into blocks, it divides the data stream corresponding to the image sub-slice into the multiple sub-blocks in byte order. When the remaining space of a sub-block is insufficient to store the data stream corresponding to an image sub-slice, blank data is used to fill the remaining space.

6. The method according to claim 1, characterized in that: The computing system caches the data streams corresponding to each digital pathology image so that the multiple computing tasks can perform the computational analysis on each digital pathology image in parallel by reusing the data streams. After completing the computational analysis of each digital pathology image, the data streams are deleted from the cache.

7. The method according to claim 1, characterized in that: The size of the sub-block is greater than the size of the data stream.

8. The method according to claim 1, characterized in that: The byte size of the sub-block is positively correlated with the byte size of the digital pathology image.

9. The method according to claim 1, characterized in that: Multiple parsing units are deployed in a load-balanced manner to distribute the data requests among the parsing units in order to improve load capacity.

Images