Method, device and readable storage medium for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom

Abstract

The present invention proposes a method, device, and readable storage medium for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom. By constructing a multi-resolution pyramid, the medical image is downsampled layer by layer into multiple resolution levels, each layer is divided into standard-sized blocks, and dynamic loading based on the window and cache management through the LRU algorithm, solving the problems of GB-level image loading lag and memory overflow. The WebSocket protocol is used to achieve real-time synchronization of multi-user annotation operations, supporting a three-level semantic labeling system including lesion area, cell type, and tissue classification, and combining a timestamp priority strategy to resolve annotation conflicts and support version backtracking. A medical compliance log system is designed to record case information and other information, and the entire operation process is traceable through local Sqlite cache and cloud HIPAA encryption synchronization. The present invention significantly improves the efficiency of medical image browsing and annotation collaboration, meeting medical data compliance requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and computer-aided diagnosis, and in particular to a method, device, and readable storage medium thereof for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom. Background Art

[0002] In the field of medical imaging diagnosis, efficient browsing and accurate annotation of GB-level pathological images such as whole slide images (WSI) are core requirements for clinical diagnosis and multidisciplinary consultation. Existing technologies have the following shortcomings:

[0003] 1. Inefficient image rendering: The application uses fixed slicing rules, which prevent the slicing granularity from being dynamically adjusted based on network bandwidth and viewport range. This results in loading delays in high-latency areas, memory usage of up to 2-3GB, and a browser plug-in crash rate exceeding 15%.

[0004] 2. Isolation of annotation functions: The pathology image annotation module is separated from the browsing function, requiring physicians to frequently switch interfaces, resulting in low operational efficiency. The annotation system only supports basic tags and lacks the ability to structuredly annotate multi-layer semantic tags (such as lesion grade and cell type).

[0005] 3. Lack of collaborative mechanism: When multiple doctors consult, they rely on repeated transmission of annotation files. There is a lack of real-time synchronization mechanism, annotation conflicts are difficult to resolve, and historical versions cannot be traced.

[0006] 4. Insufficient compliance: The existing system does not record and mark operation logs or the logs are incomplete, which cannot meet medical compliance requirements such as HIPAA, and it is difficult to achieve full-link traceability of operation processes.

[0007] In summary, existing technologies have significant shortcomings in real-time browsing, collaborative annotation and compliance management of large-scale medical images, and there is an urgent need for a systematic solution that takes into account efficiency, collaboration and compliance. Summary of the Invention

[0008] The embodiments of the present invention provide a method, device and readable storage medium for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom. These methods address the problems existing in current technologies, such as loading jams and memory overflows caused by fixed slices, separation of annotation functions, lack of multi-user collaborative mechanisms and lack of medical compliance logs, which make it difficult to meet the needs of efficient annotation and collaborative consultation of GB-level medical images.

[0009] The core technology of this invention is to realize lightweight rendering of medical images by constructing an adaptive multi-resolution pyramid, build a multi-user collaborative annotation framework by combining WebSocket real-time synchronization technology, and design an operation log evidence storage system that complies with medical compliance, so as to realize real-time browsing of GB-level medical images, multi-layer semantic annotation and full-process traceability.

[0010] In a first aspect, the present invention provides a method for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom, the method comprising the following steps:

[0011] S1. Preprocess the medical image to generate a multi-resolution pyramid. The multi-resolution pyramid includes multiple layers of decreasing resolution obtained by downsampling the original image layer by layer. Each layer of the image is divided into blocks of standard size.

[0012] S2, based on the current window area, dynamically load the image blocks of the corresponding level in the multi-resolution pyramid, and manage the block data in the memory through the cache elimination strategy;

[0013] S3, synchronize multi-user annotation operations through real-time communication protocols, perform structured annotation of medical images based on a multi-layer semantic tag system, and adopt an annotation version management strategy to resolve multi-user annotation conflicts;

[0014] S4. Record and annotate the operation log, which includes case ID, user ID, operation time, operation type, and affected area information, and realizes local caching of the log and encrypted synchronization with the cloud.

[0015] Furthermore, step S1 specifically includes:

[0016] Convert medical images into a standard format that supports multi-resolution pyramid structures and extract image metadata;

[0017] The Lanczos resampling algorithm is used to downsample the current layer image to generate the next layer image, and the edge blocks are filled to a standard size.

[0018] Furthermore, in step S3, the multi-layer semantic labeling system includes three levels of labels, which are:

[0019] First-level label used to mark the type of lesion area;

[0020] Secondary tags for annotating cell types;

[0021] A three-level label used to indicate organizational hierarchy.

[0022] Furthermore, in the S3 step, the real-time communication protocol is the WebSocket protocol, and the annotation operation is transmitted in a JSON format message containing the operation type, coordinates, and user ID;

[0023] The annotation version management strategy includes a timestamp-based overwriting mechanism and a historical version backtracking function.

[0024] Furthermore, in step S4, the operation log is cached in the local Sqlite database and automatically uploaded to the compliance server at preset intervals;

[0025] Cloud encryption synchronization uses HIPAA encryption standards.

[0026] In a second aspect, the present invention provides a multimodal medical image real-time annotation and collaborative browsing device based on adaptive DeepZoom, comprising:

[0027] Preprocessing module, used for format conversion and metadata extraction of medical images;

[0028] Multi-resolution pyramid building module, used to generate a multi-resolution pyramid. The pyramid consists of multiple levels obtained by layer-by-layer downsampling, and each layer of the image is divided into blocks of standard size;

[0029] A lightweight rendering module that dynamically loads image tiles of corresponding levels based on the viewport area and manages memory through cache eviction strategies.

[0030] Collaborative annotation module, used to synchronize multi-user annotation operations through real-time communication protocols, supporting multi-layer semantic tagging systems and annotation version management;

[0031] The medical compliance log module is used to record logs containing information about cases, users, operation time, operation type, and affected areas, and to achieve encrypted synchronization between local cache and cloud.

[0032] Furthermore, the multi-resolution pyramid construction module uses the Lanczos resampling algorithm for downsampling, and the edge blocks are processed to a standard size through padding.

[0033] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the above-mentioned adaptive DeepZoom-based multimodal medical image real-time annotation and collaborative browsing method.

[0034] In a fourth aspect, the present invention provides a readable storage medium, which stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, and the process includes the above-mentioned multimodal medical image real-time annotation and collaborative browsing method based on adaptive DeepZoom.

[0035] The main contributions and innovations of the present invention are as follows:

[0036] 1. Rendering efficiency is significantly improved:

[0037] Based on adaptive DeepZoom technology, window area tiles are dynamically loaded, reducing memory usage from 2-3GB to <1GB, shortening the first loading time by 80% (from 5 seconds to <1 second), and reducing the freeze rate from 30% to <5%, solving the problem of crashes when browsing GB-level images.

[0038] 2. Breakthrough in annotation efficiency and collaboration:

[0039] It integrates annotation and browsing functions, supports structured annotation of three-level semantic tags (lesion area → cell type → tissue grade); realizes multi-user real-time synchronization based on WebSocket, and synchronizes annotation operations to all clients in seconds, avoiding file transfer and improving consultation efficiency by more than 70%.

[0040] 3. Medical compliance assurance:

[0041] Structured logs containing multi-dimensional information such as case ID, operation time, and comparison of before and after values ​​are recorded. Through local Sqlite cache and HIPAA encryption synchronization on the cloud, the entire annotation operation process is traceable, meeting strict medical data compliance requirements.

[0042] 4. Versatility and scalability:

[0043] It supports standard formats such as SVS and TIFF, is compatible with multimodal medical images (such as pathological slices, MRI / CT), and adapts to different network environments (automatically adjusting slice granularity in high-latency scenarios), and has broad clinical application prospects.

[0044] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below so that other features, objects, and advantages of the invention are more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0046] Figure 1 is a flowchart of a multimodal medical image real-time annotation and collaborative browsing method based on adaptive DeepZoom according to an embodiment of the present invention;

[0047] Figure 2 FIG. 4 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The implementations described in the following exemplary embodiments are not intended to represent all implementations consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0049] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0050] Existing technologies have problems such as loading delays and memory overflow caused by fixed slices, separation of annotation functions, lack of multi-user collaboration mechanism and lack of medical compliance logs, which makes it difficult to meet the needs of efficient annotation and collaborative consultation of GB-level medical images.

[0051] Based on this, the present invention realizes lightweight rendering of medical images by constructing an adaptive multi-resolution pyramid, and combines WebSocket real-time synchronization technology to build a multi-user collaborative annotation framework to solve the problems existing in the existing technology.

[0052] Example 1

[0053] The present invention aims to propose a multimodal medical image real-time annotation and collaborative browsing method based on adaptive DeepZoom. Figure 1 , the method comprising:

[0054] Step 1: Raw image preprocessing

[0055] The input whole-slice image (WSI) format is SVS, with a resolution of 26520×26016, a scanning resolution of 0.24 μm / pixel, and a real size of 6.34 mm×6.22 mm.

[0056] The pre-processing unit converts the data into a TIFF format that supports the pyramid structure and extracts metadata (resolution, color space RGB / RGBA, pixel size, etc.).

[0057] The format of the input whole-slice image (WSI) can be SVS or TIFF.

[0058] Step 2: Pyramid level generation

[0059] Hierarchical design: Generate a 5-layer pyramid (usually 5-10 layers). The 0th layer is the original layer (26520×26016). The first layer is downsampled to 13260×13008 through 2x Lanczos, with a scanning resolution of 0.48μm / pixel. And so on, until the resolution of the 4th layer meets the 16μm / pixel requirement.

[0060] Blocking and padding: Each layer of the image is divided into 256×256 pixel tiles. Incomplete tiles at the edges (such as the last row with a width of 156 pixels) are padded with white pixels to the standard size of 256×256 to ensure block integrity.

[0061] Compressed storage: Each tile is compressed with JPEG and stored with a spatial index grid to record the tile coordinates.

[0062] Step 3: Dynamic loading and memory management

[0063] The client uses the WebAssembly (WASM) rendering engine to calculate the set of tiles to be loaded (such as 4×4 tiles) based on the current window coordinates (such as the 1024×1024 range of the second layer of the pyramid corresponding to the browser display area).

[0064] The LRU algorithm is used to manage the cache. When the memory usage is close to 1GB, the least recently used tile is eliminated to ensure that the memory is stable within 800MB.

[0065] In this embodiment, multi-user collaborative annotation is implemented as follows:

[0066] Scenario: Three doctors conduct real-time consultation and annotation on the same case.

[0067] Step 1: Labeling system application

[0068] Select “tumor region” in the primary label and draw a rectangular region (x=100, y=200, width 50, height 30) on the image layer 0.

[0069] The secondary label marks the cell type in this area as "cancer cell", and the tertiary label marks the tissue grade as "G2".

[0070] Among them, the labeling system is generally a three-level labeling system, which is specifically defined as follows:

[0071] 1. Primary label: lesion area (e.g., tumor / non-tumor);

[0072] 2. Secondary label: cell type (e.g., lymphocyte / cancer cell);

[0073] 3. Third-level label: organizational classification (such as G1 / G2 / G3).

[0074] Step 2: Operation synchronization and conflict resolution

[0075] Doctor A submits the annotation operation, generating a JSON message:

[0076] {

[0077] "type": "add_rect",

[0078] "id": "rect_1",

[0079] "x": 100,

[0080] "y": 200,

[0081] "width": 50,

[0082] "height": 30,

[0083] "user": "user_a",

[0084] "timestamp": "2025-05-14T15:30:00.123"

[0085] }

[0086] After receiving the message, the server verifies the user's permissions and broadcasts it to the clients of doctors B and C. Each client updates the local annotation status and renders it.

[0087] If doctor B submits a new annotation in the same area, the server will overwrite the old annotation with the newer annotation (timestamp 2025-05-14T15:30:05.456) according to the timestamp priority strategy, and record the version history for backtracking.

[0088] The timestamp generation rule is: yyyy-MM-dd HH:mm:ss.fff;

[0089] The version number is defined based on the rule submitted by the timestamp, such as the format of 202504191721101, and the storage structure is the annotated protocol structure json format.

[0090] In this embodiment, the medical compliance log system is implemented as follows:

[0091] Operation scenario: A physician modifies the tissue grading labeling of a certain area.

[0092] Step 1: Logging

[0093] The record fields include:

[0094] case_id: "P001"

[0095] user_id: "doc_zhang"

[0096] timestamp: "2025-05-14T15:35:10.789"

[0097] operation: "modify_label"

[0098] old_value: "G1"

[0099] new_value: "G2"

[0100] affected_region: {x: 1024, y: 2048, width: 512, height: 512}

[0101] Step 2: Storage and Synchronization

[0102] Uncommitted logs are cached locally through the Sqlite database, and a cloud encryption synchronization task is triggered every 5 minutes.

[0103] During synchronization, log data is encrypted with AES-256 (HIPAA compliant) and transferred to a compliant server for storage, ensuring that operation records are traceable and cannot be tampered with.

[0104] In this embodiment, the system integration and effect verification results are as follows:

[0105] Hardware environment: The client is an ordinary PC (8GB memory), the server is configured with a 2-core 4GB cloud host, and the network bandwidth is 10Mbps.

[0106] The performance data is as follows:

[0107]

[0108] Data source: Comparison data between the traditional solution and the improved solution obtained through AB testing. It can be seen that the present invention has the following advantages over the traditional method:

[0109] 1. Memory optimization:

[0110] Traditional solutions: rely on extensive resource release strategies (such as scheduled cleanup) or redundant data caching.

[0111] This invention reduces the memory usage through precise object lifecycle management (such as reference counting + weak references) and on-demand loading (LazyLoad).

[0112] 2. Solution to loading jam:

[0113] Traditional solution: serial loading or blocking IO.

[0114] The present invention eliminates lag through asynchronous loading, resource chunking, or parallel computing (such as GPU acceleration).

[0115] In this embodiment, for ease of understanding, the following is a supplementary explanation of the professional terms and technical details involved in the present invention, which is explained based on the content of the application documents and in combination with common knowledge in the field:

[0116] 1. Whole Slide Image (WSI)

[0117] Definition: Digital scanning is the process of converting traditional pathology slides (such as glass slides) into high-resolution digital images. The resolution can usually reach tens of thousands of pixels (such as 26520×26016) and the file size can reach GB levels. The microstructure of the tissue sample (such as cell morphology and lesion area) can be fully preserved.

[0118] Features: It supports full-size browsing and local high-precision magnification. It is the core data carrier for pathological diagnosis, teaching and research, but it has extremely high requirements for storage, transmission and rendering technologies.

[0119] 2. DeepZoom Technology

[0120] Core principle: By constructing a multi-resolution pyramid (image pyramid), the original high-resolution image is downsampled layer by layer into multiple low-resolution levels (such as 5-10 layers), and each layer is further divided into fixed-size image blocks (tiles, such as 256×256 pixels).

[0121] Advantages:

[0122] On-demand loading: Dynamically load tiles at the corresponding level and position based on the user's browsing window range, avoiding memory overflow caused by loading the entire image at once;

[0123] Adaptive rendering: loads high-resolution layer tiles when zooming in, and loads low-resolution tiles when zooming out, balancing image quality and performance.

[0124] 3. WebAssembly (WASM)

[0125] Definition: A binary instruction format that allows compiled code written in high-level languages ​​(such as C / C++ and Rust) to run efficiently in a browser, with performance close to that of native applications.

[0126] Role in the present invention:

[0127] Implement a dynamic loading algorithm for image pyramids, directly operate memory to optimize tile rendering efficiency, and resolve JavaScript performance bottlenecks when processing GB-level images;

[0128] Cooperate with the browser Canvas or WebGL interface to achieve fast rendering and updating of the window area.

[0129] 4. HIPAA (Health Insurance Portability and Accountability Act)

[0130] Compliance requirements:

[0131] Data encryption: Encrypt transmitted and stored medical data (such as annotation logs). This invention uses algorithms such as AES-256 to meet transmission encryption requirements.

[0132] Access control: restricts unauthorized users from accessing medical data. The logging system implements this through user authentication (such as tokens) and session management.

[0133] Audit trail: Completely record the history of data operations. This invention meets audit requirements through structured logs (including timestamps, user IDs, and operation details).

[0134] 5. LRU algorithm (Least Recently Used)

[0135] Cache eviction strategy:

[0136] When the number of tiles cached in memory exceeds the threshold, the tiles that have not been accessed for the longest time are eliminated, and the tiles that are accessed frequently are retained;

[0137] The present invention is used to manage image tile cache, reducing memory usage from 2-3GB of traditional solutions to <1GB, avoiding browser crashes.

[0138] 6. Lanczos resampling

[0139] Image scaling algorithm:

[0140] When generating pyramid levels, the upper layer image is downsampled (e.g., reduced by 2 times) through the Lanczos filter, achieving both high reconstruction accuracy and anti-aliasing effect, which is suitable for detail preservation in medical images.

[0141] Compared with algorithms such as bilinear and bicubic interpolation, it strikes a balance between speed and quality, avoiding artifacts or detail loss caused by downsampling.

[0142] 7. Multimodal medical images

[0143] Definition: Medical images of the same patient or tissue acquired using different imaging techniques, including:

[0144] Pathological images (WSI), MRI (magnetic resonance imaging), CT (computed tomography), ultrasound images, etc.;

[0145] The present invention supports unified rendering and annotation of multimodal data and is compatible with standard formats such as SVS and TIFF through a preprocessing module.

[0146] 8. Semantic Tagging System

[0147] Three-level hierarchical structure:

[0148] Level 1 label: macroscopic lesion classification (such as “tumor area” and “normal tissue”), used to define the nature of the annotated area;

[0149] Secondary labels: mesoscopic cell types (e.g., “cancer cell,” “lymphocyte,” “epithelial cell”), which refine the cell composition within the region;

[0150] Level 3 label: Microscopic tissue grading (such as G1 / G2 / G3, representing the degree of tumor differentiation), providing a quantitative basis for pathological diagnosis.

[0151] Advantages: Structured annotated data facilitates subsequent data analysis (such as AI model training and case statistics), meeting the multi-dimensional needs of clinical diagnosis.

[0152] 9. WebSocket Protocol

[0153] Real-time communication technology:

[0154] Establish a persistent connection between the client and the server, supporting bidirectional real-time data transmission (such as synchronization of annotation operations) with latency as low as milliseconds;

[0155] The present invention is used to broadcast annotation operation messages (such as "add rectangle" and "modify label") to ensure that the annotation status of multiple users is consistent in real time.

[0156] 10. Spatial Index Grid

[0157] Data management mechanism:

[0158] Grid-number the tile positions of each layer of the pyramid (such as row numbers and column numbers) and establish a mapping relationship between "layer-coordinate-tile file";

[0159] Supports quick query of the tile set corresponding to the window area, improving loading efficiency and avoiding blind retrieval of all tiles.

[0160] 11. A / B Testing

[0161] AB testing is a statistical method that verifies the effectiveness of a solution through controlled experiments. The core idea is to randomly divide users or test samples into two groups:

[0162] Group A (control group): using existing solutions (such as traditional medical image browsing systems);

[0163] Group B (experimental group): used the improved solution (such as the adaptive DeepZoom system of the present invention).

[0164] By comparing the performance of the two groups on the same indicators (such as loading time, memory usage, user operation efficiency, etc.), the advantages of the improvement plan can be scientifically evaluated. The core steps include:

[0165] Control variables: Ensure that the only difference between the two test environments is the technical solution to be verified (e.g., the pyramid rendering engine of the present invention vs. the traditional fixed slicing method);

[0166] Data collection: Record two sets of performance data under the same operating scenario (e.g., "memory usage 2-3GB vs <1GB" mentioned in the example);

[0167] Statistical analysis: Hypothesis testing (such as t-test) was used to determine whether the differences were statistically significant and to exclude accidental factors.

[0168] Specific applications in the present invention:

[0169] Rendering performance verification:

[0170] Control group: traditional scheme (fixed slices, extensive memory management);

[0171] Experimental group: the solution of the present invention (adaptive pyramid, LRU cache, WASM dynamic loading).

[0172] Test indicators: memory usage (2-3GB vs <1GB), first loading time (5 seconds vs <1 second), and lag rate (30% vs <5%).

[0173] It can also include collaborative annotation efficiency verification:

[0174] Control group: single-user annotation + file transfer collaboration;

[0175] Experimental group: multi-user real-time synchronous annotation (WebSocket protocol + conflict resolution strategy).

[0176] Test indicators: synchronization delay of annotation operations, time required to resolve conflicts during multi-user collaboration, and improvement rate of physician operation efficiency.

[0177] Compliance log reliability verification can also be included:

[0178] Control group: no log or unstructured log record;

[0179] Experimental group: the structured log system of the present invention (Sqlite cache + HIPAA encryption synchronization).

[0180] Test indicators: log integrity (field coverage), encrypted transmission time, and compliance audit passability.

[0181] Example 2

[0182] Based on the same concept, the present invention also proposes a multimodal medical image real-time annotation and collaborative browsing device based on adaptive DeepZoom, comprising:

[0183] Preprocessing module, used for format conversion and metadata extraction of medical images;

[0184] Multi-resolution pyramid construction module, used to generate a multi-resolution pyramid. The pyramid consists of multiple levels obtained by layer-by-layer downsampling. Each layer of the image is divided into blocks of standard size. Downsampling is performed using the Lanczos resampling algorithm, and edge blocks are padded to a standard size.

[0185] A lightweight rendering module that dynamically loads image tiles of corresponding levels based on the viewport area and manages memory through cache eviction strategies.

[0186] The collaborative tagging module is used to synchronize multi-user tagging operations through a real-time communication protocol, supporting a multi-layer semantic tagging system and tagging version management. It implements real-time broadcasting of tagging operations based on the WebSocket protocol, and the multi-layer semantic tagging system includes a three-level tagging hierarchy.

[0187] The medical compliance log module is used to record logs containing information about cases, users, operation time, operation type, and affected areas, and to synchronize local caches with cloud encryption. Cloud encryption synchronization uses HIPAA encryption standards, and local cache is implemented through a Sqlite database.

[0188] Example 3

[0189] This embodiment also provides an electronic device, referring to Figure 2 , includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0190] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits for implementing the embodiments of the present invention.

[0191] Memory 404 may include a large-capacity memory 404 for data or instructions. By way of example, and not limitation, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to the data processing device. In certain embodiments, memory 404 is non-volatile memory. In certain embodiments, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0192] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0193] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any one of the multimodal medical image real-time annotation and collaborative browsing methods based on adaptive DeepZoom in the above embodiments.

[0194] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0195] Transmission device 406 can be used to receive or transmit data via a network. Specific examples of such networks may include wired or wireless networks provided by the electronic device's communications provider. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0196] The input / output device 408 is used to input or output information. In this embodiment, the input information may be 11111, and the output information may be 11111, etc.

[0197] Example 4

[0198] This embodiment also provides a readable storage medium, which stores a computer program. The computer program includes program code for controlling a process to execute a process. The process includes the method for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom according to embodiment one.

[0199] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be repeated here.

[0200] In general, various embodiments may be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0201] Embodiments of the present invention can be implemented by computer software, which is executable by the data processor of the mobile device, such as in the processor entity, or is implemented by hardware, or is implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer executable components configured to perform the embodiment when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logical circuits, boxes and functions, or a combination of program steps and logical circuits, boxes and functions. The software can be stored in physical media such as memory chips or storage blocks implemented in the processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. Physical media is non-transient media.

[0202] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0203] The above embodiments merely illustrate several embodiments of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. A multimodal medical image real-time annotation and collaborative browsing method based on adaptive DeepZoom, characterized by: The following steps are involved: S1. Preprocessing a medical image to generate a multi-resolution pyramid, wherein the multi-resolution pyramid includes multiple layers of decreasing resolution obtained by downsampling the original image layer by layer, and each layer of the image is divided into blocks of standard size; S2. Based on the current window area, dynamically load the image blocks of the corresponding level in the multi-resolution pyramid, and manage the block data in the memory through the LRU cache elimination strategy; S3, synchronize multi-user annotation operations through real-time communication protocols, perform structured annotation of medical images based on a multi-layer semantic tag system, and adopt an annotation version management strategy to resolve multi-user annotation conflicts; The real-time communication protocol is the WebSocket protocol, and the annotation operation is transmitted in a JSON format message containing the operation type, coordinates, and user ID; The annotation version management strategy includes a timestamp-based overwriting mechanism and a historical version backtracking function; the multi-layer semantic tag system includes three levels of tags, which are: First-level label used to mark the type of lesion area; Secondary tags for annotating cell types; Three-level labels for marking organizational levels; S4. A log is created to record the annotation operation, which includes the case ID, user ID, operation time, operation type, and affected area information, and implements local caching of the log and encrypted synchronization in the cloud. The log is cached in the local Sqlite database and automatically uploaded to the compliance server at preset intervals. The cloud-based encrypted synchronization adopts the HIPAA encryption standard.

2. The method for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom according to claim 1, characterized in that: Step S1 specifically includes: Convert medical images into a standard format that supports multi-resolution pyramid structures and extract image metadata; The Lanczos resampling algorithm is used to downsample the current layer image to generate the next layer image, and the edge blocks are filled to the standard size.

3. A multimodal medical image real-time annotation and collaborative browsing device based on adaptive DeepZoom, characterized by: include: Preprocessing module, used for format conversion and metadata extraction of medical images; A multi-resolution pyramid construction module is used to generate a multi-resolution pyramid, wherein the pyramid includes multiple levels obtained by layer-by-layer downsampling, and each layer of the image is divided into blocks of standard size; A lightweight rendering module that dynamically loads image tiles of the corresponding level based on the viewport area and manages tile data in memory through an LRU cache eviction strategy. Collaborative annotation module, which is used to synchronize multi-user annotation operations through a real-time communication protocol, perform structured annotation of medical images based on a multi-layer semantic tagging system, and adopt an annotation version management strategy to resolve multi-user annotation conflicts; The real-time communication protocol is the WebSocket protocol, and the annotation operation is transmitted in a JSON format message containing the operation type, coordinates, and user ID; The annotation version management strategy includes a timestamp-based overwriting mechanism and a historical version backtracking function; the multi-layer semantic tag system includes three levels of tags, which are: First-level label used to mark the type of lesion area; Secondary tags for annotating cell types; Three-level labels for marking organizational levels; The medical compliance log module is used to record logs containing information about cases, users, operation time, operation type, and affected areas, and to achieve local caching and cloud-based encrypted synchronization; the logs are cached in the local Sqlite database and automatically uploaded to the compliance server at preset intervals; the cloud-based encrypted synchronization adopts the HIPAA encryption standard.

4. The multimodal medical image real-time annotation and collaborative browsing device according to claim 3, characterized in that: The multi-resolution pyramid construction module uses the Lanczos resampling algorithm to perform downsampling, and the edge blocks are filled to the standard size.

5. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the adaptive DeepZoom-based multimodal medical image real-time annotation and collaborative browsing method according to any one of claims 1 to 2.

6. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes the adaptive DeepZoom-based multimodal medical image real-time annotation and collaborative browsing method according to any one of claims 1 to 2.

Images