Multi-modal medical image real-time labeling and collaborative browsing method and device based on adaptive DeepZoom and readable storage medium of multi-modal medical image real-time labeling and collaborative browsing method and device

Abstract

The invention provides a multi-modal medical image real-time labeling and collaborative browsing method and device based on adaptive DeepZoom and a readable storage medium thereof. A multi-resolution pyramid is constructed, a medical image is downsampled into multiple resolution levels layer by layer, each layer is divided into standard size blocks, dynamic loading is carried out based on a window, and cache management is carried out through an LRU algorithm. The problems of loading lagging and memory overflow of the GB-level image are solved; real-time synchronization of multi-user labeling operation is achieved through a WebSocket protocol, a three-level semantic label system containing lesion areas, cell types and tissue grading is supported, labeling conflicts are solved in combination with a timestamp priority strategy, and version backtracking is supported; a medical compliance log system is designed, information such as cases is recorded, and the whole operation process can be traced through synchronization of local Sqlite cache and cloud HIPAA encryption. According to the method, the medical image browsing efficiency and the labeling collaboration are remarkably improved, and the medical data compliance requirement is met.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing and computer-aided diagnosis, and particularly to a method, device and readable storage medium for real-time annotation and collaborative browsing of multi-modal medical images based on adaptive DeepZoom. Background Art

[0002] In the field of medical image diagnosis, efficient browsing and accurate annotation of GB-level pathological images such as whole-slide images (WSI) are the core requirements of clinical diagnosis and treatment and multi-disciplinary consultations. The existing technologies have the following defects: 1. Low image rendering efficiency: Using a fixed slicing rule, it is impossible to dynamically adjust the slice granularity according to the network bandwidth and viewport range, resulting in loading lags in high-latency areas, with a memory occupancy of up to 2 - 3 GB and a browser plugin crash rate exceeding 15%.

[0003] 2. Isolation of annotation functions: The pathological image annotation module is separated from the browsing function, and doctors need to frequently switch interfaces, resulting in low operation efficiency; the annotation system only supports basic tags and lacks the structured annotation ability for multi-layer semantic tags (such as lesion grading, cell type).

[0004] 3. Lack of collaborative mechanism: When multiple doctors conduct consultations, they rely on repeated transmission of annotation files and lack a real-time synchronization mechanism, making it difficult to solve annotation conflicts and impossible to trace historical versions.

[0005] 4. Insufficient compliance: The existing systems do not record annotation operation logs or the logs are incomplete, unable to meet medical compliance requirements such as HIPAA, and it is difficult to achieve full-link traceability of the operation process.

[0006] In summary, the existing technologies have significant deficiencies in real-time browsing, collaborative annotation, and compliance management of large-scale medical images, and there is an urgent need for a systematic solution that takes into account efficiency, collaboration, and compliance. Summary of the Invention

[0007] Embodiments of the present invention provide a method, device and readable storage medium for real-time annotation and collaborative browsing of multi-modal medical images based on adaptive DeepZoom, aiming at problems such as loading lags and memory overflow caused by fixed slicing in current technologies, separation of annotation functions, lack of multi-user collaborative mechanisms, and lack of medical compliance logs, which are difficult to meet the requirements of efficient annotation and collaborative consultations of GB-level medical images.

[0008] The core technology of the present invention mainly realizes lightweight rendering of medical images by constructing an adaptive multi-resolution pyramid, constructs a multi-user collaborative annotation framework in combination with WebSocket real-time synchronization technology, and designs an operation log evidence storage system that complies with medical compliance, so as to achieve real-time browsing, multi-layer semantic annotation, and full-process traceability of GB-level medical images.

[0009] In a first aspect, the present invention provides a real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom, and the method includes the following steps: S1. Preprocess the medical image to generate a multi-resolution pyramid, which includes multiple levels with decreasing resolutions obtained by downsampling layer by layer from the original image, and each layer of the image is divided into standard-sized blocks; S2. Dynamically load the image blocks of the corresponding level in the multi-resolution pyramid based on the current viewport area, and manage the block data in memory through a cache eviction policy; S3. Synchronize multi-user annotation operations through a real-time communication protocol, perform structured annotation on the medical image based on a multi-layer semantic label system, and adopt an annotation version management strategy to solve multi-user annotation conflicts; S4. Record the annotation operation log, which includes case identification, user identification, operation time, operation type, and affected area information, and implement local caching and cloud encryption synchronization of the log.

[0010] Further, the specific steps of S1 include: Convert the medical image into a standard format that supports the multi-resolution pyramid structure, and extract the image metadata; Use the Lanczos resampling algorithm to downsample the current layer of the image to generate the next-level image, and the edge blocks are processed by padding to the standard size.

[0011] Further, in step S3, the multi-layer semantic label system includes three levels of labels, which are in sequence: The first-level label for annotating the type of lesion area; The second-level label for annotating the cell type; The third-level label for annotating the tissue grade.

[0012] Further, in step S3, the real-time communication protocol is the WebSocket protocol, and the annotation operation is transmitted as a JSON format message including the operation type, coordinates, and user ID; The annotation version management strategy includes a timestamp-based overwrite mechanism and a historical version backtracking function.

[0013] Further, in step S4, the operation log is cached through a local Sqlite database and automatically uploaded to a compliance server at preset intervals; The cloud encryption synchronization adopts the HIPAA encryption standard.

[0014] In a second aspect, the present invention provides a real-time annotation and collaborative browsing device for multi-modal medical images based on adaptive DeepZoom, including: A preprocessing module for format conversion and metadata extraction of medical images; A multi-resolution pyramid construction module for generating a multi-resolution pyramid, which includes multiple levels obtained by successive downsampling, and each level of image is divided into standard-sized blocks; A lightweight rendering module for dynamically loading corresponding-level image blocks based on the viewport area and managing memory through a cache eviction strategy; A collaborative annotation module for synchronizing multi-user annotation operations through a real-time communication protocol, supporting a multi-layer semantic label system and annotation version management; A medical compliance log module for recording logs containing case, user, operation time, operation type, and affected area information, and implementing local cache and cloud encryption synchronization.

[0015] Furthermore, the multi-resolution pyramid construction module uses the Lanczos resampling algorithm for downsampling, and the edge blocks are processed by padding to the standard size.

[0016] In a third aspect, the present invention provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the above-mentioned real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom.

[0017] In a fourth aspect, the present invention provides a readable storage medium, in which a computer program is stored. The computer program includes program codes for controlling a process to execute the process, and the process includes the above-mentioned real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom.

[0018] The main contributions and innovations of the present invention are as follows: 1. Significantly improved rendering efficiency: Based on the adaptive DeepZoom technology, the tiles in the viewport area are dynamically loaded, and the memory occupancy is reduced from 2 - 3GB to <1GB. The first loading time is shortened by 80% (from 5 seconds to <1 second), and the stuttering rate is reduced from 30% to <5%, solving the problem of GB-level image browsing crashes.

[0019] 2. Breakthrough in annotation efficiency and collaboration: Integrate the annotation and browsing functions, support structured annotation of three-level semantic labels (lesion area → cell type → tissue grade); realize multi-user real-time synchronization based on WebSocket, and the annotation operations are synchronized to all clients in seconds, avoiding file transmission, and the consultation efficiency is improved by more than 70%.

[0020] 3. Guarantee of medical compliance: Record structured logs containing multi-dimensional information such as case IDs, operation times, and pre- and post-value comparisons. Through local Sqlite caching and cloud HIPAA encryption synchronization, the entire process of annotation operations can be traced, meeting strict medical data compliance requirements.

[0021] 4. Generalizability and extensibility: Support standard formats such as SVS and TIFF, be compatible with multi-modal medical images (such as pathological sections, MRI / CT), adapt to different network environments (automatically adjust slice granularity in high-latency scenarios), and have broad clinical application prospects.

[0022] Details of one or more embodiments of the present invention are set forth in the following drawings and description to make other features, objects, and advantages of the present invention more concise and understandable. Brief Description of the Drawings

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The illustrative embodiments and descriptions thereof are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flowchart of a method for real-time annotation and collaborative browsing of multi-modal medical images based on adaptive DeepZoom according to an embodiment of the present invention; Figure 2 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present invention. Detailed Embodiments

[0024] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0025] It should be noted that: In other embodiments, the steps of the corresponding methods are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0026] The prior art has problems such as loading lags and memory overflows caused by fixed slicing, separated annotation functions, lack of multi-user collaboration mechanisms, and lack of medical compliance logs, making it difficult to meet the requirements for efficient annotation and collaborative consultation of GB-level medical images.

[0027] Based on this, the present invention realizes lightweight rendering of medical images by constructing an adaptive multi-resolution pyramid, and constructs a multi-user collaborative annotation framework in combination with WebSocket real-time synchronization technology to solve the problems existing in the prior art.

[0028] Embodiment 1 The present invention aims to propose a real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom. Specifically, referring to Figure 1 , the method includes: Step 1: Preprocessing of the original image Input a whole-slide image (WSI) in the SVS format, with a resolution of 26520×26016, a scanning resolution of 0.24μm / pixel, and a true size of 6.34mm×6.22mm.

[0029] Convert it to the TIFF format that supports the pyramid structure through a preprocessing unit, and extract metadata (resolution, color space RGB / RGBA, pixel size, etc.).

[0030] Among them, the format of the input whole-slide image (WSI) can be SVS or TIFF.

[0031] Step 2: Generation of pyramid levels Level design: Generate 5 pyramid levels (usually 5 - 10 levels). The 0th level is the original level (26520×26016), the 1st level is downsampled by a factor of 2 using Lanczos to 13260×13008, with a scanning resolution of 0.48μm / pixel, and so on until the resolution of the 4th level meets the requirement of 16μm / pixel.

[0032] Tiling and padding: Each layer of the image is divided into tiles of 256×256 pixels. Incomplete tiles at the edges (such as the last row with a width of 156 pixels) are padded with white pixels to the standard size of 256×256 to ensure the integrity of tiling.

[0033] Compressed storage: Each tile is compressed using JPEG and stored with a spatial index grid recording the tile coordinates.

[0034] Step 3: Dynamic loading and memory management The client uses a WebAssembly (WASM) rendering engine to calculate the set of tiles to be loaded (such as 4×4 tiles) according to the current viewport coordinates (such as the 1024×1024 range corresponding to the 2nd level of the pyramid in the browser display area).

[0035] The LRU algorithm is used to manage the cache. When the memory occupancy approaches 1 GB, the least recently used tiles are evicted to ensure that the memory is stabilized within 800 MB.

[0036] In this embodiment, the multi-user collaborative annotation implementation method is as follows: Scenario: Three physicians conduct real-time consultation and annotation on the same case.

[0037] Step 1: Application of the label system Select the first-level label "tumor area" and draw a rectangular area (x = 100, y = 200, width 50, height 30) on the 0th layer of the image.

[0038] The second-level label annotates the cell type in this area as "cancer cells", and the third-level label annotates the tissue grade as "G2".

[0039] Among them, the label system generally has three levels of labels, and the specific definitions are as follows: 1. First-level label: lesion area (such as tumor / non-tumor); 2. Second-level label: cell type (such as lymphocyte / cancer cell); 3. Third-level label: tissue grade (such as G1 / G2 / G3).

[0040] Step 2: Operation synchronization and conflict resolution Physician A submits an annotation operation and generates a JSON message: { "type": "add_rect", "id": "rect_1", "x": 100, "y": 200, "width": 50, "height": 30, "user": "user_a", "timestamp": "2025-05-14T15:30:00.123" } After receiving the message, the server verifies the user's permission and broadcasts it to the clients of Physicians B and C. Each client updates the local annotation status and renders it.

[0041] If Physician B submits a new annotation in the same area, the server, according to the timestamp priority strategy, overwrites the old annotation with the newer annotation (timestamp 2025-05-14T15:30:05.456), and at the same time records the version history for traceability.

[0042] Among them, the timestamp generation rule: yyyy-MM-dd HH:mm:ss.fff; The version number is defined according to the rule of submitting the timestamp, such as the format of 202504191721101, and the storage structure is the marked protocol structure in json format.

[0043] In this embodiment, the implementation method of the medical compliance log system is as follows: Operation scenario: A doctor modifies the tissue grading label of a certain area.

[0044] Step 1: Log recording The recorded fields include: case_id: "P001" user_id: "doc_zhang" timestamp: "2025-05-14T15:35:10.789" operation: "modify_label" old_value: "G1" new_value: "G2" affected_region: {x: 1024, y: 2048, width: 512, height: 512} Step 2: Storage and synchronization Locally, the unsubmitted logs are cached through the Sqlite database, and a cloud encryption synchronization task is triggered every 5 minutes.

[0045] During synchronization, the log data is encrypted with AES-256 (meeting HIPAA requirements) and transmitted to the compliance server for storage to ensure that the operation records are traceable and cannot be tampered with.

[0046] In this embodiment, the system integration and effect verification results are as follows: Hardware environment: The client is an ordinary PC (8GB of memory), the server is configured with a 2-core 4GB cloud host, and the network bandwidth is 10Mbps.

[0047] The performance data is as follows:

[0048] Among them, the data source: Obtain the comparison data of the traditional scheme and the improved scheme through A / B testing. It can be seen that the present invention has the following advantages compared with the traditional method: 1. Memory optimization: Traditional scheme: Rely on a rough resource release strategy (such as cleaning regularly) or redundant data caching.

[0049] The present invention: By precise object lifecycle management (such as reference counting + weak references) and lazy loading, the memory occupancy level is reduced.

[0050] 2. Solution to loading lag: Traditional solution: Serial loading or blocking I / O is adopted.

[0051] The present invention: By asynchronous loading, resource chunking, or parallel computing (such as GPU acceleration), the lag is eliminated.

[0052] In this embodiment, for the convenience of understanding, the following is a supplementary explanation of the professional terms and their technical details involved in the present invention, which is described based on the content of the application file and combined with common knowledge in the field: 1. Whole Slide Image (WSI) Definition: A traditional pathological section (such as a glass section) is converted into a high-resolution digital image through a digital scanner. The resolution can usually reach tens of thousands of pixels (such as 26520×26016), and the file size reaches the GB level. It can completely retain the microscopic structure of the tissue sample (such as cell morphology, lesion area).

[0053] Features: It supports full-size browsing and local high-precision magnification. It is the core data carrier for pathological diagnosis, teaching, and research, but has extremely high requirements for storage, transmission, and rendering technologies.

[0054] 2. DeepZoom technology Core principle: By constructing a multi-resolution pyramid (image pyramid), the original high-resolution image is downsampled layer by layer into multiple low-resolution levels (such as 5 - 10 layers), and each layer is further divided into fixed-size image blocks (tiles, such as 256×256 pixels).

[0055] Advantages: Lazy loading: According to the viewport range browsed by the user, the corresponding level and position tiles are dynamically loaded, avoiding memory overflow caused by loading the entire image at once; Adaptive rendering: High-resolution level tiles are loaded when magnifying at a high magnification, and low-resolution tiles are loaded when reducing the browsing, balancing image quality and performance.

[0056] 3. WebAssembly (WASM) Definition: A binary instruction format that allows the compiled code of high-level languages (such as C / C++, Rust) to run efficiently in the browser, with performance close to native applications.

[0057] Role in the present invention: Implement a dynamic loading algorithm for image pyramids, directly operate on memory to optimize the tile rendering efficiency, and solve the performance bottleneck of JavaScript when processing GB-level images; Cooperate with the browser Canvas or WebGL interface to achieve fast rendering and updating of the viewport area.

[0058] 4. HIPAA (Health Insurance Portability and Accountability Act) Compliance requirements: Data encryption: Implement encryption for medical data (such as annotation logs) during transmission and storage. This invention uses algorithms such as AES-256 to meet the transmission encryption requirements; Access control: Restrict unauthorized users from accessing medical data. The logging system achieves this through user authentication (such as Token) and session management; Audit trail: Completely record the history of data operations. This invention meets the audit requirements through structured logs (including timestamps, user IDs, operation details).

[0059] 5. LRU algorithm (Least Recently Used) Cache eviction policy: When the number of tiles cached in memory exceeds the threshold, evict the tiles that have not been accessed for the longest time recently and retain the tiles that are frequently accessed; In this invention, it is used to manage the image tile cache, reducing the memory occupancy from 2 - 3GB in the traditional solution to <1GB, avoiding browser crashes.

[0060] 6. Lanczos resampling Image scaling algorithm: When generating pyramid levels, downsample the upper-layer image (such as reducing by a factor of 2) through the Lanczos filter, which has both high reconstruction accuracy and anti-aliasing effects and is suitable for retaining details of medical images; Compared with algorithms such as bilinear and bicubic interpolation, it achieves a balance between speed and quality, avoiding artifacts or detail loss caused by downsampling.

[0061] 7. Multi-modal medical images Definition: Refers to medical images of the same patient or tissue obtained through different imaging techniques, including: Pathological images (WSI), MRI (Magnetic Resonance Imaging), CT (Computed Tomography), ultrasound images, etc.; This invention supports the unified rendering and annotation of multi-modal data and is compatible with standard formats such as SVS and TIFF through a preprocessing module.

[0062] 8. Semantic Tag System Three - level hierarchical structure: First - level tags: Macro - lesion classification (such as "tumor area", "normal tissue"), used to define the nature of the annotation area; Second - level tags: Mesoscopic cell types (such as "cancer cells", "lymphocytes", "epithelial cells"), which refine the cell composition within the area; Third - level tags: Microscopic tissue grading (such as G1 / G2 / G3, representing the degree of tumor differentiation), providing a quantitative basis for pathological diagnosis.

[0063] Advantages: Structured annotation data facilitates subsequent data analysis (such as AI model training, case statistics), meeting the multi - dimensional requirements of clinical diagnosis.

[0064] 9. WebSocket Protocol Real - time communication technology: Establish a persistent connection between the client and the server, supporting two - way real - time data transmission (such as synchronization of annotation operations), with a latency as low as milliseconds; In the present invention, it is used to broadcast annotation operation messages (such as "add rectangle", "modify label") to ensure real - time consistency of the annotation status among multiple users.

[0065] 10. Spatial Index Grid Data management mechanism: Grid - number the tile positions of each layer of the pyramid (such as row number, column number), establishing a mapping relationship of "level - coordinate - tile file"; Support fast query of the tile set corresponding to the viewport area, improving the loading efficiency and avoiding blind retrieval of all tiles.

[0066] 11. AB Testing (A / B Testing) AB testing is a statistical method for verifying the effectiveness of a solution through a controlled experiment. The core idea is to randomly divide users or test samples into two groups: Group A (control group): Use the existing solution (such as a traditional medical image browsing system); Group B (experimental group): Use the improved solution (such as the adaptive DeepZoom system of the present invention).

[0067] By comparing the performance of the two groups on the same metrics (such as loading time, memory occupancy, user operation efficiency, etc.), scientifically evaluate the advantages of the improved solution. Its core steps include: Controlling variables: Ensure that the only difference between the two test environments is the technical solution to be verified (such as the pyramid rendering engine of the present invention vs the traditional fixed slicing method); Data collection: Record the performance data of two groups under the same operation scenarios (such as "memory occupancy 2 - 3GB vs <1GB" mentioned in the embodiments); Statistical analysis: Determine whether the differences are statistically significant through hypothesis testing (such as t - test) to exclude accidental factors.

[0068] Specific applications in the present invention: Rendering performance verification: Control group: Traditional solution (fixed slicing, extensive memory management); Experimental group: The solution of the present invention (adaptive pyramid, LRU cache, WASM dynamic loading).

[0069] Test metrics: Memory occupancy (2 - 3GB vs <1GB), first - load time (5 seconds vs <1 second), stuttering rate (30% vs <5%).

[0070] It can also include collaborative annotation efficiency verification: Control group: Single - user annotation + file - transfer collaboration; Experimental group: Multi - user real - time synchronous annotation (WebSocket protocol + conflict - resolution strategy).

[0071] Test metrics: Synchronization delay of annotation operations, time consumed for conflict resolution during multi - user collaboration, proportion of improvement in physician operation efficiency.

[0072] It can also include compliance log reliability verification: Control group: No log or unstructured log recording; Experimental group: The structured log system of the present invention (Sqlite cache + HIPAA - encrypted synchronization).

[0073] Test metrics: Log integrity (field coverage rate), encryption - transmission time, compliance audit pass rate.

[0074] Embodiment 2 Based on the same concept, the present invention also proposes a multi - modal medical image real - time annotation and collaborative browsing device based on adaptive DeepZoom, including: A pre - processing module for performing format conversion and metadata extraction on medical images; A multi - resolution pyramid construction module for generating a multi - resolution pyramid, where the pyramid includes multiple levels obtained by successive downsampling, and each layer of the image is divided into standard - size blocks; Lanczos resampling algorithm is used for downsampling, and edge blocks are processed by padding to the standard size; A lightweight rendering module for dynamically loading corresponding - level image blocks based on the viewport area and managing memory through a cache - eviction strategy; A collaborative annotation module for synchronizing multi-user annotation operations through a real-time communication protocol, supporting a multi-layer semantic tag system and annotation version management; implementing real-time broadcasting of annotation operations based on the WebSocket protocol, and the multi-layer semantic tag system includes three levels of tag hierarchies; A medical compliance log module for recording logs containing information such as cases, users, operation times, operation types, and affected areas, and implementing local caching and cloud encryption synchronization; cloud encryption synchronization uses the HIPAA encryption standard, and local caching is implemented through a Sqlite database.

[0075] Embodiment III This embodiment also provides an electronic device. Refer to Figure 2 , which includes a memory 404 and a processor 402. A computer program is stored in the memory 404, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0076] Specifically, the above-mentioned processor 402 may include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present invention.

[0077] Among them, the memory 404 may include a mass storage 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 404 may include removable or non-removable (or fixed) media. Where appropriate, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. Where appropriate, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0078] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0079] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the above-described adaptive DeepZoom-based multi-modal medical image real-time annotation and collaborative browsing methods.

[0080] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0081] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0082] The input / output device 408 is used to input or output information. In this embodiment, the input information can be 11111, etc., and the output information can be 11111, etc.

[0083] Embodiment 4 This embodiment also provides a readable storage medium. The readable storage medium stores a computer program, and the computer program includes program codes for controlling a process to execute the process. The process includes the adaptive DeepZoom-based multi-modal medical image real-time annotation and collaborative browsing method according to Embodiment 1.

[0084] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated herein.

[0085] In general, various embodiments can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the present invention is not limited thereto. Although various aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0086] Embodiments of the present invention can be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components that are configured to perform the embodiments when the program runs. The one or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any box in the logical flow as shown in the figures can represent a program step, or interconnected logical circuits, boxes, and functions, or a combination of program steps and logical circuits, boxes, and functions. The software can be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media are non-transitory media.

[0087] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as within the scope described in this specification.

[0088] The above embodiments merely represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the appended claims.

Claims

1. A real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom, characterized in that Including the following steps: S1. Preprocess the medical image to generate a multi-resolution pyramid, which includes multiple levels with decreasing resolutions obtained by downsampling layer by layer from the original image, and each layer of the image is divided into standard-sized blocks; S2. Dynamically load the image blocks at the corresponding level in the multi-resolution pyramid based on the current window area, and manage the block data in memory through a cache eviction strategy; S3. Synchronize multi-user annotation operations through a real-time communication protocol, perform structured annotation on the medical image based on a multi-layer semantic label system, and adopt an annotation version management strategy to resolve multi-user annotation conflicts; S4. Record the annotation operation log, which includes case identification, user identification, operation time, operation type, and affected area information, and implement local caching and cloud encryption synchronization of the log.

2. The real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom according to claim 1, wherein, The specific steps of S1 include: Convert the medical image into a standard format that supports the multi-resolution pyramid structure and extract image metadata; Use the Lanczos resampling algorithm to downsample the current layer of the image to generate the next layer of the image, and fill the edge blocks to the standard size.

3. A real-time annotation and collaborative browsing method for multimodal medical images based on adaptive DeepZoom according to claim 1, characterized in that, In step S3, the multi-layer semantic label system includes three levels of labels, which are in sequence: The first-level label for annotating the type of lesion area; The second-level label for annotating the cell type; The third-level label for annotating the tissue grade.

4. A real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom, characterized in that, In step S3, the real-time communication protocol is the WebSocket protocol, and the annotation operation is transmitted as a JSON format message containing the operation type, coordinates, and user ID; The annotation version management strategy includes a timestamp-based overwrite mechanism and a historical version backtracking function.

5. A real-time annotation and collaborative browsing method for multi-modal medical images based on adaptive DeepZoom according to any one of claims 1 to 4, characterized in that, In step S4, the operation log is cached through a local Sqlite database and automatically uploaded to a compliance server at preset intervals; The cloud encryption synchronization adopts the HIPAA encryption standard.

6. A real-time annotation and collaborative browsing device for multi-modal medical images based on adaptive DeepZoom, characterized in that, Including: A preprocessing module for converting the format of the medical image and extracting metadata; A multi-resolution pyramid construction module for generating a multi-resolution pyramid, which includes multiple levels obtained by downsampling layer by layer, and each layer of the image is divided into standard-sized blocks; A lightweight rendering module for dynamically loading the corresponding level of image blocks based on the window area and managing memory through a cache eviction strategy; A collaborative annotation module for synchronizing multi-user annotation operations through a real-time communication protocol, supporting a multi-layer semantic label system and annotation version management; A medical compliance log module for recording logs containing case, user, operation time, operation type, and affected area information, and implementing local caching and cloud encryption synchronization.

7. The multimodal medical image real-time annotation and collaborative browsing device according to claim 6, characterized in that, The multi-resolution pyramid construction module uses the Lanczos resampling algorithm for downsampling, and fills the edge blocks to the standard size.

8. The multimodal medical image real-time annotation and collaborative browsing device according to claim 6, characterized in that, The collaborative annotation module realizes real-time broadcasting of annotation operations based on the WebSocket protocol, and the multi-layer semantic label system includes three levels of label hierarchies.

9. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom according to any one of claims 1 to 5.

10. A readable storage medium, characterized in that, A computer program is stored in the readable storage medium, and the computer program includes program code for controlling a process to execute the process, and the process includes the method for real-time annotation and collaborative browsing of multimodal medical images based on adaptive DeepZoom according to any one of claims 1 to 5.

Images