Pathological image multi-user terminal collaborative labeling consultation method and device

Abstract

The invention provides a pathological image multi-user terminal collaborative annotation consultation method and device, and the method comprises the following steps: rendering a pathological image in a front-end interface of each user terminal, and building a historical annotation database in each user terminal; the sending terminal transmits the differential annotation information of the pathological image to a receiving terminal; and the receiving terminal restores the differential annotation information into real-time annotation information and synchronizes the real-time annotation information to a corresponding front-end interface through an OT algorithm. According to the scheme, real-time collaborative labeling of multiple terminals is achieved through WebRTC and OT algorithms, the problem of labeling conflicts is solved by introducing the permission level, and the efficiency of consultation on the pathological images is improved.

Description

Technical Field

[0001] This application relates to the field of digital pathology technology, and in particular, to a method and device for collaborative annotation and consultation of pathological images on multiple user terminals. Background Art

[0002] In the field of medical pathological diagnosis, the consultation of digital pathological slides, as the core link for disease confirmation, is of self-evident importance. At present, traditional pathological consultations are mainly carried out by physically transmitting glass slides offline or sharing static images. However, this mode has many significant technical bottlenecks, seriously restricting the development of pathological diagnosis.

[0003] In terms of collaboration efficiency, in the traditional mode, multiple experts cannot synchronously annotate the same pathological image in real time. Once there are conflicts in annotation operations, it is also difficult to effectively solve them, resulting in a fragmented consultation process. According to statistics, the average time for a single consultation exceeds 40 minutes, which greatly reduces the diagnostic efficiency and may lead to delays in the patient's condition. At the technical level, existing remote systems, such as general video conferencing platforms like Zoom, lack professional pathological annotation tools and cannot record the history of annotation operations, making it difficult to meet the professional requirements of pathological diagnosis; while traditional DICOM systems have relatively serious latency problems during data transmission. The latency for transmitting 1 whole-slide pathological image can reach 100 - 300ms, making real-time interactive diagnosis difficult to achieve. In terms of network transmission, DICOM systems not only have high latency, but also have a packet loss rate of more than 20% in a weak network environment, seriously affecting the stability and integrity of data transmission, and thus affecting the accuracy and timeliness of pathological consultations.

[0004] Therefore, how to improve the quality and efficiency of pathological slide consultations is an urgent problem to be solved in the existing technology. Summary of the Invention

[0005] The embodiments of this application provide a method and device for collaborative annotation and consultation of pathological images on multiple user terminals. By using WebRTC and OT algorithms, real-time collaborative annotation of multiple terminals is achieved, and permission levels are introduced to solve the problem of annotation conflicts, improving the efficiency of consulting pathological images.

[0006] In a first aspect, the embodiments of this application provide a method for collaborative annotation and consultation of pathological images on multiple user terminals, and the method includes: Render the pathological image on the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; Take the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals. Obtain the real-time annotation information of the pathological image by the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; The receiving terminals restore the differential annotation information to real-time annotation information and synchronize the real-time annotation information to the corresponding front-end interfaces through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of two or more sending terminals for the same region in the pathological image at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

[0007] In a second aspect, an embodiment of the present application provides a multi-user terminal collaborative annotation and consultation device for pathological images, including: A rendering module, configured to render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; An annotation transmission module, configured to take the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals, obtain the real-time annotation information of the pathological image by the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; A synchronization module, where the receiving terminals restore the differential annotation information to real-time annotation information and synchronize the real-time annotation information to the corresponding front-end interfaces through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of two or more sending terminals for the same region in the pathological image at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute a multi-user terminal collaborative annotation and consultation method for pathological images.

[0009] The main contributions and innovations of the present invention are as follows: The embodiments of this application combine differential compression transmission of real-time annotation information and OT algorithm synchronization to achieve real-time sharing of annotation information, significantly improving the efficiency of multi-user collaborative annotation. Moreover, the permission level mechanism can solve the problem of annotation conflicts in the same area and ensure the authority of annotation results. The embodiments of this application use pyramid layering to divide the pathological image into tiles with metadata, construct a tile retrieval mapping table, and preferentially render the tiles in the currently visible area, which can optimize the rendering efficiency and enhance the user viewing experience. In terms of data management, the embodiments of this application use the MongoDB database to establish a historical annotation database to store historical annotation information, use an annotation data cache pool to save the annotation data structure, and combine a hash verification mechanism to query annotation operations, which is not only convenient for data management and traceability but also can avoid duplicate transmission and processing of annotation information, improving the accuracy and efficiency of data processing. From the perspective of transmission efficiency, the embodiments of this application obtain differential annotation information by taking the difference between real-time annotation information and historical annotation information for transmission, which can reduce the amount of data transmission, improve the transmission efficiency, and reduce the network load.

[0010] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects, and advantages of this application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of this application, and constitute a part of this application. The illustrative embodiments and descriptions thereof of this application are used to explain this application and do not constitute an improper limitation of this application. In the drawings: Figure 1 is a flowchart of a method for multi-user terminal collaborative annotation and consultation of pathological images according to an embodiment of this application; Figure 2 is a flowchart of real-time annotation data synchronization based on repeatability verification according to an embodiment of this application; Figure 3 is a flowchart of a pathological image analysis model for marking lesion areas according to an embodiment of this application; Figure 4 is a flowchart of performing annotation operations simultaneously through multiple sending terminals according to an embodiment of this application; Figure 5 The figure is a synchronization flowchart of multi-user conflict operations according to an embodiment of this application; Figure 6 is a structural block diagram of a multi-user terminal collaborative annotation and consultation device for pathological images according to an embodiment of this application; Figure 7 is a schematic hardware structure diagram of an electronic device according to an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0013] It should be noted that: In other embodiments, the steps of the corresponding method are not necessarily executed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may also be combined into a single step for description in other embodiments.

[0014] Embodiment 1 An embodiment of the present application provides a method for collaborative annotation and consultation of pathological images by multiple user terminals. It realizes real-time collaborative annotation of multiple terminals through WebRTC and OT algorithms, and introduces permission levels to solve the problem of annotation conflicts, improving the efficiency of consulting pathological images. Specifically, referring to Figure 1 , the method includes: Render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; Take the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals. Obtain the real-time annotation information of the pathological image of the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; The receiving terminal restores the differential annotation information to real-time annotation information and synchronizes the real-time annotation information to the corresponding front-end interface through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of the same area of the pathological image from two or more sending terminals at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

[0015] In some embodiments, the user terminal and the front-end interface are constructed through the Vue framework. The front-end interface supports diversified operations such as drawing, erasing, zooming in, and zooming out, and provides professional pathological annotation tools, including arrow, rectangle, circular annotation box, and text annotation functions, meeting the professional needs of pathological diagnosis.

[0016] Further, in the step of rendering the pathological image on the front-end interface of each user terminal, the pathological image is divided into multiple tiles by pyramid layering, and each tile is attached with metadata, where the metadata includes the layer information and coordinate information of the tile, and a tile retrieval mapping table is constructed. The tile retrieval mapping table records the mapping between each view area of the pathological section and the tile metadata. The current visible area of the front-end interface is obtained in real time, and the tiles within the current visible area are preferentially rendered based on the tile retrieval mapping table.

[0017] Specifically, the traditional method is to directly load and render the pathological image of the entire scene, which takes hundreds of milliseconds to several seconds. However, in this solution, by fragmenting the pathological image, when the user makes the first load on the front-end interface, only a small number of tiles within the current field of view need to be obtained, thus greatly shortening the time for loading and rendering.

[0018] Moreover, this solution will preferentially render the tiles within the current visible area of the front-end interface through the tile retrieval mapping table to ensure that when the expert zooms in on the image details or pans, the high-resolution tiles can be quickly rendered, avoiding frame freezes or blank areas and improving the fluency of the annotation operation.

[0019] In some specific embodiments, the historical annotation database is a MongoDB database, and the historical operation information of the current user terminal on the pathological image is also saved in the historical annotation database.

[0020] Specifically, saving the historical operation information of the current user terminal on the pathological image in the historical annotation database can facilitate the user to review the historical annotation information at any time and enhance the usability of this solution.

[0021] Further, the pathological image is fragmented and stored in the historical annotation database, thereby improving the retrieval efficiency and access efficiency of the pathological image.

[0022] In some specific embodiments, the sending terminal in this solution can be one or more, and each user terminal can be both a sending terminal and a receiving terminal at the same time. For example, when there are two user terminals annotating the pathological image, then these two user terminals are both sending terminals and send their own annotation operations to the receiving terminal. At the same time, these two user terminals are also both receiving terminals to receive the annotation operations sent by the other party.

[0023] In some specific embodiments, an annotation data cache pool is established in each user terminal, and an annotation data structure is generated for each annotation information on the corresponding user terminal and saved in the annotation data cache pool. Among them, the annotation data structure includes the annotation coordinates, annotation type, annotation time of the annotation operation, and the hash fingerprint of the corresponding image fragment.

[0024] Further, a real-time annotation data structure is generated based on the real-time annotation information of the sending terminal, and query is performed in the annotation data cache pool corresponding to the sending terminal by means of hash verification. If there is an annotation operation in the annotation data cache pool with the same hash fingerprint, the same annotation coordinates, and the same annotation type as the real-time annotation data structure, the transmission of the real-time annotation information is stopped.

[0025] Further, if there is no annotation operation in the annotation data cache pool that is the same as the real-time annotation data structure, the real-time annotation information is transmitted to each receiving terminal, and the real-time annotation information is stored in the historical annotation database.

[0026] Further, when the differential annotation information fails to be transmitted to the receiving terminal, the differential annotation information is temporarily stored in the annotation data cache pool for data retransmission.

[0027] Specifically, in a weak network environment, there will be some problems where differential annotation information fails to be transmitted to the receiving terminal, resulting in data asynchronization. Therefore, when the differential annotation information fails to be transmitted to the receiving terminal, the differential annotation information is temporarily stored in the annotation data cache pool for data retransmission to solve the problem of data asynchronization.

[0028] Specifically, an IndexedDB is used to establish the annotation data cache pool, and the hash fingerprint of the image shard is generated using the SHA-256 algorithm.

[0029] Further, a dynamic expiration policy is set in the annotation data cache pool, and a data access frequency threshold is set. The data structure with a data access frequency greater than the frequency threshold is permanently saved, while the remaining low-frequency data is automatically cleared after a set time.

[0030] Further, when the receiving terminal receives the differential annotation information and restores it to real-time annotation information, a real-time annotation data structure is generated from the real-time annotation information, and query is performed in the annotation data cache pool corresponding to the receiving terminal by means of hash verification. If there is an annotation operation in the annotation data cache pool with the same hash fingerprint, the same annotation coordinates, and the same annotation type as the real-time annotation data structure, the real-time annotation information is discarded.

[0031] That is to say, in this solution, duplicate judgments are respectively performed at the sending end and the receiving end. Only when the duplicate judgments at both ends pass, the real-time annotation information is synchronously annotated to the front-end interface through the OT algorithm to ensure that duplicate data is not transmitted. The flowchart of real-time annotation data synchronization based on duplicate verification is as Figure 2 shown.

[0032] In some specific embodiments, a pre-trained pathological image analysis model is locally deployed on the user terminal, and the pathological image analysis model analyzes and marks the lesion area of the pathological image.

[0033] Specifically, the pathological image analysis model adopts a lightweight YOLO or other object detection models to detect suspicious lesion areas in pathological images through an AI model, so as to provide auxiliary diagnosis suggestions for users.

[0034] Specifically, since the pathological image analysis model is deployed locally on the user terminal, the annotation information of the user is fed back to the pathological image analysis model for real-time training. The flowchart of the pathological image analysis model for marking the lesion area is as Figure 3 shown.

[0035] Specifically, the pathological image analysis model in this solution directly analyzes pathological images locally, avoiding uploading sensitive data to the backend server and reducing the risk of data leakage.

[0036] In some specific embodiments, this solution uses WebRTC to implement the transmission of differential annotation information between different user terminals.

[0037] In some specific embodiments, the full pathological image after the user's real-time annotation of the pathological image is used as the real-time annotation information, and the differential annotation information is obtained by taking the difference between the real-time annotation information and the historical annotation information with the latest timestamp in the corresponding historical annotation database.

[0038] That is to say, the differential annotation information only includes a small part of the latest annotation information of the sending terminal, and there is no other large amount of duplicate data, thus improving the data transmission time.

[0039] In some specific embodiments, the receiving terminal fuses the differential annotation information with the historical annotation information with the latest timestamp in the corresponding historical annotation database to obtain the real-time annotation information.

[0040] Specifically, since the annotation in this solution is synchronous, the historical annotation information saved locally on each user terminal is the same. When generating the differential annotation information, the real-time annotation information is differentiated from the historical annotation information with the latest timestamp in the historical annotation database. Therefore, only by fusing the differential annotation information with the historical annotation information with the latest timestamp in the corresponding historical annotation database can the real-time annotation information be obtained.

[0041] In some specific embodiments, when the receiving end simultaneously receives the real-time annotation information of the same area in the pathological image from two or more sending terminals, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized, and the real-time annotation information of the sending terminal with a lower permission level is synchronized in the corresponding front-end interface in a semi-transparent form.

[0042] Exemplarily, in the medical field, the present solution divides the permission levels into five levels: 5, 4, 3, 2, and 1. Level 5 is for chief physicians, Level 4 is for associate chief physicians, Level 3 is for resident physicians, Level 2 is for general doctors, and Level 1 is for interns. Therefore, the flowchart of the simultaneous annotation operations of multiple sending terminals in this solution is as follows Figure 4 shown, and the synchronization flowchart for multi-user conflict operations is as follows Figure 5 shown. Prioritizing the rendering of the annotation operations of high-permission-level users can make the synchronized annotations more accurate.

[0043] Furthermore, when the receiving end simultaneously receives real-time annotation information of the same region in a pathological image from two or more sending terminals, and the permission levels of the sending terminals to which the two pieces of real-time annotation information belong are the same, the real-time annotation information with a later timestamp is preferentially synchronized. Similarly, the real-time annotation information with an earlier timestamp is synchronized in the corresponding front-end interface in a semi-transparent form.

[0044] Specifically, if two pieces of real-time annotation information with the same permission level and the same annotation operation are received, it may indicate that the real-time annotation information with an earlier timestamp was not transmitted in a timely manner due to reasons such as the network. Therefore, the real-time annotation information with a later timestamp is preferentially synchronized.

[0045] In some specific embodiments, the OT algorithm is a conventional multi-terminal synchronization algorithm, which is used to ensure that multi-user annotation operations are presented consistently on each terminal. This solution will not be introduced in detail here.

[0046] Embodiment 2 Based on the same concept, referring to Figure 6 , the present application also proposes a multi-user terminal collaborative annotation and consultation device for pathological images, including: A rendering module, which is used to render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal. The historical annotation database is used to store the historical annotation information of the pathological image; An annotation transmission module, which is used to use the user terminal performing the annotation operation as the sending terminal and all user terminals except the current sending terminal as the receiving terminals, obtain the real-time annotation information of the pathological image of the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; A synchronization module, the receiving terminal restores the differential annotation information to real-time annotation information and synchronizes the real-time annotation information to the corresponding front-end interface through the OT algorithm. Among them, when the receiving end simultaneously receives real-time annotation information of the same region in a pathological image from two or more sending terminals, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

[0047] Embodiment III This embodiment also provides an electronic device. Referring to Figure 7 , it includes a memory 404 and a processor 402. A computer program is stored in the memory 404, and the processor 402 is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0048] Specifically, the above-mentioned processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0049] Among them, the memory 404 may include a mass storage 404 for data or instructions. By way of example and not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 404 may include removable or non-removable (or fixed) media. In a suitable case, the memory 404 may be internal or external to the data processing device. In a particular embodiment, the memory 404 is a non-volatile memory. In a particular embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). In a suitable case, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM), or a flash memory, or a combination of two or more of these. In a suitable case, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM may be a fast page mode dynamic random access memory (FPMDRAM), an extended date out dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0050] The memory 404 can be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402.

[0051] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the multi-user terminal collaborative annotation and consultation methods for pathological images in the above embodiments.

[0052] Optionally, the above electronic device may further include a transmission device 406 and an input / output device 408. Among them, the transmission device 406 is connected to the above processor 402, and the input / output device 408 is connected to the above processor 402.

[0053] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above network may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0054] The input / output device 408 is used to input or output information. In this embodiment, the input information can be pathological images, differential annotation information, etc., and the output information can be real-time annotation information, etc.

[0055] Optionally, in this embodiment, the above processor 402 can be set to execute the following steps through a computer program: Render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; Take the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals. Obtain the real-time annotation information of the pathological image by the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; The receiving terminal restores the differential annotation information to real-time annotation information and synchronizes the real-time annotation information to the corresponding front-end interface through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of the same region of the pathological image from two or more sending terminals at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

[0056] It should be noted that for the specific examples in this embodiment, reference can be made to the examples described in the above embodiments and optional implementation manners, and details thereof will not be repeated herein.

[0057] Generally, various embodiments can be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the present invention can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor, or other computing device. However, the present invention is not limited thereto. Although the various aspects of the present invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, the blocks, devices, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuits or logic, general hardware or a controller or other computing device, or some combination thereof.

[0058] Embodiments of the present invention can be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. A computer software or program (also referred to as a program product), including software routines, applets, and / or macros, can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product can include one or more computer-executable components that are configured to perform the embodiments when the program runs. The one or more computer-executable components can be at least one software code or a part thereof. Additionally, in this regard, it should be noted that any block in the logical flow, as Figure 7 described, can represent a program step, or an interconnected logical circuit, block, and function, or a combination of a program step and a logical circuit, block, and function. The software can be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs. The physical media are non-transitory media.

[0059] Those skilled in the art should understand that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as falling within the scope described in this specification.

[0060] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A multi-user terminal collaborative annotation and consultation method for pathological images, characterized in that, It includes the following steps: Render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; Take the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals. Obtain the real-time annotation information of the pathological image by the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; The receiving terminal restores the differential annotation information to real-time annotation information and synchronizes the real-time annotation information to the corresponding front-end interface through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of the same region in the pathological image from two or more sending terminals at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

2. The multi-user terminal collaborative annotation and consultation method for pathological images according to claim 1, wherein, In the step of rendering the pathological image in the front-end interface of each user terminal, the pathological image is divided into multiple tiles by pyramid layering, and each tile is attached with metadata, where the metadata includes the layer information and coordinate information of the tile, construct a tile retrieval mapping table, the tile retrieval mapping table records the mapping between each view area of the pathological section and the tile metadata, obtain the current visible area of the front-end interface in real time, and preferentially render the tiles within the current visible area based on the tile retrieval mapping table.

3. A multi-user terminal collaborative annotation and consultation method for pathological images according to claim 1, characterized in that, The historical annotation database is a MongoDB database, and the historical operation information of the current user terminal on the pathological image is also saved in the historical annotation database.

4. A multi-user terminal collaborative annotation and consultation method for pathological images according to claim 1, characterized in that, Establish an annotation data cache pool in each user terminal, generate an annotation data structure body for each annotation information on the corresponding user terminal and save it in the annotation data cache pool, where the annotation data structure body includes the annotation coordinates, annotation type, annotation time of the annotation operation, and the hash fingerprint of the corresponding image shard.

5. A multi-user terminal collaborative annotation and consultation method for pathological images according to claim 4, characterized in that, Generate a real-time annotation data structure body based on the real-time annotation information of the sending terminal, query in the annotation data cache pool corresponding to the sending terminal by means of hash verification. If there is an annotation operation in the annotation data cache pool with the same hash fingerprint, the same annotation coordinates, and the same annotation type as the real-time annotation data structure body, stop the transmission of the real-time annotation information.

6. A method for collaborative annotation and consultation of pathological images by multi-user terminals according to claim 4, characterized in that, When the receiving terminal receives the differential annotation information and restores it to real-time annotation information, generate a real-time annotation data structure body for the real-time annotation information, query in the annotation data cache pool corresponding to the receiving terminal by means of hash verification. If there is an annotation operation in the annotation data cache pool with the same hash fingerprint, the same annotation coordinates, and the same annotation type as the real-time annotation data structure body, discard the real-time annotation information.

7. A method for collaborative annotation and consultation of pathological images by multiple user terminals according to claim 1, characterized in that, Take the full-scale pathological image after the user performs real-time annotation on the pathological image as the real-time annotation information, and perform a difference between the real-time annotation information and the historical annotation information with the latest timestamp in the corresponding historical annotation database to obtain the differential annotation information.

8. A multi-user terminal collaborative annotation and consultation method for pathological images according to claim 1, characterized in that, The receiving terminal fuses the differential annotation information with the historical annotation information with the latest timestamp in the corresponding historical annotation database to obtain real-time annotation information.

9. A multi-user terminal collaborative annotation and consultation device for pathological images, characterized in that, It includes: A rendering module, configured to render the pathological image in the front-end interface of each user terminal, set a permission level for each user terminal, and establish a historical annotation database in each user terminal, where the historical annotation database is used to store the historical annotation information of the pathological image; An annotation transmission module, configured to use the user terminal performing the annotation operation as the sending terminal, and all user terminals except the current sending terminal as the receiving terminals, obtain the real-time annotation information of the pathological image by the sending terminal, perform differential compression on the real-time annotation information and the historical annotation information in the historical annotation database of the corresponding sending terminal to obtain differential annotation information, and transmit the differential annotation information to the receiving terminals; A synchronization module, the receiving terminal restores the differential annotation information to real-time annotation information and synchronizes the real-time annotation information to the corresponding front-end interface through the OT algorithm. Among them, when the receiving end receives the real-time annotation information of the same region of the pathological image from two or more sending terminals at the same time, the real-time annotation information of the sending terminal with a higher permission level is preferentially synchronized.

10. An electronic device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is set to run the computer program to execute a multi-user terminal collaborative annotation and consultation method for pathological images according to any one of claims 1-8.

Images