Multi-terminal cooperative processing method for huge image and application of multi-terminal cooperative processing method

Abstract

The invention provides a huge image multi-terminal cooperative processing method and application thereof, and relates to the field of distributed system concurrency control. In order to solve the problems of high transmission delay and coarse concurrent lock granularity in multi-terminal collaboration of a huge image, the method comprises the following steps of: reducing dimensions of global vector coordinates of an operation terminal into local relative coordinates relative to a bottom layer tile so as to generate a local differential data packet; the server side executes directional pushing or asynchronous weak synchronization based on the viewport state and the zoom hierarchy matching result; and generating a self-adaptive dynamic bounding box based on the cursor position, performing intersection mapping on the dynamic bounding box and the spatial index tree, applying a micro-area read-write lock and performing broadcasting, so that the conflicting micro-area enters a read-only mode. The method is mainly used for efficient collaborative annotation of large-scale images, network bandwidth consumption is effectively reduced, and fine-grained non-inductive concurrency is achieved.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics, distributed system concurrency control and network communication data synchronization, and in particular to a method for multi-terminal collaborative processing of large-format images and its application. Background Technology

[0002] With the development of digital healthcare and remote sensing technologies, the application of ultra-high resolution (large-format) images, such as whole-slice digital pathology images (WSI) and high-precision satellite remote sensing images, is becoming increasingly widespread. In these scenarios, multiple experts are typically required to conduct remote real-time collaborative consultations and annotations. Most existing collaborative processing solutions employ a vector data synchronization mechanism based on a global coordinate system. This means that after annotation is completed on the client side, the complete vector path, containing global floating-point coordinates, is broadcast to all collaborating users. To cope with the data volume of large-format images, some existing technologies employ image pyramids and task region segmentation techniques. These divide the image into multiple static regions and assign them to different users. On the client side, the bounding boxes of the vector data are extracted and their intersection with the viewport range is determined. Only the vector data within the viewport is rendered on the client side, reducing the rendering load on a single end.

[0003] Therefore, there is an urgent need for a multi-terminal collaborative processing method for large-format images based on spatial index trees and tile difference, and its application, to solve the problems existing in the current technology. Summary of the Invention

[0004] This invention provides a method for multi-terminal collaborative processing of large-format images and its application, addressing the problems of severe network congestion caused by full transmission, concurrent blocking of multiple users on the same screen due to coarse locking granularity, and overload of front-end computing load caused by transmitting global floating-point coordinates in existing collaborative systems when processing large-format images.

[0005] The core technology of this invention is to reduce the global vector coordinates to local relative coordinates based on tiles and perform differential transmission, combined with the server-side directional interception and distribution based on viewport and LOD awareness, and the fine-grained micro-area read-write lock mechanism based on the intersection mapping of the cursor dynamic sliding bounding box and the spatial index tree, to achieve ultra-low latency and seamless concurrent collaboration of huge images.

[0006] In a first aspect, the present invention provides a method for multi-device collaborative processing of large-format images, the method comprising the following steps:

[0007] Real-time acquisition and maintenance of the global viewport state of the current screen viewport boundary and scaling level of each operating terminal; When the first operation terminal generates a vector drawing operation on the giant image, it reduces the global vector coordinates corresponding to the vector drawing operation to local relative coordinates relative to the tile it is located, and generates a local differential data packet by combining the identification information of the corresponding tile and sends it to the server. After receiving the local differential data packet, the server performs spatial intersection calculation and scaling level matching based on the global viewport state. If it is determined that the operation area corresponding to the local differential data packet is not within the current viewport range of other receiving operation terminals or does not meet the scaling level matching conditions, the real-time data push to the receiving operation terminal is suspended. The server builds a spatial index tree in memory to manage the cursor position of each operation terminal. When it detects that the cursor of the first operation terminal enters the preset micro-area that moves with the cursor and is dynamically generated, the preset micro-area is used as a spatial query window to perform intersection mapping with the spatial index tree nodes. Micro-area read-write locks are applied to the nodes that intersect and the lock status is broadcast to other operation terminals. After receiving the locked status, other operating terminals set the corresponding micro-area rendering to anti-conflict read-only mode and intercept the upload of newly generated operation commands in the read-only mode micro-area; when the cursor of the first operating terminal moves out of the preset micro-area, the corresponding micro-area read-write lock is released.

[0008] Furthermore, after suspending the real-time data push to the receiving end, it also includes: Local differential data packets that do not meet the push conditions are temporarily stored in a dedicated queue outside the viewport of the receiving operation terminal. The preload buffer is calculated around the actual physical viewport of the receiving operation end. When the viewport movement of the receiving operation end is detected, causing the edge of the preload buffer to spatially intersect with the coordinates recorded in the dirty area queue outside the viewport, the receiving operation end is actively woken up and the corresponding local differential data packet is pushed to the receiving operation end.

[0009] Furthermore, if it is determined that the operating area corresponding to the local differential data packet is not within the current viewport range of other receiving operating terminals or does not meet the scaling level matching condition, including: If the scaling level of the receiving operation terminal is determined to be greater than the preset macro threshold, the vector data corresponding to the local differential data packet is thinned or transformed by the bounding rectangle to generate dimensionality-reduced simplified data. The dimensionality-reduced simplified data is pushed to the receiving operation terminal to replace the original microscopic vector data packet.

[0010] Furthermore, the global vector coordinates corresponding to the vector drawing operation are reduced in dimensionality to local relative coordinates relative to the tile they reside in, and local differential data packets are generated by combining the identification information of the corresponding tile, including: When it is detected that a continuously drawn vector line segment crosses the physical boundary of different tiles, the vector line segment is forcibly truncated at the intersection boundary, and the line segment points distributed inside different tiles are mapped to generate local relative coordinates relative to the origin of their respective tiles. Local differential data packets include a header and a payload. The header contains the operation instruction type, the physical timestamp of the operation terminal, and the tile index number compressed using dimensionality reduction encoding. The payload contains the scaling level and local relative coordinates. The local relative coordinates are stored using a first-bit width data type, which is smaller than the second-bit width representing the global vector coordinates.

[0011] Furthermore, the mechanism applies micro-area read-write locks to intersecting nodes and broadcasts the lock status to other operating terminals, and also includes a hysteresis buffering mechanism to prevent lock oscillations. The preset micro-area is a dynamically sliding bounding box whose area adaptively adjusts with the scaling level; After the first operating terminal acquires the micro-area read / write lock, it expands the locked logical boundary outwards according to preset parameters to form a hysteresis buffer zone. When the second operating terminal attempts to preempt the read / write lock of the micro-area, it must be determined that its cursor has crossed the outer boundary of the hysteresis buffer and the first operating terminal is inactive before the switch of the lock state can be triggered.

[0012] Furthermore, the method also includes a concurrent conflict rollback mechanism to prevent network latency: Before obtaining confirmation of locking from the server, the operator uses an optimistic rendering strategy to push vector data into the local unconfirmed optimistic queue and drive the local local rendering engine to draw. If the server determines that a race condition has occurred, it sends a rejection control packet containing a temporary conflict identifier and regional authority status data to the terminal that caused the conflict. After receiving the rejection control packet, the operator locates the illegal vector line segment in the unconfirmed optimistic queue based on the conflict temporary identifier, calls the rendering engine's reverse compositing mode to perform local reverse pixel erasure of the illegal vector line segment, and applies the region's authoritative state data to perform state replay.

[0013] Furthermore, the corresponding micro-area rendering is set to a conflict-preventing read-only mode, and the upload of newly generated operation commands within the read-only mode micro-area is intercepted, including: The operator dynamically generates a local shadow layer above the locked micro-area; when the user draws within the locked micro-area, the generated local vector data is redirected to the shadow layer to be rendered in a preset warning style, and its push into the system's main rendering engine is intercepted. When a user draws a vector line segment that crosses the locked boundary, the geometric intersection point between the vector line segment to be drawn and the bounding box of the locked micro-area is calculated in real time. Line segments located inside the bounding box are truncated and stripped to the shadow layer, while only legal line segments located outside the bounding box are generated into local differential data packets and sent to the server.

[0014] Secondly, the present invention provides a large-format image multi-terminal collaborative processing device, comprising: The viewport awareness module is used to acquire and maintain the global viewport state of the current screen viewport boundary and scaling level of each operating terminal in real time. The differential transmission module is used to reduce the global vector coordinates corresponding to the vector drawing operation to local relative coordinates relative to the tile it is located when the first operation terminal generates a vector drawing operation on the giant image, and generate a local differential data packet by combining the identification information of the corresponding tile and sending it to the server. The distribution filtering module is used to perform spatial intersection calculation and scaling level matching based on the global viewport state after the server receives the local differential data packet. If it is determined that the operation area corresponding to the local differential data packet is not within the current viewport range of other receiving operation terminals or does not meet the scaling level matching conditions, the real-time data push to the receiving operation terminal is suspended. The space lock management module is used by the server to build a space index tree in memory to manage the cursor position of each operation terminal. When the cursor of the first operation terminal enters the preset micro-area that moves with the cursor and is dynamically generated, the preset micro-area is used as a space query window to perform intersection mapping with the space index tree nodes. Micro-area read-write locks are applied to the nodes that intersect and the lock status is broadcast to other operation terminals. The read-only interception module is used to set the corresponding micro-area rendering to anti-conflict read-only mode after other operating terminals receive the lock status, and to intercept the upload of newly generated operation instructions in the read-only mode micro-area; when the cursor of the first operating terminal moves out of the preset micro-area, the corresponding micro-area read-write lock is released.

[0015] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described method for multi-terminal collaborative processing of large-format images.

[0016] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the above-described large-format image multi-terminal collaborative processing method.

[0017] The main contributions and innovations of this invention are as follows: 1. Reduce network transmission load and collaboration latency: This invention uses server-side targeted distribution based on viewport and LOD awareness to intercept data pushes that are not within the viewport or do not match the scaling level from the source. Combined with dimensionality reduction and bit width compression transmission of local relative coordinates of tiles, it effectively reduces the data packet size of a single interaction and the overall broadcast frequency, thus alleviating network transmission latency and bandwidth pressure in large-scale image collaboration scenarios.

[0018] 2. Improve the fine-grainedness and operational consistency of concurrent collaboration: This invention abandons static coarse-grained layer locks or region allocations, and adopts a micro-area read-write lock based on the intersection mapping of the cursor's dynamic sliding bounding box and the spatial index tree. This makes the granularity of concurrent control precise to the tiny area that moves with the cursor, and is combined with a hysteresis buffer mechanism to prevent lock oscillation, thus realizing collaborative annotation by multiple people on the same screen without interference.

[0019] 3. Avoiding front-end precision loss and reducing computational overhead: This invention uses a tile relative coordinate increment mechanism and the spatial boundary constraints of fixed tile size to replace global double-precision floating-point data with short integer data for coordinate transmission and calculation. This avoids precision loss and computational overflow caused by the front-end graphics engine processing massive global large floating-point coordinates, and reduces the front-end rendering computational resource consumption.

[0020] 4. Reduce the overhead of concurrent conflict redrawing and maintain display stability: This invention uses an unconfirmed optimistic queue and a reverse rendering compensation mechanism to perform local reverse pixel erasure and state replay only on illegal vector line segments when network latency or race conditions occur. This avoids screen flickering caused by full image redrawing and maintains visual continuity and precise isolation of network data when drawing across locked boundaries.

[0021] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a multi-terminal collaborative processing method for large-format images according to an embodiment of the present invention; Figure 2 This is an architectural diagram of a large-format image multi-terminal collaborative processing device according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0025] Example 1 This embodiment provides a multi-terminal collaborative processing method for large-format images based on spatial index trees and tile difference. This method is applicable to concurrent rendering and collaborative annotation scenarios for large-format images such as whole-slice digital pathology images (WSI) and high-precision satellite remote sensing images. Figure 1 and Figure 2 As shown, the specific steps include: Step S1: Global Viewport State Awareness and Maintenance The server (full name: Collaboration Control Server) continuously builds and maintains a frequently accessed "global user viewport status table" in high-speed memory (e.g., based on Redis or Node.js process local cache). For each online operator in the current collaboration room, this status table uses the user's unique identifier (User_ID) as the hash key, and its mapped record values ​​contain the following core fields: The current co-working room identifier, the current screen's absolute physical coordinate bounding box (Viewport_AABB, consisting of four global floating-point coordinate values ​​[X_min, Y_min, X_max, Y_max], i.e., viewport boundary coordinates), the current zoom level (Current_LOD, which determines the threshold of visible detail), and the timestamp of the viewport's last update.

[0026] To ensure high-frequency updates of the server status while avoiding "network storms" caused by mouse dragging events on the front end, a debouncing and throttling reporting mechanism with displacement threshold judgment is adopted between the client and the server: The underlying event listener on the client side captures the user's panning or zooming operations in real time. Whenever a viewport change event is triggered, the client algorithm first calculates the absolute difference in displacement between the "current viewport AABB" and the "last successfully reported viewport AABB" on the horizontal and vertical coordinates. The client will only trigger a reporting action if any of the following conditions are met: 1) Displacement exceeding limits: The absolute difference between the horizontal or vertical displacement exceeds the preset displacement threshold (for example, set to 10% of the physical width of the bottom tile, i.e., the viewport has been significantly displaced). 2) Static state: The operator detects that no new viewport change events have been generated within a preset time period (e.g., 200 milliseconds), that is, the dragging or zooming action has stopped; 3) Level changes: Any step change occurs in the scaling level (Current_LOD).

[0027] When any of the above conditions are met to trigger a report, the operator encapsulates the latest [X_min, Y_min, X_max, Y_max] and Current_LOD into a lightweight viewport update request packet and sends it to the server via the WebSocket link.

[0028] Upon receiving the request packet, the server performs an atomic state refresh operation: using the latest viewport boundary coordinates and scaling level, it overwrites the old record value corresponding to the User_ID in the hash table and writes the current system timestamp to the Last_Update_Time field. Through this mechanism of "client displacement determination and debouncing + server atomic overwrite," the server memory always maintains a "low-latency and high-precision" mirror of the viewport states of each operating terminal, providing a reliable global data source for the spatial intersection calculation in the subsequent step S3.

[0029] Step S2: Local coordinate dimensionality reduction and differential data packet construction When the first operating terminal (i.e., the first client) performs vector drawing operations on the giant image, the system breaks away from the traditional method of transmitting a global floating-point coordinate set and performs coordinate dimensionality reduction and differential transformation. The giant image is cut into fixed-size physical logical grids (such as 256×256 or 512×512 pixel tiles) at the bottom layer.

[0030] Specifically, when continuously drawing vector line segments on the operating end, the client algorithm performs geometric clipping calculations in real time between the vector line segment and the boundary of the underlying tile mesh (e.g., using variants of the Cohen-Sutherland or Liang-Barsky algorithms). When a drawn vector line segment is detected to cross the physical boundary of different tiles, the vector line segment is forcibly truncated at the intersection boundary. Subsequently, local coordinate dimensionality reduction mapping is performed: the global floating-point coordinates of the top-left vertex of the target tile are extracted as the local coordinate origin of the tile, and the line segment sampling points distributed inside the tile are converted from global absolute coordinates to local relative offsets relative to this local origin through vector subtraction.

[0031] Because the physical size of the underlying tiles is fixed (e.g., 512×512 pixels), and attributes such as line width have upper limits in the local space, the numerical range of local relative offsets is strictly limited to a very small known interval. Based on this geometric constraint, the client executes a bit-width compression strategy when encapsulating data: encapsulating the dimensionality-reduced data into a custom binary structure to generate a local differential data packet. This data packet contains a header and a payload. The header contains the operation instruction type, user identifier, physical timestamp of the operation terminal, and tile index number using dimensionality reduction encoding (such as Morton code / Z-order curve code for one-dimensional compression); the payload contains the scaling level at the time of the operation, the number of consecutive incremental coordinate points, and the local relative coordinate array. For example, the data packet structure is defined as:

[0032] in, The physical timestamp when the data packet was generated for the client; It identifies the tile location, which serves as the basic spatial reference frame for data packets, replacing the traditional absolute coordinates of the global map; As a globally unique identifier for collaborative participants, in conjunction with (Such as drawing, erasing, locking requests, and other command types), driving the "dual-track rendering mechanism" on the rendering end; Δx and Δy occupy only a very small amount of byte space (such as 8-bit integers can represent coordinate differences within a range of 256), compressing the coordinate array that originally required tens of KB to a few bytes, achieving ultra-low latency network transmission.

[0033] As can be seen, in this structure, the X-axis and Y-axis offsets of local relative coordinates are stored using a first-bit wide data type (e.g., a 16-bit short integer Int16, occupying 2 bytes), instead of the conventional second-bit wide data type (e.g., a 64-bit double-precision floating-point number Float64, occupying 8 bytes) used to represent global vector coordinates. Because the numerical range is controllable, Int16 is sufficient to represent local offsets without precision loss, thus avoiding precision loss and computational overflow caused by the front-end graphics engine processing massive global floating-point numbers. Simultaneously, it compresses the network transmission volume of single-point coordinates by more than 75%.

[0034] Step S3: Directed distribution and asynchronous weak synchronization based on spatial computation and LOD matching After receiving the local differential data packet from the first operator, the server combines the tile index number and scaling level in the packet header to reconstruct the local bounding box (Action_AABB) of the operation in the global coordinate system. Subsequently, the server iterates through the viewport state information of other receiving operators (such as the second operator, also known as the rendering operator) within the current collaborative room, executing a coarse-to-fine hierarchical matching and distribution logic: First, a coarse LOD (Level of Detail) screening and degradation push are performed: The server reads the Current_LOD from the second operating terminal. If it determines that the scaling level is greater than the preset macro threshold (i.e., it is in a macro low-magnification perspective), then microscopic detail lines are not only invisible, but will also cause rendering noise and bandwidth waste. The server directly intercepts the original high-precision data packets and switches to degradation weak synchronization logic: triggering a vector thinning algorithm (such as the Douglas-Peucker algorithm, where the tolerance increases with the LOD), or converting complex vectors into dimensionality-reduced simplified data such as bounding rectangles and center heatsinks, and only pushing this simplified data to the second operating terminal to complete macroscopic degradation synchronization.

[0035] Next, precise AABB intersection calculation and real-time push are performed: If the LOD of the second operation terminal is within the microscopic visible range, the server extracts its Viewport_AABB and uses the axis-aligned bounding box intersection test algorithm to determine whether the projections of Action_AABB and Viewport_AABB on the X and Y axes overlap. If spatial overlap exists, it is determined that the operation is visible to the second operation terminal, and the differential data packet is pushed into its real-time WebSocket push queue for rapid delivery.

[0036] Finally, the server performs temporary storage of the dirty area outside the viewport and progressive asynchronous weak synchronization: If the AABB (Area, Scale, and Block) do not intersect, the server intercepts the network and temporarily stores the data packets in a dedicated "dirty area queue outside the viewport" for this second operation terminal. For the data stored in the dirty area queue, the server executes a progressive asynchronous weak synchronization mechanism: 1) Predictive Wake-up for Translation: The server expands outwards from the Viewport_AABB of the second operating terminal by a preset ratio (e.g., 20% in both width and height of the screen) to calculate and virtually create a preload buffer. When the second operating terminal performs translation and dragging, causing the viewport to update, the server refreshes the Viewport_AABB and simultaneously calculates whether the edge of the new preload buffer intersects with the coordinates recorded in the dirty area queue. If they intersect, the server actively wakes up the push thread and pushes the corresponding data packets in batches to the operating terminal, achieving predictive on-demand loading. 2) Silent fallback during idle time: When the system detects that the idle time of the event queue in the collaborative room exceeds the set threshold (e.g., no high-frequency interaction within 500ms), the server will silently push the remaining data of the dirty queue to the local cache of each operator with low priority. This serves as a fallback strategy for eventual consistency, ensuring that the local machine has complete underlying data when the user suddenly zooms or jumps.

[0037] Step S4: Micro-area dynamic read / write lock based on spatial index tree The server constructs a spatial index tree based on an R-tree or quadtree in memory, with its leaf nodes mapped to the underlying physical logical grid (e.g., aggregating multiple adjacent tiles into a minimum lockable spatial unit).

[0038] When the cursor on the first operating device moves on the screen, the system does not lock onto a fixed grid. Instead, it dynamically generates a bounding box that moves with the cursor, centered on the cursor's real-time physical coordinates. The area of ​​this bounding box undergoes adaptive elastic scaling: its side length is inversely proportional to the scaling level Current_LOD of the current operating device or mapped according to a preset step function (the sensing range is scaled up proportionally from a macroscopic perspective and shrunk to the fine stroke level from a microscopic perspective).

[0039] To avoid the bottleneck of global traversal calculation in the R-tree caused by high-frequency cursor displacement, the server establishes a "last hit node cache" for each operation. When cursor displacement causes new dynamic bounding box coordinates, the server first performs an O(1) complexity cache hit test: extract the four extreme coordinates of the new dynamic bounding box, compare them with the AABB boundary values ​​of the last hit parent node recorded in the cache a constant number of times, and determine whether the new bounding box is completely contained within the parent node's region. If so, it proves that no cross-node jump has occurred in the locked range, and there is no need to backtrack the tree structure; only the intersection state needs to be updated in the local subtree. If a large-scale jump causes a cache miss, the server returns to the root node to perform a regular O(log n) space intersection query.

[0040] The server uses a dynamic bounding box as a query window to perform intersection mapping with leaf nodes, adds a read-write lock mark to the memory attributes of the matched leaf nodes, and broadcasts the lock status to other operating terminals. When the cursor moves out and no longer intersects, the lock mark is instantly erased, realizing "locking as you draw, and erasing after use".

[0041] To prevent lock state oscillations caused by frequent cursor movement between adjacent micro-area boundaries on multiple operating terminals, this embodiment introduces a hysteresis buffer mechanism. After the first operating terminal acquires the lock, its locked logical boundary expands outward according to preset parameters (such as expanding outward by half the width of the minimum locking unit), forming an invisible "hysteresis buffer zone." When the second operating terminal attempts to preempt this area, even if its cursor enters the original locking boundary of the first operating terminal, it will not immediately trigger a lock transfer; a dual condition must be met: The cursor must cross the outer hysteresis line of the expanded hysteresis buffer zone, and the first operating end must be in an inactive state (such as the mouse button being lifted for more than a preset number of milliseconds) in order to trigger the one-way switching of the lock state.

[0042] Step S5: Dual-track shadow layer interception and reverse compensation rollback When the second operating terminal receives the micro-area lock status, its client rendering engine dynamically generates an independently composed "shadow layer" above the main rendering canvas. When the user presses the mouse to draw within this locked area, the generated local vector data is not pushed into the main rendering engine, but is redirected to the shadow layer by the event interceptor for downgraded rendering with semi-transparent or warning colors (such as red dashed lines), and network reporting commands are directly intercepted and discarded.

[0043] For a continuous drawing that crosses the locked boundary, the client executes a vector space clipping algorithm in real time: calculates the geometric intersection of the currently drawn line segment and the bounding box of the locked micro-area, splits the line segment in two at the intersection, strips the line segment inside the bounding box to the shadow layer and intercepts and reports it, and only generates local differential data packets for the legitimate line segments outside the bounding box and reports them to the server, thereby ensuring the continuity of user experience while avoiding unauthorized writes.

[0044] To address network race conditions with ultra-low latency (such as lock state broadcasts arriving later than user operations due to network jitter), this embodiment employs a concurrent conflict rollback mechanism to prevent network latency. When the user makes a stroke, there is no need to wait for the server's lock acknowledgment message; instead, an optimistic rendering strategy is directly adopted: the vector data is pushed into the local Unacknowledged Optimistic Queue (UOQ) and the main rendering engine is driven to redraw. If the server, acting as a global sequencer, determines that the operation has intruded into another user's locked area, a race condition is identified, and a rejection control packet (NACK) containing a temporary conflict identifier and area authority status data is rapidly sent to the user.

[0045] Upon receiving a rejection control packet, the operator triggers a precise local rollback: based on a temporary identifier, the illegal vector line segment is located in the UOQ, and the inverse compositing mode of the underlying rendering engine is invoked (e.g., setting the global compositing operation attribute of the graphics context to destination-out). The path of the illegal vector line segment is used as a clipping mask, and only the local pixel area covered by the line segment on the canvas is zeroed out to remove the alpha channel, thereby avoiding flickering and performance overhead caused by full redrawing. Subsequently, the regional authoritative state data carried in the NACK is applied to perform state replay on the current tile to restore the correct image, and the illegal request is destroyed from the UOQ, achieving seamless error correction.

[0046] Example 2 Based on the same concept, the present invention also proposes a multi-terminal collaborative processing device for large-format images, comprising: Based on the same inventive concept as Embodiment 1, such as Figure 2 As shown, this embodiment provides a multi-terminal collaborative processing device for large-format images based on spatial index trees and tile difference. This device operates on various nodes of a distributed system and specifically includes: The viewport awareness and distribution module, deployed on the server, is used to perceive and maintain the viewport status information of each operation terminal in real time. Based on the viewport status information, it performs spatial intersection calculation and scaling level matching with the operation bounding boxes of other operation terminals to execute targeted real-time push or asynchronous weak synchronization mechanism (such as dirty area queue wake-up, idle silent push and LOD degradation). The local coordinate difference module is deployed on the operation terminal and communicates with the viewport perception and distribution module. When the operation terminal generates vector operation data, it uses a vector space clipping algorithm to segment the operation data and maps the global vector coordinates to local relative coordinates relative to the underlying fixed-size tiles. It combines information such as tile index number to construct a local difference data packet with a custom binary protocol and sends it to the viewport perception and distribution module. The micro-area space lock control module is deployed in collaboration between the server and the operator. The server is responsible for generating a dynamic bounding box based on the cursor of the operator and performing O(1) cached dimension reduction intersection mapping with the R tree nodes in memory. It applies a micro-area read-write lock with a hysteresis buffer mechanism and broadcasts it to the operator. The operator is responsible for receiving the lock status, intercepting illegal instructions through the shadow layer, and using the Unconfirmed Optimistic Queue (UOQ) and reverse pixel erasure algorithm to handle delayed concurrent conflicts. The read-only interception module, deployed on the operator's end, is used to set the corresponding micro-area rendering to anti-conflict read-only mode after receiving a locked state, and to intercept the upload of newly generated operation commands in the read-only mode micro-area; when the cursor of the locked operator's end moves out of the preset micro-area, it triggers the server to release the corresponding micro-area read-write lock; The differential data packets generated by the local coordinate differential module are filtered and directed through the viewport perception and distribution module. Meanwhile, the micro-area spatial lock control module dynamically adjusts the lock state according to the cursor position on the operating terminal. The viewport perception and distribution module decides whether to intercept data in conflict areas based on the lock state of the micro-area spatial lock control module. The read-only interception module works with the micro-area spatial lock control module to complete the client's interaction blocking and rendering isolation. The four components work together to achieve ultra-low latency and seamless concurrent collaboration for large-scale images.

[0047] Example 3 This embodiment also provides an electronic device, see reference. Figure 3 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0048] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0049] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0050] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0051] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the large-format image multi-terminal collaborative processing methods in the above embodiments.

[0052] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0053] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0054] Input / output device 408 is used to input or output information.

[0055] Example 4 This embodiment also provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the large-format image multi-terminal collaborative processing method according to Embodiment 1.

[0056] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0057] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0058] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 1 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0059] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for multi-device collaborative processing of large-format images, characterized in that, Includes the following steps: Real-time acquisition and maintenance of the global viewport state of the current screen viewport boundary and scaling level of each operating terminal; When the first operating terminal generates a vector drawing operation on the giant image, it reduces the global vector coordinates corresponding to the vector drawing operation to local relative coordinates relative to the tile it is located, and generates a local differential data packet by combining the identification information of the corresponding tile and sends it to the server. After receiving the local differential data packet, the server performs spatial intersection calculation and scaling level matching based on the global viewport state. If it is determined that the operation area corresponding to the local differential data packet is not within the current viewport range of other receiving operation terminals or does not meet the scaling level matching conditions, the real-time data push to the receiving operation terminal is suspended. The server constructs a spatial index tree in memory to manage the cursor position of each operation terminal. When it detects that the cursor of the first operation terminal enters a preset micro-area that moves with the cursor and is dynamically generated, the preset micro-area is used as a spatial query window to perform intersection mapping with the nodes of the spatial index tree. Micro-area read-write locks are applied to the nodes that intersect and the lock status is broadcast to other operation terminals. After receiving the locked state, other operating terminals set the corresponding micro-area rendering to anti-conflict read-only mode and intercept the upload of newly generated operation instructions in the read-only mode micro-area; when the cursor of the first operating terminal moves out of the preset micro-area, the corresponding micro-area read-write lock is released.

2. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, After suspending the real-time data push to the receiving operation terminal, the method further includes: Local differential data packets that do not meet the push conditions are temporarily stored in the viewport outside the dirty area queue belonging to the receiving operation terminal. A preload buffer is calculated around the actual physical viewport of the receiving operation terminal. When the viewport movement of the receiving operation terminal is detected, causing the edge of the preload buffer to spatially intersect with the coordinates recorded in the viewport external dirty area queue, the receiving operation terminal is actively woken up and the corresponding local differential data packet is pushed to the receiving operation terminal.

3. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, If it is determined that the operating region corresponding to the local differential data packet is not within the current viewport range of other receiving operating terminals or does not meet the scaling level matching condition, including: If it is determined that the scaling level of the receiving operation terminal is greater than the preset macro threshold, the vector data corresponding to the local differential data packet is thinned or transformed by the bounding rectangle to generate dimensionality-reduced simplified data. The reduced-dimensional data is pushed to the receiving operation terminal to replace the original microscopic vector data packet.

4. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, The global vector coordinates corresponding to the vector drawing operation are reduced in dimensionality to local relative coordinates relative to the tile in which they are located, and a local differential data packet is generated by combining the identification information of the corresponding tile, including: When it is detected that a continuously drawn vector line segment crosses the physical boundary of different tiles, the vector line segment is forcibly truncated at the intersection boundary, and the line segment points distributed inside different tiles are mapped to generate local relative coordinates relative to the origin of their respective tiles. The local differential data packet includes a header and a payload. The header includes the operation instruction type, the physical timestamp of the operation terminal, and the tile index number compressed using dimensionality reduction encoding. The payload includes the scaling level and the local relative coordinates. The local relative coordinates are stored using a first-bit width data type, which is smaller than the second-bit width representing the global vector coordinates.

5. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, It applies micro-area read-write locks to intersecting nodes and broadcasts the lock status to other operators. It also includes a hysteresis buffer mechanism to prevent lock oscillations. The preset micro-region is a dynamically sliding bounding box whose area adaptively adjusts with the scaling level; When the first operating terminal acquires the micro-area read / write lock, it expands the locked logical boundary outwards according to preset parameters to form a hysteresis buffer zone. When the second operating terminal attempts to preempt the read / write lock of the micro-area, it is necessary to determine that its cursor has crossed the outer boundary of the hysteresis buffer and the first operating terminal is in an inactive state before the switching of the lock state can be triggered.

6. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, The method also includes a concurrent conflict rollback mechanism to prevent network latency: Before obtaining confirmation of locking from the server, the operator uses an optimistic rendering strategy to push vector data into the local unconfirmed optimistic queue and drive the local local rendering engine to draw. If the server determines that a race condition has occurred, it sends a rejection control packet containing a temporary conflict identifier and regional authority status data to the terminal that caused the conflict. After receiving the rejection control packet, the operator locates the illegal vector line segment in the unconfirmed optimistic queue according to the conflict temporary identifier, calls the rendering engine's reverse compositing mode to perform local reverse pixel erasure of the illegal vector line segment, and applies the region authoritative state data to perform state replay.

7. The method for multi-terminal collaborative processing of large-format images as described in claim 1, characterized in that, Set the corresponding micro-area rendering to a conflict-preventing read-only mode, and intercept the upload of newly generated operation commands within the read-only mode micro-area, including: The operator dynamically generates a local shadow layer above the locked micro-area; when the user draws within the locked micro-area, the generated local vector data is redirected to the shadow layer to be rendered in a preset warning style, and its push into the system's main rendering engine is blocked. When a user draws a vector line segment that crosses the locked boundary, the geometric intersection point between the vector line segment to be drawn and the bounding box of the locked micro-area is calculated in real time. Line segments located inside the bounding box are truncated and stripped to the shadow layer, and only legal line segments located outside the bounding box are generated into the local differential data packet and sent to the server.

8. A multi-terminal collaborative processing device for large-format images, characterized in that, include: The viewport awareness module is used to acquire and maintain the global viewport state of the current screen viewport boundary and scaling level of each operating terminal in real time. The differential transmission module is used to reduce the global vector coordinates corresponding to the vector drawing operation to local relative coordinates relative to the tile it is located when the first operation terminal generates a vector drawing operation on the giant image, and generate a local differential data packet by combining the identification information of the corresponding tile and sending it to the server. The distribution filtering module is used to perform spatial intersection calculation and scaling level matching based on the global viewport state after the server receives the local differential data packet. If it is determined that the operation area corresponding to the local differential data packet is not within the current viewport range of other receiving operation terminals or does not meet the scaling level matching conditions, the real-time data push to the receiving operation terminal is suspended. The space lock management module is used by the server to build a space index tree in memory to manage the cursor position of each operation terminal. When the cursor of the first operation terminal is detected to enter the preset micro-area that is dynamically generated as the cursor moves, the preset micro-area is used as a space query window to perform intersection mapping with the nodes of the space index tree. Micro-area read-write locks are applied to the nodes that intersect and the lock status is broadcast to other operation terminals. The read-only interception module is used to set the corresponding micro-area rendering to anti-conflict read-only mode after other operating terminals receive the locked state, and to intercept the upload of newly generated operation instructions in the read-only mode micro-area; when the cursor of the first operating terminal moves out of the preset micro-area, the corresponding micro-area read-write lock is released.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the large-format image multi-terminal collaborative processing method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling the process to execute the process, the process including the large-format image multi-terminal collaborative processing method according to any one of claims 1 to 7.

Images