Online collaborative revision and visualization method for three-dimensional brain atlas annotation

Abstract

The application provides an online collaborative revision and visualization method for three-dimensional brain atlas annotation, which is executed in cooperation with system tools, the system tools adopt a three-layer Web architecture with front-end and back-end separation, including a front-end display layer, a back-end service layer and a data storage layer; the data storage layer is responsible for the persistent storage of I and L, I is a brain atlas original image, and L is brain atlas annotation information; the front-end display layer is responsible for rendering of a user interaction interface and collection of annotation operations; after receiving an annotation instruction sent by the front-end display layer, the back-end service layer calls a corresponding image processing algorithm to complete annotation calculation, and pushes the result to the data storage layer. The application realizes efficient editing of three-dimensional brain atlas annotation information, automatic generation of intermediate frame transition effects, and multi-user online collaborative revision and visualization.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an online collaborative revision and visualization method for three-dimensional brain atlas annotation. Background Technology

[0002] A stereotactic brain atlas (hereinafter referred to as a "brain atlas") is a digital map in the field of neuroscience, used for spatial navigation in stereotactic brain surgery and for determining the spatial location information of neuroscience research objects such as functional circuits, cell distribution, and omics data. A brain atlas typically contains three elements: images, annotations, and a coordinate system. Among these three elements, the annotations are the most crucial. They describe the boundaries and locations of anatomical structures and are generally saved as manually drawn vector graphics or bitmaps filled with pixels of the same grayscale within the same anatomical structure, analogous to administrative divisions on a map.

[0003] In routine research, brain atlases have become indispensable localization and navigation tools in neuroscience research. Ideally, there should be an interactive relationship between brain atlases and the neuroscience community: neuroscientists, as users, should not passively use the atlas as a localization map; their latest research findings can themselves become the most direct and effective source of feedback for optimizing and refining the atlas annotations. However, a standard stereoscopic brain atlas typically contains around one hundred standard anatomical sections, with the number of brain regions on each section ranging from dozens to hundreds. The annotation workload is extremely large, while the number of anatomical experts capable of performing the annotation work is extremely limited, thus severely restricting the update speed of brain atlases. Moreover, as "books," brain atlases still follow the traditional publishing model, with publishers inviting one or two anatomical experts to update them every few years. This inevitably limits the annotation content of brain atlases to the personal experience and perspective of a few anatomical experts, resulting in a large amount of new advances in neuroscience research not being synchronized with newly published brain atlases.

[0004] All these factors ultimately reduce the ideal "interaction between brain maps and the brain science community" to a one-way release of brain maps. Brain scientists, as users of brain maps, can only passively wait for irregular updates from anatomical experts. Not only is real-time updates unreliable, but the latest research findings in the field are also likely to be missed due to the subjective judgment of the anatomical experts.

[0005] In recent years, with the development of mesoscopic brain imaging technology, studies such as spatial transcriptomics and single-neuronal projection circuits have provided clues for brain functional partitioning that far exceed those of traditional neuroanatomy. In just over a decade, many classic brain regions in traditional brain atlases have been divided into more subregions, or their boundaries have been redefined due to more functional clues. The advancements in brain anatomy during this period far surpass those of the past century. However, the publication methods of traditional brain atlases have remained unchanged. Against this backdrop, how to complete large-scale brain region annotation work through efficient and collaborative annotation tools to overcome the bottlenecks and problems of updating traditional brain atlas publications has become one of the core challenges in the brain atlas mapping process.

[0006] Several tools have been developed to address this issue, but each has its own limitations.

[0007] In the field of online collaborative annotation, WEBKNOSSOS is an open-source online annotation tool for large 3D images, widely used in the collaborative analysis of electron microscopy data. However, WEBKNOSSOS was originally designed to serve neuronal signal tracing in electron microscopy data and lacks auxiliary labeling strategies for atlas mapping scenarios. Therefore, its interaction method is not suitable for brain atlas annotation scenarios involving brain region segmentation. Similar issues exist with BrainBox, a web-based collaborative annotation tool for magnetic resonance imaging (MRI) brain data. Another tool called Cytomine is a web-based collaborative analysis platform for large-scale biomedical images, supporting multi-user online annotation and quality control, but it is primarily geared towards browsing and annotating single slices.

[0008] In the field of brain mapping visualization and analysis, BrAVe is an open-source, cross-species 3D brain mapping visualization framework that supports cross-modal, multi-scale brain mapping data integration and analysis. However, BrAVe only provides visualization and analysis functions and does not support annotation and editing operations. NeuroAtlas is an AI-based brain mapping and brain region segmentation framework that achieves automated brain region identification and segmentation through deep learning models, but it relies on AI-automated segmentation and does not support interactive manual annotation by researchers based on their own expertise. In the field of general medical image analysis, 3D Slicer is a widely used open-source platform. Its NeuroSegmentation extension module supports the segmentation of neural structures, but as desktop software, 3DSlicer does not support online collaborative editing via the web. Summary of the Invention

[0009] Based on the aforementioned deficiencies in the existing technology, this application provides an online collaborative revision and visualization method for 3D brain map annotation, which enables efficient editing of 3D brain map annotation information, automatic generation of intermediate frame transition effects, and multi-user online collaborative revision and visualization.

[0010] To achieve the above objectives, this application provides an online collaborative revision and visualization method for 3D brain map annotation. This method relies on system tools for execution, which employ a three-tiered web architecture with a front-end and back-end separation, comprising a front-end presentation layer, a back-end service layer, and a data storage layer. The data storage layer is responsible for the persistent storage of I and L, where I is the original brain map image and L is the brain map annotation information. The front-end presentation layer, built on the React framework, is responsible for rendering the user interface and capturing annotation operations. Its core is the canvas area, which uses HTML5 Canvas for real-time rendering of I and L. User mouse operations on the canvas are captured by the front-end presentation layer and converted into annotation commands, which are then sent to the back-end service layer. The back-end service layer, built on the Python FastAPI framework, provides RESTful APIs for image processing, annotation calculation, and data management. The API interface allows the backend service layer to receive annotation instructions from the frontend presentation layer, call the corresponding image processing algorithm to complete the annotation calculation, and push the results to the data storage layer. The three layers communicate with each other via HTTP / WebSocket protocol. The frontend presentation layer initiates a data read request via HTTP protocol and receives real-time annotation updates pushed by the backend service layer via WebSocket protocol, realizing the synchronization of annotation status between multiple terminals.

[0011] In one embodiment, I provides image texture information of brain tissue as a reference for classifying different brain anatomical structures; L has the same size as I but is stored separately. L is stored in multi-frame TIF / TIFF format, with each frame corresponding to a two-dimensional cross-section of the brain tissue. Pixels belonging to the same brain region in L are filled with the same gray value, and each gray value uniquely corresponds to a brain region. The data storage layer also has T and Nom. T is a "brain region-gray value" mapping table, where each record represents a unique gray value assigned to a certain brain region. Nom is a naming tree, a tree structure describing the hierarchical relationship between various brain regions in the whole brain. The leaf nodes in the naming tree correspond to the brain regions at the finest level. L only records the gray values ​​of the leaf nodes. The annotation range of non-leaf node brain regions is obtained by querying the naming tree and merging the annotations of all leaf node brain regions it contains.

[0012] In one embodiment, the method includes: before performing annotation operations, the user first selects a working area, the system tool performs connected component analysis to limit the scope of subsequent operations, the connected component analysis is based on the currently valid L, and the set of pixels in L that have the same gray value and are spatially connected are identified as independent brain regions, and the several brain regions selected by the user together form the working area, the connected component analysis adopts the eight-connected neighborhood determination criterion, a pixel and its adjacent pixels in the above, below, left, right and four diagonal directions belong to the same connected component.

[0013] In one embodiment, the system tools have three complementary annotation tools, each suitable for different annotation scenarios: a lasso tool for large-area annotation of brain regions, a brush tool for fine modification or small-area annotation of brain regions, and an eraser tool for erasing already drawn annotations. The method includes: after switching to the lasso tool, the user first right-clicks on the target brain region to set the grayscale value of the drawing target, then holds down the left mouse button and drags to draw the area within the selection. During the dragging process, the system displays the drawing trajectory in real time. After releasing the mouse, the system calculates the filling area based on the drawn outline and displays a fill preview mask; after switching to the brush tool, the user holds down the left mouse button and drags to start drawing. Releasing the mouse automatically saves the drawing content. The user can use the left square bracket key to decrease the brush diameter and the right square bracket key to increase the brush diameter. The change in brush diameter is reflected in the cursor size in real time; after switching to the eraser tool, the user holds down the left mouse button and drags to erase the content of the area. The diameter of the eraser tool is also adjusted using the square bracket key.

[0014] In one embodiment, the system tool has a brain region segmentation function, allowing the user to manually segment a brain region into multiple sub-regions. The method includes: the user selects the segmentation function, holds down the left mouse button and drags to draw a segmentation line, and after releasing the mouse, the system segments the selected brain region in the working area into multiple sub-regions according to the segmentation line, with each sub-region displayed in a different color; the user clicks on each sub-region in the right-hand scope panel, sets the target grayscale value in the pop-up input box, and finally clicks "Apply Modification" to complete the operation. The segmentation result automatically takes effect on all frames, and white anchor points are displayed on the segmentation line. The user can drag the anchor points to move their positions, right-click to delete anchor points, and double-click to insert new anchor points.

[0015] In one embodiment, the system tool has a mirror mode, where the user only needs to annotate one side, and the other side is automatically mirrored and synchronized. The method includes: after the user enables the mirror mode, the system displays a vertical mirror axis in the center of the image, and the area to the left of the axis becomes uneditable. When the user performs annotation operations in the right area, the system uses a pixel coordinate mirror mapping algorithm to synchronize the annotation operations on the right to the corresponding position on the left in real time. The mirror mapping is implemented as follows: for the coordinates (x, y) of the right-side annotated pixel, its mirror coordinates are calculated as (x' = 2 * axis_x - x, y' = y), where axis_x is the X coordinate of the mirror axis. The system synchronously assigns the grayscale value of the right-side pixel to the pixel at the mirror coordinate.

[0016] In one embodiment, the system tool has an interpolation standard function that automatically generates transition interpolation frames between keyframes. The interpolation annotation function is based on the distance transformation of the Signed Distance Function (SDF). The method includes: after the user annotates on multiple frames, the system enters the interpolation annotation stage; first, for the annotation area drawn by the user on the keyframe, the system converts it into a binary mask M, where the pixel value within the annotation area is 1 and the pixel value outside the area is 0; then, M is subjected to positive and negative distance transformations respectively: the positive distance transformation calculates the Euclidean distance from each pixel outside the annotation area to the nearest annotation boundary pixel and assigns a negative value; the negative distance transformation calculates the Euclidean distance from each pixel within the annotation area to the nearest annotation boundary pixel and assigns a positive value. The two are added together to obtain the SDF field, whose zero isosurface corresponds exactly to the boundary of the annotation area. For the SDF fields S1 and S2 of two adjacent keyframes, the SDF field of the intermediate frame t is: St = (1 - t) * S1 + t * S2,t is the normalized frame index, with a value range of 0 to 1. Taking the zero isosurface of St will give the boundary of the labeled area of ​​the intermediate frame.

[0017] In one embodiment, the system tool has an export function. The method includes: after the user completes the annotation, he / she selects to export, specifies the start frame number, end frame number and output file name in the pop-up settings window, and the system integrates all the annotation results of the user within the specified frame range into a standard OME-TIFF format file for export. The annotation results include lasso annotation, pen drawing, brain region segmentation and interpolation-generated intermediate frames. During the export process, the system first sorts the annotation data of all frames by frame number, then reads the pixel data of the annotated image L frame by frame, encodes it into an image frame in OME-TIFF format, and writes metadata such as frame size, pixel physical resolution and number of channels in the file header.

[0018] In one embodiment, the method uses the WebSocket protocol to achieve real-time synchronization of annotation status between multiple terminals. When a user completes annotation modification on a terminal, the front-end display layer packages the new annotation information L_new in the form of key-value pairs, where the "key" is the three-dimensional coordinates (x, y, z) of the modified pixel, and the "value" is the modified grayscale value. L_new is sent to the back-end service layer via the WebSocket protocol. The back-end service layer immediately sends a response to the front-end display layer indicating that the data has been received. According to the data block division rules of L, the back-end service layer groups the coordinates of the same data block in L_new into the same group. For any record in L_new, the block name of the data block to which the point belongs is calculated based on the coordinates (x, y, z). Taking the X-axis direction as an example, the block start coordinate X1 = floor(x / cube_size) * cube_size, and the block end coordinate X1' = min(floor(x / cube_size + 1) * cube_size, x_max), cube_size is a preset fixed value, and x_max is the horizontal dimension of the image size at the original resolution. The same applies to the Y and Z directions. After the coordinate transformation is completed, all records in L_new in the backend server layer are grouped by data blocks to form revision data L'new. The backend service layer uses the new annotation grayscale values ​​recorded in L'new to replace the old grayscale values ​​at the same coordinate positions in L, completing the update of the annotation information. After that, the backend service layer analyzes the display parameters of all currently connected terminals, determines which terminals' display content needs to be updated accordingly, and actively pushes the latest version of the annotation information to these terminals. After receiving the update, each terminal re-renders the annotation image on the canvas to achieve real-time synchronization between multiple terminals. In concurrent editing scenarios, the backend service layer adopts an optimistic locking mechanism for updating annotation data. The revised data that arrives later will overwrite the data that arrives earlier, ensuring that all terminals eventually converge to a consistent annotation state.

[0019] The online collaborative revision and visualization method for 3D brain map annotation described in this application integrates various interactive annotation tools such as lasso annotation, pen annotation, brain region segmentation, and mirror symmetry annotation through a front-end and back-end separated web architecture. Combined with a keyframe interpolation algorithm based on the signed distance function (SDF) and a multi-terminal synchronization mechanism, it achieves efficient editing of multi-frame TIF format 3D brain map annotation information, automatic generation of intermediate frame transition effects, and multi-user online collaborative revision and visualization. It is applicable to brain map revision and iterative optimization in neuroscience research, especially subregion segmentation and brain region boundary modification in multi-user collaborative editing scenarios. Compared with existing technologies, it has the following beneficial effects: First, it natively supports multi-frame TIF format. This invention directly reads and processes multi-frame TIF / TIFF format brain atlas images without format conversion, avoiding information loss and additional storage overhead during data conversion. This enables this invention to directly serve the annotation of mesoscopic-level brain atlases generated by optical microscopy imaging.

[0020] Second, SDF-based keyframe interpolation significantly improves annotation efficiency. Users only need to manually annotate the first and last frames (or several keyframes), and the system can automatically generate smooth transition annotations for intermediate frames. Compared to the traditional method of manual annotation frame by frame, this method can reduce the annotation workload by one to two orders of magnitude, while ensuring the smoothness and continuity of transition boundaries through SDF distance transformation.

[0021] Third, mirror-symmetric annotation fully utilizes the left-right symmetry of brain images. When annotating left-right symmetrical brain images, users only need to annotate one side, and the other side is automatically mirrored and synchronized, doubling the annotation efficiency while ensuring complete symmetry between the left and right sides.

[0022] Fourth, multiple tools work together to meet the needs of different annotation scenarios. The lasso tool is suitable for rapid annotation of large areas, the brush tool is suitable for fine-tuning of boundaries, the segmentation tool supports subdividing a brain region into multiple sub-regions, and the eraser tool supports the correction of local annotations. The organic combination of multiple tools enables users to efficiently complete the entire annotation process from coarse to fine.

[0023] Fifth, the online collaborative revision mechanism supports multi-user participation in brain atlas annotation. The WebSocket-based real-time synchronization mechanism enables researchers located in different areas to collaborate on the same set of brain atlas data, each responsible for annotating different brain regions or areas, with the system automatically merging the revision results of all users. This provides a technological foundation for the brain science community to pool collective wisdom and iteratively optimize brain atlas annotation.

[0024] Sixth, the front-end and back-end separated web architecture lowers the barrier to entry. Users can access the system through a browser to complete all annotation operations without installing additional client software. The front-end React framework ensures the responsiveness of the interactive interface and cross-platform compatibility, while the back-end FastAPI framework provides efficient asynchronous processing capabilities to handle the processing needs of large-scale image data. Attached Figure Description

[0025] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this application in any way. Furthermore, the shapes and scales of the components in the drawings are merely illustrative to aid in understanding this application and do not specifically limit the shapes and scales of the components. Those skilled in the art, guided by the teachings of this application, can select various possible shapes and scales to implement this application according to specific circumstances. In the drawings: Figure 1 A schematic diagram of the system tool architecture upon which the online collaborative revision and visualization method for 3D brain atlas annotation provided in this application is based; Figure 2 is a schematic diagram of the interface of the system tool on which the online collaborative revision and visualization method for three-dimensional brain map annotation provided in this application is based, wherein (a) is the main interface and (b) is the toolbar in the main interface. Figure 3 A flowchart illustrating the lasso annotation process in an online collaborative revision and visualization method for 3D brain atlas annotation provided in this application; Figure 4 is a schematic diagram of the lasso annotation effect in an online collaborative revision and visualization method for 3D brain map annotation provided in this application, wherein (a) is a schematic diagram before annotation and (b) is a schematic diagram after annotation; Figure 5 A schematic diagram of the pen annotation process in an online collaborative revision and visualization method for 3D brain map annotation provided in this application; Figure 6 is a schematic diagram of the effect of pen annotation in an online collaborative revision and visualization method for 3D brain map annotation provided in this application, wherein (a) is a schematic diagram before annotation and (b) is a schematic diagram after annotation; Figure 7 A flowchart illustrating the segmentation annotation process in an online collaborative revision and visualization method for 3D brain atlas annotation provided in this application; Figure 8 is a schematic diagram of the segmentation annotation effect in an online collaborative revision and visualization method for 3D brain map annotation provided in this application, wherein (a) is a schematic diagram before annotation and (b) is a schematic diagram after annotation; Figure 9 is a schematic diagram of the effect of the mirror function in an online collaborative revision and visualization method for three-dimensional brain map annotation provided in this application, wherein (a) is a schematic diagram of the effect when the mirror function is turned on, and (b) is a schematic diagram of the effect when the mirror function is turned off. Figure 10 is a schematic diagram of the effect of interpolation annotation in an online collaborative revision and visualization method for three-dimensional brain map annotation provided in this application. Part (a) is the first image before interpolation annotation, part (b) is the second image before interpolation annotation, which also serves as the eighth image after interpolation annotation, and parts (c)-(h) are the first to sixth images obtained by interpolation annotation, which also serve as the second to seventh images after interpolation annotation. Figure 11 A flowchart derived from an online collaborative revision and visualization method for 3D brain atlas annotation provided in this application; Figure 12 This is a flowchart illustrating the online collaborative revision and multi-terminal synchronization process in an online collaborative revision and visualization method for 3D brain atlas annotation provided in this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0027] The applicant's research suggests that, in order to solve the aforementioned technical problems and support the brain science research community in carrying out large-scale brain region annotation work efficiently and collaboratively, so as to integrate the latest brain anatomy research results into the brain atlas as soon as possible, the corresponding brain image annotation and visualization tools need to meet the following requirements: 1) Support multiple users to access and edit online at the same time; 2) A variety of different annotation tools to support different usage scenarios such as subregion division and brain region boundary correction; 3) Support auxiliary labeling strategies for atlas drawing scenarios such as left and right hemisphere mirroring and multi-frame interpolation to improve the efficiency of brain region annotation.

[0028] Therefore, this application provides an online collaborative revision and visualization method for 3D brain atlas annotation. This method relies on system tools for execution, the system tools (see [link to relevant documentation]). Figure 1The diagram (shown) adopts a three-tiered web architecture with a front-end and back-end separation, comprising a front-end presentation layer, a back-end service layer, and a data storage layer. The data storage layer is responsible for the persistent storage of I and L, where I is the original brain map image and L is the brain map annotation information. The front-end presentation layer, built on the React framework, is responsible for rendering the user interface and capturing annotation operations. Its core is the canvas area, which uses HTML5 Canvas to achieve real-time rendering of I and L. User mouse operations on the canvas are captured by the front-end presentation layer and converted into annotation commands, which are then sent to the back-end service layer. The back-end service layer, built on the Python FastAPI framework, provides RESTful API interfaces for image processing, annotation calculation, and data management. After receiving annotation commands from the front-end presentation layer, the back-end service layer calls the corresponding image processing algorithm to complete the annotation calculation and pushes the results to the data storage layer. The three layers communicate via HTTP / WebSocket protocols. The front-end presentation layer initiates data read requests via HTTP and receives real-time annotation updates pushed by the back-end service layer via WebSocket, achieving annotation status synchronization across multiple terminals. User mouse operations on the canvas include clicking, dragging, and scrolling.

[0029] Wherein, I is the original image, used to provide image texture information of brain tissue as a reference for classifying different brain anatomical structures; L is the labeled image, with the same size as I, stored separately, L is stored in multi-frame TIF / TIFF format, each frame corresponds to a two-dimensional cross-section of brain tissue, pixels belonging to the same brain region in L are filled with the same gray value, and each gray value uniquely corresponds to a brain region; the data storage layer also has T and Nom, T is a "brain region-gray value" mapping table, each record in the table represents a unique gray value assigned to a certain brain region; Nom is a naming tree, a tree structure describing the hierarchical relationship between various brain regions in the whole brain, the leaf nodes in the naming tree correspond to the brain regions at the finest level; L only records the gray values ​​of leaf nodes, and the labeling range of non-leaf node brain regions is obtained by querying the naming tree and merging the labels of all leaf node brain regions contained therein.

[0030] Before performing annotation operations, users generally need to select a working area to limit the scope of subsequent operations. Therefore, the applicant designed a connected component analysis function for the system tool. In one embodiment, the method includes: before performing annotation operations, the user selects a working area (selecting the "Select Area" function, see Figure 2). The system tool performs connected component analysis to limit the scope of subsequent operations. The connected component analysis is based on the currently valid L, identifying the set of pixels in L with the same gray value and spatially connected as independent brain regions. Several brain regions selected by the user together form the working area. The connected component analysis adopts the eight-connected neighborhood criterion, where a pixel and its adjacent pixels in the upper, lower, left, right, and four diagonal directions belong to the same connected component. The specific process is as follows: the user clicks the "Select Area" button in the upper left corner to enter the selection mode. The system performs connected component analysis based on the annotated image L of the current frame to obtain region masks of several independent brain regions. The user clicks the brain regions to be annotated in sequence with the left mouse button. The boundaries of the selected brain regions are highlighted in yellow, indicating that they have been added to the working area. Clicking the selected area again deselects it. Once the selection is complete, subsequent annotation operations such as drawing, lassoing, and segmentation only apply to the selected area. The system automatically ignores operation requests outside the selection area to prevent accidental drawing of other brain regions. Even if the grayscale value of a brain region has been modified by previous annotation operations, connected component analysis can still correctly identify and select that brain region.

[0031] Please see Figure 1 As shown in Figure 2, in one embodiment, the system tools have three complementary annotation tools, each suitable for different annotation scenarios, including: a lasso tool for large-area annotation of brain regions, a brush tool for fine-tuning or small-area annotation of brain regions, and an eraser tool for erasing already drawn annotations. The method includes: after the user switches to the lasso tool (presses the shortcut key D), please refer to... Figure 3 As shown in Figure 4, first right-click on the target brain region to set the grayscale value of the drawing target. Then, within the selection area, hold down the left mouse button and drag to draw the area. During the dragging process, the system displays the drawing trajectory in real time. After releasing the mouse, the system calculates the fill area based on the drawn outline and displays a fill preview mask. The user can press Ctrl+S to save the current frame as a keyframe. After switching to the brush tool (press the shortcut key B), please refer to... Figure 5 As shown in Figure 6, hold down the left mouse button and drag to start drawing. Releasing the mouse button automatically saves the drawing. Users can use the left square bracket key to decrease the brush diameter and the right square bracket key to increase the brush diameter. The change in brush diameter is reflected in the cursor size in real time. After switching to the erase tool (press the shortcut key E), users can hold down the left mouse button and drag to erase the content in the area. The diameter of the erase tool can also be adjusted using the square bracket keys. The drawing content of the above three tools is displayed on the canvas in the form of a mask. It is only a preview state before the user clicks "Apply Modifications" and does not modify the underlying image data.

[0032] In one embodiment, the system tool has a brain region segmentation function, allowing the user to manually segment a brain region into multiple sub-regions. The method includes: the user selecting the segmentation function (see Figure 2). Figure 7 As shown in Figure 8, hold down the left mouse button and drag to draw the segmentation line. After releasing the mouse, the system will divide the selected brain region in the working area into multiple sub-regions according to the segmentation line. Each sub-region is colored and displayed. In the scope panel on the right, the user clicks on each sub-region in turn, sets the target grayscale value in the pop-up input box, and finally clicks "Apply Modification" to complete the operation. The segmentation result will take effect automatically on all frames, and white anchor points will be displayed on the segmentation line. The user can drag the anchor point to move its position, delete the anchor point by right-clicking, and insert a new anchor point by double-clicking.

[0033] In one embodiment, the system tool has a mirror mode, where the user only needs to annotate one side, and the other side is automatically mirrored and synchronized. The method includes: after the user activates mirror mode, as shown in Figures 2 and 9, the system displays a vertical mirror axis in the center of the image. The area to the left of the axis becomes uneditable. When the user performs annotation operations in the right area, the system uses a pixel coordinate mirror mapping algorithm to synchronize the annotation operations on the right side to the corresponding positions on the left in real time. The mirror mapping is implemented as follows: for the coordinates (x, y) of the pixel annotated on the right, its mirror coordinates are calculated as (x' = 2 * axis_x - x, y' = y), where axis_x is the X coordinate of the mirror axis. The system synchronously assigns the grayscale value of the pixel on the right to the pixel at the mirror coordinates. Mirror mode supports all annotation functions, including lasso drawing, pen drawing, and brain region segmentation. After the mirror mode is turned off, annotation operations can be performed independently on the left and right sides.

[0034] In one embodiment, the system tool has an interpolation standard function that automatically generates transition interpolation frames between keyframes. The interpolation annotation function is based on the distance transformation of the Signed Distance Function (SDF). The method includes: after the user annotates on multiple frames, the system enters the interpolation annotation stage; first, for the annotation area drawn by the user on the keyframe, the system converts it into a binary mask M, where the pixel value within the annotation area is 1 and the pixel value outside the area is 0; then, M is subjected to positive and negative distance transformations respectively: the positive distance transformation calculates the Euclidean distance from each pixel outside the annotation area to the nearest annotation boundary pixel and assigns a negative value; the negative distance transformation calculates the Euclidean distance from each pixel within the annotation area to the nearest annotation boundary pixel and assigns a positive value. The two are added together to obtain the SDF field, whose zero isosurface corresponds exactly to the boundary of the annotation area. For the SDF fields S1 and S2 of two adjacent keyframes, the SDF field of the intermediate frame t is: St = (1 - t) * S1 + t * S2,t is the normalized frame index, ranging from 0 to 1. Taking the zero isosurface of St yields the boundary of the labeled region for the intermediate frame, as shown in Figure 10, illustrating the effect of the interpolation annotation. This algorithm generates a smooth and natural transition effect, avoiding the jagged edges and breaks that may occur with simple pixel-by-pixel linear interpolation. Users can view the interpolation results at any time; if the effect is unsatisfactory, they can adjust the keyframe position or increase the number of keyframes. For annotations of brain regions not at the finest level, the system queries the named tree (Nom) for its leaf node brain regions, performs SDF interpolation on each leaf node brain region, and then merges the results to generate the interpolation annotation for that level.

[0035] In one embodiment, the system tool has an export function; please refer to [link / reference]. Figure 11 As shown, the method includes: after the user completes the annotation, they select export and specify the start frame number, end frame number, and output file name in the pop-up settings window. The system integrates all annotation results within the specified frame range into a standard OME-TIFF format file for export. The annotation results include lasso annotation, pen drawing, brain region segmentation, and interpolated intermediate frames. During the export process, the system first sorts the annotation data of all frames by frame number, then reads the pixel data of the annotated image L frame by frame, encodes it into OME-TIFF format image frames, and writes metadata such as frame size, pixel physical resolution, and number of channels into the file header. OME-TIFF is an open standard format in the field of biomedical imaging, supporting the storage of multi-frame, multi-channel image data and including rich metadata information. After the export is complete, the system displays a download link in the window, and the user can click the link to save the annotated file locally. The exported OME-TIFF file can be opened and viewed in standard image processing software such as ImageJ, Fiji, and 3D Slicer.

[0036] Please see Figure 12 As shown, in one embodiment, the method uses the WebSocket protocol to achieve real-time synchronization of annotation status between multiple terminals. When a user completes annotation modification on a terminal, the front-end display layer packages the new annotation information L_new in the form of key-value pairs, where the "key" is the three-dimensional coordinates (x, y, z) of the modified pixel, and the "value" is the modified grayscale value. L_new is sent to the back-end service layer via the WebSocket protocol. The back-end service layer immediately sends a response to the front-end display layer indicating that the data has been received. According to the data block division rules of L, the back-end service layer groups the coordinates of the same data block in L_new into the same group. For any record in L_new, the block name of the data block to which the point belongs is calculated based on the coordinates (x, y, z). Taking the X-axis direction as an example, the block start coordinate X1 = floor(x / cube_size) * cube_size, and the block end coordinate X1' = min(floor(x / cube_size + 1) * cube_size, x_max), cube_size is a preset fixed value, and x_max is the horizontal dimension of the image size at the original resolution. The same applies to the Y and Z directions. After the coordinate transformation is completed, all records in L_new in the backend server layer are grouped by data blocks to form revision data L'new. The backend service layer uses the new annotation grayscale values ​​recorded in L'new to replace the old grayscale values ​​at the same coordinate positions in L, completing the update of the annotation information. After that, the backend service layer analyzes the display parameters of all currently connected terminals, determines which terminals' display content needs to be updated accordingly, and actively pushes the latest version of the annotation information to these terminals. After receiving the update, each terminal re-renders the annotation image on the canvas to achieve real-time synchronization between multiple terminals. In concurrent editing scenarios, the backend service layer adopts an optimistic locking mechanism for updating annotation data. The revised data that arrives later will overwrite the data that arrives earlier, ensuring that all terminals eventually converge to a consistent annotation state.

[0037] The online collaborative revision and visualization method for 3D brain map annotation described in this application integrates various interactive annotation tools, such as lasso annotation, pen annotation, brain region segmentation, and mirror symmetry annotation, through a front-end and back-end separated web architecture. Combined with a keyframe interpolation algorithm based on the signed distance function (SDF) and a multi-terminal synchronization mechanism, it achieves efficient editing of multi-frame TIF format 3D brain map annotation information, automatic generation of intermediate frame transition effects, and multi-user online collaborative revision and visualization. It is applicable to brain map revision and iterative optimization in neuroscience research, especially subregion division and brain region boundary modification in multi-user collaborative editing scenarios.

[0038] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this teaching should not be determined by reference to the above description, but rather by reference to the foregoing claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the foregoing claims is not intended as a waiver of that subject matter, nor should it be construed as an indication that the applicant has not considered that subject matter as part of the disclosed subject matter.

Claims

1. An online collaborative revision and visualization method for 3D brain atlas annotation, characterized in that, The method relies on system tools for execution. The system tools adopt a three-tier Web architecture with front-end and back-end separation, including a front-end presentation layer, a back-end service layer, and a data storage layer. The data storage layer is responsible for the persistent storage of I and L, where I is the original brain map image and L is the brain map annotation information. The front-end presentation layer, built on the React framework, is responsible for rendering the user interface and capturing annotation operations. Its core is the canvas area, which uses HTML5 Canvas to render the I and L characters in real time. User mouse actions on the canvas are captured by the front-end presentation layer and converted into annotation commands, which are then sent to the back-end service layer. The back-end service layer, built on the Python FastAPI framework, provides RESTful API interfaces for image processing, annotation calculation, and data management. After receiving annotation commands from the front-end presentation layer, the back-end service layer calls the corresponding image processing algorithm to complete the annotation calculation and pushes the results to the data storage layer. The three layers communicate via HTTP / WebSocket protocols. The front-end presentation layer initiates data read requests via HTTP and receives real-time annotation updates pushed by the back-end service layer via WebSocket, enabling synchronization of annotation status across multiple terminals.

2. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, I provides image texture information of brain tissue as a reference for classifying different brain anatomical structures; L has the same size as I but is stored separately. L is stored in multi-frame TIF / TIFF format, with each frame corresponding to a two-dimensional cross-section of brain tissue. Pixels belonging to the same brain region in L are filled with the same gray value, and each gray value uniquely corresponds to a brain region. The data storage layer also has T and Nom. T is a "brain region-gray value" mapping table, where each record represents a unique gray value assigned to a certain brain region. Nom is a naming tree, a tree structure describing the hierarchical relationship between various brain regions in the whole brain. The leaf nodes in the naming tree correspond to the most refined level of brain regions. L only records the gray values ​​of leaf nodes. The annotation range of non-leaf node brain regions is obtained by querying the naming tree and merging the annotations of all leaf node brain regions it contains.

3. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, The method includes: before performing annotation operations, the user first selects a working area, the system tool performs connected component analysis to limit the scope of subsequent operations, the connected component analysis is based on the currently valid L, and the set of pixels in L with the same gray value and spatially connected are identified as independent brain regions, and the several brain regions selected by the user together form the working area, the connected component analysis adopts the eight-connected neighborhood determination criterion, a pixel and its adjacent pixels in the above, below, left, right and four diagonal directions belong to the same connected component.

4. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, The system tools include three complementary annotation tools, each suitable for different annotation scenarios: a lasso tool for large-area annotation of brain regions, a brush tool for fine-tuning or small-area annotation of brain regions, and an eraser tool for erasing already drawn annotations. The method includes: after switching to the lasso tool, the user first right-clicks on the target brain region to set the grayscale value of the drawing target, then holds down the left mouse button and drags to draw the area within the selection. During the dragging process, the system displays the drawing trajectory in real time. After releasing the mouse, the system calculates the fill area based on the drawn outline and displays a fill preview mask; after switching to the brush tool, the user holds down the left mouse button and drags to start drawing. Releasing the mouse automatically saves the drawing content. The user can use the left square bracket key to decrease the brush diameter and the right square bracket key to increase the brush diameter. The change in brush diameter is reflected in the cursor size in real time; after switching to the eraser tool, the user holds down the left mouse button and drags to erase the content of the area. The diameter of the eraser tool is also adjusted using the square bracket key.

5. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 4, characterized in that, The system tool has a brain region segmentation function, which allows users to manually divide a brain region into multiple sub-regions. The method includes: the user selects the segmentation function, holds down the left mouse button and drags to draw a segmentation line, and after releasing the mouse, the system divides the selected brain region in the working area into multiple sub-regions according to the segmentation line. Each sub-region is displayed with a different color. In the scope panel on the right, the user clicks on each sub-region in turn, sets the target grayscale value in the pop-up input box, and finally clicks "Apply Modification" to complete the operation. The segmentation result takes effect automatically on all frames, and white anchor points are displayed on the segmentation line. The user can drag the anchor points to move their positions, delete anchor points by right-clicking, and insert new anchor points by double-clicking.

6. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, The system tool has a mirror mode. Users only need to annotate one side, and the other side is automatically mirrored and synchronized. The method includes: after the user enables mirror mode, the system displays a vertical mirror axis in the center of the image. The area to the left of the axis becomes uneditable. When the user performs annotation operations in the right area, the system uses a pixel coordinate mirror mapping algorithm to synchronize the annotation operations on the right to the corresponding position on the left in real time. The mirror mapping is implemented as follows: for the coordinates (x, y) of the right-side annotated pixel, its mirror coordinates are calculated as (x' = 2 * axis_x - x, y' = y), where axis_x is the X coordinate of the mirror axis. The system synchronously assigns the grayscale value of the right-side pixel to the pixel at the mirror coordinate.

7. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, The system tool has an interpolation standard function, automatically generating transitional interpolation frames between keyframes. The interpolation annotation function is based on the distance transformation of the Signed Distance Function (SDF). The method includes: after the user annotates on multiple frames, the system enters the interpolation annotation stage; first, for the annotation area drawn by the user on the keyframe, the system converts it into a binary mask M, where the pixel value within the annotation area is 1 and the pixel value outside the area is 0; then, M is subjected to positive and negative distance transformations respectively: the positive distance transformation calculates the Euclidean distance from each pixel outside the annotation area to the nearest annotation boundary pixel and assigns a negative value; the negative distance transformation calculates the Euclidean distance from each pixel within the annotation area to the nearest annotation boundary pixel and assigns a positive value. The two are added together to obtain the SDF field, whose zero isosurface corresponds exactly to the boundary of the annotation area. For the SDF fields S1 and S2 of two adjacent keyframes, the SDF field of the intermediate frame t is: St = (1 - t) * S1 + t * S2,t is the normalized frame index, with a value range of 0 to 1. Taking the zero isosurface of St will give the boundary of the labeled area of ​​the intermediate frame.

8. The online collaborative revision and visualization method for 3D brain atlas annotation as described in claim 1, characterized in that, The system tool has an export function. The method includes: after the user completes the annotation, he / she selects to export, and specifies the start frame number, end frame number and output file name in the pop-up settings window. The system integrates all the annotation results of the user within the specified frame range into a standard OME-TIFF format file for export. The annotation results include lasso annotation, pen drawing, brain region segmentation and interpolation-generated intermediate frames. During the export process, the system first sorts the annotation data of all frames by frame number, and then reads the pixel data of the annotated image L frame by frame, encodes it into an image frame in OME-TIFF format, and writes metadata such as frame size, pixel physical resolution and number of channels in the file header.

9. The online collaborative revision and visualization method for three-dimensional brain atlas annotation as described in claim 1, characterized in that, The method described above uses the WebSocket protocol to achieve real-time synchronization of annotation status across multiple terminals. When a user completes annotation modification on a terminal, the front-end presentation layer packages the new annotation information L_new as key-value pairs, where the "key" is the 3D coordinates (x, y, z) of the modified pixel, and the "value" is the modified grayscale value. L_new is sent to the back-end service layer via the WebSocket protocol. The back-end service layer immediately sends a response to the front-end presentation layer indicating that the data has been received. According to the data block partitioning rules of L, the back-end service layer groups the coordinates of points belonging to the same data block in L_new into the same group. For any record in L_new, the block name of the data block to which the point belongs is calculated based on the coordinates (x, y, z). Taking the X-axis direction as an example, the block start coordinate X1 = floor(x / cube_size) * cube_size, and the block end coordinate X1' = min(floor(x / cube_size + 1) * cube_size, x_max), cube_size is a preset fixed value, x_max is the horizontal dimension of the image size at the original resolution, and the same applies to the Y and Z directions. After the coordinate transformation is completed, all records in L_new in the backend server layer are grouped by data blocks to form revision data L'new. The backend service layer uses the new annotation grayscale values ​​recorded in L'new to replace the old grayscale values ​​at the same coordinate positions in L, completing the update of the annotation information. After that, the backend service layer analyzes the display parameters of all currently connected terminals, determines which terminals' display content needs to be updated accordingly, and actively pushes the latest version of the annotation information to these terminals. After receiving the update, each terminal re-renders the labeled image on the canvas to achieve real-time synchronization between multiple terminals. In concurrent editing scenarios, the backend service layer adopts an optimistic locking mechanism for updating the labeled data. The revised data that arrives later will overwrite the data that arrives earlier, ensuring that all terminals eventually converge to a consistent labeled state.

Images