DICOM (Digital Imaging and Communications in Medicine) image annotation method and device applied to Web and computer equipment

Abstract

The invention relates to the technical field of image processing, and discloses a DICOM image annotation method and device applied to Web and computer equipment, and the method comprises the steps: creating an image display layer and an annotation drawing layer in response to a DICOM image annotation instruction; the image display layer and the annotation drawing layer are rendering layers in the Web; the label drawing layer is located on the upper layer of the image display layer; and loading the DICOM image on the image display layer, and recording the annotation information of the DICOM image on the annotation drawing layer. The method has the beneficial effect that the technical effect of improving the labeling efficiency is achieved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and computer device for annotating DICOM images. Background Technology

[0002] With the deep integration of Web technology and medical imaging technology, browser-based DICOM image processing technology is gradually becoming a core requirement for telemedicine and mobile diagnosis.

[0003] Currently, for the pure JavaScript parsing solution based on Canvas 2D, JavaScript can be used directly in the browser to parse DICOM images and draw annotation information through the Canvas 2D API.

[0004] However, when Canvas 2D draws DICOM images and annotation graphics on the same canvas simultaneously, multi-layer rendering becomes difficult, requiring frequent canvas clearing and redrawing, resulting in low annotation efficiency. Summary of the Invention

[0005] This application provides a DICOM image annotation method, apparatus, and computer device for the Web, which solves the technical problem of low annotation efficiency and achieves the technical effect of improving annotation efficiency.

[0006] To achieve the above objectives, the main technical solutions adopted in this application include: In a first aspect, embodiments of this application provide a DICOM image annotation method applied to the Web, the method comprising: In response to DICOM image annotation commands, an image display layer and an annotation drawing layer are created; the image display layer and the annotation drawing layer are rendering layers in the Web; the annotation drawing layer is located above the image display layer. The DICOM image is loaded in the image display layer, and the annotation information of the DICOM image is recorded in the annotation drawing layer.

[0007] In this embodiment, a DICOM image is loaded through an image display layer, and the annotation information of the DICOM image is recorded through an annotation drawing layer. This enables separate processing of the DICOM image and annotation information, allowing the image display layer and annotation drawing layer to be rendered independently without interference. Visual consistency is achieved through overlay, ensuring that the image display layer can support professional medical image display functions such as window width and window level adjustment and pseudo-color mapping, while also enabling the annotation drawing layer to support high-performance geometric drawing and complex interactive editing, thus improving the display efficiency of DICOM images and annotation information.

[0008] Secondly, embodiments of this application provide a DICOM image annotation apparatus for use on the Web, the apparatus comprising: A response module is used to create an image display layer and an annotation drawing layer in response to DICOM image annotation commands; the image display layer and the annotation drawing layer are rendering layers in the Web; the annotation drawing layer is located above the image display layer; The display module is used to load the DICOM image in the image display layer and record the annotation information of the DICOM image in the annotation drawing layer.

[0009] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method described in any of the above embodiments.

[0010] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the method described in any one of the above embodiments.

[0011] Fifthly, embodiments of this application provide a computer program product, including computer instructions, which are used to cause a computer to perform the method described in any of the above embodiments. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 A flowchart illustrating a DICOM image annotation method applied to the Web, provided as an embodiment of this application; Figure 2 A schematic diagram of a MOD hierarchical architecture provided for an embodiment of this application; Figure 3 An architecture diagram of a DICOM image annotation system applied to the Web is provided for embodiments of this application; Figure 4 A structural diagram of a two-layer DOM provided in an embodiment of this application; Figure 5 A flowchart of a viewport synchronization control method provided in an embodiment of this application; Figure 6 A flowchart illustrating a DICOM image annotation method applied to the Web, provided as an embodiment of this application; Figure 7 A structural diagram of a DICOM image annotation device applied to the Web, provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] Digital Imaging and Communications in Medicine (DICOM) is an international standard in the field of medical imaging, jointly developed by the American College of Radiology (ACR) and the National Electrical Manufacturers Association (NEMA). This standard defines the format and transmission protocols for medical images and related information, and is widely used in various medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, and X-ray.

[0016] DICOM images contain not only pixel data of medical images but also rich metadata information. This metadata includes patient information, examination information, equipment parameters, window width / window level, etc. Correctly parsing and displaying DICOM images is crucial for the diagnosis and analysis of medical images.

[0017] Traditional medical image viewers are primarily desktop applications, typically developed using languages ​​such as C++ and Java, and employ the operating system's native graphics API for image rendering. For example, this native graphics API could be Windows GDI, OpenGL, or similar technologies.

[0018] Typical examples of traditional medical image viewers include: Osirix, a medical image analysis software designed for the macOS platform, developed using Objective-C and rendered with Core Graphics and OpenGL; RadiAnt, a medical image analysis software designed for the Windows platform, developed using C++ and DirectX; and 3DSlicer, an open-source medical image processing platform that uses the Visualization Toolkit (VTK) for 3D rendering.

[0019] Traditional medical image viewers are typically desktop applications. These desktop applications have advantages such as high rendering performance and support for large-scale DICOM datasets. Their functionality is usually quite comprehensive. However, these desktop applications have disadvantages such as requiring the installation of client software, poor cross-platform compatibility, inability to be accessed directly through a browser, and inconvenience for remote consultations and mobile use.

[0020] With the development of web technologies, browser-based DICOM image viewers gradually emerged to address the problems existing in the aforementioned desktop applications. Early solutions mainly included server-side conversion and pure JavaScript parsing based on Canvas 2D.

[0021] Specifically, for the server-side conversion solution, the server can convert DICOM images to standard image formats such as JPEG and PNG, and then use a browser to convert them via HTML. The image is displayed using a label or Canvas 2D API. In this process, the browser can transmit the DICOM image to the server for conversion to the standard JPEG / PNG image format, and then display the image through the browser. Early PACS Web Viewers were typical examples of server-side conversion solutions.

[0022] However, this server-side conversion solution loses the original pixel data precision of the DICOM image when converting it to JPEG / PNG, failing to accurately display the details of medical images and resulting in the loss of medical information. Furthermore, the converted standard image does not contain the metadata of the DICOM image, making professional medical image adjustments impossible. Additionally, each image display requires server-side conversion, consuming significant server computing resources and easily leading to high server load. Moreover, the conversion process is time-consuming and cannot respond in real-time to user requests for window width and level adjustments, resulting in poor real-time performance.

[0023] One approach, based on Canvas 2D and using pure JavaScript for parsing, allows direct parsing of DICOM images within the browser using JavaScript, followed by drawing via the Canvas 2D API. In this approach, DICOM images can be parsed using JavaScript and drawn using Canvas 2D. Early versions of DICOM Web Viewer (DWV) and Cornerstone.js were based on this method.

[0024] However, Canvas 2D uses the CPU for image rendering, which suffers from low frame rates when processing large medical images. Scaling and panning operations are prone to stuttering, resulting in a significant performance bottleneck. Furthermore, each adjustment of the window width and level requires recalculating every pixel value and redrawing the entire canvas, leading to slow response times and potential interaction delays. Additionally, rendering multiple layers simultaneously on the same canvas is difficult, requiring frequent canvas clearing and redrawing, resulting in low efficiency.

[0025] Medical image annotation refers to drawing geometric shapes such as polygons, rectangles, and circles on medical images for marking lesion areas, measuring distances, and calculating areas.

[0026] With the development of medical imaging, in order to improve some problems in the medical image annotation process of early schemes, the current web-based annotation technology mainly includes drawing schemes built using Canvas 2D and vector graphics schemes built using SVG.

[0027] The Canvas 2D drawing solution primarily uses the HTML5 Canvas 2D API to draw annotation graphics. This solution uses the CPU for rendering Canvas 2D. When drawing a large number of annotation objects, performance issues and poor rendering performance are likely to occur. Furthermore, this method lacks a built-in object management and event system, requiring manual implementation of interactive logic such as picking, dragging, and selection, resulting in complex interactions. Moreover, this method struggles to separate the image layer and the annotation drawing layer; modifying annotations requires redrawing the entire canvas, leading to difficulties in layer management.

[0028] The SVG vector graphics solution uses Scalable Vector Graphics (SVG) to draw annotations. In this solution, SVG is based on DOM rendering; when there are too many annotation objects, the DOM tree becomes enormous, leading to decreased browser reflow and repaint performance, thus still exhibiting performance limitations. Furthermore, DICOM images are bitmap data, while SVG is vector data; both lack a unified coordinate system and scaling synchronization mechanism, making integration with bitmap images difficult.

[0029] Based on the above technical solutions, it can be concluded that the current web-based display and annotation technology for medical images has the following core technical problems: First, DICOM image display and annotation drawing cannot be efficiently separated. Existing solutions typically draw DICOM images and annotations on the same Canvas or SVG, which easily leads to any annotation modification requiring the entire image to be redrawn, resulting in poor performance.

[0030] Secondly, both Canvas 2D and SVG are CPU-based rendering tools and cannot utilize GPU hardware acceleration. Therefore, they are insufficient in performance when processing high-resolution medical images and complex annotations, lacking hardware-accelerated rendering capabilities.

[0031] Third, DICOM images use a medical image coordinate system, typically in mm units, while labeled graphics use a screen pixel coordinate system. Therefore, it is difficult to maintain precise synchronization between the two during scaling and translation, often resulting in misalignment and hindering viewport synchronization.

[0032] Fourth, the functionality of professional DICOM viewers cannot be reconciled with that of professional annotation tools. Specifically, professional DICOM viewers lack robust annotation editing capabilities. Furthermore, the graphics libraries of professional annotation tools such as Fabric.js and Konva.js do not support the display of professional display features of DICOM, such as window width and level, and pseudo-color mapping.

[0033] The above analysis reveals significant technical deficiencies in the integration, performance optimization, and coordinate system synchronization of DICOM image display and annotation. Therefore, a new method for DICOM image parsing, rendering, and drawing is urgently needed to address these issues in the medical image annotation process.

[0034] To address the aforementioned issues, this application proposes a web-based DICOM image display and interactive annotation technology solution based on a dual-layer, separate rendering architecture. The DICOM image parsing, rendering, and drawing methods in this solution primarily aim to achieve a seamless integration of professional-grade display and high-performance interactive annotation drawing of DICOM medical images within a web browser environment, resolving technical challenges related to rendering performance, coordinate synchronization, and layer management. This solution achieves a unified approach to professional-grade display and high-performance annotation editing of medical images through the organic combination of a DICOM professional display layer (Cornerstone.js) and a WebGL annotation drawing layer (PIXI.js), coupled with a precise viewport synchronization algorithm.

[0035] Alternatively, Three.js can be used instead of PIXI.js. The advantage of using Three.js (a 3D graphics library) for 2D rendering instead of PIXI.js is that Three.js is more powerful and can be extended to the 3D medical imaging field, such as implementing 3D reconstruction. However, since Three.js is primarily 3D-oriented, its operation in 2D scenes is more complex, and its performance is not as good as PIXI.js, which focuses on 2D. Therefore, this technical approach is suitable for systems that need to support both 2D and 3D medical image display.

[0036] Specifically, this application optimizes existing technologies in six aspects: dual-layer rendering architecture, viewport coordinate synchronization, rendering performance optimization, retention of DICOM professional functions, interactive annotation editing, and image format compatibility and conversion, resulting in the DICOM image annotation method for Web application proposed in this application.

[0037] First, this application implements a separate two-layer rendering architecture by constructing two rendering layers in the browser: an image display layer for displaying DICOM images and a annotation display layer for displaying annotations. This architecture allows the image display layer and the annotation drawing layer to render independently without interfering with each other, while maintaining a perfect visual overlay effect. The image display layer supports professional medical image display functions such as window width and level adjustment and pseudo-color mapping. The annotation drawing layer supports high-performance geometric drawing and complex interactive editing. Since the image display layer and the annotation drawing layer need to be implemented on different rendering engines, but must be perfectly aligned visually to the user, this application's two-layer rendering architecture needs to address the integration issues of different rendering technology stacks (Cornerstone.js and PIXI.js).

[0038] To address this synchronization issue, this application employs a viewport coordinate synchronization method to ensure pixel-level precise synchronization between the image display layer and the annotation drawing layer during zooming and translation operations, preventing misalignment or drift between the two layers. Since the image display layer uses a medical image coordinate system (Cornerstone Viewport), with its origin at the image center and units typically in millimeters, and the annotation drawing layer uses a screen pixel coordinate system (PIXI.js Viewport), with its origin at the upper left corner of the canvas and units typically in pixels, it is necessary to calculate the same zoom center point to ensure synchronized zooming between the two layers to guarantee viewport coordinate synchronization during zooming operations. Furthermore, to ensure viewport coordinate synchronization during translation operations, it is typically necessary to convert the screen pixel offset to a DICOM image coordinate offset, thereby ensuring that the two images remain synchronized even when their origins, units, and transformation matrices differ between the two coordinate systems.

[0039] To improve rendering performance while ensuring the accuracy of medical image display, this application utilizes GPU hardware acceleration technology for optimization, supporting real-time interactive editing of hundreds of complex polygons and enhancing rendering performance. Compared to traditional Canvas 2D rendering, which uses CPU rendering, resulting in insufficient performance when handling a large number of labeled objects, and the high programming complexity after introducing WebGL hardware acceleration, GPU hardware acceleration can better solve this performance problem. Furthermore, compared to using Cornerstone.js (Canvas 2D rendering) for the image display layer and PIXI.js (WebGL rendering) for the WebGL annotation drawing layer, which requires coordinating two different rendering mechanisms during program execution, the GPU achieves unification of the two, improving rendering efficiency and ensuring smooth interaction at 60 FPS even in scenarios with high-resolution medical images and a large number of labeled objects.

[0040] By employing the PIXI.js WebGL rendering engine in the annotation drawing layer, full utilization of GPU hardware is further achieved. Compared to traditional Canvas 2D CPU rendering, a significant performance improvement of 5-10 times is realized.

[0041] Alternatively, a single WebGL renderer can be used to render both the DICOM image and annotations via WebGL, instead of Cornerstone.js. The advantage of this approach is a unified rendering technology stack, eliminating the need for coordinate synchronization; however, the disadvantage is the need to implement specialized functions such as DICOM parsing, window width, and window level from scratch, resulting in high development costs and difficulty in achieving the professional level of Cornerstone.js. This approach is suitable for medical imaging systems with ample development resources and requiring high customization.

[0042] For example, performance comparison test data conducted under the same hardware environment clearly demonstrates this advantage: when the labeled object has 50 polygons, Canvas 2D renders at 45 FPS, while PIXI.js WebGL renders at 60 FPS, a performance improvement of 1.3 times. Furthermore, as the number of labeled objects increases to 100, 200, and 500 polygons, the Canvas 2D rendering frame rate drops to 28 FPS, 15 FPS, and 6 FPS respectively, while the PIXI.js WebGL rendering frame rate remains stable at around 60 FPS, with performance improvements of 2.1 times, 4 times, and 9.7 times respectively. From a technical perspective, Canvas 2D... CPU rendering relies on CPU computation and rasterization for each polygon drawing. PIXI.js WebGL, however, leverages the parallel computing capabilities of the GPU, enabling the simultaneous processing of hundreds of graphical objects. In practical applications, particularly in large-scale annotation scenarios such as pathological slide labeling and cell nucleus recognition, this application supports real-time interactive editing of hundreds of complex polygons. Currently, noticeable lag occurs when the number of labeled objects reaches 100, fully demonstrating the powerful performance advantages of this application.

[0043] In the design of the annotation drawing layer, this application adopts the WebGL renderer of PIXI.js. As a GPU-based rendering technology, WebGL can fully utilize the parallel computing potential of the GPU. Compared with traditional Canvas 2D or SVG rendering methods, WebGL demonstrates superior performance advantages when dealing with processing tasks involving thousands of vertices, maintaining a stable and smooth frame rate of 60 FPS, providing a solid guarantee for the efficient operation of annotation drawing.

[0044] Furthermore, this application integrates annotation drawing functionality while fully preserving the professional display capabilities of DICOM medical images, such as window width and level adjustment, pseudo-color mapping, and DICOM metadata parsing. The implementation of these capabilities does not destroy the original pixel data and metadata of the DICOM image to achieve annotation functionality. Moreover, DICOM image display and annotation drawing can be performed using different rendering engines while ensuring that DICOM professional functions remain unaffected. Furthermore, during window width and level adjustment, the DICOM image is re-rendered, and the coordinates and visibility of the annotation drawing layer must be maintained correctly during rendering.

[0045] Furthermore, during interactive annotation editing, this application ensures efficient interactive polygon annotation editing on DICOM images. This editing can include operations such as drawing, dragging, transforming, and deleting, while maintaining smoothness and accuracy. This process first requires precise mouse picking detection. The detected line of sight can accurately identify the annotation object or control point clicked by the user even when the image is zoomed. During polygon editing, vertex positions need to be updated in real time, and the graphic needs to be re-rendered. This process does not affect the display of the DICOM image. Moreover, this application can maintain efficient interaction even when performing complex user interaction logic such as boundary detection, selection, and undo / redo during dragging. During this process, annotation editing operations are independent of and do not interfere with the zooming and translation operations of the DICOM image.

[0046] Furthermore, this application also solves the image format compatibility and conversion problem by flexibly switching between DICOM professional display mode and standard image display mode to achieve lossless or controlled loss conversion of DICOM images to standard image formats (PNG, JPEG). DICOM images have a complex structure, containing pixel data, metadata, transmission syntax, and other information. Directly parsing DICOM images in the browser, without relying on server conversion, effectively preserves the information in the DICOM image. Additionally, when converting to standard image formats, it is necessary to retain the display effect after window width and level adjustments, and support users in selecting conversion format and quality parameter settings according to their needs.

[0047] Furthermore, to achieve the two-tier architecture of this application, it can efficiently manage resources such as DICOM images, annotation data, and rendering textures in a browser environment, realizing resource management and memory optimization to avoid memory leaks and performance degradation. Moreover, to ensure that this application can run normally on mainstream browsers such as Chrome, Firefox, Safari, and Edge, it also includes WebGL compatibility for different browsers to achieve differentiated management. Additionally, for ultra-large DICOM images and ultra-large-scale annotation data, this application can also ensure system stability through large-scale data processing.

[0048] Alternatively, this application can also employ server-side rendering. That is, the server is responsible for compositing the DICOM image and annotations, generating the final image, and then transmitting it to the browser. Its advantage lies in the simplicity of browser-side implementation; however, its disadvantages are also obvious: poor real-time interactivity and a high load on the server, making it unsuitable for high-frequency annotation editing scenarios. This technical approach is suitable for scenarios involving static image viewing where real-time interactive editing is not required.

[0049] In summary, this application employs a dual-layer, separate rendering architecture, separating the DICOM professional display layer from the WebGL annotation and drawing layer, achieving a fusion of professional medical imaging and interactive annotation. This application utilizes a pixel-level precise viewport synchronization algorithm, accurately converting different coordinate systems through mathematical transformations, with a synchronization error of less than 0.5 pixels, overcoming the technical challenges of dual-layer rendering. This application leverages the PIXI.js WebGL engine to achieve GPU hardware-accelerated annotation rendering, improving performance by 5-10 times and supporting large-scale annotation scenarios. This application fully retains the medical imaging functions of Cornerstone.js, ensuring diagnostic accuracy and standard compliance; it supports seamless conversion between DICOM and standard image formats with automatic annotation data adaptation, broadening its application scenarios. This application achieves cross-platform web deployment based on a browser, eliminating the need for client installation and reducing deployment and maintenance costs. In conclusion, this application comprehensively surpasses existing technologies in terms of technical performance, application value, economic benefits, and social benefits, demonstrating broad prospects and outstanding market value.

[0050] This application relates to the field of medical image processing technology, and in particular to a web browser-based DICOM image parsing, rendering, and interactive annotation technology. This application belongs to the interdisciplinary field of DICOM image parsing and display in Medical Imaging Informatics (MII), hardware-accelerated rendering technology in Web Graphics Library (WebGL) in Computer Graphics (CD), browser-based graphics processing and interaction in Web Frontend Technology (Web FE), and visualization annotation and editing of medical images in Human-Computer Interaction (HCI).

[0051] This application can be applied to scenarios such as medical image diagnostic systems, viewers of medical image archiving and communication systems (PACS), telemedicine consultation platforms, medical image auxiliary diagnostic systems, and training data annotation tools for medical image artificial intelligence (AI).

[0052] According to an embodiment of this application, a method for annotating DICOM images applied to the Web is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed on a computer device via a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here. The computer device can be a mobile terminal, a personal computer, a server, etc.

[0053] Figure 1 This is a flowchart illustrating a DICOM image annotation method applied to the Web, as provided in an embodiment of this application. The execution subject is a computer device, such as… Figure 1 As shown, the process includes the following steps: S101. In response to DICOM image annotation commands, create an image display layer and an annotation drawing layer. The image display layer and the annotation drawing layer are rendering layers in the Web. The annotation drawing layer is located above the image display layer.

[0054] For example, upon receiving a DICOM image annotation instruction, the computer device initiates the creation of two rendering layers in the web: an image display layer and an annotation drawing layer. During creation, the computer device ensures that the annotation drawing layer is located above the image display layer, so that subsequent annotation operations on the DICOM image can clearly display the annotation content without affecting the image's overall presentation.

[0055] In one implementation, a DICOM image annotation command is a user-triggered command that instructs a computer device to perform annotation operations on a DICOM image. Typically, this DICOM image annotation command is used to initiate a DICOM image annotation job.

[0056] In one implementation, the image display layer is a rendering layer in the Web specifically designed to present DICOM images. It is responsible for displaying DICOM image data in a visual form.

[0057] In one implementation, the annotation drawing layer is a rendering layer in the Web used to record the annotation information made by the user on the DICOM image. It is superimposed on the image display layer, making it convenient for the user to directly perform annotation operations on the image.

[0058] In one implementation, DICOM images are an international standard image format used for storing, transmitting, and exchanging medical images and related information. In one implementation, the rendering layer in the Web refers to the hierarchical structure used to render and display specific content in a web development environment. Different rendering layers can carry different functions and information displays.

[0059] In one implementation, the computer device can use the HTML5 Canvas element to create these two rendering layers, and control the annotation drawing layer to be located above the image display layer by setting the z-index property of the Canvas.

[0060] S102. Load the DICOM image in the image display layer and record the annotation information of the DICOM image in the annotation drawing layer.

[0061] For example, after the computer device completes the creation of the image display layer and the annotation drawing layer, it loads the specified DICOM image into the image display layer so that it can be displayed correctly. At the same time, the annotation drawing layer can be configured with corresponding data structures and storage mechanisms to record the annotation information that the user subsequently makes on the DICOM image.

[0062] In one implementation, the annotation information refers to various data generated when a user performs annotation operations on a DICOM image, including the geometric shape of the annotation (such as rectangle, circle, arrow, etc.), the position coordinates of the annotation in the image, the color used for the annotation, and the text description of the annotation.

[0063] In one implementation, the computer device can use a specialized DICOM image parsing library, such as DicomParser (a JavaScript library), to parse the DICOM file, extract the image data, and then display it in the image display layer using Canvas drawing methods or by setting the image data as the src attribute of the img element.

[0064] In another implementation, the DICOM image is converted to a common image format (such as JPEG or PNG), and then the front end displays it on the image display layer using a normal image loading method.

[0065] In one implementation, when using Canvas in the annotation drawing layer, information related to the user's annotation operation, such as mouse click position and movement trajectory, is obtained by listening to mouse events on the Canvas (such as mousedown, mousemove, mouseup), and this information is stored in a custom data structure.

[0066] In another implementation, computer devices can use third-party annotation libraries to implement annotation and store annotation information. For example, the Fabric.js third-party annotation library can be used. This library provides rich annotation functions and data management mechanisms, making it easy to record and manage annotation information.

[0067] In this example, by responding to DICOM image annotation commands, two web rendering layers are created: an image display layer and an annotation drawing layer above it. The image is loaded in the image display layer and the annotation information is recorded in the annotation drawing layer. This achieves the effect of separating and orderly performing DICOM image display and annotation operations.

[0068] In one example, the computer device can also perform data storage, a process that includes: S111. During the loading of the DICOM image, the metadata in the DICOM image is obtained and stored using a first preset format.

[0069] For example, during the loading of a DICOM image, a computer device can acquire metadata from the DICOM image. The computer device can store the metadata using a first preset format.

[0070] In one implementation, when a computer device begins loading a DICOM image, the computer device will launch a special parsing program to read the DICOM image file and extract various metadata information from it.

[0071] In one implementation, the computer device is configured with a first preset format. After acquiring the metadata, the computer device can store it using the first preset format for subsequent use and processing.

[0072] Optionally, the first preset format can be JSON format, a preset data structure, etc.

[0073] In one implementation, DICOM images are an image format under a medical digital imaging and communication standard, widely used in the field of medical imaging, and contain rich content such as patient personal information, examination information, and image parameters.

[0074] In one implementation, metadata refers to data describing various attributes of the DICOM image. Examples include the patient's name, age, examination date, image resolution, and scanning parameters. This information is crucial for medical diagnosis, image management, and research.

[0075] S112. During the process of listening to mouse and touch events in the annotation drawing layer, obtain the annotation information generated by the user based on the mouse and touch events, and store the annotation information using the second preset format.

[0076] For example, when a computer device is listening to mouse and touch events in the annotation drawing layer, it acquires annotation information generated by the computer device based on these events after detecting them. Then, the computer device can store this annotation information using a second preset format.

[0077] In one implementation, the computer device continuously monitors events such as mouse movement, clicks, and touch / swipe events on the annotation drawing layer. When these events are detected, the computer device further checks whether drawing should begin. Once drawing is detected, the computer device can store the annotation information added during the process after the user completes a drawing operation.

[0078] Optionally, the computer device can confirm the end of the current drawing when the user clicks the OK button.

[0079] Optionally, the computer device can automatically store annotation information when the duration of a single drawing by the user reaches the storage period.

[0080] In one implementation, the annotation information may include the shape, position, color, etc. of the drawing.

[0081] In one implementation, mouse and touch events are the basic ways for users to interact with computer devices. Mouse events include movement, clicking, double-clicking, and dragging. Touch events include touching, swiping, and zooming. These events trigger corresponding processing programs on the computer device.

[0082] Optionally, when the computer device determines, based on the event, that the user has started drawing, the computer device may begin recording annotation information.

[0083] In one implementation, the annotation information is the bar lines, graphics, and other annotation information left by the user on the annotation drawing layer through mouse or touch operation using annotation tools.

[0084] In this example, by acquiring and storing metadata when loading a DICOM image, listening to annotation drawing layer events to acquire and store annotation information, and displaying metadata and annotation information, lossless or controllable loss conversion of DICOM images to standard image formats is achieved. During the conversion process, the annotation data does not need to be modified and can be automatically adapted. This achieves the effect of comprehensively managing DICOM image-related data and presenting it intuitively to assist users in viewing and analyzing it.

[0085] This function utilizes pixel data from the current display state during conversion, preserving the optimal display effect of the window width and level adjusted by the doctor. The annotation coordinates are based on a pixel coordinate system, independent of the image format, ensuring precise and consistent annotation positions before and after conversion. It also supports multiple output formats such as Data URL, Blob, File, and ArrayBuffer, flexibly meeting the needs of different application scenarios.

[0086] In one example, the computer device can also display metadata and annotation information.

[0087] For example, after acquiring the metadata, the computer device can display the information in the metadata at the corresponding location in the interactive window. Also, after storing the annotation information, the computer device can render and display the annotation information in the first viewport object.

[0088] In one implementation, metadata is data that describes DICOM image attributes and user-annotated information, such as the patient's personal information and examination information.

[0089] In one implementation, the annotation information is the specific annotation content generated by the user on the annotation drawing layer through mouse or touch operations, such as drawn lines, added text, marked areas, etc.

[0090] In one example, the computer device can also improve the efficiency of the annotation drawing layer and the image display layer by setting parameters for these two layers. These settings include: S121. Set the transparency of the first viewport object on the annotation drawing layer to transparent. Set the transparency of the second viewport object on the image display layer to opaque.

[0091] For example, the computer device adjusts the transparency attribute of viewport objects in the annotation drawing layer and the image display layer, respectively. The computer device can set the transparency of a first viewport object on the annotation drawing layer to transparent. And the computer device can also set the transparency of a second viewport object on the image display layer to opaque.

[0092] In one implementation, the first viewport object on the drawing layer is annotated, and its transparency parameter is modified to make it transparent.

[0093] In one implementation, the annotation drawing layer is a virtual layer in the computer device interface used by the user to perform annotation operations. The user can draw various annotation content, such as lines and text, on this layer.

[0094] In one implementation, the first viewport object is a specific object located on the annotation drawing layer. It can be a container used to hold annotation elements drawn by the user.

[0095] In one implementation, when the first viewport object is set to transparent, the annotation drawing layer will not obscure the content displayed on the image display layer, thus improving the display effect of the image display layer. Simultaneously, the content drawn on the annotation drawing layer can be overlaid on the image display layer, allowing the two viewport objects to achieve the desired visual presentation.

[0096] In one implementation, the second viewport object on the image display layer adjusts its transparency parameter to an opaque state.

[0097] In one implementation, the image display layer is used to display image content, specifically the DICOM image. This opacity setting ensures the DICOM image is clearly displayed.

[0098] In one implementation, transparency is an attribute that describes the degree of transparency of an object, and its value is generally between 0 and 1, where 0 represents complete transparency and 1 represents complete opacity.

[0099] S122. Set the first viewport object on the annotation drawing layer to respond to all mouse and touch events. Set the second viewport object on the image display layer to disable all mouse and touch events.

[0100] For example, the computer device sets event response permissions for viewport objects in the annotation drawing layer and the image display layer. It first locates the first viewport object on the annotation drawing layer, enabling it to receive and process all types of mouse and touch events, such as clicks, movements, and drags. Then, it locates the second viewport object on the image display layer and disables its event response permissions, ensuring that this object does not react to any mouse or touch events, thereby guaranteeing that user actions are accurately applied to the first viewport object in the annotation drawing layer.

[0101] In one implementation, a first viewport object is used to hold user-annotated content, and a second viewport object is used to display the image. Therefore, fixing user operation permissions to the first viewport object makes it easier to store annotation information generated based on user operations in the annotation drawing layer where the first viewport object resides.

[0102] In one implementation, mouse and touch events are the basic ways for users to interact with computer devices. Mouse events include moving, clicking, double-clicking, dragging, etc., while touch events include touching, swiping, zooming, etc.

[0103] In this example, by adjusting the transparency of the first viewport object of the annotation drawing layer and the second viewport object of the image display layer, and setting their mouse and touch event response permissions, the annotations are clearly displayed, allowing users to accurately manipulate the annotations, and the image is displayed stably without interfering with the annotation interaction.

[0104] In one example, the computer device can also ensure content alignment between the annotation drawing layer and the image display layer through viewport synchronization. This process includes the following steps: S131. In response to a user scaling and / or moving operation on the first viewport object of the annotation drawing layer, a viewport synchronization command is generated. Here, the annotation drawing layer is a rendering layer in the Web. The first viewport object is used to display the annotation image corresponding to the DICOM image. The viewport synchronization command is used to synchronize the content areas of the first viewport object of the annotation drawing layer and the second viewport object of the image display layer. The image display layer is a rendering layer in the Web. The second viewport object is used to display the DICOM image. The annotation drawing layer is located above the image display layer.

[0105] For example, the computer device can monitor user actions on objects in the first viewport of the annotation drawing layer in real time. Once it detects that the user has performed operations such as zooming and / or moving on the first viewport object, the computer device can immediately generate viewport synchronization instructions.

[0106] In one implementation, a viewport synchronization command is used to instruct a first viewport object and a second viewport object to maintain synchronization in terms of displayed content. Optionally, the synchronized content may include offset position synchronization and scaling ratio synchronization.

[0107] In one implementation, the annotation drawing layer is a layer used for rendering and displaying the annotated image. This annotation drawing layer sits above the image display layer in the Web.

[0108] In one implementation, the image display layer is another rendering layer in the Web, primarily responsible for displaying DICOM images.

[0109] In one implementation, the first viewport object is a display window in the annotation drawing layer. This display window can show all or part of the annotation image in the annotation drawing layer based on the user's zoom level and dragging operations. The annotation image is the annotation image corresponding to the DICOM image.

[0110] In one implementation, the second viewport object is a display window in the image display layer. This display window can display all or part of the content of the DICOM image in the image display layer based on the user's zoom level and dragging operations. The display content of the second viewport object changes synchronously with the display content of the first viewport object.

[0111] In one implementation, the computer device can listen for user interaction events on the annotation drawing layer using front-end technologies such as JavaScript. For example, these user interaction events could include mouse wheel zooming, dragging, or other similar actions.

[0112] In one implementation, the computer device can generate a viewport synchronization command when it detects a user operation that causes a change in the displayed content of the first viewport object. This user operation can be scaling, moving, etc.

[0113] S132. In response to the viewport synchronization command, obtain the current scaling ratio and current offset coordinates of the content displayed in the first viewport object after the user performs scaling and / or translation operations on the first viewport object on the annotation drawing layer. The current scaling ratio represents the ratio of the content displayed in the first viewport object to the annotation image. The current offset coordinates represent the coordinates of the upper left corner of the content displayed in the first viewport object in the annotation coordinate system of the annotation drawing layer.

[0114] For example, after triggering the viewport synchronization command, the computer device needs to obtain the current parameters of the first viewport object after the user's operation. The current parameters of the first viewport object may include the current zoom level of the content displayed in the first viewport object, and the coordinates of the upper left corner of the content displayed in the first viewport object in the annotation coordinate system.

[0115] In one implementation, the current zoom level represents the ratio of the displayed content to the labeled image in the first viewport object. Specifically, the current zoom level can represent the ratio of the side length of the displayed content to the side length of the labeled image in the first viewport object.

[0116] For example, at a zoom level of 1, the first viewport object displays the entire labeled image. The ratio of the side length of the content displayed in the first viewport object to the zoom level of the labeled image is 1:1. Similarly, at a zoom level of 2, since the size of the first viewport object remains unchanged, but the labeled image is magnified by a factor of 2, the ratio of the side length of the content displayed in the first viewport object to the zoom level of the labeled image is 1:2. And so on.

[0117] In one implementation, the current offset coordinate represents the specific coordinates of the top-left corner of the content displayed in the first viewport object in the annotation coordinate system of the annotation drawing layer.

[0118] In one implementation, the computer device monitors user behavior in real time during user operation and adjusts the current parameters of the displayed content of the first viewport object based on the user behavior.

[0119] Optionally, the computer device can directly translate user actions into adjustments to current parameters. These user actions can be performed by the user controlling a mouse, or they can be performed by the user directly touching the screen.

[0120] For example, computer devices can determine the direction and magnitude of scaling adjustments directly based on the direction and amplitude of the scroll wheel's rotation when the user scrolls.

[0121] For example, a computer device can determine the real-time offset direction and distance directly based on the direction and distance of the mouse dragging when the user drags the mouse. Then, based on this real-time offset direction and distance, and the coordinates of the top-left corner of the displayed content of the first viewport object before the movement, the computer device can determine the current offset coordinates corresponding to the top-left corner of the displayed content of the first viewport object after the movement.

[0122] S133. Convert the current scaling ratio and current offset coordinates in the labeled coordinate system into the target scaling ratio and target offset of the second viewport object in the image coordinate system of the image display layer. The target scaling ratio represents the ratio of the displayed content in the second viewport object to the DICOM image. The target offset represents the coordinates of the center point of the displayed content in the second viewport object in the image coordinate system of the image display layer.

[0123] For example, the computer device needs to convert the current scaling ratio and current offset coordinates in the annotation drawing layer to the image coordinate system of the image display layer to obtain the target scaling ratio and target offset of the displayed content in the image coordinate system in the second viewport object of the image display layer.

[0124] In one implementation, the annotation coordinate system is the coordinate system used by the annotation drawing layer. The starting point of this annotation coordinate system is usually the top-left corner of the annotation image.

[0125] In one implementation, the image coordinate system is the coordinate system used by the image display layer. The origin of this image coordinate system is typically the center point of the DICOM image.

[0126] In one implementation, the target scaling ratio is the proportion of the DICOM image that the content displayed in the second viewport object should occupy in the image coordinate system. For example, when the second viewport object displays the entire DICOM image, the target scaling ratio is 1:1. Or, when the second viewport object displays 1 / 4 of the DICOM image, the target scaling ratio is 1:2.

[0127] In one implementation, the target offset is the offset of the center point of the displayed content in the second viewport object relative to the origin of the image coordinate system in the image coordinate system.

[0128] In one implementation, the computer device can use known coordinate system transformation relationships, such as matrix transformations, to convert coordinates in the labeled coordinate system into corresponding values ​​in the image coordinate system.

[0129] In another implementation, the computer device can use a preset coordinate transformation formula to convert the coordinates in the labeled coordinate system into the corresponding values ​​in the image coordinate system.

[0130] In one implementation, since the labeled image and the DICOM image need to completely overlap, the two images should have the same scaling ratio. Therefore, this target scaling ratio is the current scaling ratio.

[0131] S134. Render and display the DICOM image in the second viewport object of the image display layer according to the target scaling ratio and target offset.

[0132] For example, the computer device re-renders and displays the DICOM image in a second viewport object of the image display layer according to the converted target scaling ratio and target offset, so that the user sees content synchronized with the first viewport object in the second viewport object.

[0133] In one implementation, rendering refers to the process by which a computer device generates and displays an image on a screen based on given parameters and image data.

[0134] In one implementation, the computer device can determine the content to be displayed in the second viewport object based on the converted target scaling ratio and target offset. Then, the computer device can display this content in the second viewport object through rendering.

[0135] In this embodiment, by responding to the user's scaling and / or moving operation on the first viewport object of the annotation drawing layer and generating a viewport synchronization command, the scaling ratio and offset coordinates after the operation are obtained and converted into corresponding parameters of the image display layer. Finally, the DICOM image is rendered and displayed in the second viewport object of the image display layer, thereby achieving the effect of synchronous display of the annotation drawing layer and the viewport content area of ​​the image display layer.

[0136] In one example, the first viewport object also includes a viewport container. The viewport container includes an upper auxiliary annotation layer and a lower annotation object layer. The auxiliary annotation layer contains auxiliary meshes or auxiliary points.

[0137] For example, when a computer device constructs a first viewport object, it simultaneously creates a viewport container. This viewport container serves as an important internal structure within the first viewport object, used to hold multiple layers of image information from the labeled image. This multi-layered image information arrangement facilitates the hierarchical arrangement of various elements, thereby enabling easier visualization and interactive implementation.

[0138] First, during the annotation process, a layer needs to be set up for drawing the annotation objects. This layer is the annotation object layer in this example. Second, during the annotation process, it is usually necessary to set some auxiliary points, auxiliary grids, rulers, and other information to assist the user in drawing the annotation objects. Therefore, computer devices can also be equipped with auxiliary annotation layers.

[0139] In one implementation, the viewport container is a virtual storage space concept. In the field of computer graphics processing and visualization, the viewport container is used to organize and manage various graphical elements displayed in the viewport, ensuring that these elements are presented according to predetermined rules and layouts.

[0140] In one implementation, this embodiment does not limit the implementation method of the viewport container. The viewport container created using built-in functions provided by programming languages, graphics libraries, or frameworks is within the scope of this application.

[0141] In one implementation, the auxiliary annotation layer is located above the annotation object layer in the viewport container.

[0142] In one implementation, the hierarchical structure is designed to better organize and manage different types of annotation elements, so that they can be displayed and interacted with without interfering with each other, and can be rendered and displayed in a specific order.

[0143] In one implementation, the annotation object layer is used to place the main annotation objects. These annotation objects are typically shapes drawn by the user using tools based on the DICOM image.

[0144] In one implementation, the auxiliary annotation layer is a layer within the viewport container used to place auxiliary annotation elements. It mainly serves to assist in display and interaction, helping users to understand the content in the viewport more clearly.

[0145] Optionally, the auxiliary annotation layer can be one or more layers. For example, one auxiliary annotation layer can be a ruler, another can be a grid, and yet another can be halftone dots. Users can select one or more layers to display as needed to assist in annotation.

[0146] In one implementation, when constructing the auxiliary annotation layer, the computer device adds auxiliary elements such as auxiliary grids or auxiliary points to the auxiliary annotation layer according to specific needs and scenarios. The presence of these auxiliary elements can help users more accurately locate and measure the annotated objects, improving the accuracy and efficiency of the operation.

[0147] In one implementation, the auxiliary mesh is a grid structure composed of a series of horizontal and vertical lines. It can be overlaid on the display window of the first viewport object to provide the user with a reference coordinate system, making it convenient for the user to perform alignment, measurement and other operations.

[0148] In one implementation, the auxiliary points are a point-like grid. These auxiliary points help the user draw the boundaries of labeled objects based on the DICOM image.

[0149] This example demonstrates a two-layer, separate rendering architecture achieved by constructing a viewport container within the first viewport object, containing an upper auxiliary annotation layer and a lower annotation object layer. This represents the first time in a web browser environment that a perfect fusion of professional-grade DICOM medical image display and high-performance interactive annotation editing functions has been achieved, assisting users in drawing annotation objects and improving annotation efficiency. The displayed content can include window width and level adjustment, pseudo-color mapping, and DICOM metadata parsing.

[0150] Furthermore, compared to the current situation where Cornerstone.js only supports DICOM display and has weak annotation capabilities, and general drawing libraries (Fabric.js, Konva.js) do not support DICOM, there is a problem that the two are difficult to integrate effectively. In this application, the image display layer (Cornerstone.js) focuses on professional medical image processing, while the annotation and drawing layer (PIXI.js) is responsible for high-performance graphics drawing. The two layers work together with a precise viewport synchronization algorithm, which not only retains the professionalism of DICOM, but also brings a smooth annotation experience.

[0151] In addition, in terms of specific quantitative indicators, the DICOM window width and window level adjustment response time is less than 100ms, achieving instant response; it supports 500+ complex polygon annotations and the rendering frame rate is stable at 60 FPS; the response latency for annotation editing operations (drag and transform) is less than 16ms (single frame time).

[0152] Furthermore, in this example, the native Cornerstone.js engine is used to display DICOM images, which contrasts sharply with the server conversion solution (JPEG / PNG) in terms of functionality, and fully preserves all professional medical imaging functions.

[0153] Regarding pixel precision, the server conversion scheme can only retain 8 bits, resulting in precision loss, while this application can retain 12-16 bits completely. For window width and level adjustment, the server conversion scheme does not support it, but this application can achieve real-time adjustment. For pseudo-color mapping, the server conversion scheme also does not support it, while this application supports multiple mapping methods. Regarding DICOM metadata, the server conversion scheme leads to data loss, while this application can retain it completely. For HU value display, the server conversion scheme does not support it, but this application does.

[0154] From a medical perspective, this application preserves the original pixel precision, allowing doctors to finely observe different tissues by adjusting window width and level, avoiding the loss of key diagnostic information due to image conversion and ensuring diagnostic accuracy. It also complies with DICOM standards and medical image storage specifications, demonstrating standard compliance. Furthermore, the original DICOM data remains unmodified, meeting legal requirements for medical image archiving and thus possessing legal compliance.

[0155] Furthermore, this feature proves invaluable in practical applications. In remote consultation scenarios, DICOM images can be converted to JPEG for compressed transmission, effectively saving bandwidth. During report generation, annotated DICOM images can be exported as PNGs and inserted into medical reports. In the AI ​​training data preparation phase, DICOM images can be batch-converted to standard formats, facilitating deep learning model training.

[0156] In one example, step S133 above, converting the current scaling ratio and current offset coordinates in the annotation coordinate system into the target scaling ratio and target offset of the second viewport object in the image coordinate system of the image display layer, includes: S1331. In the labeled coordinate system, using the coordinates of the canvas center point in the web interface as a reference, determine the actual offset of the current offset coordinate. The actual offset represents the offset of the current offset coordinate relative to the canvas center point coordinate.

[0157] For example, the computer device first obtains the coordinates of the center point of the web interface canvas, which is the fixed reference position of the canvas in the annotation coordinate system. Next, it obtains the current offset coordinates, which are the coordinates of the top-left corner of the content displayed in the first viewport object within the annotation coordinate system of the annotation drawing layer. Then, by calculating the difference between the current offset coordinates and the canvas center point coordinates in the horizontal and vertical directions, the actual offset is obtained. This actual offset accurately represents the degree of deviation of the current offset coordinates relative to the canvas center point coordinates.

[0158] In one implementation, the annotation coordinate system is established with the top left corner of the annotation image in the annotation drawing layer as the origin.

[0159] In one implementation, the web interface is a webpage accessed by the user through a browser, and the canvas is the underlying base area within the web interface used to display and render images. Both the annotation drawing layer and the image display layer are rendered and displayed on the canvas.

[0160] Optionally, the size of both the drawn image and the DICOM image is the same as the size of the canvas.

[0161] In one implementation, the coordinates of the canvas center point are the coordinates of the canvas's center point. In this step, the computer device can determine the center position of the canvas based on the annotation coordinate system of the annotation drawing layer. The calculation of this center position is used to achieve subsequent coordinate transformations.

[0162] In one implementation, the current offset coordinate is the position of the top-left corner of the content displayed in the first viewport object within the annotation coordinate system during the annotation process.

[0163] In one implementation, the actual offset is the offset of the current offset coordinate relative to the center point coordinate of the canvas, which reflects the degree of deviation of the annotation element from the center of the canvas.

[0164] S1332. Divide the actual offset by the current scaling factor to obtain the target offset.

[0165] For example, the computer device may obtain the actual offset calculated in step S1031 above. The computer device can divide the actual offset by the current scaling ratio to obtain the target offset, which is the offset that the annotation element should have after taking into account the current scaling.

[0166] In one implementation, the actual offset is the offset of the current offset coordinates relative to the center point coordinates of the canvas, calculated in the previous steps.

[0167] In one implementation, the current scaling ratio reflects the ratio of the current display size of the canvas to its original size. The current scaling ratio is the degree of scaling of the canvas in its current display state on the web interface. For example, if the original canvas size is 100x100 and the current display size is 50x50, then the current scaling ratio is 1:2.

[0168] In one implementation, the target offset is the offset that the annotation element should have relative to the center point of the canvas after the scaling ratio is adjusted. It can ensure that the positional relationship of the annotation elements remains relatively accurate under different scaling ratios.

[0169] S1333. Determine the target scaling ratio based on the current scaling ratio.

[0170] For example, the computer device first obtains the current zoom level of the current web interface canvas and uses the current zoom level as the target zoom level.

[0171] In this example, by determining the actual offset in the annotation coordinate system with the center point of the canvas as a reference, calculating the target offset in combination with the current scaling ratio, and determining the target scaling ratio, the annotation elements are accurately positioned and displayed under different scaling states.

[0172] In one example, in step S1331 above, in the annotation coordinate system, using the coordinates of the canvas center point in the web interface as a reference, the actual offset of the current offset coordinate is determined, including: S13311. In the labeled coordinate system, determine the coordinates of the center point of the canvas in the Web interface.

[0173] For example, the computer device first obtains the size information of the web interface canvas, including the width and height of the canvas. Then, using this size information, through simple mathematical calculations, namely dividing the width and height of the canvas by 2 respectively, the horizontal and vertical coordinates of the canvas center point in the labeled coordinate system are obtained. These coordinate values ​​are the coordinates of the canvas center point.

[0174] In one implementation, the origin of the coordinate system is the top-left corner of the drawing board.

[0175] In one implementation, the coordinates of the canvas center point are the coordinate values ​​corresponding to the center position of the canvas in the annotation coordinate system. It is an important reference point for subsequent calculation of the position of annotation elements.

[0176] S13312. Calculate the theoretical starting coordinates of the upper left corner of the labeled image in the label coordinate system when the center position of the labeled image is placed at the center point coordinate of the drawing board.

[0177] For example, the computer device first obtains the size information of the labeled image, including the width and height of the labeled image. Then, given the coordinates of the center point of the canvas, by reverse derivation, half the width and half the height of the labeled image are subtracted from the coordinates of the center point of the canvas, respectively, to obtain the theoretical starting coordinates of the upper left corner of the labeled image in the labeled coordinate system.

[0178] In one implementation, the annotation image is an image element used when performing annotation operations on the web interface canvas. It can be a user-uploaded image, a drawn graphic, etc. The size information of the annotation image includes its width and height, which determines the size of the annotation image in the annotation coordinate system.

[0179] In one implementation, the theoretical starting coordinates are the coordinates of the upper left corner of the labeled image in the labeled coordinate system, assuming that the center position of the labeled image coincides with the center point of the drawing board. These coordinates are the key data for calculating the actual offset.

[0180] S13313. Calculate the difference between the current offset coordinate and the theoretical starting coordinate to obtain the actual offset.

[0181] For example, the computer device first obtains the current offset coordinates and the previously calculated theoretical starting coordinates. Then, it calculates the differences between the horizontal and vertical coordinates of the current offset coordinates and the horizontal and vertical coordinates of the theoretical starting coordinates. The combination of these two differences constitutes the actual offset, which represents the degree of offset of the current labeled image position relative to the theoretical starting position.

[0182] In one implementation, the current offset coordinate is the position of the top-left corner of the displayed content of the first viewport object in the annotation coordinate system during the annotation process.

[0183] In one implementation, the theoretical starting coordinates are the theoretical coordinates of the top-left corner of the labeled image in the labeled coordinate system, calculated in the previous steps.

[0184] In one implementation, the actual offset is the difference between the current offset coordinate and the theoretical starting coordinate in the coordinate direction, which reflects the deviation of the labeled image from the actual position and the theoretical position.

[0185] In this example, by sequentially determining the coordinates of the canvas center point in the annotation coordinate system, calculating the theoretical starting coordinates when the annotation image is placed at the center of the canvas, and obtaining the actual offset, the effect of accurately locating the offset of the annotation image relative to the center of the canvas is achieved.

[0186] In one example, the computer device can also convert DICOM images to standard images, thereby reducing data volume and improving display efficiency. This process includes: S141. Obtain the color information of each pixel from the DICOM image of the image display layer, and copy the color information to the created temporary canvas.

[0187] For example, a computer device can create a temporary canvas in response to a format conversion instruction. The computer device can sequentially acquire color information for each pixel from the DICOM image of a second viewport object. The computer device can copy this color information to the corresponding position on the temporary canvas. Based on this copying of color information, the computer device can perform the operation of copying the color information of the DICOM image onto the temporary canvas.

[0188] In one implementation, the temporary canvas can be an in-memory image buffer. It can be a two-dimensional array or a specific image data structure used to store pixel color information extracted from DICOM image elements.

[0189] In one implementation, a DICOM image is an image displayed in the image display layer. DICOM is an image format under a medical digital imaging and communication standard, containing patient information, examination information, and image pixel data. A pixel is the basic building block of an image; each pixel has its specific location and color information, which is usually represented by RGB (red, green, blue) values. These values ​​determine the color displayed by the pixel on the screen.

[0190] S142. Generate standard image elements for the image display layer using a temporary canvas, based on a preset standard image format.

[0191] For example, the computer device first determines a preset standard image format. This standard image format could be JPEG, PNG, etc. Then, according to the requirements of this format, the pixel color information in the temporary canvas is processed and converted accordingly. For example, color space conversion, compression, and other operations are performed. Next, a specific encoding algorithm is used to encode the processed pixel data according to the standard image format, generating image data that conforms to that format.

[0192] In one implementation, the preset standard image format is a universal image storage and transmission format. For example, JPEG is a lossy compression format that can reduce the size of image files while maintaining image quality. PNG is a lossless compression format that can preserve the original quality of the image.

[0193] In one implementation, the temporary canvas is a memory area that stores pixel color information extracted from DICOM image elements.

[0194] In one implementation, the standard image is the portion of the image display layer used to store and present image data that conforms to the standard image format.

[0195] S143. In the image display layer, after replacing the DICOM image element with a standard image element, render and display the second viewport object.

[0196] For example, the computer device first locates the DICOM image in the image display layer and removes it from the viewport object. Then, the computer device adds a generated standard image element to the image display layer, replacing the original DICOM image. Next, the computer device invokes the graphics rendering engine to render the standard image according to the properties and display settings of the second viewport object. For example, scaling and offsetting are performed. Finally, the rendered second viewport object is displayed on the screen, allowing the user to see the final image effect.

[0197] In one implementation, the second viewport object is a display area located in the image display layer, used to display image content.

[0198] In one implementation, rendering refers to the process of converting image data into pixel data that can be displayed on the screen. It includes various image processing operations, such as coordinate transformation.

[0199] In this example, by extracting the pixel color information of the DICOM image elements of the second viewport object to a temporary canvas, generating standard image elements according to the standard format, replacing the original DICOM image elements, and then rendering and displaying them, the effect of converting medical DICOM images into general standard format images and presenting them clearly is achieved.

[0200] Figure 2 This is a schematic diagram of a MOD hierarchical architecture provided in an embodiment of this application. Figure 1 Based on the illustrated embodiments, as Figure 2 As shown, with computer devices as the execution subject, this MOD hierarchy architecture includes: The computer device can first set up a PIXI.Application.stage root container in the Web. Then, based on this PIXI.Application.stage root container, the computer device can set up a Viewport.container viewport container. Based on this Viewport.container viewport container, two viewport objects can be set up. The first viewport object corresponds to the contentContainer content layer. This content layer is set up within the annotation content layer. The second viewport object corresponds to the backgroundLayer background layer. This background layer is set up within the image display layer. The contentContainer content layer can also include multiple layers. These multiple layers can be denoted as PolygonRenderer 1, PolygonRenderer 2, PolygonRenderer 3, etc. One layer can be used to store annotation information. Another layer can be used to display annotation auxiliary information.

[0201] Figure 3 This application provides a system architecture diagram for a DICOM image annotation method applied to the Web, as shown in the embodiments of this application. Figure 1 and Figure 2 Based on the illustrated embodiments, as Figure 3 As shown, with computer devices as the execution subject, the system includes: The first layer is the image display layer. This layer is located at the bottom layer. The technology stack used in this layer is a combination of Cornerstone.js, Cornerstone WADO Image Loader, and DICOM Parser. The rendering method for this layer is Canvas 2D API, which is CPU rendering. This layer can implement functions such as DICOM image parsing, professional display of medical images, and window width and level adjustment. This layer is implemented in the DOM through HTML. The element is mounted within its parent container and has its z-index set to 1. Furthermore, the size of this image display layer is identical to the original size of the DICOM image. This image display layer can be configured to disable mouse events by setting `pointer-events: none`, thus preventing interference with the upper-level annotation and interaction layers.

[0202] The second layer is the annotation drawing layer. This layer is located above the first layer. It uses the PIXI.js 8.x technology stack and is rendered via the WebGL API (GPU hardware acceleration). This layer enables polygon drawing, interactive editing, and viewport transformation. In the DOM, it is positioned within the Canvas element of PIXI.Application. Its position can be adjusted by setting its z-index to 999. Furthermore, setting `backgroundAlpha: 0` makes it transparent, allowing the underlying DICOM image to be visible. This layer also responds to all mouse and touch events, enabling web interaction with the user.

[0203] The third layer is the viewport management layer. The core component of this layer is the Viewport class. This layer enables unified management of zoom and pan states. It also handles user interaction events, calculates viewport transformation parameters, and synchronously notifies both rendering layers. These functions are achieved through coordinate system transformation and synchronous zoom algorithms.

[0204] The fourth layer is the data management layer. This layer can use the dicomParser library to parse the binary data of DICOM images. It can also use the Cornerstone WADO Image Loader to convert DICOM pixel data into Cornerstone-recognizable image objects. Furthermore, this layer can store polygon vertex coordinates, styles, metadata, and other information in JSON format.

[0205] In this embodiment, cross-platform deployment is achieved based on Web technology. It only requires a browser to run and does not require the installation of dedicated client software, supporting multiple operating systems such as Windows, macOS, and Linux. Compared to traditional desktop DICOMViewers, which require different versions to be developed for different operating systems and users to download and install them, this unified Web application can be accessed and used immediately through a browser, greatly reducing the user's installation cost.

[0206] Furthermore, in terms of application value, in telemedicine, doctors can access imaging systems from any location using a browser without carrying specialized equipment. In medical education, students can directly access teaching images on their own computers without installing software, and viewing is supported on tablets and smartphones (touchscreen compatibility required).

[0207] Furthermore, this application can significantly reduce development and maintenance costs by reusing mature open-source libraries (Cornerstone.js, PIXI.js). Based on typical project estimates, the development cycle can be shortened by more than 60%, and maintenance costs reduced by more than 50%. For development tasks such as DICOM parsing engines, window width and window level functions, and WebGL rendering engines, this solution can save 100% of the cost. Development tasks such as annotation interaction logic and viewport synchronization algorithms can also save 50% - 67% of the cost, resulting in a total saving of 87%.

[0208] In one example, when handling frequently triggered interactive events, such as mouse movement and wheel zoom, the system can also employ a throttle mechanism. This mechanism effectively avoids excessive DOM manipulation or complex calculations in each frame by reasonably limiting the execution frequency of the synchronization algorithm, for example, setting it to execute once every 16ms to match the 60Hz refresh rate. This significantly reduces CPU usage and ensures the smoothness and stability of system operation.

[0209] In one example, based on the above system architecture, the HTML structure settings for the two rendering layers in this two-layer DOM structure within the browser's DOM tree can be as follows: Figure 4 As shown.

[0210] The parent container (#pixiContainer) can achieve relative positioning by setting its position to relative, and this relative position will serve as a positioning reference between the two layers. The size of this relative position can be dynamically set by the application logic.

[0211] The DICOM element (#dicomElement) can be absolutely positioned to align with its parent container at the top left corner. The size of the DICOM element is identical to the original size of the DICOM image. For example, this size could be 512×512. The z-index can be set to 1 to ensure the DICOM element is in a lower layer. pointer-events can be set to none to disable mouse events in the layer containing the DICOM element, allowing mouse operations to pass through to the upper-layer PIXICanvas3.

[0212] The PIXI Canvas can be absolutely positioned to align with its parent container at the top left corner. The PIXI Canvas's size is identical to the parent container's; for example, it could be 871×641. The parent container's size may be larger or smaller than the DICOM image. The PIXI Canvas's z-index can be set to 999, ensuring it is visible on the upper layer. Furthermore, by setting the PIXI Canvas's background to transparent, the underlying DICOM image becomes visible.

[0213] In one example, the core issue of this application is how to maintain precise synchronization during scaling and translation when the image display layer and the annotation drawing layer use different coordinate systems. Based on this core issue, this application achieves this functionality through a viewport synchronization algorithm.

[0214] First, this application defines the DICOM image coordinate system (Cornerstone Viewport) and the annotation drawing layer coordinate system (PIXI.jsViewport).

[0215] In this DICOM image coordinate system, the origin is the image center, the positive X-axis points to the right, and the positive Y-axis points downwards. The unit is pixels or millimeters, depending on the DICOM label. In this DICOM image coordinate system, `scale` represents the scaling ratio. A scaling ratio of 1.0 represents the original size. `translation.x` represents the horizontal offset, and `translation.y` represents the vertical offset. Both horizontal and vertical offsets are relative to the image center.

[0216] The origin of the drawing layer coordinate system is located at the top left corner of the canvas. The positive X-axis points to the right, and the positive Y-axis points downwards. The unit is pixels. Zoom is the scaling factor. A scaling factor of 1 represents the original size. pan.x represents the horizontal offset, and pan.y represents the vertical offset. Both horizontal and vertical offsets are set relative to the top left corner of the canvas.

[0217] Secondly, the viewport synchronization algorithm in this application is specifically implemented through a scaling synchronization algorithm and a translation synchronization algorithm.

[0218] The scaling synchronization algorithm aims to synchronously update the DICOM image layer to the same visual scaling effect when the annotation rendering layer is scaled to pixZoom. This scaling process can be calculated using the function syncDicomToPixi. The input parameter of the function syncDicomToPixi can be the current scaling ratio. Its algorithm steps may specifically include: S201. Obtain the current state. The current state includes the current zoom level. For example, the computer device can obtain the current zoom level (pixiZoom) and current offset coordinates (pixiPan) of the annotation drawing layer (PIXI Viewport). Simultaneously, the computer device can obtain the second viewport information and the original image dimensions (imageWidth, imageHeight) of the image display layer (Cornerstone). Furthermore, the computer device can obtain the translation position, canvas size, and image size of the second viewport object.

[0219] S202, Calculate the center point of the canvas and image.

[0220] For example, a computer device can calculate the center point coordinates (canvasCenterX, canvasCenterY) of the canvas based on the canvas size.

[0221] S203. Calculate the PIXI coordinates of the image at the center of the canvas.

[0222] For example, a computer device can calculate the theoretical position of the top left corner of the labeled image in the label coordinate system when the labeled image is displayed fully centered.

[0223] S204. Calculate the actual offset.

[0224] For example, the computer device can calculate the difference between the current PIXI translation position and the theoretical centering position, i.e., the actual offset (offsetX, offsetY). The actual offset is equal to the current translation position minus the theoretical centering coordinates.

[0225] S205, Translation amount converted to DICOM coordinate system.

[0226] For example, a computer device can divide the actual offset by the current scaling factor to convert it into target offset coordinates (dicomTranslationX, dicomTranslationY) in the DICOM coordinate system. The target offset coordinates of the DICOM image are equal to the actual offset divided by the current scaling factor.

[0227] S206, Apply to the second viewport object.

[0228] For example, the computer device can construct a new second viewport object and apply the calculated target scaling and target offset coordinates to the Cornerstone engine to complete the synchronization. The parameters of the second viewport object can be output as (target scaling, target offset coordinates).

[0229] In this example, the scaling synchronization algorithm ensures a synchronization error of less than 0.5 pixels through mathematical transformations, which is imperceptible to the naked eye. Furthermore, the algorithm has low computational complexity, can be executed in every frame, supports real-time synchronization, and offers good real-time performance. Moreover, the algorithm is applicable to any combination of canvas size, image size, and scaling ratio, demonstrating excellent versatility.

[0230] The translation synchronization algorithm can use the same algorithm as the zoom synchronization algorithm. When the user drags the annotation drawing layer (PIXI Viewport), the pan value of the annotation drawing layer (PIXI Viewport) changes, triggering the viewport:pan event. Subsequently, the algorithm can recalculate the DICOM translation based on the new pan value by calling the syncDicomToPixi function and inputting the currentZoom parameter. Finally, based on the new DICOM translation, the second viewport object is updated.

[0231] In this example, a viewport synchronization algorithm, employing mathematical transformations, precisely calculates the mapping relationship between the image coordinate system and the labeled coordinate system. This achieves pixel-level synchronization between the two layers during scaling and translation operations, with a synchronization error of less than 0.5 pixels, virtually imperceptible to the naked eye. This viewport synchronization algorithm fully considers factors such as the difference between canvas and image sizes, the difference between the origins of the two coordinate systems (top left corner of the canvas and center of the image), the consistency of the scaling center point, and the conversion of offset units (pixels versus unscaled pixels). Quantitative indicators show that at any scaling ratio and translation position, the synchronization error is less than 0.5 pixels, the synchronization latency is less than 16ms, and smooth interaction at 60 FPS is supported.

[0232] Furthermore, unlike the current method of using simple scaling, which cannot effectively handle the transformation between different origins and different unit coordinate systems, resulting in obvious misalignment and drift between the two layers during scaling or translation, this application calculates intermediate parameters such as the canvas center point, the image center position, and the actual offset to accurately convert the coordinates in the annotation coordinate system to the coordinates in the image coordinate system, ensuring perfect visual alignment.

[0233] In one example, to improve system performance and resource utilization efficiency, this application may also introduce a "dirty rectangle" detection mechanism. This mechanism has intelligent judgment capabilities, triggering a redraw operation only when the viewport changes or the annotation data is updated, thus avoiding unnecessary rendering overhead. Simultaneously, for static display scenarios, the system can automatically pause the rendering loop, effectively reducing power consumption and system resource usage, thereby optimizing system performance and efficiently utilizing resources while ensuring display quality.

[0234] In one example, this application employs two key technologies for system performance optimization. The first is object pooling, which uses an object pool for unified management of polygonal objects and control point graphics. By reusing destroyed objects, it effectively reduces garbage collection (GC) operations caused by frequent object creation and destruction, thereby mitigating lag issues caused by GC and improving system smoothness. The second is texture cleanup technology, which allows the system to monitor texture resource usage in real time and promptly release unused texture resources, avoiding unnecessary memory occupation and preventing memory leaks, thus ensuring the stability and efficiency of the system in graphics processing.

[0235] In one example, such as Figure 5 As shown, the image display layer and annotation drawing layer in this application can adopt an event-driven synchronization mechanism. First, the computer device can subscribe to the viewport's zoom and pan events and establish a connection when the DicomImagePlugin is initialized.

[0236] Subsequently, the computer device can capture events through the PIXIViewport and update the internal PIXI state (zoom / pan) when the user interacts with it by performing actions such as mouse wheel scrolling / drag, triggering custom events `viewport:zoom` or `viewport:pan`. The computer device can listen for these events using the DicomImagePlugin. When the computer device receives a `zoom` event, it can obtain the new current zoom level by calling the `syncDicomToPixi` algorithm. The computer device can then call the synchronization algorithm to update the second viewport object (DICOM Viewport) to complete the zooming process and achieve synchronization between the two layers of vision. Furthermore, when the computer device receives a `pan` event, it can obtain the current offset coordinates, call the synchronization algorithm to update the second viewport object, and complete the translation process.

[0237] Figure 6 A flowchart illustrating a DICOM image annotation method applied to the Web, provided as an embodiment of this application, is shown below. Figures 1 to 5 Based on the illustrated embodiments, as Figure 6 As shown, with a computer device as the execution subject, the process includes: S301. The user selects a DICOM file (File object) or URL.

[0238] For example, after a user selects a DICOM file (File object or URL), the computer device can parse the DICOM file based on the user-selected DICOM file (File object or URL).

[0239] S302. Detect file type by file extension or magic bytes.

[0240] For example, during the DICOM image parsing process, the computer device can first use Electron's fs module to read the local DICOM image, obtain the file extension or magic bytes, and get the buffer data.

[0241] S303. Create WADO Image Loader ID.

[0242] For example, a computer device can generate a unique ImageId for the file, thus achieving ID generation. This ImageId typically uses the wadouri:xxx protocol format.

[0243] S304, Call cornerstone.loadImage.

[0244] For example, a computer device can register a file buffer with Cornerstone's WADOImageLoader, establishing a mapping between ImageId and file data, thus registering the loader. Subsequently, the computer device can call the cornerstone.loadImage(imageId) method, and Cornerstone will find the corresponding buffer based on the ImageId and parse it to load the image.

[0245] S305 and WADO Image Loader read binary data from files.

[0246] For example, after parsing, the computer device can obtain the image object and extract its metadata. This image object may include metadata such as pixel data, number of rows, number of columns, and window width / window center.

[0247] S306, dicomParser, parses DICOM datasets.

[0248] For example, a computer device can use the dicomParser tool to parse the DICOM dataset obtained in the above steps.

[0249] S307, Extract pixel data and metadata: Pixel Data, Patient Name, Study Date.

[0250] For example, a computer device can extract pixel data and metadata such as PixelData, Patient Name, and Study Date based on the parsing results.

[0251] S308. Construct a Cornerstone Image object.

[0252] For example, a computer device can create a div element in the DOM as a container for a DICOM image. The computer device can then activate the container by calling cornerstone.enable(element), enabling it to render DICOM images, thus enabling element activation.

[0253] S309. Returns the image object for Cornerstone to display.

[0254] For example, a computer device can call `cornerstone.displayImage(element, image)` to render the parsed image object into a container, thus displaying the image. The computer device can obtain default viewport parameters, adjust them as needed (such as setting an initial scaling ratio), and then apply them to the container to initialize the viewport settings.

[0255] In this embodiment, by starting from the user's selection of a DICOM file, through a series of means such as file type detection, creation of a loader ID, calling the loading method, parsing the dataset, extracting key data, constructing an image object, and finally displaying the image and supporting window width and window level adjustment, the DICOM image can be accurately parsed, flexibly displayed, and have personalized parameter control on the web page.

[0256] In one implementation, window width and window center are key parameters for DICOM image display, used to control the image's contrast and brightness.

[0257] After obtaining the viewport object of the current DICOM element, the computer device can modify the voi.windowWidth and voi.windowCenter properties of the viewport object based on user input or preset values. Then, the computer device can reapply the modified viewport object to the DICOM element, and Cornerstone will automatically re-render the image.

[0258] Among them, computer equipment can support the rapid application of preset window widths and window levels (such as lung window, bone window, diaphragm window, etc.), simply by assigning the preset values ​​to the viewport object.

[0259] For example, preset window widths and levels can include: Lung CT: window width 1500, window level -600 (highlighting lung tissue). Bone CT: window width 2000, window level 300 (highlighting bones). Soft tissue CT: window width 400, window level 40 (highlighting soft tissue).

[0260] In one example, the annotation drawing process in the annotation drawing layer may include the following steps: S401, PIXI.js application initialization.

[0261] For example, during the initialization process, the computer device can first create a PIXI.Application instance. This PIXI.Application instance can serve as the core of the WebGL rendering engine.

[0262] Secondly, the computer device can be configured with parameters. The computer device can set the width / height parameters to make the WebGL annotation rendering layer the same size as the container. The computer device can set the backgroundAlpha parameter to 0 to ensure the WebGL annotation rendering layer is completely transparent, preventing the underlying DICOM image from being obscured. The computer device can set the antialiasing parameter to enable anti-aliasing, ensuring smooth lines. The computer device can set the resolution of the WebGL annotation rendering layer according to the device pixel ratio (DevicePixelRatio), thus adapting to high-definition screens.

[0263] Furthermore, the computer device can set the generated Canvas element to absolute positioning and z-index to 999, placing the WebGL annotation drawing layer at the top level and mounting it to the parent container.

[0264] S402, ViewportContainer settings.

[0265] For example, PIXI.js uses containers to organize display objects. To this end, computer devices are designed with a multi-layered container structure. The root container (Stage) is the top-level container of the PIXI application. The viewport container is the core container, responsible for overall scaling and translation. When the user zooms or drags, only the scale and position properties of this container are modified, thus causing all child elements to transform synchronously. The background layer is used to place background images or grids. The parameter of this background layer can be set to z-index: 0. The content layer is used to place all annotation objects (PolygonRenderer). The parameter of this content layer can be set to z-index: 1.

[0266] S403, Polygon Renderer settings.

[0267] For example, the PolygonRenderer class inherits from PIXI.Container and encapsulates the rendering and interaction logic of a single polygon.

[0268] In computer devices, polygons can be described by defining a PolygonData structure, which includes a unique identifier (id), an array of vertex coordinates (points), style configurations (color, transparency, line width), and custom metadata.

[0269] First, computer devices can use PIXI.Graphics objects to draw the filled area and border of polygons. Computer devices can support custom fill color, border color, and transparency.

[0270] Secondly, computer devices can manage the state of polygons to set their states. These states can include a normal state for displaying polygons with defined styles; a state for highlighting (e.g., thickening borders) to indicate that the currently selected object is active; and an editing state for displaying control points (Handles) at polygon vertices, allowing users to drag and modify the shape.

[0271] Furthermore, computer devices can internally listen for events such as clicks, double-clicks, and right-clicks, and dispatch custom business events (such as polygonSelected and polygonEditRequested) to achieve event handling.

[0272] S404. Interactive editing is achieved by dragging control points.

[0273] For example, a computer device may first create a circular control point (Sprite or Graphic) at each vertex position when entering edit mode.

[0274] Furthermore, the computer device can record the initial position of the mouse button and the original coordinates of the corresponding vertex. At this point, the computer device can determine the start of the drag (PointerDown).

[0275] Subsequently, the computer device can perform a drag-and-drop movement (PointerMove). During this process, the computer device can calculate the increment of mouse movement (DeltaX, DeltaY) and apply the increment to the vertex coordinates, updating the vertex positions. Furthermore, the computer device can redraw the polygon shape and control point positions in real time, achieving smooth visual feedback.

[0276] Finally, the computer device can remove the motion listener after the drag-and-drop (PointerUp) is complete and trigger a "Pointer Update" event to notify the upper-layer application to save the data.

[0277] In one example, an image format conversion model is used to extract pixel data from a Cornerstone-rendered Canvas and convert it to a standard image format (PNG / JPEG). Once the user converts the DICOM image to the standard image format, the annotation drawing layer can continue working on the converted image without any data migration. This process can be performed in two steps: S501, DICOM to standard image format conversion.

[0278] For example, a computer device can obtain the underlying Canvas element used by the Cornerstone rendering engine. The computer device can create a new temporary Canvas and obtain its 2D drawing context. The computer device can draw the pixel data from the Cornerstone Canvas onto the temporary Canvas, achieving pixel copying. The computer device can call the Canvas API (toDataURL or toBlob), specifying the target format (PNG / JPEG) and quality parameters, to generate standard image data and export the format. The computer device can return the generated DataURL string or Blob object to the caller for subsequent saving or display.

[0279] S502, Adaptation of annotation drawing layer for the converted image.

[0280] For example, a computer device can replace the underlying DICOM rendering layer with standard image display components (such as img tags or ordinary Canvas) to achieve image replacement. Since the PIXI annotation drawing layer uses an independent pixel coordinate system, decoupled from the physical meaning of the underlying image, the original annotation coordinates remain valid as long as the image's pixel size remains unchanged. The computer device can then display the annotation drawing layer based on the original coordinates, achieving seamless switching between standard images. During this process, the user visually perceives almost no change in the underlying image source, and all annotation data, styles, and interaction logic can be directly reused.

[0281] Building upon the above embodiments, the DICOM image annotation method applied to the Web in this application demonstrates superior performance in processing large-scale medical image data, exhibiting powerful processing capabilities and excellent system stability. This DICOM image annotation method applied to the Web can easily handle high-resolution medical images, such as 4096×4096 CT images and 8192×8192 panoramic pathological sections. Furthermore, for large-scale annotated data, its performance indicators are outstanding under standard hardware configurations, for example, it can handle over 1000 polygons.

[0282] Furthermore, this application employs an on-demand rendering method, rendering only the visible area; annotations outside the viewport are not included in rendering calculations, effectively saving resources. Object pooling technology is used to reuse polygon renderer objects, reducing the overhead of creation and destruction. Event throttling technology reduces the processing frequency of high-frequency events (such as mouse movement), thereby ensuring the efficient and stable operation of the system. Based on the above operations, this application's memory usage is less than 2GB (covering image and annotation data), and the interactive operation latency is less than 50ms.

[0283] Based on the above embodiments, taking a medical image PACS viewer that supports DICOM image display and lesion annotation as an example, the execution process may include: Step 1: System initialization.

[0284] For example, a computer device may instantiate LabelSDK and specify the container DOM node and initial size.

[0285] Step 2: Image loading.

[0286] For example, a computer device can receive an incoming local DICOM file or a remote URL. The computer device can then automatically parse and render the DICOM image.

[0287] Step 3: Annotation drawing.

[0288] For example, a computer device can provide a "polygon tool" to the user, allowing the user to click and draw the outline of a lesion in the region of interest (ROI) during the annotation process.

[0289] Step 4: Image adjustment.

[0290] For example, a computer device can adjust the window width and level based on the user dragging the mouse to observe lung texture or bone structure more clearly.

[0291] Step 5: Data export.

[0292] For example, after annotation is completed, the computer device can export the coordinates and attributes of all polygons as JSON data and save it to the server.

[0293] Based on the above embodiments, taking the process of converting a DICOM image to PNG format as an example, the computer device can retain annotation information while completing the DICOM image format conversion. GIA annotation information can be used for AI training or report generation. This process may include: Step 1: Preprocessing.

[0294] For example, the Level 93 device can complete lesion labeling based on the doctor's adjustment of the optimal window width and window level in DICOM mode.

[0295] Step 2: Format conversion.

[0296] For example, a computer device can call the SDK's conversion interface to export the current view as a high-fidelity PNG image.

[0297] Step 3: Mode switching.

[0298] For example, a computer device can load a generated PNG image and replace the underlying DICOM view.

[0299] Step 4: Label and reuse.

[0300] For example, a computer device can keep the original annotation drawing layer unchanged and directly overlay it on a new PNG image, with the positions perfectly matching.

[0301] Step 5, follow-up processing.

[0302] For example, the computer device can continue to acquire the user's fine-tuned annotations in PNG mode, or directly package and download the image and annotation data.

[0303] Based on the above embodiments, taking a panoramic image (WSI) of a pathological slide as an example, hundreds of cell nucleus regions need to be labeled. The display process based on this panoramic image of a pathological slide may include: The computer device supports batch rendering, allowing for the simultaneous import of datasets containing 500+ cell nucleus annotations. Through WebGL batch rendering, loading time is less than 1 second. Even with all annotations displayed, zooming and panning operations maintain a smooth 60 FPS. Furthermore, the device can quickly hide or show specific types of cell annotations via API, with the system rendering only visible objects, effectively reducing GPU load.

[0304] Figure 7 A structural diagram of a DICOM image annotation device applied to the Web, provided in an embodiment of this application, is shown below. Figure 7 As shown, the DICOM image annotation device 700 for Web applications includes: Response module 701 is used to create an image display layer and a annotation drawing layer in response to DICOM image annotation commands. The image display layer and the annotation drawing layer are rendering layers in the Web. The annotation drawing layer is located above the image display layer.

[0305] Display module 702 is used to load DICOM images in the image display layer and record annotation information of DICOM images in the annotation drawing layer.

[0306] In one example, display module 702 is used for: Load the DICOM image and retrieve the metadata from the DICOM image, storing the metadata using a first preset format.

[0307] Listen for mouse and touch events in the annotation drawing layer, obtain annotation information generated by the user based on mouse and touch events, and store the annotation information using a second preset format.

[0308] In one example, a first viewport object is set on the annotation drawing layer, and a second viewport object is set on the image display layer. Display module 702 is used for: Set the opacity of the first viewport object to transparent. Then set the opacity of the second viewport object to opaque.

[0309] Set the first viewport object to respond to all mouse and touch events. Set the second viewport object to disable all mouse and touch events.

[0310] In one example, display module 702 is used for: In response to user zooming and / or moving operations on the first viewport object of the annotation drawing layer, a viewport synchronization command is generated. The first viewport object is used to display the annotation image corresponding to the DICOM image. The viewport synchronization command is used to synchronize the content areas of the first viewport object of the annotation drawing layer and the second viewport object of the image display layer. The second viewport object is used to display the DICOM image.

[0311] In response to viewport synchronization commands, this function acquires the current scaling factor and current offset coordinates of the content displayed in the first viewport object after the user performs scaling and / or panning operations on the first viewport object on the annotation drawing layer. The current scaling factor represents the ratio of the content displayed in the first viewport object to the annotation image. The current offset coordinates represent the coordinates of the top-left corner of the content displayed in the first viewport object in the annotation coordinate system of the annotation drawing layer.

[0312] The current zoom level and offset coordinates in the labeled coordinate system are converted to the target zoom level and target offset of the second viewport object in the image coordinate system of the image display layer. The target zoom level represents the ratio of the displayed content in the second viewport object to the DICOM image. The target offset represents the distance the center point of the displayed content in the second viewport object has moved.

[0313] The DICOM image is rendered and displayed in the second viewport object of the image display layer, based on the target scaling and target offset.

[0314] In one example, the first viewport object also includes a viewport container. The viewport container includes an upper auxiliary annotation layer and a lower annotation object layer. The auxiliary annotation layer contains auxiliary meshes or auxiliary points.

[0315] In one example, display module 702 is used for: In the labeled coordinate system, the actual offset of the current offset coordinate is determined by using the coordinates of the center point of the canvas in the web interface as a reference. The actual offset represents the amount of offset of the current offset coordinate relative to the coordinates of the canvas center point.

[0316] Divide the actual offset by the current scaling factor to obtain the target offset coordinates.

[0317] Determine the target scaling ratio based on the current scaling ratio.

[0318] In one example, display module 702 is used for: In the labeled coordinate system, determine the coordinates of the center point of the canvas in the web interface.

[0319] Calculate the theoretical starting coordinates of the top left corner of the labeled image in the label coordinate system when the center position of the labeled image is placed at the center point coordinate of the canvas.

[0320] Calculate the difference between the current offset coordinate and the theoretical starting coordinate to obtain the actual offset.

[0321] In one example, display module 702 is used for: The color information of each pixel is obtained from the DICOM image in the image display layer, and the color information is copied onto the created temporary canvas.

[0322] Based on a preset standard image format, standard image elements for the image display layer are generated using a temporary canvas.

[0323] In the image display layer, after replacing the DICOM image elements with standard image elements, they are rendered and loaded into the image display layer.

[0324] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0325] In this embodiment, the DICOM image annotation device applied to the Web is presented in the form of a functional unit. Here, a unit refers to an application-specific integrated circuit (ASIC), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0326] Figure 8 A structural diagram of a computer device provided in an embodiment of this application, such as... Figure 8 As shown, the computer device 800 includes one or more processors 801, a memory 802, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The processor 801 may be a central processing unit, a network processor, or a combination thereof. The memory 802 stores instructions executable by at least one processor 801 to cause the at least one processor 801 to perform the methods shown in the above embodiments. The memory 802 may include a program storage area and a data storage area.

[0327] The computer device also includes a communication interface 803 for communicating with other devices or communication networks.

[0328] This application also provides a computer-readable storage medium in which the methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and to be stored on a local storage medium after being downloaded over a network, so that the methods described herein can be stored on such software processing on a storage medium using a general-purpose computer, a special-purpose processor, or programmable or special-purpose hardware.

[0329] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method of any embodiment of this application.

[0330] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

[0331] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A DICOM image labeling method applied to a Web, characterized by, The method comprises: In response to a DICOM image annotation instruction, an image display layer and an annotation drawing layer are created; the image display layer and the annotation drawing layer are rendering layers in a Web; the annotation drawing layer is located above the image display layer; The DICOM image is loaded on the image display layer, and annotation information of the DICOM image is recorded on the annotation drawing layer.

2. The method of claim 1, wherein, The DICOM image is drawn on the image display layer, and the annotation information of the DICOM image is recorded on the annotation drawing layer, comprising: The DICOM image is loaded, and metadata in the DICOM image is obtained, and the metadata is stored in a first preset format; Mouse and touch events of the annotation drawing layer are listened to, and annotation information generated by a user based on the mouse and touch events is obtained, and the annotation information is stored in a second preset format.

3. The method of claim 1, wherein, The first viewport object is provided on the annotation drawing layer, and the second viewport object is provided on the image display layer; the method comprises: The transparency of the first viewport object is set to be transparent, and the transparency of the second viewport object is set to be opaque; The first viewport object is set to respond to all mouse and touch events, and the second viewport object is set to disable all mouse and touch events.

4. The method according to any one of claims 1-3, characterized in that, The method further comprises: In response to a user's zooming and / or moving operation on the first viewport object of the annotation drawing layer, a viewport synchronization instruction is generated; the first viewport object is used to display an annotation image corresponding to the DICOM image; the viewport synchronization instruction is used to synchronize the content area of the first viewport object of the annotation drawing layer and the second viewport object of the image display layer; the second viewport object is used to display the DICOM image; In response to the viewport synchronization instruction, after the user's zooming and / or panning operation on the first viewport object on the annotation drawing layer, the current zooming ratio and the current offset coordinates of the displayed content in the first viewport object are obtained; the current zooming ratio represents the ratio of the displayed content in the first viewport object to the annotation image; the current offset coordinates represent the coordinates of the top left corner position of the displayed content in the first viewport object in the annotation coordinate system of the annotation drawing layer; The current zooming ratio and the current offset coordinates in the annotation coordinate system are converted into a target zooming ratio and a target offset amount of the second viewport object in the image coordinate system of the image display layer; the target zooming ratio represents the ratio of the displayed content in the second viewport object to the DICOM image; the target offset amount represents the movement distance of the center point position of the displayed content in the second viewport object; According to the target zooming ratio and the target offset amount, the DICOM image is rendered and displayed in the second viewport object of the image display layer.

5. The method of claim 4, wherein, The first viewport object further comprises a viewport container; the viewport container comprises an auxiliary annotation layer located on the upper layer and an annotation object layer located on the lower layer; wherein the auxiliary annotation layer is provided with an auxiliary grid or an auxiliary point.

6. The method of claim 4, wherein, Converting the current scaling ratio and the current offset coordinate in the labeling coordinate system into a target scaling ratio and a target offset of the second viewport object in an image coordinate system of the image display layer, comprising: In the labeling coordinate system, taking the canvas center point coordinate in the Web interface as a reference, determining the actual offset of the current offset coordinate; the actual offset represents the offset of the current offset coordinate relative to the canvas center point coordinate; Using the actual offset divided by the current scaling ratio to obtain a target offset coordinate; According to the current scaling ratio, determining the target scaling ratio.

7. The method of claim 6, wherein, In the labeling coordinate system, taking the canvas center point coordinate in the Web interface as a reference, determining the actual offset of the current offset coordinate, comprising: In the labeling coordinate system, determining the canvas center point coordinate in the Web interface; Calculating the theoretical starting coordinate of the upper left corner position of the labeling image in the labeling coordinate system when the center position of the labeling image is placed at the canvas center point coordinate; Calculating the difference between the current offset coordinate and the theoretical starting coordinate to obtain the actual offset.

8. The method of any one of claims 1-3, wherein, The method further comprises: Obtaining color information of each pixel from the DICOM image of the image display layer, and copying the color information to the created temporary canvas; Using the temporary canvas to generate a standard image element of the image display layer according to a preset standard image format; After replacing the DICOM image element with the standard image element in the image display layer, rendering and loading to the image display layer.

9. A DICOM image labeling apparatus applied to a Web, characterized by, The device comprises: A response module for creating an image display layer and a labeling drawing layer in response to a DICOM image labeling instruction; the image display layer and the labeling drawing layer are rendering layers in Web; the labeling drawing layer is located above the image display layer; A display module for loading the DICOM image in the image display layer and recording labeling information of the DICOM image in the labeling drawing layer.

10. A computer device, comprising: Comprise: A memory and a processor, which are connected in communication with each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for causing a computer to execute the method of any one of claims 1 to 8.

12. A computer program product, characterised in that, Comprise computer instructions for causing a computer to execute the method of any one of claims 1 to 8.

Images