Cardiac electrophysiology simulation interactive methods, control devices, storage media and VR systems

Abstract

This application provides a cardiac electrophysiology simulation interaction method, control device, storage medium, and VR system, belonging to the field of VR technology. The method includes: acquiring cardiac electrophysiology simulation result data in VTK format; parsing the temporal volume data in the cardiac electrophysiology simulation result data to obtain cardiac voxel data and related metadata; rendering the cardiac voxel data and related metadata in a loop according to an initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment; monitoring trigger events and adjusting the rendering data according to an interactive control mechanism. This application achieves seamless integration of VTK format cardiac volume data with the Unity engine, ensuring high-performance rendering of high-precision 3D data in a virtual reality environment; this application also provides a temporal volume data rendering architecture, achieving smooth playback of large-scale cardiac electrophysiology simulation data through streaming loading and frame caching mechanisms.

Description

Technical Field

[0001] This application belongs to the field of VR technology, and specifically relates to a cardiac electrophysiology simulation interaction method, control device, storage medium and VR system. Background Technology

[0002] Cardiac electrophysiological simulation technology has been widely used in cardiac disease research and clinical diagnosis. Among related technologies, a cardiac electrophysiological experimental annotation and simulation system based on Virtual Reality (VR) technology has been developed, achieving precise localization of stimulation points and data transmission. However, the following key issues remain regarding the visualization and analysis of simulation results:

[0003] Existing cardiac electrophysiology simulation results are typically output in VTK (Visualization Toolkit) format, but this data is difficult to present and interact with directly in a VR environment. Researchers and physicians need to view the results in specialized VTK viewing software, which cannot be seamlessly integrated with the prior VR annotation environment, causing a break in the workflow. Traditional VTK viewers lack an immersive experience, requiring users to switch perspectives and time points multiple times to understand complex cardiac electrophysiological processes, increasing cognitive burden and difficulty of comprehension. Summary of the Invention

[0004] The purpose of this application is to provide a cardiac electrophysiology simulation interaction method, control device, storage medium and VR system, aiming at the seamless integration of VTK format cardiac body data with VR annotation environment.

[0005] According to a first aspect of this application, a VR-based cardiac electrophysiology simulation interaction method is provided. The cardiac electrophysiology simulation interaction method includes: acquiring cardiac electrophysiology simulation result data in VTK format; parsing the temporal volume data in the cardiac electrophysiology simulation result data to obtain cardiac voxel data and related metadata; rendering the cardiac voxel data and related metadata in a loop according to an initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment; and monitoring trigger events and adjusting the rendering data according to a preset interaction control mechanism.

[0006] In an optional implementation, the step of parsing the temporal volume data in the cardiac electrophysiological simulation result data to obtain cardiac voxel data and related metadata includes: parsing the header information in the cardiac electrophysiological simulation result data to identify the data format and data type; parsing the temporal volume data to obtain the cardiac voxel data; parsing the dimension, spacing, and origin information of the cardiac voxel data to obtain the related metadata and allocating memory space; and performing corresponding data conversion according to the data type to map the original voxel values ​​of the cardiac voxel data to a preset texture format.

[0007] In an optional implementation, after obtaining the cardiac voxel data and related metadata, the cardiac electrophysiological simulation interaction method further includes: classifying the cardiac voxel data according to user needs to enhance the visual distinction between tissues, wherein different tissues are distinguished based on the Hounsfield unit threshold.

[0008] In an optional implementation, a ray casting algorithm is used to render the heart voxel data and related metadata, including: constructing a principal ray vector for screen pixels using a ray-bounding box intersection test, calculating the intersection parameters of the principal ray vector and the axis-aligned bounding box, and determining the sampling interval of the principal ray vector; sampling along the direction of the principal ray vector with an adaptive step size within the sampling interval; obtaining voxel density values ​​through trilinear interpolation for each sampling point, and mapping the obtained voxel density values ​​to color and opacity values ​​using a predefined transfer function, and performing edge enhancement processing based on voxel gradients; detecting the intersection of the principal ray vector and the clipping plane, dynamically adjusting the sampling interval, and rendering highlight effects at the clipping edges; and accumulating the contributions of color and opacity values ​​using a front-to-back compositing method, and prematurely terminating ray propagation when the accumulated opacity contribution reaches a preset threshold to generate the final rendered image.

[0009] In an optional implementation, the step of cyclically rendering the cardiac voxel data and related metadata based on the initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment includes: extracting timestamps or sequence numbers according to the naming pattern of the cardiac electrophysiological simulation result data, constructing a time-series index table to determine the order of cyclic rendering; creating a playback controller according to the user-set initial frame rate, starting the rendering loop, and initializing a double-buffer structure; asynchronously loading the current frame data into the front buffer when playback starts, while preloading subsequent frame data into the cache queue in a background thread; monitoring the performance indicators of the VR system, dynamically calculating the optimal playback rate based on processor load and cache hit rate, and adjusting the frame interval of the playback controller; writing new frame data into the back buffer when switching frames, and swapping the front and back buffer pointers when the vertical synchronization signal is triggered to eliminate screen tearing and flickering.

[0010] In an optional implementation, the cardiac electrophysiology simulation interaction method further includes: periodically performing memory management, releasing cached frame data that are more than a preset threshold away from the current playback position according to the LRU strategy; and updating the timeline, frame count, and playback rate indicator in the user interface to ensure consistency with the real-time playback status.

[0011] In an optional implementation, the monitoring of trigger events, according to a preset interactive control mechanism, adjusts rendering data, including: creating a visual planar object around the heart simulation model as a visual indicator of the cutting plane and associating it with the shearing plane parameters of the volume renderer; monitoring the trigger events of the input device and capturing the corresponding position and posture changes; converting the linear displacement of the input device into a translation in the normal direction of the cutting plane and updating the plane offset parameters; decomposing the rotation increment of the input device into rotation matrices in the horizontal and vertical directions, combining them, and updating the plane normal vector; passing the updated plane normal vector and offset to the volume rendering shader; calculating the intersection point of the light rays and the cutting plane in the volume rendering shader, adjusting the sampling interval, and overlaying a highlight effect on the shearing edge.

[0012] According to a second aspect of this application, a control device is provided, the control device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the above-described cardiac electrophysiological simulation interactive method.

[0013] According to a third aspect of this application, a machine-readable storage medium is provided, on which instructions are stored, causing a machine to perform the above-described cardiac electrophysiology simulation interaction method.

[0014] According to a third aspect of this application, a VR system is provided, comprising a VR controller and a configured VR controller, a VR interactive interface, and the aforementioned control device. The control device is further configured to: establish an input adapter for the VR controller to capture and parse the position, rotation, and button state data of the VR controller; perform collision detection between a ray emitted by the VR controller and a heart simulation model, recording the collision point as an interaction target; realize rotation and translation operations of the heart simulation model based on the gripping state of the VR controller's trigger button, or adjust the position and angle of a virtual cutting plane using a joystick; automatically adjust the spatial position and orientation of the control panel of the VR interactive interface according to the user's head position to ensure it is accessible within the field of view; modify volume drawing parameters through interaction with the control panel and synchronously update the rendering effect in the preview window; realize play / pause, frame rate adjustment, and time jump through the VR controller's trigger button and touchpad operations; and dynamically adjust the volume drawing sampling accuracy according to the current scene complexity to maintain a stable interactive frame rate.

[0015] The VR-based cardiac electrophysiology simulation interaction method provided in this application includes: acquiring cardiac electrophysiology simulation result data in VTK format; parsing the temporal volume data in the cardiac electrophysiology simulation result data to obtain cardiac voxel data and related metadata; rendering the cardiac voxel data and related metadata in a loop according to the initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment; and monitoring trigger events and adjusting the rendering data according to a preset interaction control mechanism. This application embodiment achieves seamless integration of VTK format cardiac volume data with the Unity engine, ensuring high-performance rendering of high-precision 3D data in a virtual reality environment. This application embodiment also provides a temporal volume data rendering architecture, achieving smooth playback of large-scale cardiac electrophysiology simulation data through streaming loading and frame caching mechanisms, enabling continuous observation of the dynamic conduction process of electrical activity.

[0016] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures and processes shown in the description and the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a VR-based cardiac electrophysiology simulation interaction method provided in an exemplary embodiment of this application.

[0019] Figure 2 This is a flowchart illustrating the cardiac electrophysiological simulation interaction method of the VR system provided in the exemplary embodiments of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] As mentioned earlier, existing cardiac electrophysiology simulation results are typically output in VTK (Visualization Toolkit) format, but this data is difficult to present and interact with directly in a VR environment. Users need to view the results in specialized VTK viewing software, which cannot seamlessly integrate with the prior VR annotation environment, causing a break in the workflow. Traditional VTK viewers lack an immersive experience, and the temporal changes and spatial propagation characteristics of simulation results are difficult to understand intuitively. Users need to switch perspectives and time points multiple times to understand complex cardiac electrophysiological processes, increasing cognitive burden and difficulty of comprehension. The analysis tools for simulation results are separated from the visualization system, preventing users from viewing and analyzing data in the same environment. This separation leads to low work efficiency and increases the difficulty of data interpretation. Furthermore, the related technologies lack intuitive interactive control methods, making it difficult for users to freely adjust different perspectives, profiles, and rendering parameters according to their observation needs, affecting in-depth analysis of specific regions of electrophysiological data. There is insufficient support for efficient rendering of large-scale complex three-dimensional time-series data; common visualization software experiences significant performance degradation when dealing with high-precision cardiac electrophysiological models, impacting the user experience.

[0022] Therefore, this application provides a VR-based cardiac electrophysiology simulation interaction method. This cardiac electrophysiology simulation interaction method preferably utilizes Unity as its base engine; please refer to [reference needed]. Figure 1 The cardiac electrophysiology simulation interactive method may include the following steps:

[0023] Step S110: Obtain cardiac electrophysiological simulation results data in VTK format.

[0024] Please refer to Figure 2For example, the stimulation point acquisition subsystem can output cardiac electrophysiological simulation results data in VTK format (also referred to as VTK files or VTK data). For instance, based on NIFTI (Neuroimaging Informatics Technology Initiative, a file format designed for storing medical and neuroimaging data) data of the heart, a corresponding 3D cardiac data model can be exported using a 3D reconstruction algorithm. This 3D cardiac data model can be in Obj format. The 3D cardiac data model is then imported into Unity, and the origin of the cardiac model is aligned with the origin of the VR world. Each stimulation point is calibrated using ray projection from the VR device, determining the VR spatial coordinates of each stimulation point. These VR spatial coordinates are then converted to the stimulation point matrix coordinates corresponding to the cardiac model matrix. The stimulation point matrix coordinates are expanded to obtain a spherical region matrix coordinate set. This spherical region matrix coordinate set is then corrected to obtain the stimulation range dot matrix. Finally, the VTK file is output.

[0025] Step S120: Analyze the time-series volume data in the cardiac electrophysiology simulation results data to obtain cardiac voxel data and related metadata.

[0026] In this embodiment, the VTK file structure can be parsed and cardiac voxel data and related metadata (including dimensions, spacing, origin, etc.) can be obtained by configuring the VTKImporter function or code.

[0027] In a preferred embodiment of this application, step S120 may include: parsing the header information in the cardiac electrophysiological simulation result data, identifying the data format and data type; parsing the time-series volume data to obtain cardiac voxel data; parsing the dimension, spacing and origin information of the cardiac voxel data to obtain relevant metadata and allocating memory space; and performing corresponding data conversion according to the data type to map the original voxel values ​​of the cardiac voxel data to a preset texture format.

[0028] In the embodiments of this application, corresponding conversion mechanisms are performed for different types of VTK data (e.g., ASCII or binary format) and different data types (e.g., unsigned char, short, float, etc.) to ensure that the data can be accurately imported.

[0029] Please refer to Figure 2For example, the VTKImporter code reads the header information of the VTK file to identify the data format and data type; it parses the relevant metadata such as the dimensions, spacing, and origin of the heart voxel data and allocates memory space accordingly. Furthermore, it can perform corresponding data conversions based on the data type, mapping the raw voxel values ​​to a preset texture format. For example, it maps the raw voxel values ​​to an 8-bit texture format to serve as the standard input for GPU rendering and as a bridge connecting data preprocessing and real-time rendering.

[0030] In a preferred embodiment of this application, after step S120, the cardiac electrophysiology simulation interaction method may further include: classifying cardiac voxel data according to user needs to enhance the visual distinguishability between tissues. Specifically, different tissues are distinguished based on a Hounsfield unit threshold.

[0031] For cardiac electrophysiological simulation results data, this application embodiment also provides an optimization processing flow. For example, when a user request is received that data optimization is needed (e.g., the preset field optimizeForHeartData is true), the VTImporter code is used to classify and process data according to different feature value ranges such as blood and muscle tissue to enhance the visual distinction between tissues. Preferably, different tissues can be distinguished based on Hounsfield unit thresholds. For example, since blood is approximately +20 HU (Hounsfield units) and muscle tissue is approximately 10-40 HU, the processing background is now set to completely transparent; blood and soft tissue are mapped to the low-density range of 1-100 (e.g., CT data typically includes a wide range of HU values ​​(from -1000 to +3000), which are mapped to a smaller range available in computer graphics (e.g., a byte range of 0-255)); medium-density tissue is mapped to the range of 101-200; and high-density tissue is mapped to the range of 201-255.

[0032] Preferred embodiments of this application can also generate histogram analysis of cardiac voxel data, automatically adjust the optimal mapping parameters, and prepare transfer functions for subsequent rendering.

[0033] Step S130: Based on the initial frame rate and using a ray casting algorithm, continuously render the heart voxel data and related metadata to achieve simulation in a VR visual environment.

[0034] Please refer to Figure 2For example, a GPU-accelerated ray casting algorithm is used to achieve volumetric data rendering. This embodiment preferably achieves high-quality volume rendering by configuring the VolumeRayCastShader code. The shader of this VR system can be configured to: perform a ray-bounding box intersection test to determine the entry and exit points of each pixel ray; starting from the nearest intersection point, sample along the ray direction with an adaptive step size, using a large step size to quickly skip in blank areas and a small step size for fine sampling in high-gradient areas; for each sampling point, obtain the density value from the 3D texture and map it to color and opacity through a transfer function; calculate the voxel gradient for edge enhancement processing to make tissue boundaries clearer. Furthermore, when the shearing function is enabled, this embodiment can also detect the intersection relationship between the ray and the shearing plane, dynamically adjust the sampling interval and render the highlight effect of the shearing edge; accumulate the color contribution of all sampling points using a front-to-back compositing method, and terminate the ray early when the opacity reaches a threshold to generate the final rendering result.

[0035] In a preferred embodiment of this application, rendering heart voxel data and related metadata using a ray casting algorithm may include: Step S131, constructing a principal ray vector for screen pixels using a ray-bounding box intersection test, calculating the intersection parameters of the principal ray vector and the axis-aligned bounding box, and determining the sampling interval of the principal ray vector; Step S132, sampling along the direction of the principal ray vector with an adaptive step size within the sampling interval; for each sampling point, obtaining voxel density values ​​through trilinear interpolation, and mapping the obtained voxel density values ​​to color values ​​and opacity values ​​using a predefined transfer function, and performing edge enhancement processing based on voxel gradients; Step S133, detecting the intersection of the principal ray vector and the clipping plane, dynamically adjusting the sampling interval, and rendering highlight effects at the clipping edges; Step S134, accumulating the contributions of color values ​​and opacity values ​​using a front-to-back compositing method, and prematurely terminating ray propagation when the accumulated opacity value contribution reaches a preset threshold to generate the final rendered image.

[0036] In this embodiment, step S131, the ray-bounding box intersection test, is the initial stage of the volume rendering algorithm, used to determine the intersection parameters of the view vector and the volume data space boundary. Step S131 may include: constructing a principal ray vector (or simply ray) for each screen pixel from the view position; calculating the intersection parameters (e.g., near intersection point tNear and far intersection point tFar) of the principal ray vector and the normalized bounding box (e.g., AABB, axis-aligned bounding box). For example, for each coordinate axis, the intersection parameters of the ray with the two boundary planes corresponding to that axis are calculated; the largest near intersection point and the smallest far intersection point are compared and determined; if tFar > tNear and tFar > 0, then the ray is confirmed to intersect the bounding volume, and tNear and tFar are used as the parameter range boundaries for subsequent sampling.

[0037] For step S132, the ray sampling integration may include: based on the principle of numerical integration, using the discrete step length δ t Samples are taken progressively along the ray direction from the entry point (the position corresponding to tNear) to the exit point (the position corresponding to tFar). For each sampling point x... i =x0+t i ·d(where x0 is the ray origin, d is the ray direction, t) i The parameterized distance of the current sampling point along the ray direction (i.e., the distance traveled along the ray direction d from the starting point x0) can be obtained from the voxel grid using trilinear interpolation, yielding the scalar density value f(x) corresponding to that spatial location. i ).

[0038] For step S133, the mapping from scalar density to optical properties may include: after obtaining the scalar density value, the scalar density value can be mapped to optical properties (e.g., color and opacity) through a predefined transfer function T(f), for example, T:f(x i The mapping is )→(RGBA)i, where RGBA represents the RGB and alpha channels, and A represents opacity, not just RGB values. This mapping enables texture lookup and supports flexible definition of different visualization effects for different density ranges. For the cardiac electrophysiological simulation results data in this application embodiment, low-density areas can be predefined as low-opacity dark red (e.g., blood), medium-density areas as translucent red (e.g., myocardium), and high-density areas as opaque yellow (e.g., areas of electrical activity activation), thereby visually distinguishing different tissues and electrophysiological states.

[0039] For step S134, the front-to-back compositing method (or forward compositing method) may include: sampling and acquiring multi-point optical characteristics, and accumulating them through forward compositing to form the final pixel value. For example, for each sampling point, its RGB value is pre-multiplied by the opacity α; the accumulation is performed incrementally according to the forward compositing equation, which can be expressed as follows:

[0040] Cdst_new=Csrc+(1-αsrc)*Cdst_current (1)

[0041] Wherein, Cdst_new represents the updated target color (the color accumulated by the current pixel), which represents the new color value after synthesis, including information from the three RGB channels. It is the color value updated to the frame buffer after the current ray casting step is completed; Cdst_current represents the target color that has been accumulated, which represents the color value synthesized by all previous sampling points, i.e., the result accumulated in the previous step; Csrc represents the color contribution of the current sampling point (which can be pre-multiplied by its own alpha value), which represents the color value of the current sampling point after being mapped by the transfer function, which can be pre-multiplied by alpha (Csrc = src.rgb * src.a); αsrc represents the opacity (alpha value) of the current sampling point, which represents the opacity of the current sampling point after being mapped by the transfer function, and the value range is, for example, [0,1]. Equation (1) realizes the front-to-back volume synthesis, that is, the color contribution of each sampling point in the volume data is accumulated from near to far from the observer's perspective, while also considering the semi-transparent effect. When (1-αsrc) is close to 0, it means that the current sampling point is almost completely opaque, and the contribution of the subsequent sampling points to the final color will be very small.

[0042] Furthermore, as light attenuates during propagation, the opacity of preceding sampling points affects the contribution of subsequent sampling points to the final color. When the cumulative opacity approaches the threshold of complete opacity (e.g., 0.95), the sampling process can be terminated early to optimize the calculation.

[0043] In a preferred embodiment of this application, step S130 may include: extracting timestamps or sequence numbers based on the naming pattern of cardiac electrophysiological simulation result data, constructing a time-series index table to determine the order of loop rendering; creating a playback controller based on the user-set initial frame rate, starting the rendering loop, and initializing the double-buffer structure; asynchronously loading the current frame data into the front buffer when playback starts, while preloading subsequent frame data into the cache queue in a background thread; monitoring the performance indicators of the VR system, dynamically calculating the optimal playback rate based on processor load and cache hit rate, and adjusting the frame interval of the playback controller; writing new frame data into the back buffer when switching frames, and exchanging the pointers of the front and back buffers when the vertical synchronization signal is triggered to eliminate screen tearing and flickering.

[0044] In the VR system, to achieve a smooth display of the changes in cardiac electrophysiological activity over time, this embodiment of the application also configures TemporalVTKPlayer code to provide functions such as loading, caching, and playback control of time-series data. The TemporalVTKPlayer code is configured to: support playback speed control based on frame rate to simulate the real time scale of cardiac electrical activity; implement control functions such as loop playback, single-step forward / backward, and jumping to a specified frame; and have an optional data preloading mechanism to improve playback smoothness.

[0045] For example, the TemporalVTKPlayer code can also be configured to: scan the target directory of VTK files, automatically identify the naming pattern of time-series VTK files, and build an index. Based on the user-defined initial frame rate, a time-series playback controller (TemporalVTKPlayer) can be created and a rendering loop started. When playback is detected to have started, a double buffer is initialized and the current frame data is loaded asynchronously; the frame to be played is preloaded in a background thread, while the performance metrics of the VR system are monitored to dynamically adjust the optimal playback rate. For frame switching, this embodiment can achieve a smooth transition by first writing the newly loaded frame data to the back buffer, and then quickly swapping the front and back buffer pointers at an appropriate time to avoid flickering.

[0046] In a preferred embodiment of this application, the cardiac electrophysiology simulation interaction method may further include: periodically performing memory management, releasing cached frame data that are more than a preset threshold away from the current playback position according to the LRU strategy; and updating the timeline, frame count, and playback rate indicator in the user interface to ensure consistency with the real-time playback status.

[0047] Following the above examples, this embodiment can also periodically perform memory management, releasing cached frames that are far from the current playback position according to the LRU (Least Recently Used) strategy, keeping memory usage within a reasonable range. Furthermore, this embodiment can also synchronously update the UI state to ensure that information such as the timeline and frame count remains consistent with the actual playback position.

[0048] The preferred embodiments of this application employ an asynchronous data loading mechanism based on Unity coroutines to avoid UI freezing when loading large datasets, thus improving the user experience. Simultaneously, a double-buffered frame management mechanism is implemented to resolve visual discontinuity issues during frame switching, achieving a smooth temporal data transition effect. Furthermore, the embodiments of this application also provide an adaptive frame rate control algorithm based on performance monitoring, dynamically adjusting the playback speed according to the current rendering performance and data loading speed to ensure a smooth viewing experience.

[0049] Step S140: Monitor the triggered event and adjust the rendering data according to the preset interactive control mechanism.

[0050] This application also provides various interactive control mechanisms to enhance users' ability to observe and analyze cardiac electrophysiological data. For cutting plane control, a preferred embodiment of this application also includes ClippingPlaneController code to implement cardiac cutting in any direction. Users can adjust the position and direction of the cutting plane by dragging and dropping with a VR controller or mouse.

[0051] In a preferred embodiment of this application, step S140 may include: creating a visual planar object around the heart simulation model as a visual indicator of the cutting plane and associating it with the shear plane parameters of the volume renderer; monitoring the trigger events of the input device and capturing the corresponding position and pose changes; converting the linear displacement of the input device into a translation in the normal direction of the cutting plane and updating the plane offset parameters; decomposing the rotation increment of the input device into rotation matrices in the horizontal and vertical directions, combining them, and updating the plane normal vector; passing the updated plane normal vector and offset to the volume renderer; calculating the intersection point of the light rays and the cutting plane in the volume renderer, adjusting the sampling interval, and overlaying a highlight effect on the shear edge.

[0052] Please refer to Figure 2 For example, during initialization, a visual planar object (e.g., a semi-transparent quadrilateral) is created and attached around the heart model as a visual indicator of the cutting plane. When user activation of the cutting function is detected, this embodiment can detect the interaction state of the input device (e.g., mouse clicks or VR controller trigger events); during dragging, the change in the input device's position is calculated and converted into a movement along the plane's normal direction. This change can be applied to the parameters of the `clipPlaneOffset` code to achieve translation of the plane in the normal direction. Furthermore, this embodiment can also update the visual position of the cutting plane in real time, providing intuitive spatial feedback. For plane rotation operations, this embodiment can calculate the rotation increment of the input device and construct two orthogonal rotation matrices: one controls the horizontal tilt of the plane, and the other controls the vertical tilt. These two rotations are combined and applied to the plane's normal vector to achieve plane rotation in any direction. When a change in the cutting plane parameters is detected, this embodiment of the application can pass the new plane normal vector and offset to the volume rendering shader through the volumeRenderer.SetClipPlane() method. Inside the shader, for each ray, the intersection point of the ray and the plane is calculated, and the start and end positions of the sampling are adjusted accordingly. Highlights are added to the cutting edge to enhance visual recognition.

[0053] Accordingly, the VR-based cardiac electrophysiology simulation interaction method provided in this application includes: acquiring cardiac electrophysiology simulation result data in VTK format; parsing the temporal volume data in the cardiac electrophysiology simulation result data to obtain cardiac voxel data and related metadata; rendering the cardiac voxel data and related metadata in a loop according to the initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment; and monitoring trigger events and adjusting the rendering data according to a preset interaction control mechanism. This application embodiment achieves seamless integration of VTK format cardiac volume data with the Unity engine, ensuring high-performance rendering of high-precision 3D data in a virtual reality environment. This application embodiment also provides a temporal volume data rendering architecture, which achieves smooth playback of large-scale cardiac electrophysiology simulation data through streaming loading and frame caching mechanisms, enabling continuous observation of the dynamic conduction process of electrical activity.

[0054] Furthermore, this application embodiment also provides multiple interactive control mechanisms to enhance the user's ability to observe and analyze cardiac electrophysiological data. Regarding the interactive sectioning function provided in this application embodiment, the operator can adjust the sectioning plane in any direction in real time in a virtual reality environment through intuitive handle operation, allowing for in-depth observation of the electrophysiological characteristics of specific regions of the heart. Based on this, this application embodiment also constructs a parameterizable transfer function editing system, enabling users to dynamically adjust the visualization effects of different tissue types according to observation needs, enhancing the visual recognition of key electrophysiological features.

[0055] This application embodiment also provides a control device, which may include: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the above-described cardiac electrophysiology simulation interactive method.

[0056] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described cardiac electrophysiology simulation interaction method.

[0057] It should be noted that the control device and machine-readable storage medium described above can implement the cardiac electrophysiology simulation interaction method provided in the above embodiments. For specific implementation methods, please refer to the description of the cardiac electrophysiology simulation interaction method in the above embodiments, which will not be repeated here.

[0058] This application embodiment also provides a VR system, which may include a VR controller and its configured VR controller, a VR interactive interface, and the aforementioned control device. The control device may further be configured to: establish an input adapter for the VR controller to capture and parse the position, rotation, and button state data of the VR controller; perform collision detection between the VR controller-emitted rays and the heart simulation model, recording the collision point as the interaction target; realize the rotation and translation operations of the heart simulation model based on the gripping state of the VR controller's trigger button, or adjust the position and angle of the virtual cutting plane using a joystick; automatically adjust the spatial position and orientation of the control panel of the VR interactive interface according to the user's head position to ensure it is accessible within the field of view; modify volume drawing parameters through the interaction of the control panel and synchronously update the rendering effect in the preview window; realize play / pause, frame rate adjustment, and time jump through the VR controller's trigger button and touchpad operations; and dynamically adjust the volume drawing sampling accuracy according to the current scene complexity to maintain a stable interactive frame rate.

[0059] Please refer to Figure 2 The VR system provided in this application supports importing cardiac electrophysiological simulation results data in standard VTK format. It can parse the VTK file structure and obtain cardiac voxel data and related metadata through the VTKImporter code configured on the control device. The VR system uses a GPU-accelerated ray casting algorithm to achieve volumetric data rendering, and can achieve high-quality volume rendering effects through the VolumeRayCastShader code configured on the control device. The VR system also provides loading, caching, and playback control functions for time-series data through the TemporalVTKPlayer code configured on the control device. The TemporalVTKPlayer code can be configured to include: supporting playback speed control based on frame rate to simulate the real timescale of cardiac electrical activity; implementing control functions such as loop playback, single-step forward / backward, and jumping to a specified frame; and an optional data preloading mechanism to improve playback smoothness. The VR system also provides multiple interactive control mechanisms to enhance the user's ability to observe and analyze cardiac electrophysiological data: for cutting plane control, the ClippingPlaneController code configured on the control device can be used to achieve cardiac cutting in any direction, and the user can adjust the position and direction of the cutting plane by dragging with the VR controller or mouse.

[0060] In a preferred embodiment of this application, the VR system also provides an ergonomic VR interface, offering an intuitive control panel and visualization tools to reduce the learning curve. This VR interface can be configured to include: 1) a timeline control interface providing functions such as play / pause, frame rate adjustment, and frame skipping; 2) a volume rendering parameter adjustment panel for real-time adjustment of rendering parameters such as density, brightness, and step size; and 3) a preset scheme quick-switch button for optimized presets for different observation purposes (e.g., prominent myocardial structures, prominent vascular networks, etc.).

[0061] In a preferred embodiment of this application, the VR system initializes the input adapter of the VR controller and establishes a unified interaction interface. When the VR controller is detected to be active, the VR system can begin to capture and parse the controller's position, rotation, and button status data. Users can, for example, point to and select interactive objects using the ray on the right controller; the location where the ray collides with the heart model is automatically recorded as the interaction point. For model manipulation, users can use the controller's trigger buttons to grab the model for rotation and translation, or control the position and angle of the virtual cutting plane using the controller's joystick. The VR system can also automatically adjust the placement and angle of the UI control panel based on the user's head position to ensure easy access within the field of view. When the user adjusts the transfer function parameters through the control panel, the VR system can update the volume rendering effect in real time, while simultaneously displaying the current density distribution and mapping relationship in the preview window. For time-series playback control, users can use the controller buttons and touchpad to perform play / pause, frame rate adjustment, and time jump operations. Furthermore, the VR system can intelligently adjust the sampling accuracy based on the current rendering complexity, maintaining a stable frame rate while ensuring visual quality, providing a smooth VR interactive experience.

[0062] In a preferred embodiment of this application, the software for the VR system can be configured to include: creating and configuring a cardiac electrophysiological visualization environment using the code VTKSequenceSetup; starting the VR device and entering the virtual reality cardiac observation environment; adjusting playback parameters (e.g., speed, playback direction, etc.) via the control panel; adjusting the cross-sectional plane using the controller to observe the electrophysiological activity inside the heart; adjusting the transfer function parameters according to the observation needs to highlight the cardiac region of interest; and analyzing the propagation patterns and spatial distribution characteristics of electrical activity at different time points through time axis control.

[0063] Accordingly, the VR system provided in this application, based on immersive 3D visualization, offers a more intuitive spatial perception experience, enabling doctors and researchers to more comprehensively understand the 3D spatial distribution and conduction characteristics of cardiac electrophysiological activity, and improving the efficiency and accuracy of interpreting electrophysiological simulation results. Based on the dynamic time-series data playback function, it enables continuous observation of the evolution of cardiac electrical activity over time, breaking through the limitations of traditional static visualization, and helping to identify electrical signal conduction paths and temporal patterns, which is of great significance for the study of arrhythmia mechanisms. Based on interactive profile and transfer function adjustment functions, it enhances the ability to observe specific regions and tissue types, allowing users to flexibly adjust the visualization effects according to research needs, improving the accuracy of data analysis. The modular system design and special optimization for cardiac data enable the VR system to maintain good rendering performance while ensuring high-quality visualization effects, meeting the real-time interactive needs of complex 3D time-series data visualization. The ergonomic design of the virtual reality interface lowers the barrier to entry for the system, conforms to user operating habits, reduces learning costs, and allows doctors and researchers to focus more on the analysis and understanding of cardiac electrophysiological activity. The VR system's support for VTK standard format data ensures good compatibility with existing cardiac electrophysiology simulation software, facilitating integration into existing research and clinical workflows and improving efficiency. Optimized visualization for specific cardiac tissue types enhances the identification of key structures, helping to more accurately locate areas of abnormal electrical activity and providing more valuable reference information for the diagnosis and treatment of arrhythmias.

[0064] It is understood that the circuit structures, names, and parameters described in the above embodiments are merely examples. Those skilled in the art can also make readily conceived combinations and adjustments to the structural features of the above embodiments according to their needs, and the concept of this application should not be limited to the specific details of the above examples.

[0065] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A VR-based interactive method for cardiac electrophysiology simulation, characterized in that, The cardiac electrophysiology simulation interaction method includes: Acquire cardiac electrophysiological simulation results data in VTK format; The temporal volume data in the cardiac electrophysiology simulation results data is analyzed to obtain cardiac voxel data and related metadata. Based on the initial frame rate, and utilizing a ray casting algorithm, the cardiac voxel data and related metadata are rendered cyclically to achieve simulation in a VR visual environment; and Monitor triggered events and adjust rendering data according to preset interactive control mechanisms. The process of parsing the time-series volume data in the cardiac electrophysiological simulation results yields cardiac voxel data and related metadata, including: The header information in the cardiac electrophysiology simulation results data is analyzed to identify the data format and data type, and corresponding conversion mechanisms are executed for different types of VTK data and different data types; The time series data is parsed to obtain the cardiac voxel data; The dimensions, spacing, and origin information of the cardiac voxel data are analyzed to obtain the relevant metadata, and memory space is allocated; and Perform corresponding data conversions based on the data type to map the original voxel values ​​of the cardiac voxel data to a preset texture format.

2. The cardiac electrophysiology simulation interactive method according to claim 1, characterized in that, After obtaining the cardiac voxel data and related metadata, the cardiac electrophysiological simulation interaction method further includes: Based on user needs, the cardiac voxel data is classified to enhance the visual differentiation between tissues. Among them, different tissues are distinguished based on Hounsfield unit thresholds.

3. The cardiac electrophysiology simulation interactive method according to claim 1, characterized in that, Using a ray casting algorithm, the cardiac voxel data and related metadata are rendered, including: Using the ray-bounding box intersection test, a main ray vector is constructed for each screen pixel. The intersection parameters of the main ray vector and the axis-aligned bounding box are calculated, and the sampling interval of the main ray vector is determined. Within the sampling interval, sampling is performed along the direction of the principal ray vector with an adaptive step size; For each sampling point, the voxel density value is obtained by trilinear interpolation, and the obtained voxel density value is mapped to color value and opacity value using a predefined transfer function, and edge enhancement processing is performed based on the voxel gradient. The intersection point of the main ray vector and the shearing plane is detected, the sampling interval is dynamically adjusted, and a highlight effect is rendered at the shearing edge; and By using a front-to-back compositing method, the contributions of color values ​​and opacity values ​​are accumulated. When the accumulated contribution of opacity values ​​reaches a preset threshold, the light propagation is terminated in advance to generate the final rendered image.

4. The cardiac electrophysiology simulation interactive method according to claim 1, characterized in that, The process of cyclically rendering the heart voxel data and related metadata based on an initial frame rate and using a ray casting algorithm to achieve simulation in a VR visual environment includes: Based on the naming pattern of the cardiac electrophysiology simulation results data, timestamps or sequence numbers are extracted, and a time-series index table is constructed to determine the order of cyclic rendering. Based on the initial frame rate set by the user, a playback controller is created, the rendering loop is started, and the double buffer structure is initialized; When playback starts, the current frame data is asynchronously loaded into the front buffer, while the subsequent frame data is preloaded into the buffer queue in the background thread; Monitor the performance metrics of the VR system, dynamically calculate the optimal playback rate based on processor load and cache hit rate, and adjust the frame interval of the playback controller. During frame switching, new frame data is written to the back buffer, and the pointers of the front and back buffers are swapped when the vertical sync signal is triggered, eliminating screen tearing and flickering.

5. The cardiac electrophysiology simulation interactive method according to claim 4, characterized in that, The cardiac electrophysiology simulation interaction method also includes: Periodically perform memory management, releasing cached frame data that is more than a preset threshold away from the current playback position according to the LRU policy; and Update the timeline, frame count, and playback rate indicators in the user interface to ensure they match the real-time playback status.

6. The cardiac electrophysiology simulation interactive method according to claim 1, characterized in that, The monitored trigger event adjusts the rendering data according to a preset interactive control mechanism, including: Create visual planar objects around the heart simulation model as visual indicators of the cutting planes and associate them with the shear plane parameters of the volume renderer. Monitor the trigger events of the input device and capture the corresponding position and attitude changes; The linear displacement of the input device is converted into a translation in the direction of the cutting plane normal, and the plane offset parameters are updated. The rotation increment of the input device is decomposed into rotation matrices in the horizontal and vertical directions, and the plane normal vector is updated after combining them. Pass the updated plane normals and offsets to the volume rendering shader; The intersection of the light rays and the cutting plane is calculated in the volume rendering shader, the sampling interval is adjusted, and a highlight effect is superimposed on the shearing edge.

7. A control device, characterized in that, The control device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cardiac electrophysiological simulation interactive method according to any one of claims 1-6.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions that cause the machine to perform the cardiac electrophysiological simulation interactive method according to any one of claims 1-6.

9. A VR system, characterized in that, The VR system includes a VR controller and a configured VR controller, a VR interactive interface, and the control device as described in claim 7, wherein the control device is further configured to: An input adapter is established for the VR controller to capture and parse the position, rotation, and button status data of the VR controller. The VR controller emits rays that collide with the heart simulation model, and the collision point is recorded as the interaction target. Based on the gripping state of the trigger button on the VR controller, the rotation and translation operations of the heart simulation model can be realized, or the position and angle of the virtual cutting plane can be adjusted by the joystick; The spatial position and orientation of the control panel of the VR interactive interface are automatically adjusted according to the user's head position to ensure that it is accessible within the field of vision. The drawing parameters can be modified interactively through the control panel, and the rendering effect can be updated synchronously in the preview window; Play / pause, frame rate adjustment, and time jump can be achieved through the trigger button and touchpad operation of the VR controller; and The volume rendering sampling precision is dynamically adjusted based on the current scene complexity to maintain a stable interactive frame rate.

Images