A virtual-real fusion three-dimensional engine timeline control method, device and medium

Abstract

The application discloses a kind of virtual-real fusion three-dimensional engine time axis control method, equipment and medium, the method includes: based on virtual-real fusion scene constructs multidimensional data mapping time axis model, establishes time axis basic framework, forms initial time axis model;Based on initial time axis model, the spatiotemporal alignment processing of real-time heterogeneous data stream and historical data is executed bidirectionally synchronized, and multi-source data is filled to time axis, and complete time axis dataset is constructed;Based on complete time axis dataset, event prediction and analog deduction based on timing deep learning are executed, and future time domain is extended to time axis, and form visual simulation timeline;Based on visual simulation timeline, realize visual interaction and dynamic scene real-time rendering based on event semantics, complete time axis data human-computer interaction output, and feedback interaction instruction to time axis basic framework.The application realizes the full cycle, high accuracy, event level time dimension control to complex virtual-real fusion scene.

Description

Technical Field

[0001] This invention relates to the field of 3D engine technology, specifically to a method, device, and medium for controlling the timeline of a 3D engine that integrates virtual and real elements. Background Technology

[0002] With the rapid development of virtual reality, augmented reality, and 3D simulation technologies, 3D engines have been widely used in games, film and television, industrial design, and education and training. Timeline control, as a core function of 3D engines, is used to manage time-related events in virtual-real fusion scenes, such as animation sequences, data synchronization, and interactive responses.

[0003] However, existing 3D engine timeline systems have many limitations:

[0004] First, most systems only support linear time control, failing to effectively integrate multiple time states such as real-time data, historical playback, and predictive simulations, resulting in inefficient time management in complex scenarios. Traditional timelines employ a single-track linear structure, unable to simultaneously process multi-dimensional time data from physical simulations, IoT devices, user operations, and future predictions.

[0005] Secondly, existing technologies lack event-level visualization, making it difficult for users to intuitively edit and predict events. Traditional interfaces use purely numerical scales, preventing users from directly dragging, zooming, and clipping time events through graphical operations, thus reducing work efficiency.

[0006] Third, insufficient time synchronization accuracy in virtual-real fusion scenarios affects user experience and simulation realism. In scenarios involving real-time data streams (such as sensor data and video streams), issues such as network jitter and inconsistent sampling frequencies lead to data loss or misalignment, and existing technologies lack effective data compensation mechanisms.

[0007] Fourth, existing systems lack predictive and extrapolation capabilities, and cannot simulate future states based on historical data. In scenarios such as emergency drills and equipment failure early warnings, users need to anticipate possible future situations, but traditional engines only support the replay of events that have already occurred and lack forward-looking analysis capabilities.

[0008] For example, while mainstream engines such as Unity and Unreal Engine provide basic timeline tools, they cannot dynamically integrate real-time data streams and predictive models, which limits their application in scenarios such as emergency drills and historical reenactments.

[0009] The above-mentioned defects urgently need to be addressed. Summary of the Invention

[0010] To overcome the technical problems of existing 3D engine timeline control functions such as limited functionality, inability to effectively integrate multi-source heterogeneous data, lack of predictive and extrapolation capabilities, and non-intuitive interactive operation, this invention provides a virtual-real fusion 3D engine timeline control method, device, and medium. By constructing a three-state unified timeline model including real-time, historical, and predictive states, and combining adaptive interpolation algorithms, machine learning prediction models, and LOD (Level of Detail) dynamic rendering technology, it achieves full-cycle, high-precision, event-level time dimension control for complex virtual-real fusion scenes.

[0011] The technical solution of this invention is as follows:

[0012] In a first aspect, the present invention provides a method for controlling the timeline of a 3D engine that integrates virtual and real elements, comprising:

[0013] Based on the virtual-real fusion scenario, a time axis model for multi-dimensional data mapping is constructed, and a multi-track layer time axis basic framework including real-time state, historical state and prediction state is established to form an initial time axis model.

[0014] Based on the initial timeline model, the spatiotemporal alignment processing of bidirectional synchronous real-time heterogeneous data streams and historical data is performed to fill the timeline with multi-source data and construct a complete timeline dataset covering the past and current time domains.

[0015] Based on the complete timeline dataset, event prediction and simulation based on time-series deep learning are performed, extending the timeline to the future time domain, forming a visualized simulation timeline covering the past, present, and future time domains;

[0016] Based on the aforementioned visual simulation timeline, the system enables visual interaction based on event semantics and real-time rendering of dynamic scenes, completes the human-computer interaction output of timeline data, and feeds back the interaction commands to the timeline basic framework.

[0017] Furthermore, the timeline model for constructing multi-dimensional data mapping based on the virtual-real fusion scenario establishes a multi-track hierarchical timeline framework including real-time, historical, and predictive states, forming an initial timeline model, including:

[0018] Parse the metadata of the virtual-real fusion scene to obtain scene description information;

[0019] Call the application programming interface of the 3D engine to initialize the timeline basic framework based on the scene description information;

[0020] By using a hash mapping algorithm, an index association between time nodes and scene object attributes is established within the timeline framework, forming an addressable time node network.

[0021] Deploy an event-driven mechanism to dynamically assemble keyframe events based on the addressable time node network;

[0022] Configure LOD technology to optimize the storage weight allocation of the time nodes based on the distribution characteristics of the keyframe events;

[0023] By integrating the timeline framework, the addressable time node network, dynamically assembled keyframe events, and optimized storage weight configuration, the initial timeline model is output.

[0024] Furthermore, the deployment of the event-driven mechanism, based on the addressable time node network, enables the dynamic assembly of keyframe events, including:

[0025] Deploy an event-driven mechanism;

[0026] Monitor scene trigger signals, which include at least device alarm signals and threshold exceedance signals;

[0027] Based on the addressable time node network, keyframe events are automatically generated and inserted into the corresponding time nodes at the triggering time, realizing the dynamic assembly of keyframe events.

[0028] Furthermore, the LOD configuration technology optimizes the storage weight allocation of the time nodes based on the distribution characteristics of the keyframe events, including:

[0029] Configure LOD technology;

[0030] Analyze the distribution characteristics of the keyframe events to identify high-frequency changing regions and low-frequency stable regions;

[0031] Differential storage weights are assigned to nodes with different time precisions based on the distribution characteristics; the differential storage weights include at least allocating full state snapshot storage resources to high-frequency changing regions and incremental differential storage resources to low-frequency stable regions.

[0032] Furthermore, the multi-track hierarchy includes at least a physical simulation track, an IoT data track, a user interaction track, and a prediction and inference track, which correspond to the data carrying requirements of the real-time state, historical state, and prediction state, respectively.

[0033] Furthermore, based on the initial timeline model, the process of performing bidirectional synchronous real-time heterogeneous data stream and historical data spatiotemporal alignment processing is carried out to fill the timeline with multi-source data, constructing a complete timeline dataset covering both past and current time domains, including:

[0034] Based on the IoT data track configuration in the initial timeline model, a real-time data access channel is constructed.

[0035] Deploy a multi-protocol adapter; wherein the multi-protocol adapter includes at least an MQTT protocol adapter, a WebSocket protocol adapter, and an HTTP / 2 protocol adapter, each corresponding to different data source types and network environments;

[0036] Through the real-time data access channel, the multi-protocol adapter is invoked to subscribe to multi-source heterogeneous data and receive discrete data packets;

[0037] The discrete data packets are mapped to the current time pointer position and synchronously filled into the IoT data track to form a real-time data stream;

[0038] Monitor the data loss situation of the real-time data stream, call the adaptive interpolation algorithm to perform interpolation filling on the missing data frames, and output the interpolated real-time data stream;

[0039] Based on the physical simulation track configuration in the initial timeline model, a hierarchical index historical data playback mechanism is established to obtain a historical data access interface.

[0040] Historical data is loaded through the historical data access interface, data optimization processing is performed, and optimized historical time series data is generated.

[0041] Analyze the optimized historical time series data, identify key feature points and mark them as backtracking anchor points;

[0042] The real-time data stream after interpolation and padding is integrated with the optimized historical time series data to construct the complete timeline dataset.

[0043] Furthermore, based on the complete timeline dataset, the process of performing event prediction and simulation based on time-series deep learning extends the timeline to include the future time domain, forming a visualized simulation timeline covering the past, present, and future time domains.

[0044] Based on the predicted trajectory configuration in the initial time axis model, a prediction engine embedded in the time axis system is constructed.

[0045] The prediction engine is invoked to extract spatiotemporal feature vectors from the complete timeline dataset;

[0046] The spatiotemporal feature vector set is input into a time series prediction model to analyze the periodicity, trend, and suddenness of the data.

[0047] Based on the analysis results of the time series prediction model, virtual time nodes are generated and filled into the prediction and extrapolation trajectory to form prediction data in the future time domain.

[0048] The Monte Carlo simulation method is invoked to perform multiple rounds of iterative verification on the predicted data in the future time domain, and the confidence interval and confidence level of the prediction results are calculated.

[0049] Based on the stated confidence level, high-confidence predicted events are selected and rendered into the future domain of the timeline to form the visualized simulation timeline.

[0050] Furthermore, based on the visualized simulation timeline, the system achieves event-semantic-based visual interaction and real-time dynamic scene rendering, completes the human-computer interaction output of timeline data, and feeds back the interaction commands to the timeline framework, including:

[0051] Based on the visualized simulation timeline, a graphical timeline interface is rendered, and event blocks are generated within the graphical timeline interface as basic operation units.

[0052] The graphical timeline interface receives user input commands, performs editing operations on the event blocks, and outputs a set of edited event blocks and an operation log.

[0053] The event block set and the operation log are fed back to the timeline base framework to dynamically optimize the framework configuration parameters.

[0054] In a second aspect, the present invention provides an electronic device, comprising:

[0055] At least one processor; and,

[0056] A memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described virtual-real fusion 3D engine timeline control method.

[0058] Thirdly, the present invention provides a computer-readable storage medium, including a data storage area and a program storage area, wherein the data storage area stores created data and the program storage area stores a computer program; wherein, when the computer program is executed by a processor, it implements the virtual-real fusion three-dimensional engine timeline control method as described above.

[0059] According to the above-described solution, the beneficial effects of this invention are as follows:

[0060] 1. This invention breaks through the limitations of the traditional 3D engine's fragmented functional modules and realizes a complete technical closed loop from basic architecture to user interaction;

[0061] 2. This invention achieves independent management and collaborative processing of four types of time data through a hierarchical design of physical simulation track, IoT data track, user interaction track and prediction and inference track, which significantly improves the efficiency of time management in complex scenarios.

[0062] 3. This invention achieves millisecond-level data hole filling through adaptive interpolation algorithm and multi-level threshold strategy, ensuring spatiotemporal alignment between real-time data and historical data, and significantly improving the data synchronization accuracy in virtual-real fusion scenarios;

[0063] 4. This invention enables the system to have proactive analysis and inference capabilities through an embedded machine learning prediction engine and Monte Carlo verification mechanism, and supports hypothesis-verification interactive decision-making by combining parameter inversion function.

[0064] 5. By feeding interactive instructions back to the timeline framework, this invention achieves dynamic optimization of parameters such as LOD level and event-driven sensitivity, forming a closed-loop control for human-machine collaboration, which significantly improves user experience and system adaptability. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating a method for controlling the timeline of a 3D engine that integrates virtual and real elements, according to an embodiment of the present invention.

[0066] Figure 2 for Figure 1 A detailed flowchart of step S1 is shown below;

[0067] Figure 3 for Figure 1 A detailed flowchart of step S2 is shown below;

[0068] Figure 4 for Figure 1 A detailed flowchart of step S3 is shown below;

[0069] Figure 5 for Figure 1 A detailed flowchart of step S4 is shown below;

[0070] Figure 6 This is a schematic diagram of the structure of a 3D engine time axis control device for virtual-real fusion in one embodiment of the present invention;

[0071] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0072] To make the technical problems to be solved, the technical solutions, and the beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0073] It should be noted that the terms "comprising" and "having," and any variations thereof, in the specification and claims of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0074] like Figures 1 to 5 As shown, one embodiment of the present invention provides a timeline control method for a 3D engine that integrates virtual and real elements, characterized in that it includes:

[0075] Step S1: Construct a multi-dimensional data mapping timeline model based on the virtual-real fusion scenario, establishing a multi-track hierarchical timeline framework including real-time, historical, and predictive states, forming the initial timeline model. The multi-track hierarchy includes a physical simulation track, an IoT data track, a user interaction track, and a prediction and extrapolation track. The physical simulation track carries historical physical simulation data, supporting the storage and playback of calculation results from rigid body dynamics, fluid simulation, etc.; the IoT data track carries real-time sensor data, supporting high-concurrency access to multi-source heterogeneous data; the user interaction track carries user operation commands, supporting event-driven interactive responses; and the prediction and extrapolation track carries future prediction data, supporting simulation and extrapolation results based on machine learning. This timeline framework breaks through the limitations of traditional single-track linear timelines, achieving parallel processing and independent management of four types of time data.

[0076] This step specifically includes the following sub-steps:

[0077] Step S11: Parse the metadata of the virtual-real fusion scene to obtain scene description information. The metadata includes a three-dimensional spatial coordinate system, the hierarchical relationship of physical objects, and external data source identifiers. The three-dimensional spatial coordinate system is used to determine the spatial location reference. The hierarchical relationship of physical objects is used to describe the organizational topology of scene objects. The external data source identifiers include sensor identifiers (such as IoT device IDs) and video stream Uniform Resource Locators (URLs) used to establish a mapping relationship with external data sources. The scene description information includes spatial reference parameters, object topology structure, and data source mapping relationship.

[0078] Step S12: Call the application programming interface of the 3D engine to initialize the basic framework of the time axis based on the scene description information. Specifically, the application programming interface of the 3D engine configures the time axis spatial reference system according to the spatial reference parameters and defines the orbital hierarchy structure according to the object topology to initialize the basic framework of the time axis.

[0079] Step S13: Establish an index association between time nodes and scene object attributes within the timeline framework using a hash mapping algorithm to form an addressable time node network. Each time node records a timestamp and encapsulates the corresponding full scene state snapshot, including the object's position, rotation, scaling transformation matrix, material properties, physical parameters, etc.

[0080] Step S14: Deploy the event-driven mechanism to dynamically assemble keyframe events based on an addressable time node network. Specifically, this involves: registering listeners to monitor scene trigger signals, which include at least device alarm signals and threshold exceedance signals; and automatically generating keyframe events and inserting them into the corresponding time nodes at the trigger time based on the addressable time node network to achieve dynamic assembly of keyframe events.

[0081] Step S15: Configure LOD (Level of Detail) technology to optimize the storage weight allocation of time nodes based on the distribution characteristics of key frame events. Specifically, analyze the distribution characteristics of key frame events to identify high-frequency change areas and low-frequency stable areas; allocate differentiated storage weights to nodes with different time precisions based on the distribution characteristics: allocate complete state snapshot storage resources to high-frequency change areas and incremental differential storage resources to low-frequency stable areas, thereby achieving a balance between data integrity and storage efficiency.

[0082] Step S16: Integrate the basic timeline framework, addressable time node network, dynamically assembled keyframe events, and optimized storage weight configuration to output an initial timeline model. This initial timeline model has data carrying capacity and efficient index scheduling performance, providing the infrastructure for data filling in subsequent steps.

[0083] Step S2: Based on the initial timeline model, perform spatiotemporal alignment processing of bidirectional synchronous real-time heterogeneous data streams and historical data, fill the timeline with multi-source data, and construct a complete timeline dataset covering the past and current time domains;

[0084] This step specifically includes the following sub-steps (steps S21-S25 are for real-time data, and steps S26-S29 are for historical data):

[0085] Step S21: Based on the IoT data track configuration in the initial time axis model, construct a real-time data access channel. Specifically, based on the initial time axis model, extract the interface configuration parameters of the IoT data track and construct a high-concurrency real-time data access channel based on a circular buffer. The circular buffer is set with a fixed number of slots, each slot stores one time slice of data, and the write pointer is implemented by atomic operation to achieve circular increment.

[0086] Step S22: Deploy the multi-protocol adapter; wherein, the multi-protocol adapter includes an MQTT protocol adapter, a WebSocket protocol adapter, and an HTTP / 2 protocol adapter, which correspond to different data source types and network environments respectively; the MQTT protocol adapter is suitable for low-bandwidth and high-latency scenarios of IoT devices and supports QoS level configuration; the WebSocket protocol adapter is suitable for bidirectional real-time communication scenarios and supports persistent connections; the HTTP / 2 protocol adapter is suitable for traditional industrial data sources such as SCADA systems and supports multiplexing.

[0087] Step S23: Through the real-time data access channel, call the multi-protocol adapter to subscribe to multi-source heterogeneous data and receive discrete data packets. Multi-source heterogeneous data includes SCADA system data, environmental monitoring data, and video stream data.

[0088] Step S24: Map the discrete data packets to the current time pointer position, and associate them with the time node index of the initial time axis model, and synchronously fill them into the IoT data track to form a real-time data stream.

[0089] Step S25: Monitor the data missing status of the real-time data stream, call the adaptive interpolation algorithm to perform interpolation filling or anomaly labeling on the missing data frames, and output the real-time data stream after interpolation filling and quality assurance. Specifically, the adaptive interpolation algorithm selects the interpolation strategy based on the duration of data missing: when the missing duration is less than the first threshold (e.g., 100ms), linear interpolation is used, which has low computational overhead and low latency; when the missing duration is between the first and second thresholds (e.g., 100ms~1000ms), cubic spline interpolation is used to ensure smoothness and continuous derivatives; when the missing duration is greater than the second threshold (e.g., 1000ms), data holes are marked and an alarm is triggered, indicating a data quality problem.

[0090] Step S26: Based on the physical simulation track configuration in the initial timeline model, establish a hierarchical index historical data playback mechanism and obtain a historical data access interface. Specifically, based on the initial timeline model, extract the storage configuration parameters of the physical simulation track, establish a hierarchical index historical data playback mechanism, and obtain a historical data access interface that supports efficient retrieval. The hierarchical index includes a time range index based on a B+ tree structure and a precise time point index based on hash mapping.

[0091] Step S27: Load historical data through the historical data access interface, perform data optimization processing, and generate optimized historical time series data. Specifically, load the original historical data through the historical data access interface; perform denoising and cleaning on the original historical data, using the Kalman filter algorithm to remove sensor noise; perform redundancy removal on the denoised historical data, using sliding window detection to remove duplicate records; use the incremental compression algorithm to generate optimized historical time series data, where the incremental compression algorithm only stores the difference values ​​between adjacent time slices; and load the optimized historical time series data into the physical simulation track.

[0092] Step S28: Analyze the optimized historical time series data, identify key feature points and mark them as backtracking anchor points. Specifically, analyze the optimized historical time series data, identify key feature points, including mutation points and extreme points. Mutation points are identified by detecting that the first derivative exceeds a preset threshold, and extreme points are identified by detecting that the second derivative changes sign. Mark the key feature points as high-priority backtracking anchor points and establish an anchor point index to support rapid location.

[0093] Step S29: Integrate the real-time data stream after interpolation and the optimized historical time series data. The real-time data stream after interpolation corresponds to the current time domain, and the optimized historical time series data corresponds to the past time domain, thus constructing a complete time axis dataset.

[0094] Through the above-mentioned steps of synchronizing real-time and historical data, a complete timeline dataset covering both past and current time domains is constructed, providing a data foundation for subsequent prediction and extrapolation.

[0095] Step S3: Based on the complete timeline dataset, perform event prediction and simulation based on time-series deep learning, extend the future time domain to the timeline, and form a visualized simulation timeline covering the past time domain, the current time domain, and the future time domain.

[0096] This step specifically includes the following sub-steps:

[0097] Step S31: Based on the prediction trajectory configuration in the initial time axis model, construct a prediction engine embedded in the time axis system. Specifically, based on the initial time axis model, extract the interface configuration parameters of the prediction trajectory and construct a prediction engine embedded in the time axis system. The prediction engine adopts an encoder-decoder architecture and is embedded in the time axis system to achieve seamless data exchange with the data trajectory. Initialize the model loading interface and inference calculation unit of the prediction engine to obtain a prediction engine with spatiotemporal feature extraction and future state inference capabilities.

[0098] Step S32: Call the prediction engine to extract spatiotemporal feature vectors from the complete timeline dataset. Specifically, call the data preprocessing interface of the prediction engine to load historical time series data covering the past and current time domains from the complete timeline dataset; use a sliding window mechanism to extract spatiotemporal feature vectors. The window length of the sliding window is dynamically adjusted periodically according to the data to ensure the temporal relevance of the extracted features; finally, output a standardized set of spatiotemporal feature vectors.

[0099] Step S33: Input the spatiotemporal feature vector set into the time series prediction model to analyze the periodicity, trend, and suddenness of the data. Specifically, the spatiotemporal feature vector set is input into the time series prediction model, which adopts a long short-term memory network or a Transformer architecture, respectively suitable for long-term dependency modeling and complex attention mechanism scenarios. The time series prediction model performs forward inference calculation to analyze the periodicity, trend, and suddenness of the data, and outputs the data pattern analysis results and the model's hidden state parameters.

[0100] Step S34: Based on the analysis results of the time series prediction model, generate virtual time nodes and fill them into the prediction and extrapolation trajectory to form prediction data in the future time domain. Specifically, based on the data pattern analysis results and the model's hidden state parameters, perform sequence generation inference; generate a series of virtual time nodes in the direction of the future time axis. The virtual time nodes represent the possible future states calculated by the system, including the predicted equipment state evolution curves and the simulated physical process trajectories; fill the virtual time nodes into the prediction and extrapolation trajectory to form prediction data in the future time domain.

[0101] Step S35: Call the Monte Carlo simulation method to perform multiple rounds of iterative verification on the prediction data in the future time domain, and calculate the confidence interval and confidence level of the prediction results. Specifically, call the Monte Carlo simulation method to inject Gaussian noise into the network parameters of the time series prediction model to generate multiple sets of perturbation model parameters; perform independent prediction inference based on each set of perturbation model parameters to obtain multiple sets of prediction result samples; statistically analyze the distribution characteristics of multiple sets of prediction result samples, calculate the confidence interval and confidence level of the prediction results, and output a set of prediction results with confidence level labels.

[0102] Step S36: Filter high-confidence predicted events based on confidence levels and render them to the future domain of the timeline to form a visual simulation timeline. Specifically: Set a confidence threshold, filter predicted events that meet the confidence threshold based on the confidence level, and mark them as high-confidence predicted events; render the high-confidence predicted events to the future domain of the timeline, using visual encoding different from historical and real-time data, presented in a dashed projection or semi-transparent overlay manner; integrate historical data from the past time domain, real-time data from the current time domain, and predicted data from the future time domain to form a visual simulation timeline covering the past, present, and future time domains.

[0103] Step S37: Deploy the parameter inversion interface to establish a real-time feedback channel between user input and the prediction engine. Specifically, this involves: deploying the parameter inversion interface, which connects the user interaction layer and the prediction engine's input layer; receiving parameter adjustment instructions from the user through the parameter inversion interface, which include at least device load adjustment, environmental condition modification, or constraint change; injecting the parameter adjustment instructions into the prediction engine to trigger the re-execution of the prediction inference process in sub-steps S33-S36; updating the prediction curve in real time and rendering it to the future domain time axis to achieve hypothesis-verification interactive analysis; and finally outputting the optimized prediction results to achieve the technical goal of extending the future time domain to the time axis.

[0104] Through the above operations, a visualized simulation timeline covering the past, present, and future time domains is formed, providing a complete data foundation for subsequent interactive outputs.

[0105] Step S4: Based on the visualized simulation timeline, realize visualized interaction and dynamic scene rendering based on event semantics, complete the human-computer interaction output of timeline data, and feed back the interaction instructions to the timeline basic framework.

[0106] This step specifically includes the following sub-steps:

[0107] Step S41: Based on the visualized simulation timeline, render a graphical timeline interface (GUI) and generate event blocks within the GUI as basic operation units. Specifically: Based on the visualized simulation timeline, extract the visualization attribute parameters of the three-domain data and render a semantically aware GUI. The GUI adopts a layered layout structure, with the horizontal direction representing the timeline and the vertical direction displaying the physical simulation track, IoT data track, user interaction track, and prediction and inference track in layers. Event blocks are generated within the GUI as basic operation units. The visual attributes of the event blocks include the width representing the duration, the height representing the importance level, and the color representing the event type and time domain affiliation. A two-way binding relationship is established between the event blocks and the underlying time nodes. The two-way binding relationship ensures real-time synchronization between interface operations and data status, and outputs a set of semantically meaningful event blocks with operability.

[0108] Step S42: Receive user-inputted interactive commands through the graphical timeline interface, perform editing operations on event blocks, and output a set of edited event blocks and an operation log. Specifically: Receive user-inputted interactive commands through the input interface of the graphical timeline interface. Interactive commands include mouse events, touch gestures, or virtual reality controller signals; parse the operation type and operation object of the interactive commands; perform editing operations on the event blocks. Editing operations include dragging (to adjust the time position or change the event type across tracks), zooming (to adjust the duration), clipping (to split event blocks), merging (to merge adjacent event blocks), and two-finger zooming (to adjust the timeline display ratio); update the status attributes of the event blocks and the data parameters of the corresponding time nodes, and output a set of edited event blocks and an operation log.

[0109] Step S43: Feed back the event block set and operation logs to the timeline base framework and dynamically optimize the framework configuration parameters. Specifically, this involves: transmitting the event block set and operation logs to the timeline base framework through the feedback channel; parsing user behavior patterns in the operation logs, including high-frequency operation area identification, event type preference distribution, and time granularity adjustment frequency; dynamically optimizing the framework configuration parameters based on the user behavior patterns, including: adjusting the LOD level to adapt to the rendering load, adjusting the trigger threshold of the event-driven mechanism to adapt to the event density, and adjusting the bucket size of the hash mapping algorithm to adapt to the data scale; finally, outputting the optimized timeline base framework configuration.

[0110] Through the above steps, the full-cycle visualization of timeline data in a virtual-real fusion scenario is achieved, and a technical closed loop is formed through a feedback mechanism.

[0111] The virtual-real fusion 3D engine timeline control method in this embodiment breaks through the limitations of the traditional 3D engine's fragmented functional modules, realizing a complete technical closed loop from basic architecture to user interaction. Through the hierarchical design of physical simulation track, IoT data track, user interaction track, and prediction and inference track, it achieves independent management and collaborative processing of four types of time data, significantly improving time management efficiency in complex scenarios. Through adaptive interpolation algorithms and multi-level threshold strategies, it achieves millisecond-level data hole filling, ensuring spatiotemporal alignment between real-time data and historical data, significantly improving data synchronization accuracy in virtual-real fusion scenarios. Through the embedded machine learning prediction engine and Monte Carlo verification mechanism, the system has the ability to actively analyze and infer, and combined with parameter inversion function, it supports hypothesis-verification interactive decision-making. By feeding interactive commands back to the timeline basic framework, it achieves dynamic optimization of parameters such as LOD level and event-driven sensitivity, forming a closed-loop control of human-machine collaboration, significantly improving user experience and system adaptability.

[0112] like Figure 6As shown, one embodiment of the present invention provides a timeline control device for a 3D engine that integrates virtual and real elements, comprising:

[0113] Model building module 1 constructs a timeline model for multi-dimensional data mapping based on the virtual-real fusion scenario, establishes a multi-track level timeline basic framework including real-time state, historical state, and prediction state, and forms an initial timeline model;

[0114] Data processing module 2, based on the initial timeline model, performs spatiotemporal alignment processing of bidirectional synchronous real-time heterogeneous data streams and historical data, fills the timeline with multi-source data, and constructs a complete timeline dataset covering the past and current time domains;

[0115] Prediction and simulation module 3, based on a complete timeline dataset, performs event prediction and simulation based on time series deep learning, extends the timeline to the future time domain, and forms a visualized simulation timeline covering the past time domain, the current time domain, and the future time domain;

[0116] The visualization interaction module 4, based on the visualization simulation timeline, realizes visualization interaction based on event semantics and real-time rendering of dynamic scenes, completes the human-computer interaction output of timeline data, and feeds back the interaction commands to the timeline basic framework.

[0117] The virtual-real fusion 3D engine time axis control device provided in this application embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0118] like Figure 7 The diagram shown is a structural schematic of an electronic device for implementing the virtual-real fusion three-dimensional engine time axis control method of the present invention.

[0119] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as a timeline control program for a virtual-real fusion 3D engine.

[0120] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a timeline control program for a virtual-real fusion 3D engine) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0121] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a 3D engine timeline control program for virtual-real fusion, but also to temporarily store data that has been output or will be output.

[0122] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0123] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0124] Figure 7 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 7 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0125] For example, a power supply, although not shown, may also include a power source (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power sources, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.

[0126] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0127] The memory 11 in the electronic device stores the timeline control program for the virtual-real fusion 3D engine, which is a combination of multiple computer programs.

[0128] For the specific implementation method of the above computer program by processor 10, please refer to Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0129] Furthermore, if the modules / units integrated into an electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, a computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0130] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following functions: Figure 1 The flowchart illustrates the timeline control method for a 3D engine that integrates virtual and real elements.

[0131] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0132] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0134] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0135] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0136] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0137] Furthermore, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0138] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

[0139] The present invention has been described above with reference to the accompanying drawings. Obviously, the implementation of the present invention is not limited to the above-described manner. Any improvements made using the inventive concept and technical solution of the present invention, or the direct application of the inventive concept and technical solution of the present invention to other situations without modification, are all within the protection scope of the present invention.

Claims

1. A method for controlling the timeline of a 3D engine that blends virtual and real elements, characterized in that, include: Based on the virtual-real fusion scenario, a time axis model for multi-dimensional data mapping is constructed, and a multi-track layer time axis basic framework including real-time state, historical state and prediction state is established to form an initial time axis model. Based on the initial timeline model, the spatiotemporal alignment processing of bidirectional synchronous real-time heterogeneous data streams and historical data is performed to fill the timeline with multi-source data and construct a complete timeline dataset covering the past and current time domains. Based on the complete timeline dataset, event prediction and simulation based on time-series deep learning are performed, extending the timeline to the future time domain, forming a visualized simulation timeline covering the past, present, and future time domains; Based on the aforementioned visual simulation timeline, the system enables visual interaction based on event semantics and real-time rendering of dynamic scenes, completes the human-computer interaction output of timeline data, and feeds back the interaction commands to the timeline basic framework.

2. The time axis control method for virtual-real fusion in a 3D engine according to claim 1, characterized in that, The timeline model based on the virtual-real fusion scenario constructs a multi-dimensional data mapping, establishing a multi-track hierarchical timeline framework including real-time, historical, and predictive states, forming an initial timeline model, including: Parse the metadata of the virtual-real fusion scene to obtain scene description information; Call the application programming interface of the 3D engine to initialize the timeline basic framework based on the scene description information; By using a hash mapping algorithm, an index association between time nodes and scene object attributes is established within the timeline framework, forming an addressable time node network. Deploy an event-driven mechanism to dynamically assemble keyframe events based on the addressable time node network; Configure LOD technology to optimize the storage weight allocation of the time nodes based on the distribution characteristics of the keyframe events; By integrating the timeline framework, the addressable time node network, dynamically assembled keyframe events, and optimized storage weight configuration, the initial timeline model is output.

3. The time axis control method for virtual-real fusion in a 3D engine according to claim 2, characterized in that, The deployment event-driven mechanism, based on the addressable time node network, enables the dynamic assembly of keyframe events, including: Deploy an event-driven mechanism; Monitor scene trigger signals, which include at least device alarm signals and threshold exceedance signals; Based on the addressable time node network, keyframe events are automatically generated and inserted into the corresponding time nodes at the triggering time, realizing the dynamic assembly of keyframe events.

4. The time axis control method for virtual-real fusion in a 3D engine according to claim 2, characterized in that, The LOD configuration technology optimizes the storage weight allocation of the time nodes based on the distribution characteristics of the keyframe events, including: Configure LOD technology; Analyze the distribution characteristics of the keyframe events to identify high-frequency changing regions and low-frequency stable regions; Differential storage weights are assigned to nodes with different time precisions based on the distribution characteristics; the differential storage weights include at least allocating full state snapshot storage resources to high-frequency changing regions and incremental differential storage resources to low-frequency stable regions.

5. The time axis control method for virtual-real fusion in a 3D engine according to claim 1, characterized in that, The multi-track hierarchy includes at least a physical simulation track, an IoT data track, a user interaction track, and a prediction and inference track, which correspond to the data carrying requirements of the real-time state, historical state, and prediction state, respectively.

6. The time axis control method for virtual-real fusion in a 3D engine according to claim 5, characterized in that, Based on the initial timeline model, the process involves performing bidirectional synchronous real-time heterogeneous data stream and historical data spatiotemporal alignment processing to fill the timeline with multi-source data, constructing a complete timeline dataset covering both past and current time domains, including: Based on the IoT data track configuration in the initial timeline model, a real-time data access channel is constructed. Deploy a multi-protocol adapter; wherein the multi-protocol adapter includes at least an MQTT protocol adapter, a WebSocket protocol adapter, and an HTTP / 2 protocol adapter, each corresponding to different data source types and network environments; Through the real-time data access channel, the multi-protocol adapter is invoked to subscribe to multi-source heterogeneous data and receive discrete data packets; The discrete data packets are mapped to the current time pointer position and synchronously filled into the IoT data track to form a real-time data stream; Monitor the data loss situation of the real-time data stream, call the adaptive interpolation algorithm to perform interpolation filling on the missing data frames, and output the interpolated real-time data stream; Based on the physical simulation track configuration in the initial timeline model, a hierarchical index historical data playback mechanism is established to obtain a historical data access interface. Historical data is loaded through the historical data access interface, data optimization processing is performed, and optimized historical time series data is generated. Analyze the optimized historical time series data, identify key feature points and mark them as backtracking anchor points; The real-time data stream after interpolation and padding is integrated with the optimized historical time series data to construct the complete timeline dataset.

7. The time axis control method for virtual-real fusion in a 3D engine according to claim 5, characterized in that, Based on the complete timeline dataset, the process involves performing event prediction and simulation based on time-series deep learning, extending the timeline to the future time domain, and forming a visualized simulated timeline covering the past, present, and future time domains. This includes: Based on the predicted trajectory configuration in the initial time axis model, a prediction engine embedded in the time axis system is constructed. The prediction engine is invoked to extract spatiotemporal feature vectors from the complete timeline dataset; The spatiotemporal feature vector set is input into a time series prediction model to analyze the periodicity, trend, and suddenness of the data. Based on the analysis results of the time series prediction model, virtual time nodes are generated and filled into the prediction and extrapolation trajectory to form prediction data in the future time domain. The Monte Carlo simulation method is invoked to perform multiple rounds of iterative verification on the predicted data in the future time domain, and the confidence interval and confidence level of the prediction results are calculated. Based on the stated confidence level, high-confidence predicted events are selected and rendered into the future domain of the timeline to form the visualized simulation timeline.

8. The time axis control method for virtual-real fusion in a 3D engine according to claim 5, characterized in that, The process, based on the visualized simulation timeline, enables event-semantic-based visual interaction and real-time dynamic scene rendering, completes the human-computer interaction output of timeline data, and feeds back interaction commands to the timeline infrastructure framework, including: Based on the visualized simulation timeline, a graphical timeline interface is rendered, and event blocks are generated within the graphical timeline interface as basic operation units. The graphical timeline interface receives user input commands, performs editing operations on the event blocks, and outputs a set of edited event blocks and an operation log. The event block set and the operation log are fed back to the timeline base framework to dynamically optimize the framework configuration parameters.

9. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the virtual-real fusion three-dimensional engine time axis control method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, It includes a data storage area and a program storage area. The data storage area stores the created data, and the program storage area stores the computer program. When the computer program is executed by the processor, it implements the virtual-real fusion three-dimensional engine time axis control method as described in any one of claims 1 to 8.

Images