Reproduction method and system for accessing Space Timeline by using XR terminal

Abstract

The invention discloses a reproduction method and system for accessing Space Timeline by using an XR terminal, and the method comprises the steps: collecting multi-dimensional interaction data of a user executing an operation process in a virtual scene through a multi-mode perception interface and a user-defined event script of XR equipment, and fusing the multi-dimensional interaction data into a structured operation spectrum file *. STimeline; based on the file *. STimeline, automatically constructing an immersive operation scene which has the same structure as the original operation scene and contains an interaction anchor point in the target XR equipment; and the user performs multi-view observation, interactive practice and test operation in the scene, and obtains objective judgment and feedback based on the intelligent analysis module. According to the method, intelligent acquisition, immersion reproduction and data-driven optimization of the operation process in the XR environment are realized, and finally the immersion learning and training effects are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual technology, and in particular to a reproduction method for accessing a Space Timeline using an XR terminal. Background Art

[0002] With the development of extended reality (XR) technologies such as virtual reality (VR) and augmented reality (AR), immersive interactive applications have been widely used in education and training, manufacturing, medical care and other fields. Among them, an important demand is to capture / record, reproduce and optimize the operation process in the real world in the XR environment to achieve immersive learning and training.

[0003] Existing solutions are mainly based on technologies such as motion capture, trajectory recording, and video recording to collect and reproduce real operations in XR scenarios. For example, data gloves and inertial sensors are used to collect hand movements, generate virtual hand models in VR environments, and implement visual teaching of simple operations. Patent US9652037B2 discloses a method of using a somatosensory device to record full-body movements and reproduce them in real time on a virtual portrait.

[0004] However, these solutions mainly focus on the collection and reproduction of the operation actions themselves, ignoring the multi-dimensional interaction between the user and the virtual environment during the operation. Specifically, they lack analysis and representation of user behavior intentions, making it difficult to describe the semantics and rules of the operation; they do not fully utilize the multi-channel perception and feedback capabilities of the XR environment, making it impossible to provide an immersive experience; and the reproduction process lacks targeted guidance and evaluation, making it difficult to help users effectively master and internalize operation skills.

[0005] In addition, existing solutions often rely on specific hardware and software platforms, lack standardized representation of operation data, and are difficult to share and reuse across platforms. Therefore, a new technical solution is urgently needed that can comprehensively collect multi-dimensional interactive data of user operations in the XR environment, intelligently build immersive operation scenarios, guide users to observe, practice, and evaluate the operation process, and provide objective judgment and feedback, ultimately significantly improving the effect of immersive learning and training. Summary of the invention

[0006] In view of the above technical problems, the present invention provides a reproduction method and system for accessing Space Timeline using an XR terminal.

[0007] The first embodiment of the present invention provides a reproduction method for accessing Space Timeline using an XR terminal, the method comprising: collecting multi-dimensional interactive data of a user's operation process in a virtual scene through a multimodal perception interface and a custom event script of the XR device, and fusing the data into a structured operation spectrum file *.sTimeline; based on the file *.sTimeline, automatically constructing an immersive operation scene containing interactive anchor points with the same structure as the original operation scene in a target XR device; the user observes from multiple perspectives, interactively practices and tests operations in the scene, and obtains objective judgment and feedback based on an intelligent analysis module.

[0008] Optionally, the step of collecting multi-dimensional interactive data of the user's operation process in the virtual scene includes: continuously recording data of the user's head posture, hand movements, and voice commands, adding timestamps, and converting them into a standard format.

[0009] Optionally, the step of automatically constructing an immersive operation scene containing interaction anchors with the same structure as the original operation scene in the target XR device based on the file *.sTimeline includes: parsing the file *.sTimeline, extracting the key events and decision nodes recorded therein, and mapping them as interaction anchors and branch anchors in the virtual scene respectively, each anchor carrying corresponding data metadata, and the metadata including at least timestamp, spatial coordinates, associated objects, and interaction type; connecting all anchor points into a directed graph in order of timestamps, and the directed graph is called an operation flow graph, which is used to intuitively represent the spatiotemporal trajectory and decision branches of the operation; automatically reconstructing the virtual environment according to the environmental data in the *.sTimeline, and binding the operation flow graph anchors to the corresponding virtual objects.

[0010] Optionally, templated UI elements are automatically added to the operation flow chart, wherein the UI elements include text labels, icons, progress bars for indicating the operation process, and road signs and signboards for identifying decision branches.

[0011] Optionally, the steps of the user observing from multiple perspectives, interactively practicing and testing operations in the scene, and obtaining objective judgment and feedback based on the intelligent analysis module include: based on the scene timeline reconstruction, through observation mode, exploration mode, practice mode and assessment mode, supporting users to access and expand the operation process contained in SpaceTimeline in multiple dimensions.

[0012] Optionally, the method also includes: collecting the user's original operation data in the XR environment, generating an initial SpaceTimeline genealogy file, each original genealogy file is assigned a unique root identifier as an anchor point for subsequent incremental expansion; preprocessing the collected original genealogy data; different users, based on the original genealogy, open up new branch paths at key decision nodes through the exploration mode; performing multi-dimensional evaluation on the branch paths contributed by users, the evaluation including coverage, innovation, reliability, and universality; aligning and semantically matching the screened high-quality branches with the original genealogy in time and space, and merging them into the total genealogy in an incremental manner; when the total genealogy is continuously expanded and improved under the collaboration of crowdsourcing and reaches the preset saturation and stability, it is used as a new generation of original genealogy to start a new round of crowdsourcing optimization iterations.

[0013] Optionally, the preprocessing includes semantic extraction, noise filtering and data interpolation to improve data quality and consistency.

[0014] The second embodiment of the present invention also provides a reproduction system for accessing Space Timeline using an XR terminal, the system comprising: a multi-dimensional operation acquisition module, which collects multi-dimensional interactive data of the user's operation process in the virtual scene through the multi-modal perception interface and custom event script of the XR device, and merges it into a structured operation spectrum file *.sTimeline; an immersive scene construction module, which automatically constructs an immersive operation scene with the same structure as the original operation scene in the target XR device based on the file *.sTimeline; an interactive event detection module, in which the user observes from multiple perspectives, interactively practices and tests operations in the scene, and obtains objective judgment and feedback based on the intelligent analysis module.

[0015] Optionally, the interactive event detection module is also used to reconstruct the scene timeline, and to support users to access and expand the operation process contained in the SpaceTimeline in multiple dimensions through observation mode, exploration mode, practice mode and assessment mode.

[0016] Optionally, the system also includes a spatial crowdsourcing module for collecting users' original operation data in the XR environment and generating an initial SpaceTimeline genealogy file, each original genealogy file is assigned a unique root identifier as an anchor point for subsequent incremental expansion; preprocessing the collected original genealogy data; different users, based on the original genealogy, open up new branch paths at key decision nodes through the exploration mode; multi-dimensional evaluation of the branch paths contributed by users, the evaluation including coverage, innovation, reliability, and universality; spatial and temporal alignment and semantic matching of the screened high-quality branches with the original genealogy, and incrementally merging them into the total genealogy; when the total genealogy is continuously expanded and improved under the collaboration of crowdsourcing and reaches the preset saturation and stability, it is used as a new generation of original genealogy to start a new round of crowdsourcing optimization iteration.

[0017] Compared with the existing technology, the present invention comprehensively collects multi-dimensional interactive data of user operations in the XR environment, taking into account both continuous actions and discrete events, providing a data basis for comprehensive understanding and optimization of operations; constructing immersive operation scenarios, providing multi-perspective observation and interactive practice, and using XR's spatial telepresence to improve the effect of operation learning and training; integrating observation, practice, and assessment, introducing objective data analysis and feedback, and realizing precise and personalized operation skill training and evaluation. The present invention aims to realize intelligent collection, immersive reproduction, and data-driven optimization of the operation process in the XR environment, and significantly improve the efficiency and level of immersive operation training in various industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 The present invention is a structural schematic diagram of a reproduction system for accessing Space Timeline using an XR terminal.

[0019] Figure 2 The figure is a flow chart of one embodiment of a method for reproducing a Space Timeline using an XR terminal according to the present invention.

[0020] Figure 3 The present invention is a flowchart diagram of another embodiment of a method for reproducing a Space Timeline using an XR terminal.

[0021] Figure 4 The present invention is a flowchart diagram of another embodiment of a method for reproducing a Space Timeline using an XR terminal.

[0022] Figure 5 The present invention is a flowchart diagram of another embodiment of a method for reproducing a Space Timeline using an XR terminal. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] The present invention provides a reproduction system for accessing Space Timeline using an XR terminal, please refer to Figure 1 As shown, the system includes a multi-dimensional acquisition module, a scene construction module and an interactive event detection module, wherein the multi-dimensional acquisition module collects the multi-dimensional interactive data of the user's operation process in the virtual scene through the multi-modal perception interface of the XR device and the custom event script, and integrates it into a structured operation spectrum file *.sTimeline.

[0027] The scene construction module automatically builds an immersive operation scene with the same structure as the original operation scene in the target XR device based on the *.sTimeline file.

[0028] In the interactive event detection module, users can observe from multiple perspectives, interactively practice and test operations in the scene, and obtain objective judgment and feedback based on the intelligent analysis module.

[0029] In one embodiment of the present invention, the interactive event detection module is also used to reconstruct the scene timeline, and support users to access and expand the operation process contained in SpaceTimeline in multiple dimensions through observation mode, exploration mode, practice mode and assessment mode.

[0030] In one embodiment of the present invention, the system also includes a spatial crowdsourcing module, which is used to collect the user's original operation data in the XR environment and generate an initial SpaceTimeline genealogy file, each original genealogy file is assigned a unique root identifier as an anchor point for subsequent incremental expansion; the collected original genealogy data is preprocessed; different users, based on the original genealogy, open up new branch paths at key decision nodes through the exploration mode; the branch paths contributed by users are evaluated in multiple dimensions, and the evaluation includes coverage, innovation, reliability, and universality; the screened high-quality branches are aligned in time and space and semantically matched with the original genealogy, and are incrementally incorporated into the total genealogy; when the total genealogy is continuously expanded and improved under the collaboration of crowdsourcing and reaches the preset saturation and stability, it is used as a new generation of original genealogy to start a new round of crowdsourcing optimization iterations.

[0031] Please refer to Figure 2 , Figure 3 As shown, the present invention also provides a reproduction method for accessing Space Timeline using an XR terminal. The method is based on a reproduction system for accessing Space Timeline using an XR terminal. The method includes the following steps:

[0032] Step S10, through the multimodal perception interface and custom event script of the XR device, collect the multi-dimensional interactive data of the user's operation process in the virtual scene, and merge it into a structured operation spectrum file *.sTimeline.

[0033] The present invention collects the behavioral data of user operations in the XR environment in a multi-channel and multi-granular manner, and converts it into a structured and serialized operation representation through time and space multi-dimensional mapping, semantic extraction and other technologies. On this basis, the immersive rendering and interactive capabilities of XR devices are used to build an immersive operation observation and practice environment for users, and an intelligent analysis and feedback mechanism is introduced to achieve an immersive, interactive, and intelligent operation learning and training closed loop.

[0034] Compared with the existing technology, the present invention comprehensively collects multi-dimensional interactive data of user operations in the XR environment, taking into account both continuous actions and discrete events, providing a data basis for comprehensive understanding and optimization of operations; constructing immersive operation scenarios, providing multi-perspective observation and interactive practice, and using XR's spatial telepresence to improve the effect of operation learning and training; integrating observation, practice, and assessment, introducing objective data analysis and feedback, and realizing precise and personalized operation skill training and evaluation. The present invention aims to realize intelligent collection, immersive reproduction, and data-driven optimization of the operation process in the XR environment, and ultimately significantly improve the efficiency and level of immersive operation training in various industries.

[0035] The present invention utilizes the multimodal perception interface of the XR device to collect continuous interactive data of the user's operations in the virtual scene in real time. Through the HMD's 6DoF spatial tracking, hand tracking, voice recognition and other technologies, the user's head posture, hand movements, voice commands and other data are continuously recorded, timestamped and converted into a standard format. For mixed reality applications, the Spatial Mapping API can also be called to obtain the three-dimensional grid data of the real environment; the Image Tracking API can be called to identify the feature points and postures of real objects.

[0036] It is worth mentioning that the present invention also records the decision-making behavior data of users at key nodes, such as the direction selection at the branch selection point:

[0037] For example, in the assembly operation training of a certain equipment, key decision nodes may include: when installing the power module, whether to choose built-in battery or external adapter power supply; when connecting the motherboard and peripherals, whether to choose flexible screw fixation or integrated snap-on installation; when debugging the system, whether to check the hardware first and then test the software, or use parallel testing of hardware and software.

[0038] The present invention records the user's decision choices at these preset key nodes and compares them with the standard answers. At the same time, the corresponding operation branch paths are dynamically generated according to the user's choices. In this way, the decision data of the user's operation can be fully collected, including not only the continuous interaction and discrete events at the action execution level, but also the selection behavior at the strategic decision level.

[0039] By tracking and analyzing the user's decision-making patterns throughout the entire operation process, it is possible to evaluate their understanding and judgment of key decision points, identify possible knowledge blind spots and lack of experience, and provide targeted guidance and feedback. For example, if a user repeatedly chooses the wrong solution when assembling a power supply, the system can push relevant principle explanations and case analysis to help them consolidate this knowledge point.

[0040] In addition, the present invention aligns continuous data streams, discrete events, and decision-making behavior data on the time axis and integrates them into a unified *.sTimeline operation spectrum file. This file fully records the user's multi-channel operation and decision data in the XR environment in a universal time series data format.

[0041] The present invention utilizes the multi-channel sensing of XR devices to collect user behavior and decision-making data in all directions; integrates continuous data streams, semantic events, and key decisions, taking into account the integrity of the operation process, the interpretability of key nodes, and the diversity of decision paths; and introduces a universal time series data format to improve the interoperability and reuse value of operation data.

[0042] Step S20: Based on the *.sTimeline file, an immersive operation scene including an interaction anchor point having the same structure as the original operation scene is automatically constructed in the target XR device.

[0043] For details, please refer to Figure 4 As shown, step S20 specifically includes the following steps:

[0044] Step S21, parse the file *.sTimeline, extract the key events and decision nodes recorded therein, and map them to interaction anchor points and branch anchor points in the virtual scene respectively. Each anchor point carries corresponding data metadata, and the metadata includes at least timestamp, spatial coordinates, associated objects, and interaction type.

[0045] Step S22, connecting all anchor points into a directed graph in order of timestamps, the directed graph is called an operation flow graph, which is used to intuitively represent the spatiotemporal trajectory and decision branches of the operation.

[0046] Among them, the edges between ordinary anchor points carry semantic information such as timing and logical relationships; directed branch edges are connected between branch anchor points and their possible destination anchor points to represent different decision paths.

[0047] Step S23, automatically reconstruct the virtual environment according to the environment data in *.sTimeline, and bind the flow graph anchor points to the corresponding virtual objects. Users can manually edit the flow graph, add annotations, adjust the topology, etc.

[0048] Step S24, automatically attaching templated UI elements to the flow chart, wherein the UI elements include text labels, icons, progress bars for indicating the operation process, and road signs and signboards for identifying decision branches.

[0049] The present invention automatically maps nonlinear operation data to a virtual environment, achieving isomorphic reproduction of real operations in the target domain; introduces a flow graph structure to characterize the spatiotemporal characteristics, logical relationships, and decision branches of operations, supporting semantic-level navigation, retrieval, and deduction; and uses UI elements to prompt key information and decision paths, creating an immersive and wizard-like atmosphere.

[0050] Step S30, the user observes from multiple perspectives, interactively practices and tests operations in the scene, and obtains objective judgment and feedback based on the intelligent analysis module.

[0051] The present invention is based on scene timeline reconstruction, and supports users to access and expand the operation process contained in SpaceTimeline in multiple dimensions through observation mode, exploration mode, practice mode and assessment mode.

[0052] Observation mode: Load *.sTimeline data to drive the virtual Avatar to reproduce the operation. At the branch node, the system can select a path based on the strategy, such as the most commonly used path, random selection, or real-time user decision-making. The user adjusts the viewing angle and time progress controls in the XR terminal to grasp the overall context of the operation from a third-person perspective. The system forms a dynamic follow-up anchor point based on the spatial relationship between the user and the virtual Avatar. After the Avatar's action exceeds the threshold, the scene is adjusted in real time to adapt to the optimal rendering posture, creating an immersive observation experience of side-by-side observation and panoramic roaming. Among them, the branch node is the key decision point in the operation spectrum. The branch node connects multiple optional subsequent operation paths. The branch node reflects the diversity of operation plans. Typical branch node scenarios include assembly sequence selection, medical plan judgment, etc.

[0053] Observation mode: The system loads *.sTimeline data to drive the virtual Avatar to reproduce the operation. *.sTimeline uses a graph structure to model the operation process, which not only includes action sequence nodes, but also introduces branch nodes representing key decision points.

[0054] Branch nodes identify decision bifurcations in the operation process, revealing the choice space faced by the operation at that node. For example, in equipment assembly operations, branch nodes may correspond to the binary choice of "preferentially installing module A or module B"; in medical surgery operations, branch nodes may correspond to the multiple choices of "using incision X, incision Y, or incision Z". Branch nodes connect multiple optional subsequent operation paths, representing the diversity of operation plans.

[0055] Exploration mode: Users wear XR terminals and immerse themselves in the playback of *.sTimeline operation clips. At branch nodes, users can interactively choose to follow the original path or open up new exploration directions. When the exploration decision is triggered, the system will terminate the *.sTimeline playback and instead record the user's multi-channel interaction data on the new path in real time. Users can end the exploration at any time and decide whether to merge the branch operation spectrum into the main spectrum. If fusion is selected, the system will merge the branch data into the main spectrum with space-time constraint increments to achieve crowd-sourced expansion of SpaceTimeline. Otherwise, the branch operation spectrum will be stored independently for subsequent reuse. This process embodies the concept of spatial crowdsourcing, allowing users distributed in different time and space to collaborate and enrich the operation knowledge base.

[0056] Practice mode: The XR terminal guides users to simulate the key operation paths in the *.sTimeline in a virtual environment in real time. At the branch nodes, the terminal interface presents the expected consequences of alternative solutions to assist in decision-making. The system analyzes the user's multi-channel interactive data stream in real time based on artificial intelligence algorithms, evaluates its consistency with the standard *.sTimeline, and guides adjustments through instant feedback of audio, video, touch and other signals. The terminal performs semantic mapping of the user's practice data with the *.sTimeline to evaluate the mastery level.

[0057] Assessment mode: The XR terminal interface temporarily hides auxiliary information, and the user independently completes the full process operation in *.sTimeline. The user needs to make independent decisions at branch nodes based on environmental clues and experience. The system continuously collects multi-dimensional interaction data of the user's entire process, matches it with the standard operation spectrum of *.sTimeline in time and space, integrates the general evaluation indicator model, and forms a comprehensive and objective skill level analysis report. The report provides immersive visual analysis through the XR interface, diagnoses problems, and proposes targeted improvement plans.

[0058] Observation mode of the present invention: Utilizes the spatial computing and real-time rendering capabilities of XR to present the overall operation in an immersive roaming perspective, with multiple levels and angles. Exploration mode: Introduces interactive branch decision-making and real-time branch operation spectrum generation, and applies the spatial crowdsourcing mechanism to achieve continuous optimization of the SpaceTimeline knowledge base. Practice mode: Based on AI-driven real-time data analysis and multi-channel feedback, highly targeted immersive skill training and quantitative evaluation are achieved. Assessment mode: Collects full-process operation data, builds a level assessment model for spatiotemporal-semantic matching, and forms a global immersive skill assessment and improvement closed loop.

[0059] In the observation mode of the present invention, users can switch to the exploration mode at any time to open up new branch paths. In the exploration mode, users can pause recording, call up the observation perspective, and refer to other branch solutions. In the practice mode, new solutions generated by exploration can be used to consolidate innovative operational skills. The assessment mode incorporates multi-branch data generated by exploration to dynamically evaluate innovative practical capabilities.

[0060] The operational knowledge spectrum in the SpaceTimeline of the present invention is continuously expanded, optimized, internalized, and iterated in interactive accesses of different modes, forming a dynamic closed loop of knowledge extraction, fusion, application, and evaluation, which embodies the synergistic enhancement of group wisdom and machine intelligence.

[0061] Users can adjust the viewing angle and time progress controls in the XR terminal to fully understand the operation context from a third-person perspective. The system forms a dynamic follow-up anchor point based on the spatial relationship between the user and the virtual Avatar, and adjusts the scene in real time to adapt to the optimal rendering posture when the Avatar's movement exceeds the threshold, creating an immersive viewing experience of side-by-side observation and panoramic roaming.

[0062] In one embodiment of the present invention, please refer to Figure 5 As shown, the method further comprises the following steps:

[0063] Step S41, collect the user's original operation data in the XR environment, generate an initial SpaceTimeline lineage file, and each original lineage file is assigned a unique root identifier as an anchor point for subsequent incremental expansion.

[0064] Step S42, preprocessing the collected raw pedigree data. The preprocessing includes semantic extraction, noise filtering and data interpolation to improve data quality and consistency. The preprocessed pedigree file will serve as the starting point for crowdsourcing collaborative optimization.

[0065] Step S43: Different users, based on the original pedigree, open up new branch paths at key decision nodes through the exploration mode. These branch paths reflect the optimization and expansion of the operation process based on the user's own experience and judgment, enriching the breadth and depth of the operation pedigree.

[0066] Step S44: Perform a multi-dimensional evaluation on the branch paths contributed by the user, including coverage, innovation, reliability, and universality. By setting the evaluation threshold, high-quality excellent branches are automatically screened out and included in the subsequent incremental pedigree expansion.

[0067] Step S45: perform spatiotemporal alignment and semantic matching of the selected high-quality branches with the original pedigree, and merge them into the overall pedigree in an incremental manner. During the merging process, the system will detect and resolve pedigree conflicts to ensure the continuity and consistency of the overall pedigree.

[0068] Step S46: When the total pedigree is continuously expanded and improved under crowdsourcing collaboration and reaches the preset saturation and stability, it will be used as the new generation of original pedigree to start a new round of crowdsourcing optimization iteration. At the same time, the system will use data mining algorithms to identify key common features, evolution trends of operation modes and other macro insights from the total pedigree to provide directional guidance for optimization iteration.

[0069] Driven by the collective intelligence of the group, the SpaceTimeline pedigree file of the present invention has achieved large-scale collection, incremental expansion, and iterative optimization, and has continuously evolved into a comprehensive and finely structured operational knowledge base. This innovative mechanism effectively bridges the limitations of single expert knowledge and makes full use of the collective experience of the user group.

[0070] Different from traditional simple operation videos or text descriptions, SpaceTimeline encodes the time, space, semantic multi-dimensional information of the operation process and its internal connections in a graph structure to form a structured and semantic operation knowledge base, providing a foundation for subsequent access, expansion, analysis, and evaluation.

[0071] Based on the XR environment, the present invention constructs four complementary interactive access modes of observation, exploration, practice, and assessment, and makes full use of technologies such as spatial computing, real-time rendering, and multi-channel feedback to create an immersive, intuitive and vivid operational knowledge presentation and interactive experience, thereby improving the user's participation and immersion in cognition, exploration, operation, and evaluation.

[0072] The present invention introduces dynamic expansion of operation spectrum driven by spatial crowdsourcing: interactive branch decision-making and real-time branch operation spectrum generation are realized in exploration mode, allowing users to participate in the enrichment and expansion of operation knowledge in exploratory learning; and an incremental branch fusion mechanism is designed under the time and space constraints of the main spectrum, which maintains the dynamic evolution of the knowledge base while aggregating multi-branch operation content.

[0073] The SpaceTimeline representation of the present invention can be combined with the four modes, and the four modes can also be combined with each other, for example:

[0074] SpaceTimeline representation + observation mode: immersive operation and global observation;

[0075] SpaceTimeline representation + exploration mode: space crowdsourcing driven expansion of the operational spectrum;

[0076] SpaceTimeline representation + practice mode: personalized operation skills precision training;

[0077] SpaceTimeline characterization + assessment mode: quantitative evaluation of full-process operation capabilities;

[0078] Observation mode + exploration mode: seamless link from demonstration observation to innovative exploration;

[0079] Exploration mode + practice mode: consolidation and internalization of innovative operation plans;

[0080] Practice mode + assessment mode: a spiral of training, evaluation, diagnosis and improvement;

[0081] Exploration mode + spatial crowdsourcing integration: incremental accumulation of spatiotemporal constraints of branch exploration experience;

[0082] Assessment mode + human-machine collaborative analysis: synergistic enhancement of quantitative evaluation and intelligent diagnosis.

[0083] In one embodiment of the present invention, the multi-dimensional acquisition module can integrate technologies such as motion capture based on computer vision and speech understanding based on natural language processing to achieve more comprehensive and natural user behavior perception and digitization. The knowledge representation module can introduce description forms such as causal diagrams and knowledge graphs to capture the causal dependencies and contextual semantics between operating elements, giving SpaceTimeline a richer and more fine-grained semantic structure. The interactive access module can design high-order interactive modes such as collaborative creation, interpretation and reflection, and support multi-person collaboration, creative thinking and metacognitive cultivation. The human-computer collaboration module can introduce new interactive interfaces such as brain-computer interfaces, and optimize collaborative strategies through user modeling and adaptive learning technologies.

[0084] In practical applications, the present invention can be connected with workflow management systems, expert knowledge bases, etc. of specific industries to realize the automatic generation and update of operation pedigrees. For example, in the industrial field, equipment operation manuals, fault diagnosis guides, etc. can be converted into SpaceTimelines; in the medical field, expert diagnosis and treatment experience, surgical operations, etc. can be encoded into SpaceTimelines.

[0085] It should be understood that the "one embodiment" mentioned in the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the "one embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0086] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A reproduction method for accessing Space Timeline using an XR terminal, characterized in that: The method comprises: Through the multimodal perception interface and custom event scripts of XR devices, the multi-dimensional interactive data of the user's operation process in the virtual scene is collected and integrated into a structured operation spectrum file *.sTimeline; Based on the *.sTimeline file, an immersive operation scene with the same structure as the original operation scene and containing interaction anchor points is automatically constructed in the target XR device; Users can observe from multiple perspectives, interactively practice and test operations in the scene, and receive evaluation and feedback based on the intelligent analysis module.

2. The method for reproducing Space Timeline using an XR terminal according to claim 1, characterized in that: The step of collecting multi-dimensional interactive data of the user's operation process in the virtual scene includes: The data of the user's head posture, hand movements, and voice commands are continuously recorded, timestamped, and converted into a standard format.

3. The method for reproducing the Space Timeline using an XR terminal according to claim 1, characterized in that: The step of automatically constructing an immersive operation scene containing interaction anchor points having the same structure as the original operation scene in the target XR device based on the *.sTimeline file includes: Parse the file *.sTimeline, extract the key events and decision nodes recorded therein, and map them to interaction anchor points and branch anchor points in the virtual scene respectively. Each anchor point carries corresponding data metadata, which includes at least timestamp, spatial coordinates, associated objects, and interaction type. All anchor points are connected into a directed graph in order of timestamps. The directed graph is called an operation flow graph, which is used to intuitively represent the spatiotemporal trajectory and decision branches of the operation. According to the environment data in *.sTimeline, the virtual environment is automatically rebuilt and the operation flow graph anchors are bound to the corresponding virtual objects.

4. The method for reproducing Space Timeline using an XR terminal according to claim 3, characterized in that: The method further comprises: Templated UI elements are automatically added to the operation flow chart, wherein the UI elements include text labels, icons, progress bars for indicating the operation process, and road signs and signboards for identifying decision branches.

5. The method for reproducing the Space Timeline using an XR terminal according to claim 1, characterized in that: The steps of the user observing from multiple perspectives, interactively practicing and testing operations in the scene, and obtaining judgment and feedback based on the intelligent analysis module include: Based on the reconstruction of the scene timeline, through observation mode, exploration mode, practice mode and assessment mode, it supports users to access and expand the operation process contained in SpaceTimeline in multiple dimensions.

6. The method for reproducing the Space Timeline using an XR terminal according to claim 5, characterized in that: The method further comprises: Collect the user's original operation data in the XR environment and generate the original SpaceTimeline lineage file. Each original lineage file is assigned a unique root identifier as an anchor point for subsequent incremental expansion. Preprocess the collected raw pedigree data; Different users, based on the original pedigree, open up new branch paths at decision nodes through the exploration mode; Conduct multi-dimensional evaluation on branch paths, including coverage, innovation, reliability, and universality; The screened branches are spatiotemporally aligned and semantically matched with the original genealogy and merged into the total genealogy in an incremental manner.

7. The method for reproducing Space Timeline using an XR terminal according to claim 6, characterized in that: The preprocessing includes semantic extraction, noise filtering and data interpolation to improve data quality and consistency.

8. A reproduction system for accessing Space Timeline using an XR terminal, characterized in that: The system comprises: The multi-dimensional operation acquisition module collects the multi-dimensional interaction data of the user's operation process in the virtual scene through the multi-modal perception interface of the XR device and the custom event script, and integrates it into a structured operation spectrum file *.sTimeline; The immersive scene construction module automatically builds an immersive operation scene with the same structure as the original operation scene in the target XR device based on the *.sTimeline file; In the interactive event detection module, users can observe from multiple perspectives, interactively practice and test operations in the scene, and obtain objective judgment and feedback based on the intelligent analysis module.

9. The reproduction system for accessing Space Timeline using an XR terminal according to claim 8, characterized in that: The interactive event detection module is also used to reconstruct the scene timeline, and supports users to access and expand the operation process contained in SpaceTimeline in multiple dimensions through observation mode, exploration mode, practice mode and assessment mode.

10. The reproduction system for accessing Space Timeline using an XR terminal according to claim 9, characterized in that: The system also includes a spatial crowdsourcing module for collecting users' original operation data in the XR environment and generating original SpaceTimeline genealogy files, each of which is assigned a unique root identifier as an anchor point for subsequent incremental expansion; preprocessing the collected original genealogy data; based on the original genealogy, different users open up new branch paths at decision nodes through the exploration mode; multi-dimensional evaluation of branch paths, including coverage, innovation, reliability, and universality; and spatially aligning and semantically matching the screened branches with the original genealogy, and merging them into the total genealogy in an incremental manner.

Images