A three-dimensional AR assisted mode switching system for mapping a position of a physical model

Abstract

The application belongs to the technical field of image data processing, and relates to a three-dimensional AR auxiliary mode switching system for entity model position state mapping, which comprises the following modules: a data processing module for generating a normalized position state vector flow; a motion feature extraction module for obtaining a discrete motion feature set; an encrypted dynamics message generation module for obtaining encrypted dynamics messages; a data verification interlocking module for obtaining encrypted dynamics messages in a permission state; an intention reasoning mapping module for obtaining intention reasoning result data; a rendering instruction set compiling module for compiling a rendering instruction set; and a multi-level synchronous display module for distributing data streams carrying time stamps to multi-level display terminals for synchronous rendering output. The application solves the problems of limited interactive intention recognition accuracy and insufficient augmented reality mode switching adaptability caused by the lack of deep analysis of continuous dynamics characteristics of entity models.

Description

Technical Field

[0001] This invention belongs to the technical field of image data processing and relates to a three-dimensional AR-assisted mode switching system for entity model position mapping. Background Technology

[0002] Mixed reality and augmented reality technologies simulate a multi-sensory interactive 3D space for users by overlaying virtual 3D models onto the real environment, incorporating visual and tactile sensations. In sophisticated and complex interactive scenarios, users typically need to interact with the virtual world using physical models. Accurately capturing the positional changes of these physical models and triggering corresponding augmented reality assistance modes has become a key technological direction for improving the naturalness of human-computer interaction and the accuracy of system responses.

[0003] Chinese patent publication number CN113689577A discloses a method, system, device, and medium for matching virtual 3D models with physical models. This technical solution primarily involves acquiring 3D information of a target object marked with points, constructing a virtual 3D model in the real coordinate system using rigid body transformation, and then combining this with the positional information of the marked points on the physical model in the virtual reality coordinate system to perform a 3D transformation and overlap, thereby achieving spatial fusion between the two. Simultaneously, the solution also discloses the ability to identify discrete operation commands represented by user gestures and specify basic response actions such as rotation or scaling for the target object.

[0004] However, the aforementioned existing technologies have limitations when dealing with highly dynamic and continuous interactive tasks. Their triggering mechanisms mainly rely on static spatial coordinate alignment or simple rigid action command recognition, ignoring the continuous dynamic characteristics of the entity model generated during user manipulation, such as instantaneous acceleration changes, rates of motion change, and oscillation frequencies—deeply hidden physical behavior states. This lack of contextual logic for continuous physical actions can easily lead to low accuracy in recognizing complex user manipulation intentions, making it difficult to dynamically and adaptively adjust the rendering performance and mode switching of the augmented reality environment based on the actual certainty of the user's actions.

[0005] To address the aforementioned issues, this invention aims to solve the technical problems in the prior art, which suffer from limited accuracy in interactive intent recognition and insufficient adaptability in augmented reality mode switching due to the lack of in-depth analysis of the continuous dynamic characteristics of entity models. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a 3D AR-assisted mode switching system for entity model position mapping.

[0007] A 3D AR-assisted mode switching system for entity model position mapping includes: The data processing module acquires the spatial positional signals of the entity model and performs dimensionless processing to generate a normalized positional vector flow. The motion feature extraction module extracts a subset of time-series signals from the normalized position vector stream, and then extracts feature combinations from the time-series signal subset to obtain a discrete motion feature set. The encrypted dynamic message generation module extracts the dynamic parameters of the discrete motion feature set, encapsulates and hashes the discrete motion feature set and dynamic parameters to obtain the encrypted dynamic message; The data verification interlock module intercepts encrypted dynamic messages, performs verification and comparison on the encrypted dynamic messages, and obtains the encrypted dynamic messages in the permission state. The intent reasoning mapping module extracts the main payload data of the encrypted dynamics message under the permission state, outputs the target intent category and confidence value, and obtains the intent reasoning result data; The rendering instruction set compilation module parses the intent reasoning result data, calculates the visual parameters of the mesh primitives based on the target intent category and confidence value, and compiles the rendering instruction set. The multi-level synchronous display module extracts the network address of the display terminal from the rendering instruction set, calculates a unified screen output timestamp by combining it with the global master clock signal, and distributes the data stream carrying the timestamp to the multi-level display terminals for synchronous rendering output.

[0008] A further aspect of this invention involves acquiring the spatial positional signals of an entity model and performing dimensionless processing to generate a normalized positional vector flow, comprising the following steps: Acquire a six-degree-of-freedom positional data stream containing three-dimensional spatial coordinates and attitude angles from the physical model; Filter and denoise the six-degree-of-freedom positional data stream to obtain the spatial positional signal, and calculate the velocity and acceleration parameters of the spatial positional signal; Extract the physical scale parameters of the interactive system; By using physical scale parameters, minimum-maximum normalization mapping is performed on the three-dimensional spatial coordinates, attitude angles, and time derivatives of each order to generate a normalized state vector flow.

[0009] A further aspect of this invention involves extracting a subset of time-series signals from the normalized position vector flow, and then extracting feature combinations from this subset to obtain a discrete motion feature set. This process includes the following steps: Set a dynamic time window and use the dynamic time window to cut the normalized state vector stream into a subset of time signals with a sliding step size; Extract peak extrema, trough extrema, and rate of change parameters from a subset of time-series signals to obtain feature combinations; The feature combination is compared with the basic dynamic template in the pre-stored physical benchmark feature library; When the feature combination satisfies a set of conditions set by the basic dynamics template, a corresponding semantic identifier is attached to the time-series signal subset to generate a discrete motion feature set.

[0010] A further aspect of this invention involves extracting dynamic parameters from a discrete motion feature set, encapsulating and hashing the discrete motion feature set and dynamic parameters to obtain an encrypted dynamic message, including the following steps: Extract the temporal sequence, peak acceleration, and oscillation frequency parameters corresponding to the discrete motion feature set as dynamic parameters; The dynamic parameters follow a preset communication structure protocol and are packaged and encapsulated into structured dynamic data packets; The structured dynamics data packets are serialized into a byte stream and a hash algorithm is executed to generate an encrypted verification sequence. The encrypted verification sequence is injected into the leading region of the structured dynamics data packet and then merged to generate an encrypted dynamics message.

[0011] A further aspect of this invention involves extracting the temporal sequence, peak acceleration, and oscillation frequency parameters corresponding to the discrete motion feature set as dynamic parameters, including the following steps: The maximum absolute value of the second-order time derivative contained in the discrete motion feature set within the time window is calculated as the peak acceleration; The Fast Fourier Transform algorithm is applied to the first-order time derivative contained in the discrete motion feature set to analyze the spectrum, and the frequency corresponding to the energy peak point is extracted as the oscillation frequency parameter. An incrementing counter is used to assign corresponding sequence numbers to the discrete motion feature set, which serve as the temporal arrangement order.

[0012] A further aspect of this invention involves intercepting encrypted dynamics messages, performing verification and comparison on the encrypted dynamics messages, and obtaining encrypted dynamics messages in a permitted state, comprising the following steps: Intercept encrypted dynamics messages and extract the encrypted verification sequence and main payload data; Read the preset verification seed parameters and append them to the main payload data to form a byte stream to be verified; Perform a hash operation on the byte stream to be verified to generate a test comparison value; Perform bitwise operations to compare the encrypted verification sequence with the test comparison value; If it is determined that there is no discrepancy between the bitmasks of the two, then the permission state is activated.

[0013] A further aspect of this invention involves extracting the main payload data of the encrypted dynamics message under a permitted state, outputting the target intent category and confidence level, and obtaining intent inference result data, including the following steps: Input the main load data into a pre-trained Bayesian inference network model to calculate the posterior probability distribution of the preset target intent category; Information entropy is calculated based on the posterior probability distribution and used as the information entropy of the manipulation intention; The intent category corresponding to the posterior probability peak is selected as the target intent category, and the posterior probability peak is used as the confidence value. Combine the target intent category, confidence level, and oscillation frequency parameter to generate intent inference result data.

[0014] A further aspect of this invention involves parsing the intent reasoning result data, calculating the visual parameters of the mesh primitives based on the target intent category and confidence level, and compiling a rendering instruction set, including the following steps: Search the local 3D asset library based on the target intent category and load the basic augmented reality scene mesh primitives as primitives; Based on the confidence level and the preset smoothing curve, the transparency coefficient and contour luminescence intensity of the output primitives are calculated as visual performance attributes. The frame rate of the graphics primitives is adjusted based on the oscillation frequency parameters carried by the intent reasoning result data. Merge and bind primitives, transparency coefficients, outline glow intensity values, and screen refresh rate, and compile to generate a rendering instruction set.

[0015] A further aspect of this invention involves extracting the display terminal network address from the rendering instruction set, calculating a unified screen output timestamp in conjunction with a global master clock signal, and distributing the data stream carrying the timestamp to multi-level display terminals for synchronous rendering output. This includes the following steps: The display terminal's network address is parsed from the rendering instruction set; Generate foreground guide images and 3D environment model data based on the rendering instruction set; Obtain the global master clock signal and calculate a unified screen output timestamp based on the global master clock signal and the screen refresh rate; A foreground guide image carrying a timestamp is directionally projected onto a foreground transparent screen display terminal configured in the first hardware layer; The environmental 3D model data with timestamps is pushed to the background giant screen display terminal configured in the second hardware layer.

[0016] A further aspect of this invention involves calculating a unified screen output timestamp based on the global master clock signal and the screen refresh rate, including the following steps: Call the system's configured clock signal synchronizer to obtain the current global master clock signal; Extract the comprehensive latency compensation value that includes the estimated time consumption of data rendering, encoding, and network transmission; The time value corresponding to the global master clock signal is added to the comprehensive delay compensation value to obtain the screen output timestamp; The output timestamp is encoded into the data frame header of the data stream.

[0017] In summary, the present invention has the following beneficial technical effects: 1. By extracting peak extrema, trough extrema, and rate of change parameters from a subset of time-series signals to obtain feature combinations, and comparing them with a basic dynamic template to generate a discrete motion feature set, the original continuous spatial positional signal is transformed into a structured physical state representation. Through the quantitative classification of user control behaviors, standardized dynamic data support is provided for subsequent intent parsing, solving the problem of existing technologies lacking in-depth extraction of continuous physical dynamic features.

[0018] 2. A hash-based interlocking mechanism for encrypted dynamic message generation and data verification is introduced, constructing a closed-loop verification link during data flow. After extracting and encapsulating dynamic parameters, an encrypted verification sequence is generated. A second hash operation is performed to compare bits before downstream module calls, ensuring the integrity and tamper-proof nature of the main payload data during internal system transmission. This reduces the risk of erroneous operations due to data anomalies or corrupted instructions, and improves the operational reliability of the underlying communication of the interactive system.

[0019] 3. A pre-trained Bayesian inference network model is used to perform probabilistic inference on the subject load data containing dynamic features, outputting the target intent category with peak posterior probability and the corresponding confidence score. This overcomes the limitations of traditional static rigid rule matching, enabling probabilistic mapping and uncertainty quantification based on the dynamic features of manipulation behavior, effectively improving the recognition accuracy of implicit interaction intents in complex manipulation scenarios.

[0020] 4. Based on the target intent category and confidence level, the transparency coefficient and contour luminescence intensity of the mesh primitives are calculated using a smooth mapping curve. The screen refresh rate is dynamically adjusted using oscillation frequency parameters, achieving dynamic coupling between augmented reality visual parameters and the determinism of user control intent. Simultaneously, a unified screen output timestamp is calculated using a global master clock signal to drive synchronous rendering across multiple display terminals. This overcomes the shortcomings of traditional triggering mechanisms in terms of adaptability, improving the smoothness of augmented reality mode switching and the naturalness of human-computer interaction. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0022] Figure 1This discloses a schematic diagram of the framework in the embodiments of this application.

[0023] Figure 2 This discloses a flowchart of an embodiment of this application.

[0024] Figure 3 This document discloses a schematic diagram showing the trend of the normalized linear acceleration vector components changing with the sampling point sequence within a dynamic time window in the embodiments of this application, as well as the trigger comparison diagram of the eigenvalues ​​and the basic dynamic template conditions.

[0025] Figure 4 This is a schematic diagram of the smooth mapping of the change in confidence value of arbitrary graph inference of the contour luminescence intensity coefficient in the embodiments of this application. Detailed Implementation

[0026] The following is in conjunction with the appendix Figure 1 - Figure 4 A preferred description of the present invention is provided below.

[0027] See attached document Figure 1 - Figure 2 This invention proposes a 3D AR-assisted mode switching system for entity model position mapping, comprising the following modules: The data processing module acquires the spatial positional signals of the entity model and performs dimensionless processing to generate a normalized positional vector flow. The motion feature extraction module extracts a subset of time-series signals from the normalized position vector stream, and then extracts feature combinations from the time-series signal subset to obtain a discrete motion feature set. The encrypted dynamic message generation module extracts the dynamic parameters of the discrete motion feature set, encapsulates and hashes the discrete motion feature set and dynamic parameters to obtain the encrypted dynamic message; The data verification interlock module intercepts encrypted dynamic messages, performs verification and comparison on the encrypted dynamic messages, and obtains the encrypted dynamic messages in the permission state. The intent reasoning mapping module extracts the main payload data of the encrypted dynamics message under the permission state, outputs the target intent category and confidence value, and obtains the intent reasoning result data; The rendering instruction set compilation module parses the intent reasoning result data, calculates the visual parameters of the mesh primitives based on the target intent category and confidence value, and compiles the rendering instruction set. The multi-level synchronous display module extracts the network address of the display terminal from the rendering instruction set, calculates a unified screen output timestamp by combining it with the global master clock signal, and distributes the data stream carrying the timestamp to the multi-level display terminals for synchronous rendering output.

[0028] In one embodiment of the present invention, the data processing module is configured to perform the following steps: Acquire a six-degree-of-freedom positional data stream containing three-dimensional spatial coordinates and attitude angles from the physical model; Filter and denoise the six-degree-of-freedom positional data stream to obtain the spatial positional signal, and calculate the velocity and acceleration parameters of the spatial positional signal; Extract the physical scale parameters of the interactive system; By using physical scale parameters, minimum-maximum normalization mapping is performed on the three-dimensional spatial coordinates, attitude angles, and time derivatives of each order to generate a normalized state vector flow.

[0029] The central processing unit activates the sensor array bound to the physical model. The sensor array integrates an inertial measurement unit and an ultra-wideband positioning module, sampling at a preset frequency. Continuously acquire six-degree-of-freedom positional data streams from the solid model. Sampling frequency. The typical value range is 60Hz to 120Hz, and the setting is based on the premise of ensuring that the user's operation actions can be captured while taking into account the computational load of the interactive system.

[0030] A six-DOF positional data stream is a time series containing six independent variables. It should be noted that the positional state used in this implementation is equivalent to the pose commonly used in this field, referring to the comprehensive physical state of the entity model's spatial position and orientation in three-dimensional space, specifically the position components in the Cartesian coordinate system. And Euler angles, which describe the attitude of an object, namely yaw angles. Pitch angle With roll angle The six-degree-of-freedom positional data stream is represented as a discrete-time series, where each data point at any given time contains three-dimensional spatial coordinates. With attitude angle .

[0031] To eliminate the inherent random noise and drift of the sensor array, the central processing unit (CPU) uses a Kalman filter to perform recursive state estimation on the original sequence of the six-DOF positional data stream, outputting a set of smoothed spatial positional signals. Based on the smoothed spatial positional signals, the CPU uses the backward difference method to calculate the first-order time derivative of the smoothed spatial positional signals, i.e., the linear velocity vector. With angular velocity vector The first-order time derivative characterizes the instantaneous rate of change of the solid model's position; and further, the second-order time derivative of the smoothed spatial position signal, i.e., the linear acceleration vector, is calculated. With angular acceleration vector The second-order time derivative characterizes the changing trend of the velocity of the solid model.

[0032] The central processing unit loads preset physical scale parameters from the system configuration file of the interactive system. These physical scale parameters define the boundary range of the interactive space, specifically a set of constants pre-calibrated based on the physical space boundaries of the physical model's movement. Assuming the application scenario is a desktop interactive sandbox with physical dimensions of 2.0m long, 1.5m wide, and 1.0m high, the maximum value of the three-dimensional spatial coordinate components is... The values ​​are 2.0m, 1.5m, and 1.0m, with a minimum of 0m; the range of attitude angles is usually set to [−π,π] or [0,2π]; the maximum and minimum values ​​of each order of time derivative are set based on experimental statistics or theoretical estimates of the boundary operation amplitude that the user may perform in the interaction space.

[0033] Using physical scale parameters, the central processing unit performs minimum-maximum normalization on the three-dimensional spatial coordinates, attitude angles, and time derivatives of various orders obtained through calculation. The dimensionless mapping process can be represented by the following normalization formula:

[0034] In the normalization formula, The original data to be normalized can be a component of three-dimensional spatial coordinates, attitude angles, or time derivatives of various orders. and These represent the theoretical minimum and theoretical maximum values ​​that the components can reach in the physical space of the interactive system, respectively. The theoretical minimum and theoretical maximum values ​​together constitute the physical scale parameter. The normalized result is constrained to the range [0,1] to obtain the calculated value. The normalization operation ensures that features with different physical units and dimensions are on the same order of magnitude, avoiding weight bias caused by differences in numerical scale during subsequent model training and pattern matching.

[0035] Through a dimensionless mapping process, physical quantities of all dimensions are linearly mapped to a unified numerical range, generating a normalized state vector flow composed of multidimensional floating-point vectors. At any given moment, this state vector flow is an 18-dimensional characteristic vector, containing the normalized values ​​of all components of the three-dimensional spatial coordinates, attitude angles, first-order time derivatives, and second-order time derivatives.

[0036] For example, suppose the sampling frequency of the interactive system is... 100Hz, time step It takes 0.01 seconds. At any given time, the spatial positional signal acquired by the sensor array and smoothed by a Kalman filter is... To simplify the example, only show and The initial linear velocity vector, initial angular velocity vector, initial linear acceleration vector, and initial angular acceleration vector in the other dimensions are all [missing information]. .

[0037] In the next moment The spatial positional signal acquired by the sensor array and smoothed by a Kalman filter is... The central processing unit first calculates the first-order time derivative, for example... The linear velocity in the direction is yaw rate is .

[0038] The central processing unit calculates the second-order time derivative, and the linear acceleration vector is... The angular acceleration vector is The processing unit calls the physical scale parameters for normalization, assuming... , ; , ; , ; , ; , ; , .

[0039] exist At time t, the values ​​of the corresponding components in the normalized potential vector flow are calculated as follows: ; ; ; ; ; After normalizing all eighteen dimensional components, the result is... The normalized state vector at time t is passed downstream as a data point in the normalized state vector stream.

[0040] In one embodiment of the present invention, the motion feature extraction module is configured to perform the following steps: Set a dynamic time window and use the dynamic time window to cut the normalized state vector stream into a subset of time signals with a sliding step size; Extract peak extrema, trough extrema, and rate of change parameters from a subset of time-series signals to obtain feature combinations; The feature combination is compared with the basic dynamic template in the pre-stored physical benchmark feature library; When the feature combination satisfies a set of conditions set by the basic dynamics template, a corresponding semantic identifier is attached to the time-series signal subset to generate a discrete motion feature set.

[0041] The central processing unit (CPU) segments and identifies short-time motion patterns in the normalized position vector stream generated by the data processing module. Internally, the CPU maintains a first-in-first-out (FIFO) queue as a dynamic time window, the length of which is... With sliding step size These are two key configurable parameters, both measured in the number of sampling points. The length of the dynamic time window. The settings need to ensure that the dynamic time window can fully cover the duration of typical short-term manipulation actions. For example, the duration of typical human short-term manipulation actions is usually between 100ms and 800ms. If the sampling frequency is 100Hz, the value of the length N can be set to 10 to 80.

[0042] The dynamic time window uses a preset sliding step. Translation is performed on the normalized potential vector flow, with a sliding step size. This determines the magnitude of the dynamic time window movement and the sliding step size. Typically smaller than length To ensure overlap between dynamic time windows, thus enabling smooth transition detection of continuous actions, the sliding step size... The typical value is length. 1 / 4 to 1 / 2.

[0043] Each time the window slides, it extracts a fixed-length normalized state vector sequence, forming a subset of the time-series signal. This subset of the time-series signal is represented in the data structure as follows: The central processing unit initiates a feature extraction program, traversing multiple dimensions of data contained in the time-series signal subset. The feature extraction program employs a local extremum detection algorithm to locate and record the local maximum and minimum values ​​of key dimensions, such as linear acceleration vector components and angular acceleration vector components, forming peak extrema and trough extrema respectively.

[0044] Simultaneously, the feature extraction program calculates the first and last differences between the linear velocity vector components and the angular velocity vector components of the time-series signal subset, using them as dimensionless rate of change parameters. The central processing unit calls the matching engine to compare the extracted peak extrema, trough extrema, and rate of change parameters one by one with the basic dynamic templates pre-stored in the physical benchmark feature library.

[0045] The physical benchmark feature library is a pre-built database containing various basic dynamic templates characterizing fundamental physical motion laws, such as linear acceleration, emergency braking, and uniform turning. Each basic dynamic template consists of a set of conditions that define the quantitative characteristics of the corresponding motion mode. For example, the conditions for the basic dynamic template of emergency braking can be defined as follows: the normalized linear acceleration vector component has a value less than a preset negative threshold. The extreme value of the trough, and the rate of change parameter of the normalized linear velocity vector component is a negative value less than the preset negative rate of change threshold.

[0046] The typical value range is -0.5 to -0.8. This range is set based on experimental statistics of the mechanical characteristics of the human body in routine operation. It is used to ensure that the system can effectively filter out routine smooth deceleration actions and accurately capture the explosive force characteristics when the user pulls back in an emergency. The typical value of the negative threshold of the rate of change is -0.1. It is set based on the lower limit of the change in speed that causes a step decay, so as to distinguish it from the slow deceleration caused by natural friction or drift in the physical model.

[0047] When the characteristic combination of peak extrema, trough extrema, and rate of change parameters satisfies a set of conditions set by a certain basic dynamics template, the rule-based matching engine succeeds in matching. The central processing unit encapsulates the successfully matched subset of time-series signals and adds semantic identifiers corresponding to the basic dynamics template to generate a discrete motion feature set representing the physical state. The discrete motion feature set outputs a structured data object, which not only contains the original subset of time-series signals but also includes semantic identifiers of the basic dynamics template, such as the semantic identifier for emergency braking.

[0048] For example, based on the data processing module example, the central processing unit continuously receives a normalized position vector stream. The length of the dynamic time window is set. Each sampling point, sliding step size There are [number] sampling points. Assume that at a certain moment, the dynamic time window captures a subset of the time-series signal. To illustrate the calculation process, the focus is on the normalized linear acceleration vector within this subset. Components and normalized linear velocity vector Components. Within the dynamic time window. The sequence value of the component , The sequence value of the component Central processing unit traversal The sequence values ​​of the components are discovered through local extrema detection. The maximum value of the component is 0.85, and the extracted 0.85 is the peak extreme value.

[0049] If no local minimum value below the preset extreme value judgment threshold is found within the dynamic time window, the trough extreme value can be recorded as invalid or as the sequence starting value of 0.55. The typical value of the extreme value judgment threshold is 0.60, which is set based on the normalized fluctuation upper limit caused by the system's basis noise and minor environmental disturbances. It is used to filter out false extreme values ​​during the feature extraction process, ensuring that the extracted extreme value points reflect real and effective human manipulation intentions. Central processing unit calculation The rate of change parameter of the component, The first value of the component is 0.68, and the last value is 0.95. The change of 0.27 obtained by calculating the difference between the first and last values ​​is the rate of change parameter.

[0050] At this point, the central processing unit queries the physical reference feature library for a basic dynamic template named "Fast Forward Push". The basic dynamic template is set with the following conditions: The peak extremum of the component is greater than 0.80, and The rate of change parameter of the component is greater than 0.20. Since the peak extrema and rate of change parameters extracted from the current time series signal subset satisfy 0.85>0.80 and 0.27>0.20, which fully meet a set of conditions of the basic dynamic template, the rule-based matching engine is successful.

[0051] See appendix Figure 3 Within this dynamic time window, the normalized linear acceleration vector component reaches a peak extreme value of 0.85 at the fourth sampling point. Since this point exceeds the fundamental dynamic template condition represented by the horizontal dashed line, it triggers and generates a discrete motion feature set with specific semantic identifiers. It should be noted that, to clearly illustrate the extreme value characteristics of the curve, the shaded area in the figure only shows the first half of the dynamic time window, approximately 11 sampling points, as a partial illustration. In actual calculations, this time window includes... Each sample is taken. The central processing unit then takes the time-series signal subset containing 20 eighteen-dimensional vector data, appends a semantic tag of "fast forward" to it, and encapsulates it into a discrete motion feature set, which is then passed to subsequent processing steps.

[0052] In one embodiment of the present invention, the encrypted dynamics message generation module is configured to perform the following steps: Extract the temporal sequence, peak acceleration, and oscillation frequency parameters corresponding to the discrete motion feature set as dynamic parameters; The dynamic parameters follow a preset communication structure protocol and are packaged and encapsulated into structured dynamic data packets; The structured dynamics data packets are serialized into a byte stream and a hash algorithm is executed to generate an encrypted verification sequence. The encrypted verification sequence is injected into the leading region of the structured dynamics data packet and then merged to generate an encrypted dynamics message.

[0053] For each received discrete motion feature set, the central processing unit (CPU) first performs secondary dynamic feature extraction on the temporal signal subset contained within the discrete motion feature set. The CPU iterates through all components of the second-order time derivative of the temporal signal subset, calculating the L-infinity norm (maximum absolute value) of all components of the second-order time derivative within the dynamic time window. This maximum absolute value is used as the peak acceleration parameter in the normalized state. To obtain the numerical values ​​of the actual physical dimensions, the CPU can perform inverse normalization calculations using preset physical scale parameters. The inverse normalization calculation formula is:

[0054] In the formula, For the extracted normalized feature values, and These are the theoretical minimum and theoretical maximum values ​​among the physical scale parameters. The goal is to restore the physical true value, that is, to restore the original data scale before normalization in the data processing module.

[0055] The peak acceleration parameter combines the maximum values ​​of the linear acceleration vector and the angular acceleration vector, comprehensively reflecting the instantaneous explosive force of the user's control action, and is a key indicator for characterizing the intensity of the user's control action. Simultaneously, the central processing unit applies a Fast Fourier Transform algorithm to the first-order time derivatives of the time-series signal subset—namely, the linear velocity vector components and the angular velocity vector components—to analyze the spectra of the linear velocity vector components and angular velocity vector components, and extracts the frequencies corresponding to the energy peak points as oscillation frequency parameters.

[0056] The oscillation frequency parameter reveals the rhythmic characteristics of the control action. For example, the oscillation frequency parameter of a smooth, uniform pushing action is close to zero, while the back-and-forth alignment action will exhibit corresponding low-frequency oscillations. The central processing unit maintains an incrementing counter, assigning a corresponding sequence number to each generated discrete motion feature set. These corresponding sequence numbers constitute the temporal arrangement order.

[0057] The central processing unit, based on a pre-defined communication structure protocol, packages the temporal sequence, peak acceleration parameters, oscillation frequency parameters, and semantic identifiers of the discrete motion feature set into a structured dynamic data packet with defined fields and data types. This packet serves as the payload carrying core information. The pre-defined communication structure protocol defines a standard data format, such as a structure containing integer, string, and floating-point fields, to ensure consistent data format parsing across different modules within the interactive system.

[0058] To ensure data integrity and immutability during transmission, the central processing unit serializes all field values ​​of the structured dynamics data packet in a predetermined order, forming a continuous byte stream. It then performs a hash algorithm on this continuous byte stream to generate an encrypted verification sequence. The hash calculation process in this step can be abstracted as follows:

[0059] In the hash formula, This represents a binary byte stream formed by serializing the entire contents of a structured dynamics data packet; Represents the system's preset verification seed parameter; symbol Indicates a serial splicing operation; A one-way hash function representing public determinism is selected in this embodiment, namely Secure Hash Algorithm 256 (SHA-256). This is the calculated encrypted verification sequence, a fixed-length bit string, such as a 256-bit hexadecimal string. The hash calculation process ensures that any minor modification to the content of the Structured Dynamics data packet will change the calculated encrypted verification sequence. This provides a reliable basis for subsequent integrity verification.

[0060] The central processing unit creates a new data container, places the calculated encryption verification sequence in the preamble region, and places the structured dynamics data packet in the body region, merging them to generate an encrypted dynamics message capable of communication verification. The structure of the encrypted dynamics message draws inspiration from the frame structure of network data packets, including a verification header and a data body, achieving integrated encapsulation of data and verification information.

[0061] For example, following the example of the motion feature extraction module, the central processing unit generates a discrete motion feature set with a semantic tag of "fast forward". Assuming that the discrete motion feature set is the 58th discrete motion feature set generated after the interactive system starts, the temporal order of the discrete motion feature set is 58.

[0062] The central processing unit analyzes a subset of time-series signals within the discrete motion feature set and finds that... The normalized peak value for the component is 0.85, based on the physical scale parameters defined in the data processing module. The physical peak acceleration is obtained by inverse normalization calculation. The central processing unit performs Fast Fourier Transform analysis on the linear velocity vector components and angular velocity vector components, and finds that the dominant frequencies of the linear velocity vector components and angular velocity vector components are... The extracted main frequency is the oscillation frequency parameter.

[0063] The central processing unit constructs a structured dynamics data packet with the content {sequence:58, label:"fast forward",peak_accel:350.0,osc_freq:1.2}. To calculate the encryption verification sequence, the structured dynamics data packet is serialized, and a preset verification seed parameter is appended to the end. The serialized structured dynamics data packet and the verification seed parameter are then input into the 256-bit secure hash algorithm function to calculate a 256-bit encryption verification sequence, for example, 0xA1B2C3D4...E5F6.

[0064] The central processing unit combines the encrypted verification sequence with the structured dynamics data packet to generate an encrypted dynamics message. The structure of the encrypted dynamics message is {Header:{Checksum:"0xA1B2C3D4...E5F6"},Payload:{sequence:58,label:"Fast Forward",peak_accel:350.0,osc_freq:1.2}}, and sends the encrypted dynamics message to the system communication bus.

[0065] In one embodiment of the present invention, the data verification interlock module is used to perform the following steps: Intercept encrypted dynamics messages and extract the encrypted verification sequence and main payload data; Read the preset verification seed parameters and append them to the main payload data to form a byte stream to be verified; Perform a hash operation on the byte stream to be verified to generate a test comparison value; Perform bitwise operations to compare the encrypted verification sequence with the test comparison value; If it is determined that there is no discrepancy between the bitmasks of the two, then the permission state is activated.

[0066] The encrypted dynamic messages flowing in the communication link are captured and verified. The monitoring module deployed on the system communication bus continuously monitors the encrypted dynamic message flow. The monitoring module is a resident system service or daemon process that sniffs network data packets on a specified network port or memory bus in a non-blocking manner. When the format of the network data packet is identified as conforming to the predefined structure of the encrypted dynamic message, the network data packet is intercepted.

[0067] Upon interception, the monitoring module performs a data stripping operation, parsing the encrypted dynamics message into two parts: an encrypted verification sequence located in the preamble and a structured dynamics data packet, which constitutes the main payload data. The monitoring module then reads the preset verification seed parameters from the secure system configuration area.

[0068] The monitoring module appends the read verification seed parameter to the end or beginning of the stripped main payload data, forming a new byte stream to be verified. The monitoring module then performs the same hash algorithm as the cryptographic dynamics message generation module on this new byte stream with the appended verification seed parameter, generating a test comparison value—a hash digest independently calculated locally by the monitoring module—used for comparison with the cryptographic verification sequence provided by the transmitter. The monitoring module performs a bit-by-bit binary XOR operation on the original cryptographic verification sequence and the newly generated test comparison value. The core verification logic can be described as follows:

[0069] in, The Boolean result representing the verification decision; It is the main payload data extracted from the encrypted dynamics message; These are the system's preset verification seed parameters; symbol This indicates that the seed parameter will be verified. With main load data Perform a series splicing operation; It uses the same hash algorithm as the encryption dynamics message generation module when generating the encryption verification sequence; the calculation result This is the test comparison value; It is the original encryption verification sequence extracted from the encryption dynamics message. Conditional expression. The two 256-bit sequences are compared by bitwise XOR operations to verify whether they are completely equal.

[0070] A binary XOR operation results in all zeros, meaning that the original encrypted verification sequence and the bitmask of the test comparison value are identical, and the result is consistent, i.e., the Boolean result of the verification judgment. If true, the interactive system determines that the encrypted dynamics message is legitimate and has not been tampered with, and activates the permission status flag. The permission status flag is maintained by the listening module and can be queried by downstream modules before processing the encrypted dynamics message, allowing the encrypted dynamics message to continue to circulate within the interactive system.

[0071] Conversely, if the result of the binary XOR operation is not zero, the interactive system determines that the encrypted dynamics message is an illegal encrypted dynamics message or is corrupted, discards the illegal encrypted dynamics message, triggers the exception handling process, blocks the subsequent call chain related to the source of the illegal encrypted dynamics message, that is, terminates a series of function calls or service requests to the illegal encrypted dynamics message, thereby constructing an anti-counterfeiting interlocking mechanism for the data flow.

[0072] The core verification logic ensures that even if an attacker intercepts and modifies the main payload data... And recalculated the forged hash value. To forge new cryptographic dynamics messages, but because the attacker is unaware of the private verification seed parameters inside the interactive system... The attacker's forged hash value cannot pass the verification system. This verification checkpoint ensures the security of the encrypted dynamic message stream.

[0073] For example, following the example of the encrypted dynamics message generation module, a listening module deployed on the system communication bus captures the sent encrypted dynamics message. The listening module decomposes the encrypted dynamics message to obtain the original encrypted verification sequence. =“0xA1B2C3D4...E5F6”, and main load data ={sequence:58,label:"Fast Forward",peak_accel:350.0,osc_freq:1.2}. The listening module reads the preset checksum seed parameters from the secure system configuration area. =“a-secret-key-string”. The listening module will send the main payload data. Serialize the data and append a checksum seed parameter to the end of the serialized data. This forms a new byte stream to be verified.

[0074] The new byte stream to be verified is input into the secure hash algorithm 256 function to calculate the test comparison value. Let's assume the calculation result is... =“0xA1B2C3D4...E5F6”. The monitoring module performs a bit-by-bit binary XOR operation to compare the original encrypted verification sequence. Comparison with test value Upon discovering that the two are consistent, the monitoring module sets the permission status flag of the encrypted dynamics message to "true," allowing the encrypted dynamics message to be received and processed by the downstream intent analysis module.

[0075] If the encrypted dynamics message is tampered with during transmission, for example, if the peak_accel value of the main payload data is changed to 450.0, then the test comparison value calculated by the monitoring module will be affected. It will be a completely different hash value, such as "0x987654ED...CB21", compared to the original cryptographic check sequence. If an inconsistency is found during the comparison, the illegal encrypted dynamics message will be discarded.

[0076] In one embodiment of the present invention, the intent reasoning mapping module is configured to perform the following steps: Input the main load data into a pre-trained Bayesian inference network model to calculate the posterior probability distribution of the preset target intent category; Information entropy is calculated based on the posterior probability distribution and used as the information entropy of the manipulation intention; The intent category corresponding to the posterior probability peak is selected as the target intent category, and the posterior probability peak is used as the confidence value. Combine the target intent category, confidence level, and oscillation frequency parameter to generate intent inference result data.

[0077] Specifically, upon receiving an encrypted dynamics message with a true permission status flag, verified by the data verification interlock module, the central processing unit initiates a deep intent parsing process. This process extracts core payload data from the encrypted dynamics message, combines the numerical features contained within the payload data (primarily peak acceleration parameters and oscillation frequency parameters) into a feature vector, and inputs this feature vector as observational evidence into a pre-trained Bayesian inference network model.

[0078] The Bayesian inference network model is a directed acyclic graph (DAG). Nodes in the DAG represent random variables, and edges represent conditional dependencies between these variables. The conditional probability table is obtained by learning from a large amount of labeled historical manipulation data. The node topology of the Bayesian inference network model simulates the causal relationships of user manipulation behavior. Upon inputting observed evidence, the Bayesian inference network model executes probabilistic inference algorithms, such as confidence propagation, to calculate the posterior probability distribution of each predefined target intent category under the current observation.

[0079] Based on the posterior probability distribution, the interactive system calculates information entropy and uses the calculated information entropy value as the control intention information entropy, which is used to quantify the explicitness of the current user's control intention. Specifically, the control intention information entropy... The following information entropy formula is used for calculation:

[0080] In the formula, Observational evidence consisting of feature vectors extracted from the main load data; This represents the total number of predefined target intent categories; It is the collection of all predefined target intent categories; When observational evidence is observed Afterwards, the target intent category is The posterior probability. Information entropy of manipulation intent. It is a quantitative assessment of the certainty of the user's control actions. The smaller the value, the more concentrated the posterior probability distribution, and the clearer the user's manipulation intention; conversely, the larger the value, the clearer the manipulation intention information entropy. The larger the value, the more ambiguous the user's intention to control the device. High information entropy of control intention may mean that the user is hesitant or makes a mistake.

[0081] The central processing unit selects the target intent category corresponding to the posterior probability peak as the currently matched target intent category; the posterior probability peak then serves as the confidence value corresponding to the target intent category. Target Intent Category With confidence level The determination of can be expressed by the following discriminant formula:

[0082] In the formula, the target intent category is a discretized classification of the user's deeper operational purpose by the interaction system, such as exploratory navigation, aggressive avoidance, or target alignment; the confidence score reflects the interaction system's degree of certainty in judging the current intent, and the confidence score is a floating-point number between 0 and 1. The central processing unit combines the parsed target intent category and confidence score with the oscillation frequency parameter inherited from the original main load data, and encapsulates them together into a new data structure, namely the intent inference result data.

[0083] The intent reasoning result data integrates the target intent category, the confidence level of the judgment, and the rhythmic characteristics of the manipulation action, providing contextual information for the downstream rendering processing module, and outputting the intent reasoning result data to the subsequent rendering processing module.

[0084] For example, in the case of the data verification interlock module, the central processing unit receives an encrypted dynamic message with a permission status flag of true and extracts the main payload data as {sequence:58,label:"fast forward",peak_accel:350.0,osc_freq:1.2}. The interactive system uses the feature vector [peak_accel:350.0,osc_freq:1.2] as observation evidence input to a pre-trained Bayesian inference network model.

[0085] The pre-trained Bayesian inference network model has three predefined target intent categories: "exploratory cruise," "aggressive avoidance," and "target alignment." Based on the observed evidence of the input, the pre-trained Bayesian inference network model infers the posterior probability distributions as follows: P("exploratory cruise"|D)=0.92, P("aggressive avoidance"|D)=0.05, and P("target alignment"|D)=0.03. The central processing unit calculates that the target intent category is "exploratory cruise," corresponding to the peak posterior probability, with a confidence value of 0.92 for "exploratory cruise."

[0086] The entropy of the manipulation intent is calculated as follows: The central processing unit packages and generates intent inference result data, the content of which is {target_intent:"Exploratory Cruise",confidence:0.92,osc_freq:1.2}, and then passes the intent inference result data to the next step.

[0087] In one embodiment of the present invention, the rendering instruction set compilation module is configured to perform the following steps: Search the local 3D asset library based on the target intent category and load the basic augmented reality scene mesh primitives as primitives; Based on the confidence level and the preset smoothing curve, the transparency coefficient and contour luminescence intensity of the output primitives are calculated as visual performance attributes. The frame rate of the graphics primitives is adjusted based on the oscillation frequency parameters carried by the intent reasoning result data. Merge and bind primitives, transparency coefficients, outline glow intensity values, and screen refresh rate, and compile to generate a rendering instruction set.

[0088] Upon receiving the intent reasoning result data, the central processing unit uses the target intent category contained in the intent reasoning result data as the search keyword to query the local 3D asset library. The local 3D asset library is a pre-organized collection of resources, including model files, textures, materials, and animations, and uses the target intent category as the index label.

[0089] The local 3D asset library stores various augmented reality elements related to the interactive scene. The interactive system matches and loads the corresponding basic augmented reality scene mesh primitives according to the search keywords, which constitute the basic 3D model objects of the interactive scene. For example, if the target intention category is exploratory cruise, the basic augmented reality scene mesh primitives corresponding to the navigation arrow and dynamic light trail are loaded.

[0090] The central processing unit reads the confidence score recorded in the intent inference result data. To achieve a smooth transition in visual effects, the interactive system defines a smoothing mapping curve, which is derived from the confidence score... arrive A smooth mapping curve to visual intensity, such as an S-curve. A smooth mapping curve is a mapping function. It is used to convert confidence values, which represent uncertainty, into visually continuously changing parameters, thus avoiding abrupt changes in visual effects.

[0091] Based on the mapping points of the current confidence level values ​​on the smooth mapping curve, the central processing unit calculates and outputs two visual parameters: the transparency coefficient and the contour luminescence intensity value of the basic augmented reality scene mesh primitives. Transparency coefficient Through mapping formula Calculate the transparency factor It is between (Completely transparent) and Floating-point numbers between (completely opaque); See appendix Figure 4 The curve reflects the S-shaped non-linear growth trend of the luminous intensity coefficient as the confidence level smoothly transitions from 0 to 1. When the confidence level of the system's input intent inference reaches a high level, as shown by the horizontal coordinate of the marked point in the figure at 0.92, the corresponding vertical coordinate coefficient approaches its maximum value, thus ensuring that the visual performance attributes of the grid primitives can be smoothly and naturally enhanced dynamically according to the clarity of the intent.

[0092] Contour luminous intensity value Through mapping formula Calculation, where The preset upper limit of luminous intensity, such as 255 in RGB color values ​​or a normalization factor of 1.0, is set based on the physical limitations of the hardware color bit depth of the target display terminal and the upper limit of the luminous properties of the rendering engine's materials; the upper limit of luminous intensity controls the brightness of the edge luminous effect of the basic augmented reality scene mesh primitives during rendering; This refers to the input confidence level value.

[0093] The central processing unit (CPU) analyzes the oscillation frequency parameters carried in the intent inference result data and dynamically adjusts the rendering engine's default refresh rate based on these parameters. The dynamic adjustment of the refresh rate can be calculated using the following formula:

[0094] in, The corrected frame rate is the refresh rate of the screen. This is the default refresh rate for the rendering engine; The input is the oscillation frequency parameter; The preset gain coefficient, for example, ranges from 5 to 15 frames / Hz. The preset gain coefficient is set based on the balance between the human visual persistence effect and the rendering engine hardware performance, so as to ensure that, without causing screen tearing, it provides sufficient visual feedback smoothness to match the user's hand oscillation frequency is greater than the above-mentioned reference frequency threshold for fine-tuning operations. This is a preset reference frequency threshold, in Hz, for example, set from 0.5Hz to 1.0Hz. This setting is based on the inherent upper limit of the human hand's smooth, uniform operation frequency in ergonomics, used to distinguish between the user's regular cruising operation and high-frequency adjustments, i.e., oscillation frequencies greater than 1.0Hz. Dynamic correction of the screen refresh rate ensures a positive correlation between the refresh rate of the basic augmented reality scene mesh primitives and the oscillation frequency parameter of the user's control, thereby matching the dynamic effects of the basic augmented reality scene mesh primitives with the user's control rhythm.

[0095] The central processing unit (CPU) merges and binds four elements: the loaded basic augmented reality scene mesh primitives, the calculated transparency coefficient, the contour luminescence intensity value, and the corrected frame rate, and compiles them to generate a rendering instruction set. The rendering instruction set is a complete data package containing rendering objects, rendering attributes, and rendering timing controls; it can be parsed and executed by the graphics rendering pipeline.

[0096] For example, following the example of the intent reasoning mapping module, the central processing unit receives intent reasoning result data, the content of which is {target_intent:"Exploratory Cruise",confidence:0.92,osc_freq:1.2}. Based on the target intent category "Exploratory Cruise", the central processing unit retrieves and loads a model file named "nav_arrow.obj" and a dynamic flowing light visual effect named "flow_trail.vfx" from the local 3D asset library as the basic augmented reality scene mesh primitives.

[0097] The central processing unit reads a confidence level of 0.92. This assumes the smoothing curve used is a simple linear mapping. The calculated transparency coefficient is The contour luminous intensity value is ,in This is the preset upper limit of luminous intensity. The central processing unit resolves the oscillation frequency parameter to be 1.2Hz. Assuming the default frame rate of the rendering engine is 60fps, the gain coefficient is set. =10 and reference frequency threshold The refresh rate is 1.0Hz, and the formula for adjusting the refresh rate is: The corrected screen refresh rate is 100%. .

[0098] The central processing unit (CPU) compiles four elements—the basic augmented reality scene mesh primitives, the transparency coefficient, the contour luminescence intensity value, and the corrected frame rate—to generate a rendering instruction set. The logical content of the rendering instruction set can be represented as: {RenderObjects:["nav_arrow.obj","flow_trail.vfx"],Properties:{transparency:0.92,emission_intensity:0.92\times I_{max}},Timing:{refresh_rate:62}}. This rendering instruction set is then sent to the display driver layer.

[0099] In one embodiment of the present invention, the multi-level synchronous display module is used to perform the following steps: The display terminal's network address is parsed from the rendering instruction set; Generate foreground guide images and 3D environment model data based on the rendering instruction set; Obtain the global master clock signal and calculate a unified screen output timestamp based on the global master clock signal and the screen refresh rate; A foreground guide image carrying a timestamp is directionally projected onto a foreground transparent screen display terminal configured in the first hardware layer; The environmental 3D model data with timestamps is pushed to the background giant screen display terminal configured in the second hardware layer.

[0100] The display distribution module within the central processing unit receives and parses the rendering instruction set, extracts the preset display terminal network address from the rendering instruction set. The display terminal network address is a set of network addresses configured during the initialization of the interactive system, specifying the network identifiers of the foreground transparent screen display terminal and the background giant screen display terminal, such as Internet Protocol addresses.

[0101] The display distribution module initiates two parallel rendering pipelines. The first rendering pipeline is responsible for generating foreground content. It takes the basic augmented reality scene mesh primitives, combines the transparency factor and the contour luminescence intensity value, performs 3D rendering, and rasterizes it into a 2D image to generate the foreground guide image. The foreground guide image is the final 2D image that contains all foreground virtual elements and has applied visual effects such as the transparency factor and contour luminescence intensity value.

[0102] The second rendering pipeline loads the corresponding 3D environment model data based on the current scene logic; that is, the original 3D model and scene data that constitute the background image. The display distribution module calls the system-configured clock signal synchronizer. This synchronizer implements a resident system service for network time protocols or precision time protocols to ensure clock consistency across all nodes in the interactive system and to obtain the global master clock signal.

[0103] Based on the global master clock signal and the corrected frame rate specified in the rendering instruction set, the display distribution module calculates a unified screen output timestamp for the upcoming foreground guide image data stream and the environment 3D model data stream. To ensure precise synchronization between the foreground guide image data stream and the environment 3D model data stream, the calculation of the screen output timestamp follows the following correction model formula:

[0104] In the calibration model formula, This represents the current global master clock time obtained from the clock signal synchronizer; The comprehensive latency compensation value takes into account the estimated average time spent on data rendering, encoding, and network transmission. This refers to the calculated screen output timestamp, which is a high-precision absolute time value. By pre-calculating and adding a comprehensive delay compensation value, it is ensured that the foreground guide image data stream and the environment 3D model data stream arrive at the foreground transparent screen display terminal and the background giant screen display terminal at the corresponding screen output timestamps. All variables in the calibration model formula are in units of time, such as seconds or milliseconds, ensuring consistent dimensions and the correctness of the physical meaning of time calculations.

[0105] The output timestamp is encoded into the data frame headers of the foreground guide image data stream and the environment 3D model data stream, serving as the rendering trigger for instructing the foreground transparent screen display terminal and the background giant screen display terminal to display the corresponding frame. The display distribution module transmits the foreground guide image data stream carrying the output timestamp via the network to the foreground transparent screen display terminal configured at the first hardware level. With special optical display devices, such as transparent LED screens, the foreground transparent screen display terminal allows users to see the foreground guide image superimposed on the screen while also seeing the physical environment behind it. Simultaneously, it pushes the environment 3D model data stream, also carrying the output timestamp, to the background giant screen display terminal configured at the second hardware level. This conventional large-size, high-resolution display is used to create a macroscopic environmental atmosphere, thereby achieving a multi-depth, cross-physical screen composite augmented reality visual interaction closed loop.

[0106] For example, following the example of the rendering instruction set compilation module, the display distribution module receives the rendering instruction set and parses the display terminal network address from it as {foreground:"192.168.1.101",background:"192.168.1.102"}. Based on the rendering instruction set, the display distribution module then uses basic augmented reality scene mesh primitives such as "nav_arrow.obj" to... The transparency coefficient is used for 3D rendering and rasterization to generate a 1920×1080 pixel foreground guide image.

[0107] Simultaneously, the display distribution module loads "highway night view" as environmental 3D model data, associated with the target intent category of "exploratory cruise." At this time, the display distribution module obtains the current global master clock signal time from the clock signal synchronizer. The estimated overall delay compensation value is 1672531200.500s. The value is 0.016s. The display distribution module calculates the target's screen output timestamp as 0.016s. .

[0108] The display distribution module sends the encoded foreground guiding image data stream, marking the screen output timestamp in the data frame header of the foreground guiding image data stream as 1672531200.516, to the address 192.168.1.101 of the foreground transparent screen display terminal. The display distribution module also pushes the environment 3D model data stream, similarly marking the screen output timestamp as 1672531200.516, to the address 192.168.1.102 of the background giant screen display terminal.

[0109] After receiving the foreground guide image data stream and the environment 3D model data stream, the foreground transparent screen display terminal and the background giant screen display terminal located at these two network identifier addresses will wait for their respective local clocks to reach 1672531200.516 before synchronously displaying the frame of foreground guide image or environment 3D model data corresponding to the screen output timestamp. This is because the modified screen refresh rate required by the rendering instruction set is... The output timestamp of the next frame will be calculated as 1672531200.516+1 / 62s, approximately 1672531200.532s.

[0110] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A 3D AR-assisted mode switching system for entity model position mapping, characterized in that, include: The data processing module acquires the spatial positional signals of the entity model and performs dimensionless processing to generate a normalized positional vector flow. The motion feature extraction module extracts a subset of time-series signals from the normalized position vector stream, and then extracts feature combinations from the time-series signal subset to obtain a discrete motion feature set. The encrypted dynamic message generation module extracts the dynamic parameters of the discrete motion feature set, encapsulates and hashes the discrete motion feature set and dynamic parameters to obtain the encrypted dynamic message; The data verification interlock module intercepts encrypted dynamic messages, performs verification and comparison on the encrypted dynamic messages, and obtains the encrypted dynamic messages in the permission state. The intent reasoning mapping module extracts the main payload data of the encrypted dynamics message under the permission state, outputs the target intent category and confidence value, and obtains the intent reasoning result data; The rendering instruction set compilation module parses the intent reasoning result data, calculates the visual parameters of the mesh primitives based on the target intent category and confidence value, and compiles the rendering instruction set. The multi-level synchronous display module extracts the network address of the display terminal from the rendering instruction set, calculates a unified screen output timestamp by combining it with the global master clock signal, and distributes the data stream carrying the timestamp to the multi-level display terminals for synchronous rendering output.

2. The 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, The process of acquiring the spatial positional signals of the entity model and performing dimensionless processing to generate a normalized positional vector flow includes the following steps: Acquire a six-degree-of-freedom positional data stream containing three-dimensional spatial coordinates and attitude angles from the physical model; Filter and denoise the six-degree-of-freedom positional data stream to obtain the spatial positional signal, and calculate the velocity and acceleration parameters of the spatial positional signal; Extract the physical scale parameters of the interactive system; By using physical scale parameters, minimum-maximum normalization mapping is performed on the three-dimensional spatial coordinates, attitude angles, and time derivatives of each order to generate a normalized state vector flow.

3. The 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, Extracting a subset of time-series signals from the normalized position vector flow, and then extracting feature combinations from this subset to obtain a discrete motion feature set, includes the following steps: Set a dynamic time window and use the dynamic time window to cut the normalized state vector stream into a subset of time signals with a sliding step size; Extract peak extrema, trough extrema, and rate of change parameters from a subset of time-series signals to obtain feature combinations; The feature combination is compared with the basic dynamic template in the pre-stored physical benchmark feature library; When the feature combination satisfies a set of conditions set by the basic dynamics template, a corresponding semantic identifier is attached to the time-series signal subset to generate a discrete motion feature set.

4. A 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, Extracting dynamic parameters from a discrete motion feature set, encapsulating and hashing the discrete motion feature set and dynamic parameters to obtain an encrypted dynamic message, includes the following steps: Extract the temporal sequence, peak acceleration, and oscillation frequency parameters corresponding to the discrete motion feature set as dynamic parameters; The dynamic parameters follow a preset communication structure protocol and are packaged and encapsulated into structured dynamic data packets; The structured dynamics data packets are serialized into a byte stream and a hash algorithm is executed to generate an encrypted verification sequence. The encrypted verification sequence is injected into the leading region of the structured dynamics data packet and then merged to generate an encrypted dynamics message.

5. A 3D AR-assisted mode switching system for entity model position mapping according to claim 4, characterized in that, Extracting the temporal sequence, peak acceleration, and oscillation frequency parameters corresponding to the discrete motion feature set as dynamic parameters includes the following steps: The maximum absolute value of the second-order time derivative contained in the discrete motion feature set within the time window is calculated as the peak acceleration; The Fast Fourier Transform algorithm is applied to the first-order time derivative contained in the discrete motion feature set to analyze the spectrum, and the frequency corresponding to the energy peak point is extracted as the oscillation frequency parameter. An incrementing counter is used to assign corresponding sequence numbers to the discrete motion feature set, which serve as the temporal arrangement order.

6. A 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, Intercepting encrypted dynamics messages, performing verification and comparison on the encrypted dynamics messages to obtain encrypted dynamics messages in the permitted state, includes the following steps: Intercept encrypted dynamics messages and extract the encrypted verification sequence and main payload data; Read the preset verification seed parameters and append them to the main payload data to form a byte stream to be verified; Perform a hash operation on the byte stream to be verified to generate a test comparison value; Perform bitwise operations to compare the encrypted verification sequence with the test comparison value; If it is determined that there is no discrepancy between the bitmasks of the two, then the permission state is activated.

7. A 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, Extract the main payload data of the encrypted dynamics message under the permission state, output the target intent category and confidence value, and obtain the intent inference result data, including the following steps: Input the main load data into a pre-trained Bayesian inference network model to calculate the posterior probability distribution of the preset target intent category; Information entropy is calculated based on the posterior probability distribution and used as the information entropy of the manipulation intention; The intent category corresponding to the posterior probability peak is selected as the target intent category, and the posterior probability peak is used as the confidence value. Combine the target intent category, confidence level, and oscillation frequency parameter to generate intent inference result data.

8. A 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, The process involves parsing the intent inference results data, calculating the visual parameters of the mesh primitives based on the target intent category and confidence level, and compiling a rendering instruction set, including the following steps: Search the local 3D asset library based on the target intent category and load the basic augmented reality scene mesh primitives as primitives; Based on the confidence level and the preset smoothing curve, the transparency coefficient and contour luminescence intensity of the output primitives are calculated as visual performance attributes. The frame rate of the graphics primitives is adjusted based on the oscillation frequency parameters carried by the intent reasoning result data. Merge and bind primitives, transparency coefficients, outline glow intensity values, and screen refresh rate, and compile to generate a rendering instruction set.

9. A 3D AR-assisted mode switching system for entity model position mapping according to claim 1, characterized in that, Extract the display terminal network address from the rendering instruction set, calculate a unified screen output timestamp by combining it with the global master clock signal correction, and distribute the data stream carrying the timestamp to multi-level display terminals for synchronous rendering output, including the following steps: The display terminal's network address is parsed from the rendering instruction set; Generate foreground guide images and 3D environment model data based on the rendering instruction set; Obtain the global master clock signal and calculate a unified screen output timestamp based on the global master clock signal and the screen refresh rate; A foreground guide image carrying a timestamp is directionally projected onto a foreground transparent screen display terminal configured in the first hardware layer; The environmental 3D model data with timestamps is pushed to the background giant screen display terminal configured in the second hardware layer.

10. A three-dimensional AR-assisted mode switching system for entity model position mapping according to claim 9, characterized in that, The unified screen output timestamp is calculated based on the global master clock signal and the screen refresh rate, including the following steps: Call the system's configured clock signal synchronizer to obtain the current global master clock signal; Extract the comprehensive latency compensation value that includes the estimated time consumption of data rendering, encoding, and network transmission; The time value corresponding to the global master clock signal is added to the comprehensive delay compensation value to obtain the screen output timestamp; The output timestamp is encoded into the data frame header of the data stream.

Images