Synchronization control system for film and television and lifting motion based on multi-dimensional sensing closed-loop feedback

Abstract

The application belongs to the technical field of virtual reality intersection, and relates to a film and television and lifting motion synchronous control system based on multidimensional sensing closed loop feedback, which comprises the following modules: a data acquisition and synchronization module, which constructs a physical state vector with a time stamp; a state prediction module, which generates a double-channel prediction signal containing a predicted state mean value and a predicted covariance matrix; a basic picture retrieval module, which extracts original digital image materials to generate a basic film picture frame based on a visual picture frame index number; a visual compensation strategy generation module, which dynamically generates visual compensation strategy parameters; an adaptive rendering module, which generates a visual and sensory coupling output frame; and a system control and closed loop synchronization module, which triggers the collection instruction of the next running sequence of the physical state vector at the time node when the visual and sensory coupling output frame completes screen refresh. The application solves the problem of dynamic mismatch between physical motion and visual presentation caused by system delay and model uncertainty.

Description

Technical Field

[0001] This invention belongs to the technical field of virtual reality and relates to a synchronous control system for film and video and lifting motion based on multi-dimensional sensing closed-loop feedback. Background Technology

[0002] The synchronization of visual presentation with physical motion in immersive experiences, particularly simulators and dynamic rides in theme parks, remains a persistent technological challenge. The core issue lies in the fact that the human vestibular system is extremely sensitive to changes in acceleration and posture, while the visual system receives images presented by the display device. When there is a time delay, phase inconsistency, or dynamic mismatch between physical motion and visual imagery, sensory conflict arises, causing dizziness and discomfort for the user and diminishing the overall sense of immersion. To address this problem, common industry approaches include open-loop control and simple closed-loop feedback control.

[0003] Open-loop control schemes pre-program the motion trajectory of the lifting system and the playback sequence of the video content, with both operating independently according to a preset timeline. For example, a device simulating a fall would initiate a timed descent program while simultaneously playing a pre-rendered video of a high-speed descent. Another rudimentary closed-loop feedback method involves acquiring the device's current position or speed through sensors and using this data to select one of several pre-stored video clips for playback, or simply adjusting the video playback rate to attempt to match the physical speed. These schemes can provide basic synchronization when the device operates under stable and predictable conditions.

[0004] However, traditional open-loop control methods have inherent limitations. They cannot respond to unexpected disturbances or non-model dynamics during system operation, such as elastic deformation of mechanical components or response differences caused by load changes. Deviations will occur between the actual trajectory of physical motion and the preset trajectory, leading to asynchrony with the video feed. While some existing simple closed-loop feedback schemes introduce feedback mechanisms, their processing delays are typically significant. From sensor data acquisition to controller processing and then to the video playback system's response, the time lag across the entire chain can reach tens or even hundreds of milliseconds. In scenarios involving rapid, intense motion requiring quick responses, this delay is sufficient for passengers to perceive a disconnect between visual and tactile feedback; this deficiency is particularly pronounced during moments of abrupt changes in motion.

[0005] Based on the above problems, the present invention aims to solve the problem of dynamic mismatch between physical motion and visual representation caused by system latency and model uncertainty. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a synchronous control system for video and lifting motion based on multi-dimensional sensing closed-loop feedback.

[0007] The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback includes:

[0008] The data acquisition and synchronization module is used to acquire real-time motion status data of the lifting car, perform time alignment processing on the real-time motion status data, and construct a physical state vector with timestamps.

[0009] The state prediction module is used to input the physical state vector into the preset dynamic system prediction model for forward extrapolation and generate a dual-channel prediction signal containing the predicted state mean and the prediction covariance matrix.

[0010] The basic image retrieval module is used to parse the dual-channel prediction signal to extract the prediction state mean, retrieve the visual image frame index number that matches the prediction state mean in the preset film and television state mapping database, and extract the original digital image material based on the visual image frame index number to generate basic film and television image frames.

[0011] The visual compensation strategy generation module is used to extract the prediction covariance matrix from the dual-channel prediction signal, compare the prediction covariance matrix with the preset system tolerance threshold, and dynamically generate visual compensation strategy parameters.

[0012] The adaptive rendering module is used to input basic film and television frames and visual compensation strategy parameters into the adaptive visual effects rendering engine for pixel-level reconstruction and generate visually coupled output frames.

[0013] The system control and closed-loop synchronization module is used to push the visual coupling output frame to the terminal video device for optical rendering, and to trigger the acquisition command of the physical state vector in the next running sequence at the time node when the visual coupling output frame completes screen refresh.

[0014] A further aspect of this invention involves constructing a physical state vector with a timestamp, comprising the following steps:

[0015] Activate the multi-dimensional inertial measurement unit deployed in the lifting car to simultaneously collect the car's real-time absolute position value, triaxial linear acceleration value, and angular attitude value as real-time motion state data;

[0016] Time alignment processing is performed on the real-time absolute position values, three-axis linear acceleration values, and angular attitude values ​​of the car, and a unified timestamp is added;

[0017] The real-time absolute position value of the car, the three-axis linear acceleration value, and the angular attitude value are integrated after being appended with a unified timestamp to generate a physical state vector.

[0018] A further aspect of the present invention involves generating a dual-channel prediction signal comprising the predicted state mean and the predicted covariance matrix, including the following steps:

[0019] The physical state vector is input into a preset extended Kalman filter model, and the car state estimate at the current moment is generated through multi-dimensional fusion filtering.

[0020] The state transition equation of the extended Kalman filter model is used to perform forward extrapolation of the car state estimate at the current moment, calculate the optimal expected position data and expected speed data of the car at future time nodes, and determine the optimal expected position data and expected speed data as the predicted state mean.

[0021] By deriving the system state distribution deviation parameters during the synchronous forward extrapolation process, a predictive covariance matrix for quantifying the degree of nondeterministic fluctuations is obtained.

[0022] The predicted state mean and the predicted covariance matrix are fused to output a dual-channel prediction signal.

[0023] A further aspect of this invention involves extracting original digital image materials based on visual frame index numbers to generate basic film and television frames, comprising the following steps:

[0024] Decompile the data packets of the dual-channel prediction signal and extract the optimal expected position data from the mean of the prediction state;

[0025] In the pre-set film and television state mapping database, the optimal expected position data is compared by a state parameter polling algorithm to lock the visual image frame index number that matches the optimal expected position data;

[0026] Based on the visual frame index, the corresponding original digital image material is extracted from the preset film and television rendering engine, and the original digital image material is decoded and loaded into the video memory to generate basic film and television frames.

[0027] A further aspect of the present invention involves dynamically generating visual compensation strategy parameters, including the following steps:

[0028] Extract the prediction covariance matrix from the dual-channel prediction signal;

[0029] Extract the position covariance submatrix representing the three-dimensional spatial position state of the car from the prediction covariance matrix;

[0030] Extract the diagonal elements of the position covariance submatrix and calculate the matrix trace parameter;

[0031] Compare the matrix trace parameters with the preset system tolerance threshold;

[0032] When the matrix trace parameter is less than or equal to the system tolerance threshold, a suppression parameter with a value of zero is generated as the visual compensation strategy parameter.

[0033] When the matrix trace parameter is greater than the system tolerance threshold, the visual compensation strategy parameters are generated by using a linear mapping function based on the excess value of the matrix trace parameter and the system tolerance threshold.

[0034] A further aspect of the present invention generates a visually coupled output frame, comprising the following steps:

[0035] Control the adaptive visual effects rendering engine to synchronously load basic film and television frame data and visual compensation strategy parameters;

[0036] When the visual compensation strategy parameter is set to the suppression parameter, the original pixel array distribution of the basic video frame is maintained, and pass-through rendering is performed to maintain the original resolution and generate visually coupled output frames.

[0037] With the visual compensation strategy parameters carrying physical conversion values, a dynamic motion blur filter is applied using the preset visual focus area of ​​the basic film and television frame composition as the center, in conjunction with the intensity constant assigned by the visual compensation strategy parameters and the principal direction vector obtained by decomposing the prediction covariance matrix, to generate a visually coupled output frame.

[0038] A further aspect of the present invention involves applying a dynamic motion blur filter using the intensity constant assigned by the visual compensation strategy parameters and the principal direction vector obtained from the decomposition of the prediction covariance matrix, comprising the following steps:

[0039] Extract the position covariance submatrix representing the three-dimensional spatial position state of the car from the prediction covariance matrix;

[0040] The position covariance submatrix extracted from the prediction covariance matrix is ​​subjected to eigenvalue decomposition, and the eigenvalue equation is solved to obtain multiple eigenvalues ​​and corresponding eigenvectors. The eigenvectors represent the directions in three-dimensional physical space.

[0041] Extract the largest eigenvalue from multiple eigenvalues;

[0042] The eigenvector corresponding to the largest eigenvalue is determined as the principal direction vector, which is used to characterize the direction with the greatest spatial uncertainty in system state prediction.

[0043] A further aspect of this invention involves streaming the visually coupled output frame to a terminal video device for optical rendering, and triggering a physical state vector acquisition command for the next running sequence at the time node when the visually coupled output frame completes screen refresh, including the following steps:

[0044] A transmission link is established to continuously transmit the visual-sensing coupling output frames to the panoramic projection array deployed around the lifting car for optical image processing and physical space light mapping;

[0045] Start the synchronous monitoring thread to monitor the time point at which the visual-sensory coupling output frame completes screen refresh on the panoramic projection array;

[0046] At the instant the screen refresh is completed, a command to acquire the physical state vector of the next running sequence is sent to the multi-dimensional inertial measurement unit, driving the control cycle process to generate an information flow closed loop.

[0047] A further aspect of this invention involves time-aligning the real-time absolute position values, triaxial linear acceleration values, and angular attitude values ​​of the car, and appending a unified timestamp, including the following steps:

[0048] The receiving time of the current concurrent data point is obtained through the built-in master clock;

[0049] Determine whether the time difference between receiving the real-time absolute position value, three-axis linear acceleration value, and angular attitude value of the car is within the preset synchronization error range;

[0050] If the receiving time difference is within the synchronization error range, a unified timestamp is generated using the master clock.

[0051] A unified timestamp is appended to the real-time absolute position value, triaxial linear acceleration value, and angular attitude value of the car within the same acquisition cycle.

[0052] A further aspect of this invention involves generating visual compensation strategy parameters by using a linear mapping function to calculate the excess magnitude of the matrix trace parameter and the system tolerance threshold when the matrix trace parameter exceeds the system tolerance threshold. This includes the following steps:

[0053] The excess value is obtained by subtracting the system tolerance threshold from the matrix trace parameter;

[0054] Extract the preset gain constant, which is used to map the excess amplitude to the intensity control range that the rendering engine can recognize;

[0055] The physical conversion value is obtained by multiplying the excess value by the gain constant.

[0056] The physical conversion values ​​are confirmed as the output parameters of the visual compensation strategy.

[0057] In summary, the present invention has the following beneficial technical effects:

[0058] 1. By introducing a prediction model based on extended Kalman filtering, the motion state of the elevator car in the next control cycle can be projected forward. The model integrates real-time data from multi-dimensional inertial measurement units and uses state transition equations to calculate the optimal expected position at future time points, rather than relying on current or past measurement positions. This prediction process compensates for the inherent delays in data processing and system response, enabling the video system to retrieve and prepare frame images corresponding to the future physical state in advance. At the macro level of the control flow, the timing of visual output is aligned with the timing of physical motion, improving the accuracy of visual-motion synchronization.

[0059] 2. By utilizing the prediction covariance matrix synchronously output from the prediction model, the uncertainty of system motion can be dynamically quantified. When the car experiences severe acceleration, deceleration, or vibration, the deviation between sensor data and the motion model increases, leading to an increase in the trace of the prediction covariance matrix. Based on this change, the system automatically generates visual compensation strategy parameters when the prediction uncertainty exceeds a preset threshold. This mechanism enables the system to perceive its own prediction confidence level in real time and convert it into a quantifiable control signal, providing a basis for real-time evaluation and adaptive adjustment of synchronization quality.

[0060] 3. Based on the prediction covariance matrix and visual compensation strategy parameters, the system can perform pixel-level reconstruction of basic video frames. When the car movement is stable and the prediction confidence is high, the system outputs a pass-through rendered image that maintains the original resolution. When the movement is violent and the prediction uncertainty increases, the system applies a dynamic motion blur effect to the image based on the principal direction vector of the covariance matrix and the intensity specified by the compensation parameters. This processing can actively mask the slight visual-motion misalignment caused by prediction errors by simulating the natural visual phenomena under violent movement, transforming clear erroneous images that cause sensory conflict into blurred dynamic images that conform to physiological expectations, maintaining the continuity and comfort of the experience at the micro-sensory level. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0062] Figure 1 This discloses a schematic diagram of the framework in the embodiments of this application.

[0063] Figure 2 This discloses a flowchart of an embodiment of this application. Detailed Implementation

[0064] The following is in conjunction with the appendix Figures 1-2 A preferred description of the present invention is provided below.

[0065] See attached document Figures 1-2 This invention proposes a synchronous control system for video and lifting motion based on multi-dimensional sensing closed-loop feedback, comprising the following modules:

[0066] The data acquisition and synchronization module is used to acquire real-time motion status data of the lifting car, perform time alignment processing on the real-time motion status data, and construct a physical state vector with timestamps.

[0067] The state prediction module is used to input the physical state vector into the preset dynamic system prediction model for forward extrapolation and generate a dual-channel prediction signal containing the predicted state mean and the prediction covariance matrix.

[0068] The basic image retrieval module is used to parse the dual-channel prediction signal to extract the prediction state mean, retrieve the visual image frame index number that matches the prediction state mean in the preset film and television state mapping database, and extract the original digital image material based on the visual image frame index number to generate basic film and television image frames.

[0069] The visual compensation strategy generation module is used to extract the prediction covariance matrix from the dual-channel prediction signal, compare the prediction covariance matrix with the preset system tolerance threshold, and dynamically generate visual compensation strategy parameters.

[0070] The adaptive rendering module is used to input basic film and television frames and visual compensation strategy parameters into the adaptive visual effects rendering engine for pixel-level reconstruction and generate visually coupled output frames.

[0071] The system control and closed-loop synchronization module is used to push the visual coupling output frame to the terminal video device for optical rendering, and to trigger the acquisition command of the physical state vector in the next running sequence at the time node when the visual coupling output frame completes screen refresh.

[0072] In one embodiment of the present invention, the data acquisition and synchronization module is used to acquire real-time motion state data of the elevator car, construct a physical state vector with a timestamp, and perform the following operations:

[0073] The multi-dimensional inertial measurement unit deployed in the elevator car is activated to synchronously collect the real-time absolute position, triaxial acceleration, and angular attitude values ​​of the car at a preset frequency. The real-time absolute position, triaxial acceleration, and angular attitude values ​​of the car are time-aligned and a unified timestamp is added. The multi-source heterogeneous data with the added unified timestamp are integrated to generate a physical state vector that reflects the absolute physical motion parameters at the current moment.

[0074] Specifically, the steps of acquiring real-time motion state data of the elevator car and constructing a timestamped physical state vector are executed by the data acquisition and synchronization processing unit deployed within the elevator car control system. The data acquisition and synchronization processing unit sends data acquisition commands to the multi-dimensional inertial measurement unit integrated on the elevator car structure via a deterministic time-triggered Ethernet protocol. Once activated, the multi-dimensional inertial measurement unit simultaneously activates its internal absolute position sensor, triaxial linear accelerometer, and angular attitude sensor according to the preset frequency set in the command. The absolute position sensor measures and outputs the vertical displacement of the car relative to a predetermined reference point in real time, thus forming the raw absolute position data stream.

[0075] The three-axis linear accelerometer and angular attitude sensor respectively acquire the linear acceleration components of the car along the three orthogonal axes and the angular velocity components around these three axes, generating raw three-axis linear acceleration data streams and raw angular attitude data streams. The data acquisition and synchronization processing unit uses its built-in master clock to perform time alignment processing on each set of concurrent data points received from the multi-dimensional inertial measurement unit, that is, to add a uniform timestamp to the real-time absolute position value, three-axis linear acceleration value, and angular attitude value of the car acquired within the same acquisition cycle.

[0076] The data acquisition and synchronization processing unit integrates the multi-source heterogeneous data after time alignment processing in a structured manner to generate an ordered data set. This set is a timestamped physical state vector that reflects the absolute physical motion parameters of the car at the current moment, and it is temporarily stored in the local transmission data buffer.

[0077] The multidimensional inertial measurement unit (IMU) is a hardware module integrating multiple sensors, including a laser time-of-flight sensor for acquiring the car's real-time absolute position, a microelectromechanical system (MEMS) accelerometer for acquiring triaxial linear acceleration, and a MEMS gyroscope for acquiring angular attitude. The preset frequency is set between 200Hz and 1000Hz, a setting designed to meet the data density requirements for capturing the car's transient dynamic characteristics during high-speed start-stop or vibration processes, while avoiding unnecessary load on the control system's real-time computing resources due to excessively high data sampling rates. Timestamps are provided by a temperature-compensated crystal oscillator within the data acquisition and synchronization processing unit, achieving microsecond-level time resolution to ensure time consistency required by subsequent state estimation algorithms.

[0078] The timestamped physical state vector is a data structure that internally arranges uniform timestamps, one-dimensional real-time absolute position values ​​of the car, three-dimensional triaxial linear acceleration values, and three-dimensional angular attitude values ​​in a predetermined order. Assuming the application scenario is a high-speed amusement park ride simulating a high-speed fall, the accuracy requirements for the weighting of acceleration data acquisition and clock synchronization are high in order to capture the most sensitive vertical motion changes for passengers.

[0079] For example, at the start time t0 of a certain control cycle, the data acquisition and synchronization processing unit sends an acquisition command to the multidimensional inertial measurement unit.

[0080] Assume that at time t0, the laser time-of-flight sensor measures the real-time absolute position of the elevator car to be 35.2m. The microelectromechanical system (MEMS) accelerometer measures the triaxial linear accelerations as ax = 0.05m / s², ay = -0.02m / s², and az = -2.5m / s², indicating that the car is undergoing an accelerated descent with slight horizontal swaying. The MEMS gyroscope measures the angular attitude values ​​as roll = 0.005rad, pitch = -0.008rad, and yaw = 0.0rad. The data acquisition and synchronization processing unit receives this set of physically simultaneous measurement data, calls its internal master clock, and appends a uniform timestamp to the data set, for example, 1678886401.005000 seconds. The system integrates these data according to a predetermined format to construct a seven-dimensional physical state vector P0 with timestamps, specifically [35.2, 0.05, -0.02, -2.5, 0.005, -0.008, 0.0]. This vector is then marked as data to be processed and retrieved from the data transmission buffer.

[0081] In one embodiment of the present invention, the state prediction module is used to receive a physical state vector and inject it into a dynamic system prediction model, calculate and generate a dual-channel prediction signal containing the predicted state mean and the predicted covariance matrix, and perform the following operations:

[0082] The physical state vector is input into a pre-set extended Kalman filter model. Multi-dimensional fusion filtering is used to eliminate the instantaneous sensor jitter caused by the elastic deformation of physical structural components. The state transition equation of the extended Kalman filter model is used to perform forward extrapolation of the nonlinear motion trend at the end of the next control cycle. The optimal expected position and expected speed data of the car at the future time node are calculated and determined as the predicted state mean. The system state distribution deviation parameters during the forward extrapolation process are calculated synchronously to derive the predicted covariance matrix that quantifies the degree of nondeterministic fluctuation. The predicted state mean and the predicted covariance matrix are fused using a communication protocol to output a dual-channel predicted signal.

[0083] Specifically, upon receiving the timestamped physical state vector, the dynamic system prediction model performs calculations to generate a dual-channel prediction signal containing the predicted state mean and prediction covariance matrix. First, the dynamic system prediction model uses the received timestamped physical state vector as the measurement value at the current moment and injects it into a preset extended Kalman filter model. The extended Kalman filter model performs its core update step, using multi-dimensional observation data in the timestamped physical state vector to correct the predicted state at the previous moment. Through multi-dimensional fusion filtering, a corrected car state estimate at the current moment is generated. This process effectively reduces sensor measurement noise and instantaneous jitter caused by the instantaneous elastic deformation of the car's physical structural components during high-speed movement.

[0084] The dynamic system prediction model invokes the state transition equations of the extended Kalman filter model to perform a forward extrapolation on the corrected current car state estimate. This extrapolation process simulates the nonlinear motion trend of the car in the next control cycle, calculating the optimal expected position and expected velocity data of the car at a future time node. This vector containing multidimensional motion parameters is determined as the predicted state mean. During this forward extrapolation, the model simultaneously calculates the propagation of the state covariance. That is, based on the Jacobian matrix of the state transition equations and the preset process noise covariance, it calculates the system state distribution deviation parameters during the forward extrapolation process and derives a prediction covariance matrix that quantifies the degree of prediction uncertainty fluctuation.

[0085] The dynamic system prediction model encapsulates the calculated predicted state mean and prediction covariance matrix using a predefined communication protocol, integrating them into a structured data packet, which is then output as a dual-channel prediction signal. The forward derivation process of the extended Kalman filter model follows the following state prediction equation and covariance prediction equation:

[0086] The formula for calculating the predicted state mean is:

[0087]

[0088] The extended Kalman filter model refers to a recursive algorithm for state estimation of nonlinear systems. In this embodiment, it is used to fuse multi-source sensor data and predict the future motion state of the car. State transition equations It is a nonlinear function describing the car's kinematics, which calculates the state at the next moment based on the current state and control input. Indicates in The optimal state estimate after correction of the measured values ​​at time 1. express The control input vector that is constantly applied to the system, such as the drive voltage of the motor or the opening command of the hydraulic valve. That is, the predicted state mean, which is the model's prediction of the mean state. Prior estimates of the system state at time t.

[0089] The formula for calculating the predicted covariance matrix is:

[0090]

[0091] in, Is The state estimation covariance matrix after correction at each time step. It is a state transition equation exist The Jacobian matrix calculated at point . It is the process noise covariance matrix, used to describe the system uncertainties that the model cannot fully model, such as unpredicted changes in air resistance. Its value is set based on offline experimental calibration and statistical analysis. It involves transposing the Jacobian matrix that describes the system dynamics. The dual-channel prediction signal encapsulates two core data blocks: the prediction state mean and the prediction covariance matrix.

[0092] For example, the dynamic system prediction model receives a generated timestamped physical state vector P0 with values ​​[35.2, 0.05, -0.02, -2.5, 0.005, -0.008, 0.0]. The model uses this as a measurement. Update the internal state, assuming the updated optimal state estimate is obtained. At the midpoint, the vertical position is 35.198m, and the vertical velocity is -1.505m / s. The model then initiates a forward extrapolation for the next control cycle, with the control cycle time Δt set to 0.005s. The state transition equations are constructed based on kinematic principles. Calculate the mean of the predicted state For example, the predicted position value at the next moment is 35.198 + (-1.505 × 0.005) = 35.190475 m, and the predicted velocity value at the next moment is -1.505 + (-2.5 × 0.005) = -1.5175 m / s. The generated predicted state mean is a vector containing position and velocity [35.1905, -1.5175].

[0093] Since the car is currently in a non-steady-state operating range with non-zero acceleration, the predicted covariance matrix calculated by the model... The values ​​of the diagonal elements will increase accordingly. The system predicts the covariance matrix. Extract the position covariance submatrix representing the three-dimensional spatial position state. After calculation, the obtained position covariance submatrix is ​​a 3×3 matrix [[0.002, 0.001, 0.000], [0.001, 0.006, 0.000], [0.000, 0.000, 0.030]], where the diagonal elements 0.002m², 0.006m², and 0.030m² represent the variance of the car's predicted position at the next moment along the three orthogonal spatial axes, respectively, quantifying the uncertainty of the prediction.

[0094] The vector [35.1905, -1.5175] and the matrix [[0.002, 0.001, 0.000], [0.001, 0.006, 0.000], [0.000, 0.000, 0.030]] are packaged to generate a dual-channel prediction signal and sent to the downstream steps.

[0095] In one embodiment of the present invention, the basic image retrieval module is used to analyze the dual-channel prediction signal in real time, extract the mean of the prediction state, retrieve and generate basic video image frames in the video mapping engine, and perform the following operations:

[0096] The data packets of the dual-channel prediction signal are decompiled, and the mean value of the predicted state representing the expected state of the car's motion at the next moment is extracted. The preset film and television state mapping database compares the optimal expected position data in the mean value of the predicted state through a state parameter polling algorithm, and locks the visual image frame index number that matches the expected physical position. Based on the visual image frame index number, the corresponding original digital image material is extracted from the film and television rendering engine to generate the basic film and television image frame without the introduction of visual diffusion effect compensation.

[0097] Specifically, the parsing of the dual-channel prediction signal and the generation of basic video frames are performed by the video mapping engine. The data parsing unit of the video mapping engine receives the dual-channel prediction signal from the dynamic system prediction model. This unit decompiles the data packets of the dual-channel prediction signal according to a predefined communication protocol, separating two data components. That is, the mean of the predicted state, which represents the expected state of the car's motion at the next moment, is extracted to a temporary register, and the prediction covariance matrix is ​​directly routed to an independent parallel processing channel for use in subsequent steps.

[0098] The state retrieval unit of the video mapping engine reads the predicted state mean from a temporary register and extracts the optimal expected position data in scalar form. This unit then performs a search operation in the preset video state mapping database using a state parameter polling algorithm. This algorithm uses the optimal expected position data as the query keyword and compares it against the video state mapping database to locate the visual frame index number that is closest to or within the same preset interval as the expected physical position. Upon receiving the visual frame index number, the image extraction unit of the video mapping engine uses it as an address pointer to access the video rendering engine, which stores all the original digital image materials. Based on the visual frame index number, the video rendering engine locates and extracts the corresponding original digital image material from its internal high-speed solid-state storage array. This material is decoded and loaded into the graphics processor's video memory, forming a basic video frame composed of the original pixel array without any visual diffusion effect compensation, awaiting subsequent rendering processing.

[0099] The video state mapping database is a static lookup table loaded into memory during system initialization. This table uses the physical location of the elevator car as the key and the corresponding frame number as the value to construct a one-to-one mapping relationship. For example, the position resolution can be set to correspond to one frame per 0.01m. In this embodiment, the state parameter polling algorithm employs either direct address access based on a hash table or binary search based on a sorted array to ensure that the conversion from the optimal expected position data to the visual frame index is completed within microseconds. The optimal expected position data refers to the scalar value representing the vertical coordinate of the elevator car in the predicted state mean vector. The video rendering engine is a server or hardware module that manages massive pre-rendered digital image sequences and possesses high-throughput data reading capabilities. The basic video frame is a two-dimensional pixel matrix stored in video memory, and its data content completely reflects the single-frame image information of the original video file.

[0100] For example, the video mapping engine receives the generated dual-channel prediction signal. First, its data parsing unit unpacks the signal, extracts the mean of the predicted state, which is a vector [35.1905, -1.5175], and stores it in a register; simultaneously, it extracts the position covariance submatrix [[0.002, 0.001, 0.000], [0.001, 0.006, 0.000], [0.000, 0.000, 0.030]]. The state retrieval unit separates the optimal expected position data from the mean of the predicted state, which is 35.1905m. The system then uses this value as a keyword to query the video state mapping database. Assume that one of the mapping rules stored in the database is: the car state with a physical location range of [35.190m, 35.200m) corresponds to the visual frame index number 5824.

[0101] The state parameter polling algorithm, through comparison, finds that 35.1905 falls within this range, thus locking in and outputting the visual frame index number as 5824. The image extraction unit sends index number 5824 to the video rendering engine, which then locates the image file numbered 5824 on its storage medium, such as a 16-bit high dynamic range image format file with a resolution of 3840x2160, decodes it, and loads it into video memory. This unprocessed pixel data in video memory is the basic video frame, ready to be used in subsequent rendering steps.

[0102] In one embodiment of the present invention, the visual compensation strategy generation module is used to compare the prediction covariance matrix in the dual-channel prediction signal with the system tolerance threshold, dynamically generate visual compensation strategy parameters that control the intensity of the visual occlusion effect, and perform the following steps:

[0103] Extract the prediction covariance matrix from the dual-channel prediction signal to measure the degree of drastic motion transition and the degree of environmental abrupt interference; extract the diagonal elements of the prediction covariance matrix to calculate the matrix trace parameter, and compare its value with a pre-set system tolerance threshold representing the stable operation range in real time; when the matrix trace parameter drops to the system tolerance threshold or below, generate a suppression parameter with a value of zero; when the matrix trace parameter crosses the system tolerance threshold, use a linear mapping function to convert and output the visual compensation strategy parameter to guide the spatial blur intensity based on the excess value amplitude.

[0104] Specifically, the comparison of the predicted covariance matrix with the system tolerance threshold, and the dynamic generation of visual compensation strategy parameters, are performed by the visual compensation strategy generation unit. This unit extracts the predicted covariance matrix, which measures the degree of drastic motion transitions and environmental abrupt interference, from the output dual-channel prediction signal and loads it into a dedicated computation register. This unit executes the calculation process for the matrix trace parameter, which involves algorithmically accessing all diagonal elements of the predicted covariance matrix and summing them. The result is determined as the matrix trace parameter. The unit reads a pre-set system tolerance threshold, used to characterize the stable operating range, from its internal non-volatile configuration memory area and compares the real-time calculated matrix trace parameter with the read system tolerance threshold.

[0105] Based on the comparison results, the visual compensation strategy generation unit executes condition judgment and parameter generation logic. When the value of the matrix trace parameter is less than or equal to the system tolerance threshold, it indicates that the car is running smoothly and the prediction confidence is high. At this time, the unit generates a suppression parameter with a value of zero or zero. Conversely, when the value of the matrix trace parameter exceeds the system tolerance threshold, it indicates that the car is experiencing violent dynamics or unpredictable disturbances. The unit then uses a preset linear mapping function to convert the values ​​based on the excess range between the two values ​​and outputs the visual compensation strategy parameter values ​​used to guide the subsequent spatial blur intensity.

[0106] The calculation of the matrix trace parameter and the visual compensation strategy parameter follows the formulas below:

[0107] The formula for calculating the trace parameter of a matrix is:

[0108]

[0109] in, It is the prediction covariance matrix output from the state prediction module. The extracted position covariance submatrix is ​​usually as follows: Matrix. Since the complete state vector contains variables with different physical dimensions such as position, velocity, and attitude, the position covariance submatrix is ​​extracted and calculated, ensuring that the diagonal elements... It has a unified physical unit. Matrix trace parameter It is the trace of the covariance submatrix of the location, which compresses multi-dimensional spatial location uncertainty information into a single scalar, intuitively reflecting the absolute uncertainty level of the system's location prediction in three-dimensional physical space.

[0110] The generation logic of the visual compensation strategy parameters can be represented by a piecewise function:

[0111]

[0112] in, That is, the trace parameter of the matrix. These are the final output parameters for the visual compensation strategy. System tolerance threshold. This constant was calibrated based on statistical analysis of extensive historical operational data and passenger subjective comfort feedback experiments. Assuming the application scenario is a simulated emergency evasive maneuver by a space fighter, this threshold is set relatively sensitively to minimize dizziness while maintaining immersion. For example, this threshold is set to 0.01 when the matrix trace parameter... If this value is exceeded, it is considered that the deviation between the actual movement of the system and the prediction model may exceed the range of what passengers can perceive. This represents the preset linear gain coefficient. The uncertainty deviation, which is dimensionless (square dimension) of the position error, is mapped to a dimensionless intensity parameter, with units of . .

[0113] Suppression parameter The value is usually set to 0, indicating that no visual compensation effect needs to be applied. The conversion coefficient of the linear mapping function. It is a gain constant used to adjust the compensation intensity, mapping the excess magnitude of the matrix trace parameter to an intensity control range that the rendering engine can recognize, such as a normalized interval of 0 to 1.

[0114] For example, the visual compensation strategy generation unit receives the position covariance submatrix generated by the state prediction module. The values ​​are [[0.002, 0.001, 0.000], [0.001, 0.006, 0.000], [0.000, 0.000, 0.030]]. This unit calculates the matrix trace parameter τ, which is the sum of the diagonal elements: τ = 0.002 + 0.006 + 0.030 = 0.038. The system reads the system tolerance threshold τth from the configuration, whose preset value is 0.01. The system compares the calculated matrix trace parameter 0.038 with the system tolerance threshold 0.01. Since 0.038 is greater than 0.01, the system determines that the current car movement is violent and visual compensation needs to be enabled. The system executes a linear mapping function to calculate the visual compensation strategy parameters. Assuming mapping coefficients If the value is set to 10, the excess range is 0.038 - 0.01 = 0.028, resulting in the final calculated visual compensation strategy parameters. This value of 0.28 will be used as a non-zero intensity instruction and output to the next step of the adaptive visual rendering engine to control the intensity of dynamic motion blur.

[0115] In one embodiment of the present invention, the adaptive rendering module is used to start the adaptive visual effects rendering engine to reconstruct the basic film and television frame at the pixel level according to the visual compensation strategy parameters, generate a visually coupled output frame, and perform the following steps:

[0116] The adaptive visual effects rendering engine is controlled to simultaneously load the basic video frame to be processed and the visual compensation strategy parameters that guide the macro rendering intensity; when the visual compensation strategy parameters are suppression parameters, the original pixel array distribution of the basic video frame is maintained, and pass-through rendering is performed to preserve the high-frequency details of the global structure and maintain the original resolution.

[0117] With the visual compensation strategy parameters carrying physical conversion values, a dynamic motion blur filter is applied with the visual focus area of ​​the basic film and television frame composition as the center, in conjunction with the intensity constant assigned by the visual compensation strategy parameters and the principal direction vector obtained by the decomposition of the prediction covariance matrix, to generate a visual-sensory coupling output frame to cover up the vestibular sensory temporal misalignment deviation.

[0118] Specifically, the pixel-level reconstruction of the basic video frame and the generation of the visually coupled output frame are executed by the adaptive visual effects rendering engine on the graphics processor. The adaptive visual effects rendering engine synchronously loads two core inputs via its internal data bus: one is the basic video frame to be processed, and the other is the visual compensation strategy parameters used to guide the macroscopic rendering intensity. The rendering engine executes a conditional branching process, and its execution path depends entirely on the specific value of the visual compensation strategy parameters. When the visual compensation strategy parameters are zero (a suppression parameter), the rendering engine triggers a pass-through rendering path that maintains the original resolution. In this path, the rendering pipeline is configured in a no-operation mode, meaning that the original pixel array distribution of the loaded basic video frame is not modified in any way, preserving all high-frequency details and structures contained in the image, and directly copying them to the output frame buffer.

[0119] When the visual compensation strategy parameters carry specific non-zero physical conversion values, the rendering engine activates a pixel-level reconstruction path. In this path, the engine uses the preset visual focus area of ​​the basic video frame as the center, jointly scheduling the intensity constant assigned by the visual compensation strategy parameters, and the principal direction vector obtained by decomposing the prediction covariance matrix generated by the state prediction module, to apply a dynamic motion blur filter. Regardless of the processing path, the final set of pixels output is defined as a visual-sensory coupling output frame. This frame aims to mask or compensate for vestibular sensory temporal misalignment caused by deviations between physical motion and the ideal model through visual means.

[0120] When performing pixel-level reconstruction, the process of obtaining the principal direction vector involves the eigenvalue decomposition of the position covariance submatrix, which solves the following eigenvalue problem:

[0121]

[0122] This equation is used to obtain the spatial direction with the greatest positional uncertainty. This represents the position covariance submatrix extracted from the prediction covariance matrix. It represents the characteristic vector in three-dimensional space, and the dimensionless pure spatial direction vector in the physical state space. Represented as a scalar factor, quantized along its corresponding eigenvector. The magnitude of the position error variance in the direction. After eliminating the interference from velocity and attitude dimensions, the largest eigenvalue is... The corresponding "main direction vector" is directly called by the adaptive visual effects rendering engine as the physical space direction for applying pixel motion blur in the graphics pipeline.

[0123] The adaptive visual effects rendering engine refers to a collection of shader programs running on a graphics processor that can change its post-processing effects in real time based on input parameters. The suppression parameter is set to 0 in this embodiment. Maintaining the original resolution for pass-through rendering is the simplest rendering mode, with the lowest computational cost. The visual focus area is a pixel region within a predefined range of the frame, typically the geometric center of the image. This region receives weaker or no processing when a blur effect is applied to simulate the focusing effect of the human eye. The dynamic motion blur filter is a post-processing shader that simulates motion blur by weighted averaging of pixel colors in a specific direction.

[0124] The intensity constant is the parameter of the generated visual compensation strategy. The value of this directly controls the length or radius of the blur kernel. (Main direction vector) From the prediction covariance matrix The decomposed eigenvalues ​​and the largest eigenvalues The associated feature vectors represent the direction with the greatest uncertainty in system state prediction, and this direction is used as the direction for applying motion blur. The visual-sensory coupling output frame is the final image after adaptive rendering, ready to be streamed to the display device.

[0125] For example, the adaptive visual effects rendering engine loads the generated base video frame, image number 5824, and the calculated visual compensation strategy parameter, which has a value of 0.28. Since the parameter value 0.28 is not zero, the rendering engine determines that pixel-level reconstruction is required. The engine retrieves the position covariance submatrix stored in shared memory. Its value is [[0.002, 0.001, 0.000], [0.001, 0.006, 0.000], [0.000, 0.000, 0.030]]. To determine the physical direction of motion blur application, the engine performs feature decomposition. By solving the eigenvalue equation, three eigenvalues ​​and their corresponding eigenvectors are obtained. The largest eigenvalue, λmax, is equal to 0.030, and its corresponding eigenvector is the principal direction vector. The vector is [0, 0, 1], which indicates that the main uncertainty is concentrated in the vertical lifting direction.

[0126] The rendering engine invokes the dynamic motion blur shader, passing in two key parameters: an intensity constant of 0.28 and a blur direction set to the primary direction vector [0, 0, 1]. The shader uses the center of the image as the visual focus area and samples and weights each pixel of the base video frame along [0, 0, 1] (i.e., the vertical direction), with the degree of blur determined by 0.28. The resulting pixel matrix is ​​the visually coupled output frame, which exhibits a directional visual motion blur effect that matches the car's violent vertical motion.

[0127] In one embodiment of the present invention, the system control and closed-loop synchronization module is used to push the visually coupled output frame to the terminal video device for optical rendering, and drive the control cycle process to generate an information flow closed loop, performing the following steps:

[0128] A transmission link is established to continuously transmit the visual-sensing coupling output frames to the panoramic projection array deployed around the lifting car for optical image processing and physical space light mapping; the time node when the visual-sensing coupling output frames complete screen refresh is monitored, and at this projection instant, the acquisition command of the physical state vector of the next running sequence is sent to the multi-dimensional inertial measurement unit.

[0129] The continuous loop of data-driven feedback and predictive compensation, which crosses from the end physical perception system to the top visual rendering system, induces the high-dimensional blurred dynamic images received by the passenger's optic nerve to directly match the high-vibration spatial acceleration impact experienced by the inner ear vestibular organ.

[0130] Specifically, the presentation of the visually coupled output frames and the closed-loop drive of the entire control process are handled by the main control and streaming module. This module establishes a stable transmission link with the panoramic projection array deployed around the elevator car via a high-speed digital video interface, such as Fiber Optic DisplayPort (a high-definition digital display interface). The generated visually coupled output frames are transmitted to the panoramic projection array in real time as a continuous frame sequence. After receiving the video stream, each projection unit in the panoramic projection array performs its own optical image processing, accurately mapping the image content onto its assigned screen area in the form of physical rays, collectively forming a panoramic image with edge blending. At the same time, the main control and streaming module internally initiates a synchronization monitoring thread, which is specifically used to monitor the exact time point when the visually coupled output frames complete screen refresh on the panoramic projection array.

[0131] Once a vertical synchronization signal is detected, indicating that the optical rendering of a frame has been physically completed, the thread sends a new acquisition command to the multi-dimensional inertial measurement unit deployed on the car at the instant of projection, requesting it to start the acquisition of the physical state vector for the next running sequence. This action marks the formation of a complete data flow loop. The system continuously and cyclically executes this data-driven feedback and predictive compensation alternation link from the end physical perception system, i.e., the multi-dimensional inertial measurement unit, across to the top visual rendering system, i.e., the adaptive visual effects rendering engine, and finally back to the optical rendering of the physical world. This continuous cyclical process aims to use technical means to induce the high-dimensional blurred dynamic images received by the passenger's optic nerve to directly match the high jitter spatial acceleration impact experienced by the inner ear vestibular organ in terms of temporal and dynamic characteristics, thereby achieving deep coupling of visual and kinematic senses.

[0132] The transmission link bandwidth ensures uncompressed transmission of high-resolution, high-frame-rate video signals, such as meeting the data transmission requirements of a 120Hz refresh rate at 4K ultra-high-definition display resolution. The panoramic projection array consists of multiple laser projectors with geometric correction and edge blending functions, ensuring distortion-free and uniformly bright projected images. The screen refresh time is a key time marker guiding the synchronization rhythm of the entire system, captured through hardware interrupts or clock synchronization protocols. The acquisition command for the physical state vector in the next running sequence is completely consistent with the acquisition command in content and format, marking the start of a new "perception-prediction-rendering-presentation" cycle. The alternating link of data-driven feedback and predictive compensation is the macroscopic manifestation of the core control logic of this invention, describing the complete path of information from the physical world to the digital world, processed, and then back to the physical world, influencing perception.

[0133] For example, within a certain control cycle, the generated visual-coupled output frame is packaged into a data packet by the main control and streaming module. The main control and streaming module then transmits this frame data via optical fiber to a circular projection array consisting of three projectors inside the car. The three projectors respectively calculate the images for the left, right, and front views and project them onto the circular screen, collectively forming a complete dynamic image. Assume the system refresh rate is 200Hz, meaning the image is updated every 5ms. The synchronization monitoring thread captures the screen refresh completion signal occurring at time t1 = 1678886401.010000 seconds.

[0134] At this instant, the thread immediately triggers the instruction generation unit, sending a new acquisition instruction to the multi-dimensional inertial measurement unit. Upon receiving the instruction, the multi-dimensional inertial measurement unit initiates a new round of sensor data acquisition, generating the next physical state vector. This new vector will then undergo prediction by the state prediction module, image indexing by the basic image retrieval module, compensation strategy generation by the visual compensation strategy generation module, and rendering reconstruction by the adaptive rendering module, finally being projected at approximately t2 = 1678886401.015000 seconds. This complete process, occurring within the 5ms interval from t1 to t2, is executed continuously and cyclically, ensuring that every frame seen by the passenger is based on a prediction of the actual motion they have just experienced, achieving continuous dynamic synchronization between vision and tactile sensation.

[0135] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0136] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A synchronous control system for video and lifting motion based on multi-dimensional sensing closed-loop feedback, characterized in that, include: The data acquisition and synchronization module is used to acquire real-time motion status data of the lifting car, perform time alignment processing on the real-time motion status data, and construct a physical state vector with timestamps. The state prediction module is used to input the physical state vector into the preset dynamic system prediction model for forward extrapolation and generate a dual-channel prediction signal containing the predicted state mean and the prediction covariance matrix. The basic image retrieval module is used to parse the dual-channel prediction signal to extract the prediction state mean, retrieve the visual image frame index number that matches the prediction state mean in the preset film and television state mapping database, and extract the original digital image material based on the visual image frame index number to generate basic film and television image frames. The visual compensation strategy generation module is used to extract the prediction covariance matrix from the dual-channel prediction signal, compare the prediction covariance matrix with the preset system tolerance threshold, and dynamically generate visual compensation strategy parameters. The adaptive rendering module is used to input basic film and television frames and visual compensation strategy parameters into the adaptive visual effects rendering engine for pixel-level reconstruction and generate visually coupled output frames. The system control and closed-loop synchronization module is used to push the visual coupling output frame to the terminal video device for optical rendering, and to trigger the acquisition command of the physical state vector in the next running sequence at the time node when the visual coupling output frame completes screen refresh.

2. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, Constructing a timestamped physical state vector involves the following steps: Activate the multi-dimensional inertial measurement unit deployed in the lifting car to simultaneously collect the car's real-time absolute position value, triaxial linear acceleration value, and angular attitude value as real-time motion state data; Time alignment processing is performed on the real-time absolute position values, three-axis linear acceleration values, and angular attitude values ​​of the car, and a unified timestamp is added; The real-time absolute position value of the car, the three-axis linear acceleration value, and the angular attitude value are integrated after being appended with a unified timestamp to generate a physical state vector.

3. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, The process of generating a two-channel prediction signal, which includes the predicted state mean and the predicted covariance matrix, involves the following steps: The physical state vector is input into a preset extended Kalman filter model, and the car state estimate at the current moment is generated through multi-dimensional fusion filtering. The state transition equation of the extended Kalman filter model is used to perform forward extrapolation of the car state estimate at the current moment, calculate the optimal expected position data and expected speed data of the car at future time nodes, and determine the optimal expected position data and expected speed data as the predicted state mean. By deriving the system state distribution deviation parameters during the synchronous forward extrapolation process, a predictive covariance matrix for quantifying the degree of nondeterministic fluctuations is obtained. The predicted state mean and the predicted covariance matrix are fused to output a dual-channel prediction signal.

4. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, The process of extracting raw digital image materials based on visual frame index numbers to generate basic film and television frames includes the following steps: Decompile the data packets of the dual-channel prediction signal and extract the optimal expected position data from the mean of the prediction state; In the pre-set film and television state mapping database, the optimal expected position data is compared by a state parameter polling algorithm to lock the visual image frame index number that matches the optimal expected position data; Based on the visual frame index, the corresponding original digital image material is extracted from the preset film and television rendering engine, and the original digital image material is decoded and loaded into the video memory to generate basic film and television frames.

5. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, Dynamically generating visual compensation strategy parameters includes the following steps: Extract the prediction covariance matrix from the dual-channel prediction signal; Extract the position covariance submatrix representing the three-dimensional spatial position state of the car from the prediction covariance matrix; Extract the diagonal elements of the position covariance submatrix and calculate the matrix trace parameter; Compare the matrix trace parameters with the preset system tolerance threshold; When the matrix trace parameter is less than or equal to the system tolerance threshold, a suppression parameter with a value of zero is generated as the visual compensation strategy parameter. When the matrix trace parameter is greater than the system tolerance threshold, the visual compensation strategy parameters are generated by using a linear mapping function based on the excess value of the matrix trace parameter and the system tolerance threshold.

6. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, Generating a visually coupled output frame includes the following steps: Control the adaptive visual effects rendering engine to synchronously load basic film and television frame data and visual compensation strategy parameters; When the visual compensation strategy parameter is set to the suppression parameter, the original pixel array distribution of the basic video frame is maintained, and pass-through rendering is performed to maintain the original resolution and generate visually coupled output frames. With the visual compensation strategy parameters carrying physical conversion values, a dynamic motion blur filter is applied using the preset visual focus area of ​​the basic film and television frame composition as the center, in conjunction with the intensity constant assigned by the visual compensation strategy parameters and the principal direction vector obtained by decomposing the prediction covariance matrix, to generate a visually coupled output frame.

7. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 6, characterized in that, Applying a dynamic motion blur filter to the intensity constant assigned by the joint visual compensation strategy parameters and the principal direction vector obtained from the decomposition of the prediction covariance matrix includes the following steps: Extract the position covariance submatrix representing the three-dimensional spatial position state of the car from the prediction covariance matrix; Eigenvalue decomposition is performed on the position covariance submatrix extracted from the prediction covariance matrix, and the eigenvalue equation is solved to obtain multiple eigenvalues ​​and corresponding eigenvectors. The eigenvectors represent the directions in three-dimensional physical space. Extract the largest eigenvalue from multiple eigenvalues; The eigenvector corresponding to the largest eigenvalue is determined as the principal direction vector, which is used to characterize the direction with the greatest spatial uncertainty in system state prediction.

8. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 1, characterized in that, The visually coupled output frame is streamed to the terminal video device for optical rendering, and the physical state vector acquisition command for the next running sequence is triggered at the time node when the visually coupled output frame completes screen refresh. This includes the following steps: A transmission link is established to continuously transmit the visual-sensing coupling output frames to the panoramic projection array deployed around the lifting car for optical image processing and physical space light mapping; Start the synchronous monitoring thread to monitor the time point at which the visual-sensory coupling output frame completes screen refresh on the panoramic projection array; At the instant the screen refresh is completed, a command to acquire the physical state vector of the next running sequence is sent to the multi-dimensional inertial measurement unit, driving the control cycle process to generate an information flow closed loop.

9. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 2, characterized in that, The real-time absolute position values, three-axis linear acceleration values, and angular attitude values ​​of the car are time-aligned and a unified timestamp is added, including the following steps: The receiving time of the current concurrent data point is obtained through the built-in master clock; Determine whether the time difference between receiving the real-time absolute position value, three-axis linear acceleration value, and angular attitude value of the car is within the preset synchronization error range; If the receiving time difference is within the synchronization error range, a unified timestamp is generated using the master clock. A unified timestamp is appended to the real-time absolute position value, triaxial linear acceleration value, and angular attitude value of the car within the same acquisition cycle.

10. The video and lifting motion synchronization control system based on multi-dimensional sensing closed-loop feedback according to claim 5, characterized in that, When the matrix trace parameter is greater than the system tolerance threshold, the visual compensation strategy parameters are generated by converting the excess value of the matrix trace parameter and the system tolerance threshold using a linear mapping function, including the following steps: The excess value is obtained by subtracting the system tolerance threshold from the matrix trace parameter; Extract the preset gain constant, which is used to map the excess amplitude to the intensity control range that the rendering engine can recognize; The physical conversion value is obtained by multiplying the excess value by the gain constant. The physical conversion values ​​are confirmed as the output parameters of the visual compensation strategy.

Images