A method and system for processing data of an MR device

Abstract

The embodiment of the application provides a kind of MR equipment data processing method and system, it is related to MR equipment data processing technical field, method includes obtaining the instantaneous running frequency data and initial asynchronous time warp transformation matrix of the graphics processor and unit matrix;The variance of the graphics processor in the preset time window is calculated based on the instantaneous running frequency data, and a stability index is obtained;The initial asynchronous time warp transformation matrix and unit matrix are weighted and fused based on the stability index, and a target asynchronous time warp transformation matrix is obtained;According to the target asynchronous time warp transformation matrix and the preset rendering picture, a target rendering picture is determined.The embodiment of the application can improve the accuracy of asynchronous time warp and improve the visual stability of MR equipment.

Description

Technical Field

[0001] This application relates to the field of MR device data processing technology, and in particular to an MR device data processing method and system. Background Technology

[0002] In related technologies, head motion prediction mechanisms used to compensate for head rotation delay rely on two core inputs: historical data on the user's head posture and historical data on rendering delay. Historical head posture data is used to infer the user's movement trends, while historical rendering delay data is used to estimate the time required for image generation. This allows for pre-rendering and adjustment of the image when the user turns their head, thereby reducing perceived delay.

[0003] However, both input sources received by the prediction mechanism were interfered with. On the one hand, the head pose data was mixed with signals generated by electromagnetic noise, making it impossible for the system to accurately determine whether the head was actually experiencing slight tremors or was merely affected by sensor noise. On the other hand, the rendering latency data exhibited irregular and severe jitter, making it impossible for the system to accurately predict when the next frame would be ready. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a data processing method and system for MR devices, designed to improve the accuracy of asynchronous time warp and enhance the visual stability of MR devices.

[0005] In a first aspect, embodiments of this application provide a method for processing MR device data, including:

[0006] Obtain the instantaneous operating frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and identity matrix;

[0007] Based on the instantaneous operating frequency data, the variance of the graphics processor within a preset time window is calculated to obtain a stability index;

[0008] The initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused based on the stability index to obtain the target asynchronous time warp transformation matrix; the target rendering image is determined based on the target asynchronous time warp transformation matrix and the preset rendering image.

[0009] According to some embodiments of this application, when the operating clock frequency of the graphics processor exhibits a smooth and continuous unidirectional drift trend; the step of calculating the variance of the graphics processor within a preset time window based on the instantaneous operating frequency data to obtain a stability index includes:

[0010] Obtain the instantaneous operating frequency sequence of the graphics processor within a preset time period;

[0011] Trend analysis is performed on the instantaneous operating frequency sequence to obtain the trend slope;

[0012] The stability index is obtained based on the slope of the trend.

[0013] According to some embodiments of this application, the step of weighted fusion of the initial asynchronous time-warped transformation matrix and the identity matrix based on the stability index to obtain the target asynchronous time-warped transformation matrix includes:

[0014] Obtain the instantaneous stability index and weighted fusion stability index of the current frame;

[0015] Based on the difference between the instantaneous stability index and the weighted fusion stability index, and a preset stability index change limit, the instantaneous stability index is adjusted to obtain a smooth stability index.

[0016] The initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused according to the smoothness stability index to obtain the target asynchronous time warp transformation matrix.

[0017] According to some embodiments of this application, the preset stability index variation limit is obtained through the following steps:

[0018] Obtain the scene type information of the current application and the instantaneous angular velocity or angular acceleration of head movement, wherein the scene type information includes requirements for visual stability, requirements for response speed, and instantaneous angular velocity or angular acceleration of head movement;

[0019] Based on the visual stability requirements, the response speed requirements, and the instantaneous angular velocity or angular acceleration of the head movement, a preset stability index variation limit is obtained.

[0020] According to some embodiments of this application, the step of weightedly fusing the initial asynchronous time-warped transformation matrix and the identity matrix based on the smoothness stability index to obtain the target asynchronous time-warped transformation matrix includes:

[0021] Acquire at least two different types of system operational reliability assessment information, wherein the system operational reliability assessment information includes the operating status information of the graphics processor and the data quality information of the inertial measurement unit;

[0022] Based on the reliability assessment information of at least two different types of systems and the preset fusion rules, a comprehensive reliability index is obtained;

[0023] The initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused according to the smoothness stability index and the comprehensive reliability index to obtain the target asynchronous time warp transformation matrix.

[0024] According to some embodiments of this application, obtaining a comprehensive reliability index based on the at least two different types of system operational reliability assessment information and preset fusion rules includes:

[0025] Obtain characteristic information of the current application scenario, wherein the characteristic information includes an emphasis on visual stability or response speed;

[0026] Acquire head movement activity information, wherein the activity information includes head angular velocity or angular acceleration;

[0027] Based on the characteristic information of the application scenario and the activity information of the head movement, the relative weights of the graphics processor's operating status information and the inertial measurement unit's data quality information in the fusion process are adjusted to obtain the target relative weights.

[0028] A comprehensive reliability index is obtained based on the target relative weight, the graphics processor's operating status information, and the inertial measurement unit's data quality information.

[0029] According to some embodiments of this application, adjusting the relative weights of the graphics processor's operating state information and the inertial measurement unit's data quality information during the fusion process, based on the characteristic information of the application scenario and the activity information of the head movement, to obtain the target relative weights, includes:

[0030] The first weight adjustment direction for obtaining the characteristic information of the application scenario and the second weight adjustment direction for obtaining the activity information of the head movement;

[0031] Based on the first weight adjustment direction, the second weight adjustment direction, and the preset priority, the target weight dominant direction is obtained;

[0032] Based on the dominant direction of the target weight, the relative weights of the graphics processor's operating status information and the inertial measurement unit's data quality information during the fusion process are adjusted to obtain the target relative weight.

[0033] According to some embodiments of this application, after obtaining the target weight dominance direction based on the first weight adjustment direction, the second weight adjustment direction, and the preset priority, the method further includes:

[0034] Obtain the instantaneous rate of change of the target asynchronous time warp transformation matrix determined by the target weight-dominant direction;

[0035] When the instantaneous rate of change is greater than the threshold of visual stability, the priority of the current target weight dominant direction is reduced.

[0036] According to some embodiments of this application, it also includes:

[0037] When the instantaneous rate of change is greater than the preset priority adjustment trigger first threshold, and the duration for which the instantaneous rate of change is greater than the preset priority adjustment trigger first threshold is greater than the preset trigger duration, the priority of the current target weight dominant direction is reduced.

[0038] When the instantaneous rate of change is less than or equal to the preset priority adjustment trigger second threshold, and the duration of the instantaneous rate of change being less than or equal to the preset priority adjustment trigger second threshold is greater than the preset recovery duration, the priority of the target weight dominant direction is restored.

[0039] Secondly, embodiments of this application provide an MR device data processing system, comprising:

[0040] The acquisition module is used to acquire the instantaneous operating frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and identity matrix;

[0041] The calculation module is used to calculate the variance of the graphics processor within a preset time window based on the instantaneous operating frequency data, and obtain a stability index.

[0042] The weighted fusion module is used to perform weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix based on the stability index to obtain the target asynchronous time warp transformation matrix.

[0043] The determination module is used to determine the target rendering image based on the target asynchronous time warp transformation matrix and the preset rendering image.

[0044] According to the technical solution of this application embodiment, at least the following beneficial effects are achieved: The embodiment first obtains the instantaneous operating frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and identity matrix; based on the instantaneous operating frequency data, it calculates the variance of the graphics processor within a preset time window to obtain a stability index; based on the stability index, it performs a weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix to obtain a target asynchronous time warp transformation matrix; based on the target asynchronous time warp transformation matrix and a preset rendering image, it determines the target rendering image. This application embodiment can improve the accuracy of asynchronous time warp and enhance the visual stability of MR devices.

[0045] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0046] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0047] Figure 1 A flowchart illustrating a data processing method for an MR device provided in one embodiment of this application;

[0048] Figure 2 A schematic diagram of the process for obtaining the stability index in one embodiment of this application;

[0049] Figure 3 This is a schematic diagram of the process for obtaining the target asynchronous time warp transformation matrix according to one embodiment of this application;

[0050] Figure 4 A schematic diagram of the process for obtaining a preset stability index change limit provided in one embodiment of this application;

[0051] Figure 5 A schematic diagram of the process for obtaining the target asynchronous time warp transformation matrix provided in another embodiment of this application;

[0052] Figure 6 A schematic diagram illustrating the process of obtaining a comprehensive reliability index provided in one embodiment of this application;

[0053] Figure 7 A schematic diagram illustrating the process of obtaining the target relative weights, provided for another embodiment of this application;

[0054] Figure 8 This is a schematic diagram illustrating the subsequent process of obtaining the target weight dominant direction according to one embodiment of this application;

[0055] Figure 9 A flowchart illustrating a data processing method for an MR device provided in another embodiment of this application;

[0056] Figure 10 This is a schematic diagram of an MR device data processing system provided in one embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical methods, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] It should be noted that the meaning of "multiple" (or "more than") in the description of the embodiments of this application refers to two or more, and "greater than," "less than," "exceeding," etc. are understood to exclude the number itself, while "above," "below," "within," etc. are understood to include the number itself. If "first," "second," etc. are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0059] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: the existence of a alone, the existence of b alone, the existence of c alone, the simultaneous existence of a and b, the simultaneous existence of a and c, the simultaneous existence of b and c, or the simultaneous existence of a, b, and c, where a, b, and c can be single or multiple.

[0060] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0061] Based on the above, this application proposes a data processing method and system for MR devices, aiming to improve the accuracy of asynchronous time distortion and enhance the visual stability of MR devices.

[0062] The MR device data processing method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application that implements the MR device data processing method, but is not limited to the above forms.

[0063] This application can be applied to numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices. It should be noted that in various specific embodiments of this invention, when processing is required based on data related to the characteristics of an object (e.g., user attributes or sets of attribute information), permission or consent from the corresponding object is obtained first, and the collection, use, and processing of this data comply with relevant laws and standards. Furthermore, when the embodiments of the present invention need to obtain the attribute information of an object, they will obtain the separate permission or separate consent of the corresponding object through pop-up windows or redirection to a confirmation page. After obtaining the separate permission or separate consent of the corresponding object, they will then obtain the relevant data of the object necessary for the embodiments of the present invention to operate normally.

[0064] See Figure 1 , Figure 1 This is a schematic flowchart illustrating an MR device data processing method according to an embodiment of this application. The MR device data processing method provided in this embodiment includes, but is not limited to, steps S110 to S140, which will be described in detail below.

[0065] Step S110: Obtain the instantaneous operating frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and identity matrix;

[0066] Step S120: Calculate the variance of the graphics processor within a preset time window based on the instantaneous operating frequency data to obtain the stability index;

[0067] Step S130: Based on the stability index, the initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused to obtain the target asynchronous time warp transformation matrix;

[0068] Step S140: Determine the target rendering screen based on the target asynchronous time warp transformation matrix and the preset rendering screen.

[0069] It should be noted that instantaneous operating frequency data refers to the real-time operating frequency of the graphics processor at a specific moment. This can be obtained by reading from the graphics processor's hardware registers, acquiring it through the performance monitoring interface provided by the operating system, or querying it through the graphics driver API. The initial asynchronous time-warp transformation matrix is ​​the reference transformation matrix used for geometric correction of the rendered image during asynchronous time-warp processing. It can be pre-calculated based on historical head pose data, set according to empirical values, or generated through a calibration process. Its main purpose is to provide a default image correction parameter to compensate for head movement delay. The identity matrix is ​​a transformation matrix that does not change any vectors or matrices in matrix operations. It can be implemented as a square matrix with diagonal elements of 1 and all other elements of 0. Its main purpose is to provide a benchmark representing no transformation or minimal transformation during weighted fusion, thereby reducing image distortion when the system is unstable. The stability index is a numerical value that quantifies the degree of fluctuation in the graphics processor's operating frequency. It can be represented by statistics such as the variance, standard deviation, or coefficient of variation of the frequency data. Its main purpose is to objectively reflect the operating stability of the graphics processor within a specific time window, providing a basis for subsequent dynamic adjustments. Weighted fusion is the process of combining two or more data points with specific weights to generate a new data point. It can be implemented using linear interpolation, nonlinear function mapping, or rule-based mixing. The main purpose is to dynamically adjust the asynchronous time warp transformation matrix based on the stability of the graphics processor to balance the image correction effect with visual stability.

[0070] In one embodiment, the core clock frequency of the graphics processor is read every few milliseconds. The initial asynchronous time-warp transform matrix can be generated at device startup based on a preset head motion model and display parameters, or calculated during user calibration based on a specific action sequence. The identity matrix is ​​a standard 3x3 or 4x4 identity matrix with diagonal elements of 1 and all other elements of 0. When calculating the stability index, a preset time window of, for example, 100 milliseconds is set. Within this window, instantaneous operating frequency data points of the graphics processor are collected, and then the sample variance of these data points is calculated. If 20 frequency data points are collected within 100 milliseconds, the variance of these 20 data points is calculated as the stability index. The value of the stability index can be normalized to a range of 0 to 1, where 0 represents complete stability and 1 represents high instability. When weighted and fused with the initial asynchronous time-warp transform matrix and the identity matrix, linear interpolation can be used. If the stability index is S (ranging from 0 to 1), the target asynchronous time-warped transformation matrix can be represented as M_target = (1 - S) * M_initial + S * M_identity. Here, M_initial is the initial asynchronous time-warped transformation matrix, and M_identity is the identity matrix. When S is close to 0, M_target is close to M_initial; when S is close to 1, M_target is close to M_identity. This weighted fusion calculation can be performed in the geometry shader stage of the graphics processor or in the post-processing stage of the display pipeline. Finally, when determining the target rendering image, the generated target asynchronous time-warped transformation matrix can be used as a transformation parameter and applied to the vertex data or texture coordinates of the preset rendering image through a graphics API (such as OpenGL or DirectX), thereby achieving geometric correction of the image. A full-screen quad can be used, its vertex coordinates transformed according to the target asynchronous time-warped transformation matrix, and the original rendering texture sampled, finally output to the display buffer.

[0071] It is worth noting that by monitoring the stability of the graphics processor's operating frequency in real time and quantifying it into a stability index, this application can dynamically adjust the asynchronous time warp transformation matrix. This dynamic adjustment mechanism effectively suppresses the jitter of the rendering process delay time value. At the same time, it also reduces the interference of noise signals introduced by power supply network fluctuations coupled into the inertial measurement unit on head posture prediction. The asynchronous time warp processing can perform image correction based on more accurate head posture prediction values, thereby avoiding unstable shaking or high-frequency jitter in the MR device image and improving the user's visual stability and immersion in the mixed reality environment.

[0072] See Figure 2 , Figure 2This is a flowchart illustrating the process of obtaining a stability index in one embodiment of this application. Regarding step S130 above, when the operating clock frequency of the graphics processor exhibits a smooth and continuous unidirectional drift trend, the variance of the graphics processor within a preset time window is calculated based on the instantaneous operating frequency data to obtain the stability index, including but not limited to steps S210 to S240. Each step will be described in turn below.

[0073] Step S210: Obtain the instantaneous operating frequency sequence of the graphics processor within a preset time period;

[0074] Step S220: Perform trend analysis on the instantaneous operating frequency sequence to obtain the trend slope;

[0075] Step S230: Obtain the stability index based on the trend slope.

[0076] It's important to note that trend analysis involves processing time-series data using statistical or mathematical methods to identify the overall direction, rate, or pattern of data change over time. Techniques such as linear regression, moving averages, exponential smoothing, or Fourier transforms can be employed. The aim is to reveal the underlying long-term patterns of data change, rather than short-term random fluctuations. The trend slope, quantified by trend analysis methods, represents the rate of change of a data sequence per unit time. Specifically, it's the slope of the straight line fitted to the data points. It can be calculated using the least squares method to fit a linear model, precisely characterizing the direction and intensity of frequency drift. Obtaining a stability index based on the trend slope involves using the trend slope as the core input and calculating a value through a pre-defined mapping relationship or function. This value is used to assess the stability of the graphics processing unit's (GPU) operation. This can be achieved by comparing the absolute value of the trend slope with a preset threshold or by mapping the trend slope to a stability score between 0 and 1 using an inverse proportional function. The goal is to provide a comprehensive quantitative value to improve the accuracy of subsequent weighted fusion.

[0077] In one embodiment, when the operating clock frequency of the graphics processor in the MR device exhibits a smooth and continuous unidirectional drift trend, the following implementation can be carried out: First, a data acquisition module can continuously acquire the instantaneous operating frequency data of the graphics processor over the past 5 seconds at a frequency of 100 times per second, forming an instantaneous operating frequency sequence containing 500 data points. These data points can be stored in a circular buffer to ensure that the latest frequency information is always obtained. Next, a trend analysis module can perform linear regression analysis on this instantaneous operating frequency sequence to calculate the slope of the fitted line, i.e., the trend slope. For example, if the fitted line equation is y = ax + b, then the value of 'a' is the trend slope. Finally, a stability calculation module can determine the stability index based on this trend slope. A threshold is set: if the absolute value of the trend slope is less than 0.01 MHz / ms, the graphics processor is considered to be running stably, and the stability index can be set to 1.0. If the absolute value of the trend slope is greater than 0.01 MHz / ms, the stability index can be mapped based on the magnitude of the absolute value of the trend slope through a preset nonlinear function (e.g., an exponential decay function), so that the larger the absolute value of the trend slope, the closer the stability index is to 0. The stability index can be calculated as exp(-k * |trend slope|), where k is an adjustment coefficient. When the graphics processor frequency exhibits significant unidirectional drift, the stability index will decrease accordingly, thus favoring the identity matrix in the subsequent asynchronous time-warp transformation matrix weighted fusion process, reducing image distortion caused by inaccurate predictions.

[0078] It is worth noting that, when the graphics processor's operating clock frequency exhibits a smooth and continuous unidirectional drift trend, this application, by performing trend analysis on the instantaneous operating frequency sequence and obtaining the trend slope, can more accurately assess the true operating state of the graphics processor. This avoids misjudging continuous unidirectional frequency drift as instability, thus enabling the calculated stability index to more accurately reflect the actual operating condition of the system. Therefore, based on this more accurate stability index, weighted fusion of the asynchronous time-warp transformation matrix can generate a target asynchronous time-warp transformation matrix that better reflects the actual situation, effectively avoiding screen flickering or jitter caused by inaccurate stability assessment.

[0079] See Figure 3 , Figure 3 This is a schematic diagram of the process for obtaining the target asynchronous time warp transformation matrix according to an embodiment of this application. Regarding step S130, which involves weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix based on a stability index to obtain the target asynchronous time warp transformation matrix, this includes, but is not limited to, steps S310 to S330. Each step will be described in detail below.

[0080] Step S310: Obtain the instantaneous stability index and weighted fusion stability index of the current frame;

[0081] Step S320: Adjust the instantaneous stability index based on the difference between the instantaneous stability index and the weighted fusion stability index, and the preset stability index change limit, to obtain the smooth stability index.

[0082] Step S330: The initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused according to the smoothness stability index to obtain the target asynchronous time warp transformation matrix.

[0083] It should be noted that the instantaneous stability index refers to the stability metric calculated in the current frame without any smoothing processing. It directly reflects the operating state of the graphics processor at the current moment and can be obtained directly by calculating the variance of the graphics processor's operating frequency within a very short time window. The weighted fusion stability index is a smoothed or historically accumulated stability metric that comprehensively considers the stability information of previous frames. It has a certain degree of lag and smoothness and can be calculated by performing an exponentially weighted moving average or a simple sliding average on historical instantaneous stability indices. Its purpose is to provide a relatively stable reference benchmark to avoid instantaneous fluctuations interfering with the weighted fusion process. The preset stability index change limit is a pre-set numerical range used to limit the maximum allowable change of the instantaneous stability index in a short period of time. Its purpose is to prevent drastic fluctuations in the stability index, thereby avoiding unnecessary jitter in the image.

[0084] In one embodiment, at the start of each frame rendering cycle, the system acquires the instantaneous stability index for the current frame. For example, this instantaneous stability index can be obtained by real-time monitoring of the graphics processor's clock frequency data over the last 10 milliseconds and calculating the variance of this frequency data. Simultaneously, the system maintains a weighted fusion stability index, which can be the result of smoothing the instantaneous stability indices of the past 100 frames using an exponentially weighted moving average (EWMA) algorithm. The weight of new data can be set to 0.1, and the weight of old data to 0.9, thus ensuring that the index reflects long-term trends and has good stability. Further, to adjust the instantaneous stability index, the system calculates the difference between the current instantaneous stability index and the weighted fusion stability index. For example, if the instantaneous stability index is 0.05 and the weighted fusion stability index is 0.03, the difference is 0.02. The system also presets a stability index change limit, for example, this limit can be set to 0.01. If the calculated difference exceeds the preset stability index change limit, for example, 0.02 is greater than 0.01, the system will adjust the instantaneous stability index. Specifically, the instantaneous stability index can be limited to within ± the preset change limit of the weighted fused stability index. For example, 0.05 can be adjusted to 0.03 + 0.01 = 0.04, thus obtaining a smooth stability index. This adjustment ensures that the stability index will not undergo drastic changes in a short period of time, and even if there are spikes in the original instantaneous index, they can be effectively suppressed. Finally, the system will perform a weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix based on this smooth stability index. For example, if the smooth stability index is 0.04, the final target asynchronous time warp transformation matrix can be calculated according to a preset weighting formula, such as: Target asynchronous time warp transformation matrix = (1 - Smooth stability index) * Initial asynchronous time warp transformation matrix + Smooth stability index * Identity matrix. By using a smoothed stability metric for weighting, the final transformation matrix can be made more stable, effectively reducing unstable flickering or high-frequency jitter during rendering and improving the smoothness and comfort of the visual experience.

[0085] See Figure 4 , Figure 4 This is a schematic flowchart illustrating the process of obtaining a preset stability index change limit according to one embodiment of this application. The process of obtaining the preset stability index change limit includes, but is not limited to, steps S410 to S420, which will be described in detail below.

[0086] Step S410: Obtain the scene type information of the current application and the instantaneous angular velocity or angular acceleration of the head movement. The scene type information includes requirements for visual stability, requirements for response speed, and instantaneous angular velocity or angular acceleration of the head movement.

[0087] Step S420: Based on the requirements for visual stability, response speed, and instantaneous angular velocity or angular acceleration of head movement, obtain the preset stability index change limit.

[0088] It should be noted that scene type information refers to the category to which the application or content currently running on the MR device belongs. This information is obtained through the application list, user settings, or automatic system identification of the currently running program's tags. The purpose is to reflect users' different preferences for visual experience and interactive smoothness in different application scenarios. Visual stability requirements refer to the user's tolerance for visual instability such as screen jitter and shaking in a specific scenario, or the level of stability they expect to achieve. This can be achieved using a parameter in a preset scene configuration file or by the user manually selecting a preference in the application settings. The purpose is to guide the system's optimization direction in terms of screen smoothness. Response speed requirements refer to the user's tolerance for the delay between head movement and screen update in a specific scenario, or the response speed they expect to achieve. This can be achieved using another parameter in a preset scene configuration file or by the user manually selecting a preference in the application settings. The purpose is to guide the system's optimization direction in terms of screen latency. The instantaneous angular velocity or angular acceleration of head movement is the rotational speed or rate of change of the user's head at a certain moment. This can be obtained through the inertial measurement unit (IMU) built into the MR device.

[0089] In one embodiment, when the MR device is running a viewing application with high requirements for visual stability, the system can identify the scene type information and determine that it has high requirements for visual stability but relatively low requirements for response speed. Simultaneously, if the instantaneous angular velocity or angular acceleration of the user's head movement is small, it indicates that the head movement is gentle. Based on these inputs, the system can set a preset stability index variation limit to a small value. This small value makes the adjustment of the instantaneous stability index more sensitive and precise, thereby ensuring that the image maintains high visual stability after smoothing and reducing unnecessary jitter.

[0090] In one embodiment, when the MR device is running a fast-paced interactive game that demands high response speed, the system identifies the scene type information and determines that while response speed is critical, visual stability requirements can be appropriately relaxed. Simultaneously, if the user's instantaneous angular velocity or angular acceleration of head movement is large, it indicates intense head movement. Based on these inputs, the system sets a relatively large preset stability index variation limit. This larger value allows adjustments to the instantaneous stability index to respond more quickly to changes in head movement, thereby reducing screen latency and ensuring smooth interaction. Specifically, the system maintains a preset mapping table or algorithm model. This mapping table or algorithm model takes scene type information (including visual stability and response speed requirements) and instantaneous angular velocity or angular acceleration of head movement as input and outputs corresponding preset stability index variation limits. The system can dynamically adjust the stability index variation limits based on the actual operating environment and user behavior, thereby achieving a balance between visual stability and response speed.

[0091] See Figure 5 , Figure 5 This is a flowchart illustrating the process of obtaining the target asynchronous time warp transformation matrix according to another embodiment of this application. Regarding step S330, which involves weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix based on a smoothness stability index to obtain the target asynchronous time warp transformation matrix, this includes, but is not limited to, steps S510 to S530. Each step will be described in detail below.

[0092] Step S510: Obtain at least two different types of system operation reliability assessment information, wherein the system operation reliability assessment information includes the graphics processor's operating status information and the inertial measurement unit's data quality information;

[0093] Step S520: Obtain a comprehensive reliability index based on at least two different types of system operational reliability assessment information and preset fusion rules;

[0094] Step S530: The initial asynchronous time warp transformation matrix and the identity matrix are weighted and fused according to the smoothness stability index and the comprehensive reliability index to obtain the target asynchronous time warp transformation matrix.

[0095] In one embodiment, when acquiring at least two different types of system operational reliability assessment information, the temperature, core load rate, and power consumption data of the graphics processing unit (GPU) are monitored in real time. This data can be directly provided by the GPU's driver interface or hardware sensors, serving as the GPU's operational status information. Simultaneously, the raw output data stream of the inertial measurement unit (IMU) is continuously analyzed. The noise level is assessed by calculating the short-term variance of the gyroscope output, or the reliability of data transmission is determined by monitoring the integrity and transmission delay of data packets. These constitute the IMU data quality information. After obtaining this information, a comprehensive reliability index can be calculated according to preset fusion rules. A weighted average model is set, where the GPU's operational status information and the IMU's data quality information are assigned different initial weights. When the GPU's temperature exceeds a certain threshold, the weight of its operational status information can be reduced; or when the IMU's data noise is too high, the weight of its data quality information can be reduced. In this way, a comprehensive reliability index reflecting the overall reliability level of the current system can be obtained—a normalized value between 0 and 1, where 1 indicates a good system operation and 0 indicates an extremely poor system operation. Finally, when weighting and fusing the initial asynchronous time-warped transformation matrix and the identity matrix based on the smoothness stability index and the comprehensive reliability index, a dynamic weighting formula is adopted: Target asynchronous time-warped transformation matrix = Smoothness stability index weight * Initial asynchronous time-warped transformation matrix + Comprehensive reliability index weight * Identity matrix. The sum of the smoothness stability index weight and the comprehensive reliability index weight is 1, and these two weights can be dynamically adjusted according to the value of the comprehensive reliability index. For example, when the comprehensive reliability index is high, the weight of the smoothness stability index can be relatively high to fully utilize its reflection of frequency stability; conversely, when the comprehensive reliability index is low, the weight of the comprehensive reliability index can be increased accordingly.

[0096] It is worth noting that, in the weighted fusion of the initial asynchronous time-warped transform matrix and the identity matrix in this embodiment, not only is the smoothness and stability of the GPU's operating frequency considered, but also a comprehensive reliability index formed by the GPU's operating status information and the IMU's data quality information is fully incorporated. Even when system reliability is compromised due to poor GPU operating status or degraded IMU data quality, the weighted fusion process can be adaptively adjusted through the participation of the comprehensive reliability index, reducing reliance on unstable or unreliable data, thereby generating a more accurate and robust target asynchronous time-warped transform matrix. This effectively avoids transform matrix deviations caused by actual system reliability issues, significantly improves the rendering quality and visual stability of MR devices under complex conditions, and reduces perceptible image shaking or jitter.

[0097] See Figure 6 , Figure 6 This is a flowchart illustrating the process of obtaining a comprehensive reliability index according to an embodiment of this application. Regarding step S520, which obtains the comprehensive reliability index based on at least two different types of system operational reliability assessment information and preset fusion rules, steps S610 to S630 are included, but are not limited to. Each step will be described in detail below.

[0098] Step S610: Obtain the characteristic information of the current application scenario, wherein the characteristic information includes the emphasis on visual stability or response speed;

[0099] Step S620: Obtain head movement activity information, wherein the activity information includes head angular velocity or angular acceleration;

[0100] Step S630: Based on the characteristic information of the application scenario and the activity information of head movement, adjust the relative weights of the graphics processor's running status information and the inertial measurement unit's data quality information in the fusion process to obtain the target relative weights.

[0101] Step S640: Based on the target relative weight, the graphics processor's operating status information, and the inertial measurement unit's data quality information, obtain the comprehensive reliability index.

[0102] In one embodiment, when a user is using an application displaying a static 3D model requiring detailed observation, the system can identify that the scene places a high emphasis on visual stability; when the user is playing a fast-paced action game, the system can identify that the scene places a high emphasis on response speed. This characteristic information can be pre-stored in the application configuration file or dynamically identified based on application behavior using a machine learning model. Simultaneously, the system determines the intensity of head movement by calculating the root mean square value of the head's angular velocity or the peak value of its angular acceleration. Based on the acquired application scene characteristic information and head movement activity information, the system dynamically adjusts the relative weights of the graphics processing unit's (GPU) runtime information and the inertial measurement unit's (IMU) data quality information during the fusion process, according to a preset weight adjustment rule table or by executing a weight adjustment algorithm. If the scene characteristics indicate a high emphasis on visual stability and the head movement activity is low, the system can increase the weight of the IMU data quality information, for example, setting its weight to 0.7, while setting the weight of the GPU runtime information to 0.3. Conversely, if the scene characteristics indicate a high emphasis on response speed and high head movement activity, the system can increase the weight of the graphics processor's (GPU) operational status information, setting its weight to 0.6, while setting the weight of the inertial measurement unit (IMU) data quality information to 0.4. After determining the target relative weights, the system applies these weights to the real-time acquired GPU operational status information and IMU data quality information. If the GPU operational status information value is R_GPU, the IMU data quality information value is R_IMU, and the target relative weights are W_GPU and W_IMU respectively, then the comprehensive reliability index can be calculated as (R_GPU * W_GPU + R_IMU * W_IMU). The system can obtain a dynamically adjusted comprehensive reliability index, which can more accurately reflect the overall reliability of the current MR device under specific operating conditions, thus providing more accurate input for subsequent asynchronous time warp processing.

[0103] See Figure 7 , Figure 7 This is a flowchart illustrating the process of obtaining the target relative weights according to another embodiment of this application. Regarding step S630 above, based on the characteristics of the application scenario and the activity information of head movement, the relative weights of the graphics processor's operating state information and the inertial measurement unit's data quality information are adjusted during the fusion process to obtain the target relative weights. This includes, but is not limited to, steps S710 to S730, which will be described in detail below.

[0104] Step S710: Obtain the first weight adjustment direction of the characteristic information of the application scenario and the second weight adjustment direction of the head movement activity information;

[0105] Step S720: Based on the first weight adjustment direction, the second weight adjustment direction, and the preset priority, obtain the target weight dominant direction;

[0106] Step S730: Based on the dominant direction of the target weight, adjust the relative weights of the graphics processor's operating status information and the inertial measurement unit's data quality information during the fusion process to obtain the target relative weight.

[0107] In one embodiment, assuming the MR device is running an application requiring high visual stability, such as a virtual building walkthrough, the characteristics of the application scenario instruct the system to prioritize increasing the weight of inertial measurement unit (IMU) data quality information to ensure high stability of the image even with slight head movements. This determines the first weight adjustment direction as "increasing the weight of IMU data quality information." Simultaneously, if the user is rapidly turning their head to look around, the activity level of the head movement instructs the system to prioritize increasing the weight of graphics processor (GPU) status information to ensure the image can quickly respond to large head movements. This determines the second weight adjustment direction as "increasing the weight of GPU status information." If the first and second weight adjustment directions conflict, the system will determine the priority based on a preset priority. For example, the priority of "visual stability" can be preset to be higher than "response speed." Even with active head movements, the system will prioritize visual stability. Therefore, based on the preset priority, the target weight dominance direction will be determined as "increasing the weight of IMU data quality information." Subsequently, the system will adjust the relative weights of GPU status information and IMU data quality information during the fusion process according to this target weight dominance direction. Specifically, the weighting of inertial measurement unit (IMU) data quality information in the overall reliability index calculation can be increased, while the weighting of graphics processor (GPU) status information can be correspondingly decreased. If the initial weighting is 50:50, guided by the goal of "increasing the weight of IMU data quality information," the weighting can be adjusted to 30:70, giving IMU data quality information a greater impact on the overall reliability index. Even when there are conflicts between the application scenario and head motion information, the system can make clear and reasonable weight adjustments based on preset priorities, thereby ensuring the accuracy of the overall reliability index.

[0108] See Figure 8 , Figure 8 This is a schematic diagram of the process following the determination of the target weight dominant direction according to an embodiment of this application. Regarding step S720 above, after obtaining the target weight dominant direction based on the first weight adjustment direction, the second weight adjustment direction, and the preset priority, the process includes, but is not limited to, steps S810 to S820, which will be described in detail below.

[0109] Step S810: Obtain the instantaneous rate of change of the target asynchronous time warp transformation matrix determined by the target weight-dominant direction;

[0110] Step S820: When the instantaneous rate of change is greater than the threshold of visual stability, reduce the priority of the current target weight dominant direction.

[0111] In one embodiment, assuming a user is rapidly turning their head in a highly dynamic mixed reality application scenario, the system may initially determine a target weight-dominant direction based on the characteristics of the application scenario and the activity level of the head movement. This direction tends to prioritize responding to head movements to ensure image responsiveness. This rapid response may cause the target asynchronous temporal distortion transformation matrix to change drastically within a short period. At this point, the system acquires the instantaneous rate of change of the target asynchronous temporal distortion transformation matrix determined by the target weight-dominant direction. A dedicated matrix analysis unit continuously monitors and calculates the Euler angle change rate of the transformation matrix between consecutive frames. When the calculated instantaneous rate of change, such as a rotational change of 5 degrees per millisecond, exceeds a preset visual stability threshold, such as a threshold of 3 degrees per millisecond, the system determines that the current image change speed is too fast and may cause user discomfort. The system immediately reduces the priority of the current target weight-dominant direction. This can be achieved by adjusting the internal weight allocation logic, reducing the weight coefficient of the head movement response direction that was originally given high priority, thereby allowing the system to consider the visual stability factors represented by the graphics processor's operating state information more in the subsequent weight fusion process. Through this dynamic adjustment, even in scenarios with rapid head movements, the system can promptly detect and correct excessive distortions that may cause image instability, ensuring that the final image presented to the user remains stable, thereby improving the overall visual experience.

[0112] See Figure 9 , Figure 9 This is a schematic flowchart of an MR device data processing method provided in another embodiment of this application. It includes, but is not limited to, steps S910 to S920, which will be described in detail below.

[0113] Step S910: When the instantaneous rate of change is greater than the preset priority adjustment triggers the first threshold, and the duration of the instantaneous rate of change being greater than the preset priority adjustment triggering the first threshold is greater than the preset trigger duration, the priority of the current target weight dominant direction is reduced.

[0114] Step S920: When the instantaneous rate of change is less than or equal to the preset priority adjustment trigger second threshold, and the duration of the instantaneous rate of change being less than or equal to the preset priority adjustment trigger second threshold is greater than the preset recovery duration, the priority of the target weight dominant direction is restored.

[0115] In one embodiment, the system continuously monitors the instantaneous rate of change of the target asynchronous temporal warp transform matrix determined by the target weight dominance direction. The instantaneous rate of change can be calculated as the Euclidean distance or matrix norm difference between the target asynchronous temporal warp transform matrix of the current frame and the target asynchronous temporal warp transform matrix of the previous frame. A preset priority adjustment trigger first threshold can be set to 0.05, and a preset priority adjustment trigger second threshold can be set to 0.01. When the system detects that the instantaneous rate of change is consistently greater than 0.05, a timer is started. If the accumulated time of this timer exceeds a preset trigger duration, such as 100 milliseconds, the system triggers a priority reduction operation. Reducing the priority of the target weight dominance direction can be achieved by adjusting a weight coefficient, decreasing the weight coefficient associated with the current dominance direction from 1.0 to 0.5, or switching to a preset weight configuration that prioritizes stability. When the instantaneous rate of change is consistently less than or equal to 0.01, the system starts another timer. If the accumulated time of this timer exceeds a preset recovery duration, such as 200 milliseconds, the system triggers a priority recovery operation. Restoring the priority of the target weight's dominant direction can be achieved by either reverting the previously reduced weight coefficient to 1.0 or switching back to a preset weight configuration that prioritizes response speed. This prevents frequent reductions or restorations of priority due to brief fluctuations in the instantaneous rate of change, ensuring more stable and reliable priority adjustment decisions.

[0116] In one embodiment, even if the instantaneous rate of change occasionally exceeds a first threshold, the priority will not be reduced if the trigger duration has not been reached, thus avoiding unnecessary performance loss. Similarly, after the instantaneous rate of change stabilizes, the system waits for a period of time to confirm stability before restoring the priority, avoiding instability caused by premature restoration. This dynamic adjustment mechanism based on duration judgment enables MR devices to more effectively balance visual stability and response speed in complex and ever-changing application scenarios.

[0117] See Figure 10 , Figure 10 This is a schematic diagram of an MR device data processing system provided in one embodiment of this application. The MR device data processing system 1000 includes:

[0118] The acquisition module 1010 is used to acquire the instantaneous operating frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and identity matrix;

[0119] The calculation module 1020 is used to calculate the variance of the graphics processor within a preset time window based on instantaneous operating frequency data, and obtain a stability index.

[0120] The weighted fusion module 1030 is used to perform weighted fusion of the initial asynchronous time warp transformation matrix and the identity matrix based on the stability index to obtain the target asynchronous time warp transformation matrix.

[0121] The determination module 1040 is used to determine the target rendering image based on the target asynchronous time warp transformation matrix and the preset rendering image.

[0122] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0123] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0124] The foregoing has provided a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined in this application.

Claims

1. A method of processing data of an MR device, which is applied to a graphics processor, characterized in that, The method comprises the following steps: obtain the instantaneous running frequency data of the graphics processor and the initial asynchronous time warp transformation matrix and the unit matrix; calculate the variance of the graphics processor within a preset time window based on the instantaneous running frequency data to obtain a stability index; weight and fuse the initial asynchronous time warp transformation matrix and the unit matrix based on the stability index to obtain a target asynchronous time warp transformation matrix; determine a target rendering picture according to the target asynchronous time warp transformation matrix and a preset rendering picture; the weight and fusion of the initial asynchronous time warp transformation matrix and the unit matrix based on the stability index to obtain the target asynchronous time warp transformation matrix, comprising: obtain the instantaneous stability index and the weight fusion stability index of the current frame; adjust the instantaneous stability index according to the difference between the instantaneous stability index and the weight fusion stability index, and the preset stability index change limit value, to obtain a smoothed stability index; weight and fuse the initial asynchronous time warp transformation matrix and the unit matrix according to the smoothed stability index to obtain the target asynchronous time warp transformation matrix; the weight and fusion of the initial asynchronous time warp transformation matrix and the unit matrix according to the smoothed stability index to obtain the target asynchronous time warp transformation matrix, comprising: obtain at least two different types of system running reliability evaluation information, wherein the system running reliability evaluation information includes the running state information of the graphics processor and the inertial measurement unit data quality information; obtain a comprehensive reliability index according to the at least two different types of system running reliability evaluation information and a preset fusion rule; weight and fuse the initial asynchronous time warp transformation matrix and the unit matrix according to the smoothed stability index and the comprehensive reliability index to obtain the target asynchronous time warp transformation matrix; the obtaining of the comprehensive reliability index according to the at least two different types of system running reliability evaluation information and the preset fusion rule, comprising: obtain the characteristic information of the current application scene, wherein the characteristic information includes the emphasis on visual stability or response speed; obtain the activity information of head movement, wherein the activity information includes the head angular velocity or angular acceleration; adjust the relative weight of the running state information of the graphics processor and the inertial measurement unit data quality information in the fusion process according to the characteristic information of the application scene and the activity information of the head movement to obtain a target relative weight; obtain a comprehensive reliability index according to the target relative weight, the running state information of the graphics processor and the inertial measurement unit data quality information.

2. The method of claim 1, wherein, In the case that the running clock frequency of the graphics processor presents a slow and continuous one-way drift trend; the calculation of the variance of the graphics processor within a preset time window based on the instantaneous running frequency data to obtain a stability index, comprising: obtain the instantaneous running frequency sequence of the graphics processor within a preset period; perform trend analysis on the instantaneous running frequency sequence to obtain a trend slope; obtain a stability index according to the trend slope.

3. The method of claim 1, wherein, The preset stability index change limit value is obtained by the following steps: Obtain scene type information of the current application and instantaneous angular velocity or angular acceleration of head movement, wherein the scene type information includes a visual stability requirement, a response speed requirement, and instantaneous angular velocity or angular acceleration of head movement; According to the visual stability requirement, the response speed requirement, and the instantaneous angular velocity or angular acceleration of head movement, a preset stability index change limit value is obtained.

4. The method of claim 1, wherein, The target relative weight is obtained by adjusting the relative weight of the running state information of the graphics processor and the inertial measurement unit data quality information in the fusion process according to the characteristic information of the application scene and the activity information of the head movement, comprising: Obtain the first weight adjustment direction of the characteristic information of the application scene and the second weight adjustment direction of the activity information of the head movement; According to the first weight adjustment direction, the second weight adjustment direction and the preset priority, a target weight dominant direction is obtained; According to the target weight dominant direction, the relative weight of the running state information of the graphics processor and the inertial measurement unit data quality information in the fusion process is adjusted to obtain the target relative weight.

5. The method of claim 4, wherein, After the target weight dominant direction is obtained according to the first weight adjustment direction, the second weight adjustment direction and the preset priority, it further comprises: Obtain the instantaneous change rate of the target asynchronous time warp transformation matrix determined by the target weight dominant direction; When the instantaneous change rate is greater than the threshold of the visual stability, the priority of the current target weight dominant direction is reduced.

6. The method of claim 5, wherein, Further comprising: When the instantaneous change rate is greater than the first threshold of the preset priority adjustment trigger, and the duration of the instantaneous change rate being greater than the first threshold of the preset priority adjustment trigger is greater than the preset trigger duration, the priority of the current target weight dominant direction is reduced; When the instantaneous change rate is less than or equal to the second threshold of the preset priority adjustment trigger, and the duration of the instantaneous change rate being less than or equal to the second threshold of the preset priority adjustment trigger is greater than the preset recovery duration, the priority of the target weight dominant direction is recovered.

7. MR device data processing system for use in a graphics processor, characterized by Comprising: An acquisition module is configured to acquire instantaneous running frequency data of the graphics processor and an initial asynchronous time warp transformation matrix and a unit matrix; A calculation module is configured to calculate the variance of the graphics processor within a preset time window based on the instantaneous running frequency data to obtain a stability index; A weighted fusion module is configured to perform weighted fusion on the initial asynchronous time warp transformation matrix and the unit matrix based on the stability index to obtain a target asynchronous time warp transformation matrix; A determination module is configured to determine a target rendering picture according to the target asynchronous time warp transformation matrix and a preset rendering picture; The weighted fusion module is further configured to obtain an instantaneous stability index and a weighted fusion stability index of a current frame; According to a difference between the instantaneous stability index and the weighted fusion stability index, a preset stability index variation limit value, the instantaneous stability index is adjusted to obtain a smoothed stability index; According to the smoothed stability index, the initial asynchronous time warp transformation matrix and the unit matrix are weighted and fused to obtain a target asynchronous time warp transformation matrix; At least two different types of system operation reliability evaluation information are obtained, wherein the system operation reliability evaluation information includes running state information of a graphic processor and inertial measurement unit data quality information; According to the at least two different types of system operation reliability evaluation information and a preset fusion rule, a comprehensive reliability index is obtained; According to the smoothed stability index and the comprehensive reliability index, the initial asynchronous time warp transformation matrix and the unit matrix are weighted and fused to obtain a target asynchronous time warp transformation matrix; Characteristic information of a current application scene is obtained, wherein the characteristic information includes a focus on visual stability or response speed; Activity information of head movement is obtained, wherein the activity information includes head angular velocity or angular acceleration; According to the characteristic information of the application scene and the activity information of the head movement, a relative weight of the running state information of the graphic processor and the inertial measurement unit data quality information in the fusion process is adjusted to obtain a target relative weight; According to the target relative weight, the running state information of the graphic processor and the inertial measurement unit data quality information, a comprehensive reliability index is obtained.

Images