Motion state self-adaptive augmented reality picture anti-dizziness control method and system

Abstract

The invention discloses a motion state adaptive augmented reality picture anti-dizziness control method and system, and the method comprises the steps: obtaining an acceleration sequence and an angular velocity sequence outputted by an inertial measurement unit, and adding a sensing timestamp to the acceleration sequence and the angular velocity sequence; determining a current motion state based on the acceleration sequence and the angular velocity sequence and outputting state confidence; calculating a jitter risk index based on the energy distribution of the angular velocity sequence in combination with the state confidence; generating an image stabilization control parameter set based on the jitter risk index and the time delay estimation result, and determining a cutting budget; and determining compensation transformation for the current augmented reality rendering frame, and performing resampling and cutting on the augmented reality rendering frame by a two-dimensional graph acceleration unit on the display synthesis side according to the compensation transformation to generate an output frame and outputting the output frame to a display end for presentation. According to the technical scheme, artifacts caused by boundary cutting are restrained while the stability of the output picture is guaranteed, and therefore dizziness and visual discomfort in a motion scene are reduced.

Description

Technical Field

[0001] This invention relates to the technical field of augmented reality glasses, and in particular to a motion-adaptive augmented reality image anti-dizziness control method, system, computer device, and storage medium. Background Technology

[0002] In actual use of augmented reality glasses, users are often in motion scenarios such as walking, running or cycling. The periodic swinging and random shaking of the head and the device will be superimposed on the augmented reality overlay, causing the image to shake significantly relative to the field of vision, thus causing dizziness, discomfort or distraction.

[0003] Existing technologies typically rely on inertial sensors to stabilize the image to a certain extent, or adjust the displayed content to alleviate discomfort. However, in engineering implementation, there are still common problems that make it difficult to achieve both: On the one hand, the jitter spectrum and amplitude vary significantly under different motion states, and stabilization using a fixed or single strategy is prone to insufficient or excessive stabilization, thus reducing responsiveness; on the other hand, the latency and fluctuations of the link from rendering and compositing to display output can change the deviation between the compensation time and the actual display time, making the compensation effect unstable and further amplifying the risk of display artifacts such as black borders and insufficient cropping. Summary of the Invention

[0004] The purpose of this application is to propose a motion-adaptive augmented reality screen anti-dizziness control method, system, computer device and storage medium, which aims to ensure the stability of the output screen while suppressing artifacts caused by cropping boundaries, thereby reducing dizziness and visual discomfort in motion scenes.

[0005] To address the aforementioned technical problems, this application provides a motion-adaptive augmented reality screen anti-dizziness control method, employing the following technical solution: Acquire the acceleration and angular velocity sequences output by the inertial measurement unit, and add sensing timestamps to the acceleration and angular velocity sequences; Within the sliding time window, the current motion state is determined based on the acceleration sequence and angular velocity sequence, and the state confidence level is output. The jitter risk index is calculated based on the energy distribution of the angular velocity sequence and the state confidence level. The delay estimation result of the display link is obtained, and a set of image stabilization control parameters is generated based on the jitter risk index and the delay estimation result, and a cropping budget is determined. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed in the output image. Under the common constraints of the image stabilization control parameter set and the clipping budget, a compensation transformation for the current augmented reality rendering frame is determined, and the 2D graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

[0006] To address the aforementioned technical problems, this application also provides a motion-adaptive augmented reality anti-dizziness control system, which employs the following technical solution: The acquisition module is used to acquire the acceleration sequence and angular velocity sequence output by the inertial measurement unit, and to add a sensing timestamp to the acceleration sequence and angular velocity sequence; The determination module is used to determine the current motion state based on the acceleration sequence and angular velocity sequence within a sliding time window and output the state confidence level; The calculation module is used to calculate the jitter risk index based on the energy distribution of the angular velocity sequence and in combination with the state confidence. The generation module is used to obtain the latency estimation result of the display link, and generate a set of image stabilization control parameters and determine the cropping budget based on the jitter risk index and the latency estimation result. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed in the output image. The output module is used to determine the compensation transformation for the current augmented reality rendering frame under the common constraints of the image stabilization control parameter set and the clipping budget, and the two-dimensional graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

[0007] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution: A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the motion state adaptive augmented reality anti-dizziness control method as described above.

[0008] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below: A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the motion-state adaptive augmented reality anti-dizziness control method as described above.

[0009] Compared with the prior art, the embodiments of this application have the following main advantages: The motion-adaptive augmented reality anti-dizziness control method disclosed in this application introduces display link delay constraints based on the joint assessment of motion state and jitter risk, generates a set of image stabilization control parameters and simultaneously determines the clipping budget, and then the two-dimensional graphics acceleration unit on the display synthesis side performs resampling and clipping. This enables the augmented reality overlay image to adaptively balance stability and responsiveness in motion scenarios such as walking, running, and cycling, and suppresses display artifacts caused by black borders and insufficient clipping under the premise of controlled displacement compensation, thereby reducing the probability of dizziness and visual discomfort. Attached Figure Description

[0010] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of an embodiment of the motion-adaptive augmented reality anti-dizziness control method according to the present application; Figure 2 This is a schematic diagram of an embodiment of the motion-adaptive augmented reality anti-dizziness control system according to this application; Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] This invention is particularly applicable to augmented reality glasses with limited display and computing resources, such as monochrome HUDs with fixed output resolution, where the display synthesis side only has a 2D graphics acceleration unit for 2D resampling and cropping but lacks 3D reprojection rendering capabilities, and the main processor's computing budget for image stabilization is limited. Under these constraints, if a general image stabilization strategy relying on complex 3D graphics pipelines is still adopted, it is easy to introduce large latency or make it difficult to stably suppress black border artifacts. Therefore, this invention introduces a joint constraint of display link latency estimation and cropping budget, transforming the compensation transformation into resampling and cropping processing parameters that can be executed by the 2D graphics acceleration unit, thereby achieving low latency and stable presentation under fixed resolution output conditions.

[0014] refer to Figure 1A flowchart illustrating an embodiment of the motion-state adaptive augmented reality anti-dizziness control method according to this application is shown. The motion-state adaptive augmented reality anti-dizziness control method includes the following steps: Step S101: Obtain the acceleration sequence and angular velocity sequence output by the inertial measurement unit, and add a sensing timestamp to the acceleration sequence and angular velocity sequence.

[0015] In this embodiment, the electronic device running on which the motion-adaptive augmented reality anti-dizziness control method operates can send or receive data via wired or wireless connections. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods.

[0016] In this embodiment, the motion-adaptive augmented reality anti-dizziness control method is executed by the augmented reality glasses. The glasses include at least an inertial measurement unit, a processor, a display synthesis unit, and a two-dimensional graphics acceleration unit connected to the display synthesis unit. The inertial measurement unit outputs acceleration and angular velocity sequences reflecting the motion of the glasses. The acceleration sequence characterizes the linear acceleration change of the glasses in the corresponding coordinate axis direction at each sampling moment, and the angular velocity sequence characterizes the rotational velocity change of the glasses around the corresponding coordinate axis at each sampling moment. To ensure that subsequent steps can correlate data from different sources under the same time reference, the method in this embodiment adds a sensing timestamp to each sampling point when acquiring the acceleration and angular velocity sequences output by the inertial measurement unit. The sensing timestamp is used to identify the acquisition time corresponding to the sampling point. Preferably, the sensing timestamp is consistent with the clock reference of the display link or can be converted to the same time reference through a clock synchronization mechanism, so that it can correctly correspond to the target time of the display output when generating compensation transformation in the later stage. As an example, the inertial measurement unit can output acceleration and angular velocity data at a fixed sampling frequency. The processor calculates the data segment composed of several consecutive sampling points in a sliding time window manner. The sliding time window can have a fixed time length and be updated in a preset step size, so as to maintain the ability to track motion changes while avoiding judgment jitter caused by single-point noise.

[0017] Step S102: Determine the current motion state based on the acceleration sequence and angular velocity sequence within the sliding time window and output the state confidence level.

[0018] In this embodiment, the current motion state is determined based on the acceleration sequence and the angular velocity sequence, and the state confidence score is output. The motion state characterizes the typical motion pattern of the user while wearing glasses, and the state confidence score characterizes the reliability of the current determination result. Specifically, the processor can extract motion features that reflect differences in motion patterns within a sliding time window. For example, it can extract periodic variation features and peak-valley amplitude features of vertical acceleration based on the acceleration sequence, and extract rotational intensity features and periodic stability features based on the angular velocity sequence. These motion features are then combined with preset determination rules or a lightweight classification model to determine motion states such as walking, running, or cycling. As a reference example, in a walking scenario, vertical acceleration often exhibits a clear step frequency period with a moderate peak-to-peak value, while the root mean square of the angular velocity is relatively low and the periodic stability is high. In a running scenario, the step frequency is higher, and the vertical peak-to-peak value and angular velocity energy are significantly increased. In a cycling scenario, the vertical impact is relatively small, while some directional components of the angular velocity may exhibit a smoother but continuous trend. State confidence can be given by the margin of the decision boundary, feature consistency, or model output probability. For example, when the extracted cadence features are highly matched with the typical running range and the periodic stability meets the preset conditions, the state confidence takes a higher value; when the motion features are in the critical range between walking and running, the state confidence takes a lower value, which is used to prompt subsequent risk assessment and parameter generation to adopt a more conservative strategy, thereby reducing the problem of unstable or overly stable images caused by state misjudgment.

[0019] Step S103: Calculate the jitter risk index based on the energy distribution of the angular velocity sequence and the state confidence level.

[0020] In this embodiment, after determining the motion state and outputting the state confidence score, a jitter risk index is calculated based on the energy distribution of the angular velocity sequence and the state confidence score. Here, the energy distribution reflects the intensity and spectral characteristics of the angular velocity within a time window, while the jitter risk index characterizes the risk of significant relative shaking and discomfort in the augmented reality image under the current motion state and current jitter intensity. Specifically, the processor can calculate the energy quantization value of the angular velocity sequence within the sliding time window, for example, by calculating the root mean square of the angular velocity amplitude or performing frequency domain analysis on the angular velocity sequence to obtain the energy proportion of different frequency bands, thereby distinguishing between slower posture changes and faster jitter components. Subsequently, the energy distribution and the state confidence score are fused to obtain the jitter risk index, so that under the same angular velocity energy level, the jitter risk index corresponding to high-impact states such as running can be higher than that of walking, reflecting the impact of motion state on dizziness sensitivity and image stability requirements. As a reference example, when the energy proportion of the angular velocity sequence in the higher frequency band increases significantly and the state confidence points to running, the jitter risk index is calculated to a higher value to drive the convergence of subsequent image stabilization parameters and cropping budget; when the overall angular velocity energy is low and the state confidence points to walking, the jitter risk index is low to maintain the responsiveness of the image.

[0021] Step S104: Obtain the latency estimation result of the display link, and generate a set of image stabilization control parameters and determine the cropping budget based on the jitter risk index and the latency estimation result. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed for the output image.

[0022] In this embodiment, after obtaining the jitter risk index, the latency estimation result of the display link is acquired, and a set of image stabilization control parameters is generated based on the jitter risk index and the latency estimation result, and a clipping budget is determined. The latency estimation result of the display link is used to characterize the time delay between the generation of the current rendering frame and the final presentation of the frame on the display end. This delay usually includes rendering queuing, compositing queuing, and display refresh waiting, and may fluctuate with system load. The set of image stabilization control parameters is used to control the intensity and response characteristics of image compensation. Among them, the displacement gain is used to control the ratio of the compensation displacement to the estimated motion amount, the filter time constant is used to control the smoothness of the compensation change over time or the response speed, and the displacement upper limit is used to limit the maximum allowable compensation displacement in a single frame or per unit time. The clipping budget is used to limit the maximum allowable window offset of the output image. Its essence is a boundary margin constraint reserved to avoid black borders or insufficient clipping when performing resampling and clipping. Specifically, the processor can determine the target display time for compensation based on the latency estimation results of the display link. When the latency is large or the fluctuation is large, in order to prevent residual jitter caused by compensation lag, it is usually necessary to increase the prediction margin of compensation or narrow the compensation amplitude. At the same time, the higher the jitter risk index, the more stable the image stabilization control parameter set should be. For example, reduce the displacement gain to avoid excessive jitter tracking, increase the filter time constant to enhance smoothness, and tighten the displacement upper limit to suppress abrupt displacement. Conversely, when the jitter risk index is low, the displacement gain can be appropriately increased and the displacement upper limit can be relaxed to improve the responsiveness of the image. The determination of the cropping budget is related to the displacement upper limit and the display resolution. The processor can map the displacement upper limit to the required cropping margin. The larger the cropping margin, the larger the window offset that can be allowed for the output frame, but it will also lead to a reduction in the effective field of view or a higher scaling ratio. Therefore, this embodiment generates the cropping budget and the image stabilization control parameter set together, so that the compensation intensity and the available boundary margin are coordinated under the same constraint framework. As a reference example, in running scenarios and when the estimated link latency increases, the processor lowers the displacement gain and tightens the displacement limit, while setting the clipping budget to a more conservative maximum window offset to reduce the risk of black borders and stabilize the output image; in walking scenarios and when the latency is low, the processor appropriately relaxes the displacement limit and increases the displacement gain to make the superimposed image more consistent with changes in the field of view.

[0023] Step S105: Under the common constraints of the image stabilization control parameter set and the clipping budget, a compensation transformation for the current augmented reality rendering frame is determined, and the 2D graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

[0024] In this embodiment, after generating the image stabilization control parameter set and the clipping budget, a compensation transformation for the current augmented reality rendering frame is determined under the joint constraints of the image stabilization control parameter set and the clipping budget. The 2D graphics acceleration unit on the display compositing side then performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame, which is then displayed on the display. The compensation transformation describes the geometric adjustment relationship that should be applied to the current rendering frame to counteract the relative shaking of the image caused by the wearer's movement. Preferably, the compensation transformation can be expressed as a 2D geometric transformation, including one or more combinations of translation, scaling, or rotation. The translation component is used to compensate for the offset of the image window, the scaling component provides additional boundary margin when the clipping budget is limited, and the rotation component counteracts the image tilt caused by changes in head posture. The processor generates the target value of the compensation transformation based on the motion changes reflected by inertial measurement data and in conjunction with the image stabilization control parameter set. Simultaneously, it limits the window offset corresponding to the compensation transformation according to the clipping budget to ensure that the output image does not show black borders due to excessive offset. The 2D graphics acceleration unit (GTU) is a hardware or hardware acceleration module used on the display compositing side to perform 2D image processing. It can resample and crop rendered frames with a relatively low main processing load. Resampling refers to mapping the source image to coordinates according to a compensation transform and calculating pixel values ​​at the new coordinates. Cropping refers to selecting a target window from the source image or the resampled result that matches the output resolution. For example, when the compensation transform includes a window offset to the upper right, the GTU determines the source window position based on the compensation transform and crops the corresponding area within the allowable cropping budget. It also applies slight scaling when necessary to ensure the cropped window always falls within the effective pixel range, ultimately generating an output frame that matches the display resolution. When the compensation transform includes a small-angle rotation, the GTU completes the rotation coordinate mapping during resampling and outputs the cropped frame, resulting in a more stable augmented reality overlay on the display.

[0025] This application introduces display link latency constraints based on a joint assessment of motion state and jitter risk, generates a set of image stabilization control parameters and simultaneously determines the clipping budget, and then the two-dimensional graphics acceleration unit on the display synthesis side performs resampling and clipping. This enables the augmented reality overlay image to adaptively balance stability and responsiveness in motion scenarios such as walking, running, and cycling, and suppresses display artifacts caused by black borders and insufficient clipping under the premise of controlled compensation displacement, thereby reducing the probability of dizziness and visual discomfort.

[0026] In some optional implementations of this embodiment, after the steps of obtaining the acceleration sequence and angular velocity sequence output by the inertial measurement unit and adding sensing timestamps to the acceleration sequence and angular velocity sequence, the method further includes: Gravity component estimation and subtraction are performed on the acceleration sequence to generate a gravity-de-acceleration sequence; The angular velocity sequence is subjected to zero-bias estimation and subtraction to generate a zero-bias corrected angular velocity sequence; The gravity-de-acceleration sequence and the zero-bias correction angular velocity sequence serve as inputs for determining the motion state and calculating the jitter risk index.

[0027] In this embodiment, the acceleration sequence typically contains both a static component caused by gravity and a dynamic component caused by the wearer's movement. The gravity component projects onto different axes as the wearing posture changes. Directly using the original acceleration sequence can easily misjudge slow posture changes as motion impacts. This embodiment performs gravity component estimation and subtraction on the acceleration sequence. Gravity component estimation can be achieved through low-pass filtering, posture fusion calculation, or gravity vector update based on the stationary segment, thereby obtaining a gravity-free acceleration sequence that mainly reflects motion impacts and swaying. The angular velocity sequence may have a zero bias, meaning that it still outputs non-zero angular velocity values ​​in an ideal stationary state. This zero bias will accumulate as a significant error in integration or energy statistics. This embodiment performs zero-bias estimation and subtraction on the angular velocity sequence. The zero bias estimation can be obtained statistically during the stationary calibration phase or can be adaptively updated during operation using the low dynamic range, thereby generating a zero-bias-corrected angular velocity sequence. Through the above processing, the gravity-free acceleration sequence and the zero-bias-corrected angular velocity sequence are numerically closer to real motion changes and serve as inputs for motion state determination and jitter risk index calculation, making subsequent threshold determination and energy distribution calculation more stable. As a reference example, when a user stops or slowly turns their head, the original acceleration may change significantly along a certain axis, while the change in gravity acceleration is significantly reduced, thus avoiding misjudging a slow change in posture as running or violent shaking.

[0028] This application estimates and subtracts the gravity component from the acceleration sequence and estimates and subtracts the zero bias from the angular velocity sequence, making the input data for motion state determination and jitter risk assessment more reflective of real dynamic motion changes. This reduces the interference of slow attitude changes and sensor bias on feature calculation and energy statistics, thereby improving the stability and consistency of motion state recognition and risk assessment, and avoiding image slippage or compensation drift caused by misadjustment of image stabilization parameters due to input deviation.

[0029] In some optional implementations of this embodiment, the step of calculating the jitter risk index based on the energy distribution of the angular velocity sequence and combined with the state confidence includes: Within the sliding time window, the step frequency or tread frequency characteristics, peak-to-peak vertical acceleration, root mean square angular velocity, and periodic stability are calculated, and candidate motion states of walking, running, or cycling are determined based on the step frequency or tread frequency characteristics and the periodic stability. The system confirms switching to the candidate motion state only when the entry threshold is continuously met for a preset duration, and exits the candidate motion state only when the exit threshold is continuously met for a preset duration. The state confidence level is determined by the margin of the decision condition relative to the entry threshold and the exit threshold.

[0030] In this embodiment, the determination of the motion state adopts a multi-feature joint judgment within a sliding time window, and frequently switches between states by constraining entry threshold, exit threshold, and duration, so as to output a state confidence level with quantifiable credibility. Specifically, cadence or pedal frequency features are used to characterize the main cycle of the motion, peak-to-peak vertical acceleration is used to characterize the impact amplitude, root mean square angular velocity is used to characterize the rotation intensity, and periodic stability is used to characterize periodic consistency and rhythmic stability. Within each sliding time window, the processor calculates the above features based on the degravity acceleration sequence and the zero-bias correction angular velocity sequence, and determines the candidate motion state of walking, running, or cycling based on cadence or pedal frequency features and periodic stability. Walking usually corresponds to a lower cadence and higher periodic stability, running corresponds to a higher cadence and a larger peak-to-peak vertical acceleration, and cycling may exhibit pedal frequency features with relatively small vertical impact. To avoid short-term noise causing the state to jump back and forth between walking and running under critical conditions, this embodiment only confirms the switch to the candidate motion state when the judgment condition corresponding to the candidate motion state continuously meets the entry threshold for a preset duration, and exits the candidate motion state when the judgment condition continuously meets the exit threshold for a preset duration, thus forming a state machine-like judgment logic with hysteresis characteristics. The state confidence is used to reflect the credibility of the judgment logic, which can be determined by the margin of the judgment condition relative to the entry threshold and exit threshold. For example, when the step frequency is significantly higher than the running entry threshold and the period stability meets the running stability range, the margin is large and the state confidence is correspondingly high; when the step frequency is close to the threshold boundary or the period stability fluctuates greatly, the margin is small and the state confidence decreases, thus providing a credibility weight for the subsequent calculation of jitter risk indicators. As a reference example, when the user gradually accelerates from walking to running, the step frequency and root mean square of angular velocity will gradually increase. The user only switches to running state when the running entry condition is met for several consecutive time windows, avoiding false switching caused by short-term acceleration or a single step.

[0031] This application introduces a combination of features, including step frequency or tread frequency, peak-to-peak vertical acceleration, root mean square angular velocity, and periodic stability, within a sliding time window. It also employs entry threshold, exit threshold, and duration constraints to confirm state switching. This makes motion state recognition insensitive to short-term noise and critical fluctuations, and the output state confidence level can quantitatively reflect the reliability of the judgment. This provides a more reliable basis for subsequent jitter risk index calculation and image stabilization parameter generation, reducing image parameter jitter caused by frequent changes in motion state.

[0032] In some optional implementations of this embodiment, the step of calculating the jitter risk index based on the energy distribution of the angular velocity sequence and combined with the state confidence includes: The zero-bias correction angular velocity sequence is projected onto at least two preset frequency bands within the sliding time window to obtain high-frequency energy components and low-frequency energy components. The high-frequency energy component is given a higher weight than the low-frequency energy component, and the jitter risk index is calculated based on the weighted energy component and the state confidence, so that the jitter risk index increases as the state confidence increases.

[0033] In this embodiment, the jitter risk index is calculated using a combination of frequency band energy decomposition and state confidence fusion to simultaneously characterize the intensity of jitter and the reliability of the motion state. Specifically, the zero-biased angular velocity sequence can be projected or filtered and decomposed according to a preset frequency band within a sliding time window to obtain high-frequency and low-frequency energy components. The low-frequency energy components typically correspond to slow attitude changes or smooth rotations, while the high-frequency energy components typically correspond to high-frequency oscillations caused by rapid jitter or impacts. The processor assigns a higher weight to the high-frequency energy components than to the low-frequency energy components, making the impact of rapid jitter on risk assessment more significant. The weighted energy components are then fused with the state confidence to calculate the jitter risk index. This results in a higher risk value when the state confidence is high and the high-frequency energy is significant, and a more conservative risk value when the state confidence is low or the high-frequency energy is not obvious. This fusion relationship can be understood as follows: the more reliable the motion state determination, the more fully the jitter risk index reflects the jitter intensity under that state, thus making the generation of image stabilization parameters more targeted. As a reference example, within a time window with high confidence in the running state, if the high-frequency energy of the angular velocity remains high, the jitter risk index is increased to drive stronger smoothing and limiting; while in the critical period with low confidence in the state, even if a high-frequency energy peak appears for a short time, the jitter risk index will not be excessively amplified, so as to avoid the subjective experience of the image being tense and loose due to drastic changes in the stabilization parameters.

[0034] This application decomposes the zero-bias correction angular velocity sequence into energy components of at least two frequency bands and assigns higher weight to the high-frequency energy. Then, it integrates the high-frequency energy with the state confidence to form a jitter risk index. This enables the jitter risk assessment to distinguish between rapid jitter and slow attitude changes and to highlight the high-frequency jitter component that is more sensitive to dizziness. As a result, the image stabilization control parameters are more targeted to deal with high-impact scenarios such as running, improving the stabilization effect while avoiding excessive suppression of low-frequency natural head turning movements.

[0035] In some optional implementations of this embodiment, the step of obtaining the latency estimation result of the display link includes: Read the rendering queue depth, vertical synchronization period, and display compositing wait time, and calculate the rendering queuing latency based on the rendering queue depth and the vertical synchronization period, and calculate the compositing queuing latency based on the display compositing wait time; The delay estimation result is obtained by superimposing the rendering queuing delay and the synthesis queuing delay, and the delay estimation result and the sensing timestamp are made to use the same time base.

[0036] In this embodiment, the latency estimation result of the display link consists of rendering queuing latency and compositing queuing latency, used to characterize the time position of the current rendering frame from generation to presentation, so that the calculation of compensation transformation can be aligned with the target display time. Specifically, the rendering queue depth reflects the queuing length of frames to be processed in the rendering pipeline, the vertical synchronization period reflects the display refresh rhythm, and the display compositing wait time reflects the queuing and synchronization wait status of the compositing stage. The processor reads the rendering queue depth, vertical synchronization period, and display compositing wait time, and calculates the rendering queuing latency based on the rendering queue depth and vertical synchronization period. For example, the number of frames to be displayed corresponding to the rendering queue depth can be associated with the refresh period of each frame to obtain the estimated queuing time. At the same time, the compositing queuing latency is calculated based on the display compositing wait time. The rendering queuing latency and the compositing queuing latency are superimposed to obtain the latency estimation result. To ensure that the latency estimation result can be aligned with the inertial data in time, this embodiment uses the same time base as the latency estimation result and the sensing timestamp, or maps the two to the same time axis through a unified clock source and its conversion relationship, thereby reducing systematic deviations when performing attitude extrapolation and compensation alignment. As a reference example, when the rendering load increases and the rendering queue depth increases, the rendering queuing latency also increases. If the latency estimation result is not updated, the compensation will be aligned with an earlier moment, resulting in compensation lag. This embodiment updates the latency estimation result in a timely manner by reading the queue depth and waiting time, so that the compensation is more in line with the actual display.

[0037] This application calculates the rendering queue delay and the compositing queue delay by reading the rendering queue depth, vertical synchronization cycle, and display compositing waiting time, and then superimposes them to obtain the delay estimation result. This allows the display link delay to be dynamically changed with the system load and be plotted in real time. Furthermore, it uses the same time base as the sensor timestamp, thereby providing a reliable time reference for compensating the display time of the target alignment and reducing compensation lag, residual jitter, or compensation overshoot caused by delay fluctuations.

[0038] In some optional implementations of this embodiment, the step of determining the compensation transformation for the current augmented reality rendering frame under the common constraints of the image stabilization control parameter set and the clipping budget includes: The acceleration sequence and the angular velocity sequence are extrapolated using the time delay estimation result as the extrapolation time domain to generate the attitude prediction of the target display time; Based on the pose prediction, a two-dimensional geometric compensation transformation for the augmented reality rendering frame is determined, and the two-dimensional geometric compensation transformation is converted into resampling and clipping processing parameters that can be executed by the two-dimensional graphics acceleration unit.

[0039] In this embodiment, when determining the compensation transformation under the joint constraints of the image stabilization control parameter set and the clipping budget, attitude extrapolation based on the time delay estimation result is introduced to obtain the attitude prediction at the target display time, thereby making the compensation transformation oriented towards the actual rendering time rather than only towards the sampling time. Specifically, the time delay estimation result can be regarded as the time interval from the current rendering time to the target display time. The processor extrapolates the acceleration sequence and angular velocity sequence using this time interval as the extrapolation time domain to generate the attitude prediction at the target display time. The attitude prediction is used to characterize the attitude change trend of the glasses relative to the current time at the target display time. Based on the attitude prediction, the processor determines the two-dimensional geometric compensation transformation for the augmented reality rendering frame, wherein the two-dimensional geometric compensation transformation may include translation components and scaling components. The translation component is used to offset the equivalent offset of the attitude change on the display plane, and the scaling component is used to provide the necessary boundary margin when the clipping budget is limited. Subsequently, the two-dimensional geometric compensation transformation is converted into resampling and clipping processing parameters that can be executed by the two-dimensional graphics acceleration unit, so that the hardware acceleration unit can perform coordinate mapping, pixel interpolation, and window cropping on the rendering frame according to the processing parameters. As a reference example, when the attitude prediction indicates that there is an equivalent field of view offset to the left at the moment the target is displayed, the two-dimensional geometric compensation transformation includes translation compensation to the right. At the same time, the maximum acceptable window offset and the corresponding scaling ratio are given according to the clipping budget. The two-dimensional graphics acceleration unit sets the source window position and scaling parameters accordingly and completes the resampling output, thereby presenting a more stable superimposed image on the display end.

[0040] This application extrapolates the acceleration and angular velocity sequences using the time delay estimation results as the extrapolation time domain to generate the attitude prediction quantity for the target display time. Based on this, a two-dimensional geometric compensation transformation is constructed and converted into resampling and clipping processing parameters that can be executed by the two-dimensional graphics acceleration unit. This allows the compensation to be aligned with the actual presentation time rather than just the sampling time. At the same time, the compensation is directly applied to the hardware executable parameters on the display synthesis side, reducing the burden on the main processor and improving the real-time performance and consistency of the compensation execution, thereby improving the screen stabilization experience in motion scenes.

[0041] In some optional implementations of this embodiment, after the step of the two-dimensional graphics acceleration unit on the display compositing side performing resampling and cropping of the augmented reality rendering frame according to the compensation transformation to generate an output frame and outputting it to the display end for presentation, the method further includes: The black border risk value is calculated based on the boundary occupancy of the output frame, and the displacement limit is tightened or the scaling ratio is increased to update the cropping budget when the black border risk value exceeds a preset threshold. The residual jitter index is calculated based on the residual displacement between the output frame and the compensation transform, and the filter time constant is adjusted based on the residual jitter index.

[0042] In this embodiment, after the 2D graphics acceleration unit performs resampling and cropping on the augmented reality rendering frame according to the compensation transformation to generate the output frame and output it to the display end, the output result is further used to adaptively update the image stabilization control parameter set and the cropping budget to suppress black border artifacts and stabilize residual jitter. Specifically, the boundary occupancy of the output frame is used to reflect whether the output image is close to or touches the effective pixel boundary. For example, the black border risk value can be obtained by statistically analyzing the proportion of invalid pixels in the edge area of ​​the output frame or the width of the edge filling area. When the black border risk value exceeds a preset threshold, it indicates that the cropping budget is insufficient or the window offset is too large. In this embodiment, the displacement upper limit is tightened to reduce the maximum compensation offset of subsequent frames, or the scaling ratio is increased to increase the available boundary margin, thereby updating the cropping budget and reducing the probability of black borders. On the other hand, the residual displacement between the output frame and the compensation transform reflects the degree of relative jitter that is not eliminated after compensation. In this embodiment, the residual jitter index is calculated based on the residual displacement, and the filtering time constant is adjusted based on the residual jitter index. For example, when the residual jitter index is consistently high, the filtering time constant is increased to enhance smoothness; when the residual jitter index decreases and the risk of black borders is controlled, the filtering time constant is decreased to improve responsiveness. As a reference example, if edge occupancy increases for several consecutive frames in a running scene, the system increases the scaling ratio and tightens the displacement limit to bring the output window back to a safe range. At the same time, the filtering time constant is adjusted according to the residual jitter index to gradually converge the image stability without introducing obvious black borders.

[0043] This application actively suppresses black border risk by calculating the black border risk value based on the boundary occupancy after the output frame is generated and tightening the upper limit of displacement or increasing the scaling ratio to update the cropping budget when the threshold is exceeded. This allows the black border risk to converge adaptively with the scene. At the same time, a residual jitter index is formed based on the residual displacement between the output frame and the compensation transform, and the filtering time constant is adjusted accordingly. This allows the smoothness of the image stabilization to be dynamically corrected according to the actual compensation effect. This ensures the stability of the image while avoiding long-term over-smoothing that leads to a decrease in responsiveness, thus improving the overall robustness and usability.

[0044] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0045] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0046] Further reference Figure 2 As a response to the above Figure 1 The implementation of the method shown in this application provides an embodiment of an augmented reality anti-dizziness control system with motion state adaptation. This system embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0047] like Figure 2 As shown, the motion-adaptive augmented reality anti-dizziness control system 200 described in this embodiment includes: an acquisition module 201, a determination module 202, a calculation module 203, a generation module 204, and an output module 205. Wherein: The acquisition module 201 is used to acquire the acceleration sequence and angular velocity sequence output by the inertial measurement unit, and to add a sensing timestamp to the acceleration sequence and angular velocity sequence; The determination module 202 is used to determine the current motion state based on the acceleration sequence and angular velocity sequence within a sliding time window and output the state confidence level. Calculation module 203 is used to calculate the jitter risk index based on the energy distribution of the angular velocity sequence and in combination with the state confidence. The generation module 204 is used to obtain the latency estimation result of the display link, and generate a set of image stabilization control parameters and determine the cropping budget based on the jitter risk index and the latency estimation result. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed in the output image. The output module 205 is used to determine the compensation transformation for the current augmented reality rendering frame under the common constraints of the image stabilization control parameter set and the clipping budget, and the two-dimensional graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

[0048] The motion-adaptive augmented reality anti-dizziness control system provided in this embodiment of the invention can realize all the processes of the motion-adaptive augmented reality anti-dizziness control method of the above embodiment. The functions and technical effects of each module in the device are the same as those of the motion-adaptive augmented reality anti-dizziness control method of the above embodiment, and will not be repeated here.

[0049] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0050] The computer device 3 includes a memory 31, a processor 32, and a network interface 33 that are interconnected via a system bus. It should be noted that only the computer device 3 with components 31-33 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0051] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0052] The memory 31 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 31 may be an internal storage unit of the computer device 3, such as the hard disk or memory of the computer device 3. In other embodiments, the memory 31 may also be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Of course, the memory 31 may also include both the internal storage unit and its external storage device of the computer device 3. In this embodiment, the memory 31 is typically used to store the operating system and various application software installed on the computer device 3, such as computer-readable instructions for motion-adaptive augmented reality anti-dizziness control methods. In addition, the memory 31 can also be used to temporarily store various types of data that have been output or will be output.

[0053] In some embodiments, the processor 32 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 32 is typically used to control the overall operation of the computer device 3. In this embodiment, the processor 32 is used to execute computer-readable instructions stored in the memory 31 or to process data, such as executing computer-readable instructions for the motion-adaptive augmented reality anti-dizziness control method.

[0054] The network interface 33 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 3 and other electronic devices.

[0055] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the motion state adaptive augmented reality screen anti-dizziness control method described above.

[0056] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0057] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A motion-adaptive augmented reality anti-dizziness control method, characterized in that, Includes the following steps: Acquire the acceleration and angular velocity sequences output by the inertial measurement unit, and add sensing timestamps to the acceleration and angular velocity sequences; Within the sliding time window, the current motion state is determined based on the acceleration sequence and angular velocity sequence, and the state confidence level is output. The jitter risk index is calculated based on the energy distribution of the angular velocity sequence and the state confidence level. The delay estimation result of the display link is obtained, and a set of image stabilization control parameters is generated based on the jitter risk index and the delay estimation result, and a cropping budget is determined. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed in the output image. Under the common constraints of the image stabilization control parameter set and the clipping budget, a compensation transformation for the current augmented reality rendering frame is determined, and the 2D graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

2. The method according to claim 1, characterized in that, After acquiring the acceleration and angular velocity sequences output by the inertial measurement unit and adding sensing timestamps to the acceleration and angular velocity sequences, the method further includes: Gravity component estimation and subtraction are performed on the acceleration sequence to generate a gravity-de-acceleration sequence; The angular velocity sequence is subjected to zero-bias estimation and subtraction to generate a zero-bias corrected angular velocity sequence; The gravity-de-acceleration sequence and the zero-bias correction angular velocity sequence serve as inputs for determining the motion state and calculating the jitter risk index.

3. The method according to claim 2, characterized in that, The step of calculating the jitter risk index based on the energy distribution of the angular velocity sequence and in combination with the state confidence level includes: Within the sliding time window, the step frequency or tread frequency characteristics, peak-to-peak vertical acceleration, root mean square angular velocity, and periodic stability are calculated, and candidate motion states of walking, running, or cycling are determined based on the step frequency or tread frequency characteristics and the periodic stability. The system confirms switching to the candidate motion state only when the entry threshold is continuously met for a preset duration, and exits the candidate motion state only when the exit threshold is continuously met for a preset duration. The state confidence level is determined by the margin of the decision condition relative to the entry threshold and the exit threshold.

4. The method according to claim 3, characterized in that, The step of calculating the jitter risk index based on the energy distribution of the angular velocity sequence and in combination with the state confidence level includes: The zero-bias correction angular velocity sequence is projected onto at least two preset frequency bands within the sliding time window to obtain high-frequency energy components and low-frequency energy components. The high-frequency energy component is given a higher weight than the low-frequency energy component, and the jitter risk index is calculated based on the weighted energy component and the state confidence, so that the jitter risk index increases as the state confidence increases.

5. The method according to claim 1, characterized in that, The step of obtaining the latency estimation result of the display link includes: Read the rendering queue depth, vertical synchronization period, and display compositing wait time, and calculate the rendering queuing latency based on the rendering queue depth and the vertical synchronization period, and calculate the compositing queuing latency based on the display compositing wait time; The delay estimation result is obtained by superimposing the rendering queuing delay and the synthesis queuing delay, and the delay estimation result and the sensing timestamp are made to use the same time base.

6. The method according to claim 1, characterized in that, The step of determining the compensation transformation for the current augmented reality rendering frame under the common constraints of the image stabilization control parameter set and the clipping budget includes: The acceleration sequence and the angular velocity sequence are extrapolated using the time delay estimation result as the extrapolation time domain to generate the attitude prediction of the target display time; Based on the pose prediction, a two-dimensional geometric compensation transformation for the augmented reality rendering frame is determined, and the two-dimensional geometric compensation transformation is converted into resampling and clipping processing parameters that can be executed by the two-dimensional graphics acceleration unit.

7. The method according to claim 1, characterized in that, After the step of the 2D graphics acceleration unit on the display compositing side performing resampling and cropping of the augmented reality rendering frame according to the compensation transformation to generate an output frame and outputting it to the display end for presentation, the method further includes: The black border risk value is calculated based on the boundary occupancy of the output frame, and the displacement limit is tightened or the scaling ratio is increased to update the cropping budget when the black border risk value exceeds a preset threshold. The residual jitter index is calculated based on the residual displacement between the output frame and the compensation transform, and the filter time constant is adjusted based on the residual jitter index.

8. A motion-adaptive augmented reality anti-dizziness control system, characterized in that, include: The acquisition module is used to acquire the acceleration sequence and angular velocity sequence output by the inertial measurement unit, and to add a sensing timestamp to the acceleration sequence and angular velocity sequence; The determination module is used to determine the current motion state based on the acceleration sequence and angular velocity sequence within a sliding time window and output the state confidence level; The calculation module is used to calculate the jitter risk index based on the energy distribution of the angular velocity sequence and in combination with the state confidence. The generation module is used to obtain the latency estimation result of the display link, and generate a set of image stabilization control parameters and determine the cropping budget based on the jitter risk index and the latency estimation result. The set of image stabilization control parameters includes at least displacement gain, filter time constant and displacement upper limit. The cropping budget is used to limit the maximum window offset allowed in the output image. The output module is used to determine the compensation transformation for the current augmented reality rendering frame under the common constraints of the image stabilization control parameter set and the clipping budget, and the two-dimensional graphics acceleration unit on the display compositing side performs resampling and clipping on the augmented reality rendering frame according to the compensation transformation to generate an output frame and output it to the display end for presentation.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the motion state adaptive augmented reality screen anti-dizziness control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the motion state adaptive augmented reality anti-dizziness control method as described in any one of claims 1 to 7.

Images