Extended reality content adjustment method and apparatus based on physiological and environmental perception

Abstract

This application provides a method and apparatus for adjusting extended reality content based on physiological and environmental perception, relating to the field of extended reality. The method includes: acquiring the user's physiological data and environmental perception data of the current physical environment; processing the physiological data and environmental perception data separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data; determining the user's current physiological state based on the physiological state data, and determining the collision risk based on the real-time environmental data and the user's pose information; and adjusting the rendering parameters of the virtual reality or augmented reality content presented to the user according to the current physiological state and / or collision risk. This method achieves accurate perception and fusion processing of multimodal physiological and environmental data of the extended reality user, can predict motion sickness and discomfort in advance and dynamically adjust rendering parameters, effectively alleviating physiological discomfort, avoiding physical collisions, and balancing immersive experience with user safety.

Description

Technical Field

[0001] This application relates to the field of extended reality (including virtual reality VR and augmented reality AR), and more specifically, to an extended reality content adjustment method and apparatus based on physiological and environmental perception. Background Technology

[0002] Currently, extended reality devices can cause motion sickness in some users during immersive experiences due to conflicts between visual and vestibular signals. Existing technologies can only detect physiological signals after motion sickness occurs, but cannot predict or intervene in advance. Furthermore, extended reality headsets can obstruct the user's vision, and existing environmental perception solutions, which rely heavily on visual sensors, fail in low-light, obstructed, or transparent environments. Optical data acquisition also poses a privacy risk. In addition, current technologies only provide content interaction or simple obstacle prompts, making it difficult to optimize the immersive experience while ensuring safety, and they cannot achieve personalized adaptation based on the user's physiological characteristics. Summary of the Invention

[0003] The purpose of this application is to provide an extended reality content adjustment method and apparatus based on physiological and environmental perception, which solves the above-mentioned problems existing in the prior art, can realize accurate perception of user physiological state and environmental collision risk, dynamically adapt content rendering parameters, and effectively alleviate physiological discomfort and avoid physical collision.

[0004] Firstly, a method for adjusting extended reality content based on physiological and environmental perception is provided, which may include: Acquire user physiological data and environmental perception data of the current physical environment; The physiological data and the environmental perception data are processed separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The user's current physiological state is determined based on the physiological state data, and the collision risk is determined based on the real-time environmental data and the user's pose information. Adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

[0005] In one possible implementation, the physiological data is processed to obtain physiological state data corresponding to the physiological data, including: The baseline physiological parameters are extracted from the acquired contact-type physiological sensing signals, and the observed physiological parameters extracted from the acquired non-contact environmental sensing signals are calibrated online to obtain the physiological state data.

[0006] In one possible implementation, the environmental perception data includes at least one of visual image data, millimeter-wave radar point cloud data, and inertial measurement data; The physiological data includes at least one of heart rate data, blood oxygen data, heart rate variability data, electroencephalogram (EEG) data, and skin conductance response data.

[0007] In one possible implementation, determining the user's current physiological state based on the physiological state data includes: The physiological state data is processed using a trained physiological state recognition model, and the current physiological state is output.

[0008] In one possible implementation, determining collision risk based on the real-time environmental data and the user's pose information includes: Using a configured collision risk probability algorithm, the real-time distance between the user and the obstacles identified in the real-time environmental data is calculated to obtain the collision risk probability that characterizes the collision risk.

[0009] In one possible implementation, the current physiological state includes: motion sickness index, cognitive load index, and emotional arousal level; Based on the current physiological state and / or the collision risk, adjust the rendering parameters of the virtual reality or augmented reality content presented to the user, including: When the motion sickness index exceeds a configured first preset threshold, perform at least one of the following operations: reduce the field of view of the virtual reality or augmented reality content, or reduce the movement speed of the virtual camera; When the cognitive load index exceeds the configured second preset threshold, the image information density is reduced. When the emotional excitement level is lower than the configured third preset threshold and the current application is entertainment, the intensity of content stimulation is increased; And / or, when the probability of collision risk exceeds the risk thresholds configured for different levels, trigger visual warnings, auditory warnings, tactile warnings, or switch to a see-through reality video view.

[0010] In one possible implementation, visual alerts, auditory alerts, and tactile alerts are triggered in a tiered manner, including: When the probability of collision risk exceeds the configured first risk threshold, the visual warning is triggered; When the collision risk probability exceeds the configured second risk threshold, the auditory warning and the tactile warning are triggered.

[0011] Secondly, an extended reality content adjustment device based on physiological and environmental perception is provided, the device may include: The acquisition unit is used to acquire the user's physiological data and environmental perception data of the current physical environment; The processing unit is used to process the physiological data and the environmental perception data respectively to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The determining unit is used to determine the user's current physiological state based on the physiological state data, and to determine the collision risk based on the real-time environmental data and the user's pose information; An adjustment unit is used to adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

[0012] Thirdly, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements any of the steps described in the first aspect above.

[0013] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the steps of any of the methods described in the first aspect above.

[0014] This application provides a method and apparatus for adjusting augmented reality content based on physiological and environmental perception. The method includes: acquiring the user's physiological data and environmental perception data of the current physical environment; processing the physiological data and environmental perception data separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data; determining the user's current physiological state based on the physiological state data, and determining the collision risk based on the real-time environmental data and the user's pose information; and adjusting the rendering parameters of the virtual reality or augmented reality content presented to the user according to the current physiological state and / or collision risk. This method achieves accurate perception and fusion processing of multimodal physiological and environmental data of the augmented reality user, can predict motion sickness and discomfort in advance and dynamically adjust rendering parameters, effectively alleviate physiological discomfort, avoid physical collisions, and balance immersive experience with user safety. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an extended reality content adjustment method based on physiological and environmental perception, provided for an embodiment of this application; Figure 2 A system structure diagram for an extended reality content adjustment method based on physiological and environmental perception, provided in an embodiment of this application; Figure 3 This is a schematic diagram of the adaptive online calibration principle provided in the embodiments of this application; Figure 4 A schematic diagram of the structure of an extended reality content adjustment device based on physiological and environmental perception provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0018] Currently, extended reality (XR) devices can cause motion sickness in some users during immersive experiences due to conflicts between visual and vestibular signals. Existing technologies can only detect physiological signals after motion sickness occurs, but cannot predict or intervene in advance. Furthermore, extended reality headsets can obstruct the user's vision, and existing environmental perception solutions, which rely heavily on visual sensors, fail in low-light, obstructed, or transparent environments. Optical data acquisition also poses a privacy risk. In addition, current technologies only provide content interaction or simple obstacle prompts, making it difficult to optimize the immersive experience while ensuring safety, and they cannot achieve personalized adaptation based on the user's physiological characteristics.

[0019] Therefore, this application provides an extended reality content adjustment method based on physiological and environmental perception to solve the above-mentioned problems in the prior art. It can accurately perceive the user's physiological state and the risk of collision with the environment, dynamically adapt the content rendering parameters, and effectively alleviate physiological discomfort and avoid physical collisions.

[0020] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0021] Figure 1This is a flowchart illustrating an extended reality content adjustment method based on physiological and environmental perception, provided as an embodiment of this application. Figure 1 As shown, the method may include: Step S110: Obtain the user's physiological data and the environmental perception data of the current physical environment.

[0022] Among them, environmental perception data includes at least one of visual image data, millimeter-wave radar point cloud data, and inertial measurement data; Physiological data includes at least one of the following: heart rate data, blood oxygen data, heart rate variability data, electroencephalogram (EEG) data, and skin conductance response data.

[0023] Specifically, this step involves the synchronous acquisition of multimodal sensor data. The acquisition terminal is integrated into the faceplate, headband, and shell area of ​​the XR head-mounted display device (HMD). All sensor modules transmit data through an internal bus (I2C / SPI) or a low-latency wireless communication protocol (such as Bluetooth 5.3 or UWB). During the acquisition process, all data packets are stamped with microsecond-level high-precision timestamps to ensure the timing alignment of physiological data and environmental perception data, and to avoid subsequent processing errors caused by time differences in cross-modal data.

[0024] A. Physiological data is collected through a contact-type physiological sensing unit. The collected physiological data includes at least one of the following: heart rate data, blood oxygen data, heart rate variability data, electroencephalogram (EEG) data, and skin conductance response data. The acquisition parameters and sensing positions for each data type are optimized to ensure acquisition accuracy. Heart rate, blood oxygen, and heart rate variability data: acquired through a photoplethysmography (PPG) sensor. The sensor is embedded in the temple or nasal side area of ​​the XR head-mounted display faceplate, achieving a flexible fit with the skin. The sampling frequency is configured to 100Hz, and a dual-light path acquisition mode of green light and infrared light is adopted to reduce acquisition errors caused by skin movement and hair occlusion. Electroencephalogram (EEG) data: acquired through dry electrode EEG sensors, which are placed in the frontal and parietal lobe regions of the headband. The sampling frequency is configured to 250Hz, and the data are collected in frequency bands such as alpha, beta, and theta waves for subsequent neural state analysis. Skin conductance response data (GSR): Collected through a dual-electrode sensor embedded in the contact area behind the ear of the headband, with a sampling frequency configured at 10Hz, to detect changes in skin conductance and reflect the user's level of excitement or tension.

[0025] B. Environmental perception data is collected through non-contact environmental perception units. The collected environmental perception data includes at least one of visual image data, millimeter-wave radar point cloud data, and inertial measurement data. The field of view, detection range, and sampling parameters of each sensing module are matched with each other to achieve all-round perception of the user's surrounding physical environment in 360°. Visual image data: Acquired through an inside-out visual camera, which is embedded in the front and sides of the XR headset shell. The resolution is configured as 1080p, the field of view (FOV) is 120°, and the sampling frame rate is 30fps. It simultaneously realizes environmental image acquisition, simultaneous localization and mapping (SLAM), and user gesture tracking functions. It only acquires local feature images of the environment and does not transmit complete optical images, thus initially reducing the risk of privacy leakage. Millimeter-wave radar point cloud data: Collected through a millimeter-wave radar chip, the radar is integrated into the four corners of the XR headset shell, operates at a frequency of 60GHz, has a detection range of 0.1~10 meters, and outputs three-dimensional point cloud data. It can detect the distance, relative speed, and azimuth of obstacles. Its detection is not affected by light intensity, smoke, dust, or transparent objects. It only processes point cloud data and does not collect any optical images, thus fundamentally avoiding privacy leaks. Inertial measurement data: Acquired through an inertial measurement unit (IMU), which is integrated into the center of gravity of the XR head-mounted display. The sampling frequency is configured to 100Hz to collect data on head acceleration, angular velocity, and attitude angle, assisting in SLAM positioning, user pose tracking, and motion state determination.

[0026] In some embodiments, an infrared thermal imaging sensor is added in conjunction with a contact physiological sensing unit to achieve non-contact auxiliary acquisition of physiological parameters such as body temperature and microcirculation, thereby compensating for the acquisition error of the contact sensor when the user is exercising vigorously or sweating. Add an ultra-wideband (UWB) positioning module, which works in conjunction with inertial measurement data, to achieve absolute position acquisition of users in large physical spaces (such as immersive experience halls and industrial training sites), thereby improving pose tracking accuracy; By adding a barometric pressure sensor to collect ambient air pressure data and combining it with millimeter-wave radar data, the obstacle distance detection error under different air pressure environments can be corrected, making it suitable for special XR application scenarios such as outdoor and high-altitude environments.

[0027] Step S120: Process the physiological data and environmental perception data respectively to obtain the physiological state data corresponding to the physiological data and the real-time environmental data containing obstacle semantic information corresponding to the environmental perception data.

[0028] This step is the data preprocessing and feature extraction stage. The processing terminal is the built-in high-performance ARM processor, FPGA or external computing unit of the XR headset. Modular algorithms are used to filter, reduce noise, calibrate and fuse the raw acquired data to remove invalid data and interference signals, and extract physiological state data and real-time environmental data with practical analytical value. The environmental perception data must include obstacle semantic information after processing, and the physiological data must be calibrated and fused from multiple sources to ensure the accuracy of subsequent state and risk assessment.

[0029] Specifically, A. Process the physiological data to obtain the corresponding physiological state data, including: Reference physiological parameters are extracted from the acquired contact-based physiological sensing signals, and the observed physiological parameters extracted from the acquired non-contact environmental sensing signals are calibrated online to obtain physiological state data.

[0030] The process can be understood as follows: First, the collected raw physiological signals such as PPG, EEG, and GSR are filtered and denoised. The PPG signal is filtered by bandpass filtering (0.5~10Hz) to remove motion artifacts and baseline drift. The EEG signal is filtered by wavelet transform to remove power frequency interference (50 / 60Hz) and electromyographic interference. The GSR signal is filtered by moving average filtering to remove random noise, ensuring that the signal-to-noise ratio (SNR) of the raw physiological signals is ≥30dB, laying the foundation for subsequent parameter extraction. Baseline physiological parameters are extracted from preprocessed contact physiological sensing signals, including heart rate, blood oxygen, and heart rate variability (HRV) from PPG signals, power ratios (α / β, θ / α) of each frequency band from EEG signals, and skin conductance level (SCL) and skin conductance response (SCR) peak and frequency from GSR signals. Simultaneously, observational physiological parameters are extracted from non-contact environmental sensing signals, specifically heart rate and respiratory rate from millimeter-wave radar body motion signals (BCG). The non-contact detection characteristics of millimeter-wave radar are used to achieve auxiliary acquisition of physiological parameters. Afterwards, combined Figure 3 As shown, the baseline physiological parameters extracted from contact-type physiological sensing signals are used as the true values ​​to adaptively calibrate the observed physiological parameters extracted from non-contact environmental sensing signals. The error of the observed physiological parameters is corrected through a dynamic compensation algorithm. Finally, the calibrated observed physiological parameters are fused with the baseline physiological parameters to obtain complete physiological state data.

[0031] Furthermore, the dynamic compensation algorithm is as follows:

[0032] in, The calibrated heart rate, Estimated heart rate extracted from millimeter-wave radar BCG signals. The baseline heart rate is extracted from the contact PPG signal. and The adaptive weighting coefficients are dynamically adjusted based on signal quality (such as signal-to-noise ratio, SNR). γ is a bias compensation term with a value ranging from -5 to 5 bpm. When SNR ≥ 40 dB, α is 0.9 and β is 0.1; when 30 dB ≤ SNR < 40 dB, α is 0.8 and β is 0.2. The values ​​are also adjusted in real-time based on the user's motion state (determined by IMU data): γ is 0 when the user is stationary, 1~2 bpm during low-speed motion, and 3~5 bpm during high-speed motion.

[0033] Through the above online calibration, even in scenarios where the user is exercising vigorously or the contact sensor is briefly removed from the skin, clinically accurate physiological data can still be obtained, with heart rate measurement error ≤ ±2 bpm and heart rate variability measurement error ≤ ±5 ms, solving the problem of acquisition accuracy of single sensing methods in dynamic scenarios.

[0034] B. The core of environmental perception data processing is multi-sensor data fusion and semantic modeling. By leveraging the advantages of fusing visual image data, millimeter-wave radar point cloud data, and inertial measurement data, it overcomes the limitations of single-sensor perception and ultimately generates real-time updated 3D environmental data containing static / dynamic obstacle semantic information. The specific processing flow is as follows: Feature point extraction (using ORB / SIFT algorithm) and distortion correction are performed on visual image data to remove blurry, overexposed / underexposed invalid image frames; noise reduction and clustering are performed on millimeter-wave radar point cloud data to remove clutter points and outliers; attitude calculation (using Kalman filter) is performed on inertial measurement data to obtain the real-time pose and motion trajectory of the user's head, providing pose reference for subsequent data fusion; The coordinate transformation and registration of the 3D point cloud generated by the millimeter-wave radar and the sparse feature point cloud generated by the visual SLAM algorithm are performed. Based on the real-time pose provided by the IMU data, the extrinsic parameter matrix between the sensors is calculated to achieve spatial alignment of visual image data and millimeter-wave radar point cloud data. The registration error is ≤ ±0.05 meters, ensuring that environmental perception data from different sensors are fused in the same coordinate system. By combining the registered multi-source environmental perception data, semantic recognition and classification of obstacles in the user's surrounding environment are performed. By fusing the spatial distribution density of point cloud, the relative velocity of obstacles detected by millimeter-wave radar, and the appearance features of obstacles extracted from visual images through extended Kalman filter (EKF), static obstacles (such as walls, furniture, glass doors, and pillars) and dynamic obstacles (such as pedestrians, pets, moving furniture, and vehicles) are distinguished. A motion trajectory prediction model (using Kalman prediction algorithm) is established for dynamic obstacles to predict their motion path in the next 1 to 3 seconds. The identified obstacle information is fused with the environmental map constructed by SLAM to generate real-time environmental data containing semantic information about obstacles. This data not only includes the location, size, and distance information of obstacles, but also labels the type of obstacles (static / dynamic), the movement speed and trajectory of dynamic obstacles, and the danger level of obstacles (normal / high-risk). At the same time, it divides the environment into safe areas, risk areas, and dangerous areas, providing complete environmental information for subsequent collision risk assessment.

[0035] In some embodiments, deep learning models are used to replace traditional filtering algorithms. For example, CNN convolutional neural networks are used to denoise raw physiological signals, and Transformer models are used to fuse multi-sensor environmental data, improving signal processing and feature extraction capabilities in complex scenarios. An edge computing and cloud collaborative processing mode is introduced, where lightweight preprocessing (such as filtering and denoising) is completed locally on the XR headset, while complex calibration, fusion, and semantic recognition algorithms are completed on edge computing units or cloud servers, reducing the local computing power consumption of the XR headset and adapting to low-end and mid-range XR hardware platforms. Data caching and fault tolerance mechanisms can also be added. When a sensor module fails or data is interrupted, cached historical data and auxiliary data from other sensor modules are called to achieve continuous generation of physiological state data and real-time environmental data, avoiding system downtime caused by sensor failure.

[0036] Step S130: Determine the user's current physiological state based on physiological state data, and determine the collision risk based on real-time environmental data and the user's pose information.

[0037] This step is the state determination and risk assessment stage. Using a pre-set model and algorithm, the physiological state data and real-time environmental data processed in step S120 are analyzed, outputting quantified data on the user's current physiological state and collision risk, providing a basis for subsequent content rendering parameter adjustments. The user's pose information is obtained by fusing IMU data and SLAM positioning data, including the user's real-time location, head posture, movement speed, and direction of movement. The update frequency of the pose information is consistent with the real-time environmental data to ensure the real-time nature of the collision risk assessment.

[0038] Specifically, determining a user's current physiological state based on physiological state data includes: A trained physiological state recognition model is used to process physiological state data and output the current physiological state. Current physiological state includes: motion sickness index, cognitive load index, and emotional arousal level.

[0039] Furthermore, the Motion Sickness Index (MSI) can be categorized as follows: 0 indicates no motion sickness or discomfort, and 100 indicates severe motion sickness (symptoms such as dizziness, nausea, and vomiting). Cognitive Load Index (CLI) can be categorized as follows: 0 indicates low cognitive load (users can easily cope with XR content), and 100 indicates high cognitive load (users cannot understand or cope with XR content, and experience symptoms such as inattention and fatigue). Emotional arousal (AE) can be categorized as follows: 0 indicates that the user is calm, and 100 indicates that the user is highly excited (or tense, anxious).

[0040] Next, the physiological state data is standardized, mapping each physiological parameter to the range of 0-1 to eliminate the dimensional differences between different parameters. The standardized physiological state data is then input into a trained physiological state recognition model, which calculates through forward propagation and outputs preliminary scores for each physiological state. The preliminary scores are dynamically corrected by adjusting the weights based on the user's XR experience duration and motion state (determined by IMU data). For example, when the XR experience duration exceeds 30 minutes, the weight of the motion sickness index is increased by 20%, and when the user is moving at high speed, the weight of the emotional excitement score is increased by 15%. The corrected quantitative physiological state index is then output as the basis for physiological decisions regarding subsequent content adjustments.

[0041] Collision risks are determined based on real-time environmental data and user pose information, including: The configured collision risk probability algorithm is used to calculate the real-time distance between the user and the obstacles identified in the real-time environmental data, and to obtain the collision risk probability that represents the collision risk.

[0042] This step uses a configured collision risk probability algorithm to calculate the real-time distance between the user and the identified obstacles in the real-time environmental data, thus obtaining the collision risk probability (P) that characterizes the collision risk. collision The probability is a value between 0 and 1. The closer the value is to 1, the higher the risk of a collision between the user and the obstacle; the closer the value is to 0, the lower the risk of a collision. The collision risk calculation not only considers the real-time distance between the obstacle and the user, but also integrates the type of obstacle, the speed of the dynamic obstacle, and the user's movement speed and direction, to achieve a multi-dimensional collision risk assessment.

[0043] The collision risk probability is calculated using a distance-based exponential function algorithm, with the following formula:

[0044] in, The real-time distance between the user and the obstacle. The preset safe distance threshold (default 1.5 meters). This is the sensitivity coefficient (which can be adjusted according to the user's movement speed, with a value ranging from 1 to 3). The closer the value is to 1, the higher the risk of collision.

[0045] In some embodiments, dynamic obstacle trajectory fusion is introduced to optimize the collision risk probability. When there are dynamic obstacles in the real-time environmental data, combined with its motion trajectory prediction model, the predicted distance d between the user and the dynamic obstacle in the next 1 second is calculated. pre If pre d < d0, the collision risk probability is increased by 30% - 50% to achieve a forward-looking assessment of the dynamic collision risk and solve the problem that traditional algorithms only consider the real-time distance and cannot predict dynamic collisions.

[0046] In another embodiment, a user personalized feature library is added to the physiological state recognition model. The physiological baseline data of different users (such as resting heart rate, basic EEG band ratio) are stored in the feature library. The model adjusts the decision threshold according to the user's personalized baseline data to achieve personalized recognition of the physiological state and solve the problem of judgment errors caused by differences in physiological characteristics of different users; the reinforcement learning algorithm is used to dynamically optimize the parameters (d0, k) of the collision risk probability algorithm. According to the user's historical collision events and obstacle avoidance behavior data, the safety distance threshold and sensitivity coefficient are adjusted in real time to adapt to the motion habits and reaction speeds of different users; the multi-obstacle collision risk fusion assessment is added. When there are multiple obstacles around the user, the individual collision risk probabilities of each obstacle are calculated, and the comprehensive collision risk probability is calculated according to the relative positions of the obstacles and the user to avoid risk omission caused by single obstacle assessment.

[0047] Step S140: Adjust the rendering parameters of the virtual reality or augmented reality content presented to the user according to the current physiological state and / or collision risk.

[0048] Specifically, adjusting the rendering parameters of the virtual reality or augmented reality content presented to the user according to the current physiological state and / or collision risk includes: When the motion sickness index exceeds the configured first preset threshold, perform at least one of the following operations: reduce the field of view angle of the virtual reality or augmented reality content, reduce the moving speed of the virtual camera.

[0049] Or, when the collision risk probability exceeds the configured risk thresholds at different levels, trigger visual warnings, auditory warnings, tactile warnings, or switch to a see-through reality video view in a hierarchical manner.

[0050] In some embodiments, when the motion sickness index exceeds the configured first preset threshold (MSI > 70), perform at least one of the following operations, and the adjustment amplitude can be dynamically graded according to the score of the motion sickness index: Reduce the field of view angle of XR content from the default 110° - 120° by 10% - 20%, that is, adjust it to 88° - 108°. The higher the motion sickness index, the greater the reduction (reduce by 20% when MSI > 80, reduce by 10% when 70 < MSI ≤ 80). By reducing the field of view angle, reduce the visual input of the user and relieve the conflict between visual fatigue and vestibular sensation; Reduce the moving speed of the virtual camera in XR content by 30% - 50%. The higher the motion sickness index, the greater the reduction (reduce by 50% when MSI > 80, reduce by 30% when 70 < MSI ≤ 80). Make the moving speed of the virtual camera match the user's head movement and vestibular sensation perception, and reduce the motion sickness discomfort caused by visual movement; Add static stable virtual reference points (such as horizon, fixed logo, crosshair) in the central area of the field of view of XR content. The size of the reference point is 5% - 10% of the field of view, and the color is high - contrast white or yellow, providing a visual anchor point for the user and enhancing visual stability; Reduce the rendering frame rate of XR content from 90fps to 60fps appropriately, and at the same time turn off the dynamic blur effect of the screen, reduce the visual stimulation caused by rapid screen changes, and relieve motion sickness symptoms.

[0051] In some embodiments, the present application can be assisted and adjusted according to the cognitive load index and emotional excitement degree to achieve personalized optimization of the XR experience: When the cognitive load index exceeds the configured second preset threshold, reduce the picture information density.

[0052] That is, when the cognitive load index is too high (CLI > 80), simplify the picture complexity of XR content, reduce the number of texts and graphics in the picture, reduce the detailed rendering precision of virtual objects, and at the same time slow down the interaction rhythm of XR content, extend the user's operation response time, and reduce the user's cognitive load; When the emotional excitement degree is lower than the configured third preset threshold and the current is an entertainment application, increase the content stimulation intensity.

[0053] That is, when the emotional excitement degree is too low (AE < 30) and the XR content is a game or entertainment application, appropriately increase the picture color saturation, add dynamic special effects, or trigger random interaction events (such as item drops and plot branches in games) to increase the user's emotional excitement degree; When the emotional excitement degree is too high (AE > 90) and the XR content is a training or teaching application, reduce the color saturation of the picture, reduce dynamic special effects, play soothing background sounds, and slow down the interaction rhythm of XR content to relieve the user's tension or anxiety.

[0054] When the collision risk probability exceeds the configured risk thresholds at different levels, a combination of tiered warning triggers and adjustments to rendering parameters is used to ensure the user's physical safety while minimizing interference with the XR immersive experience. The collision risk probability thresholds are divided into a primary threshold and a secondary threshold, with the primary threshold configured as 0.3 (P... collision A threshold of >0.3 indicates a low to moderate collision risk; the secondary threshold is configured to 0.7 (P<0.3). collision A value >0.7 indicates a high risk of collision. Both threshold levels can be dynamically adjusted based on the user's movement habits and reaction speed.

[0055] Level 1 Collision Risk: Mild warning, no adjustment to core rendering parameters. When the probability of collision risk exceeds the first-level threshold, a first-level warning is triggered, which is also known as a haptic warning. This involves only a slight visual cue to alert the user to their surroundings, without adjusting the core rendering parameters of the XR content, ensuring the immersive experience remains unaffected. The first-level warning works as follows: a ripple-like pattern is displayed on the virtual ground or edge of the XR content's field of view. The ripples spread from the direction of the obstacle towards the user, are light blue, and have 70% transparency. The speed of the ripples is positively correlated with the probability of collision risk; the higher the probability, the faster the ripples spread.

[0056] Level 2 collision risk: Strong warning, adjust according to rendering parameters. When the collision risk probability exceeds the secondary threshold, a secondary strong warning is triggered, which includes both auditory and tactile warnings. Simultaneously, the rendering parameters of the XR content are adjusted appropriately to forcefully remind the user to avoid obstacles and ensure physical safety. The secondary strong warning performs at least one of the following actions, and the warning intensity can be adjusted according to the collision risk probability: The visual warning involves turning the virtual safety boundary in the XR content red and flashing it at a high frequency (2~3Hz), while displaying the outline of the obstacle in the field of vision. The outline is highlighted in red with 50% transparency, clearly indicating the location and size of the obstacle. The auditory warning is a 3D spatial sound effect warning emitted through the XR headset's headphones. The sound effect is emitted from the direction of the obstacle, at a volume of 80% to 100% of the user's current volume, and is a short and clear prompt tone (such as "Please be aware of the obstacle ahead") to avoid user discomfort caused by harsh sound effects. The tactile alert is provided by vibration feedback through the XR headset or the matching handheld controller. The vibration intensity is 80%~100% and the vibration frequency is 50~100Hz. When the obstacle is on the left / right side of the user, only the left / right side of the headset or controller vibrates, thus achieving spatial tactile cues. Rendering Parameters and View Adjustment: Temporarily switch the VR view to the see-through reality video view, overlay the real-world environment video captured by the XR headset's camera on the virtual content, with the transparency of the real-world environment video being 60% - 80%, enabling the user to clearly see the surrounding physical environment. At the same time, pause the virtual camera movement of the XR content to avoid the movement of virtual content disturbing the user's obstacle avoidance behavior; Forced Deceleration / Stop: If the XR content is a virtual walking or flying application, forcefully reduce or stop the movement speed of the virtual character until the collision risk probability is lower than the first-level threshold to prevent the user from having an actual collision due to virtual movement.

[0057] In another embodiment, the rendering parameters are adjusted based on the collaborative determination of the current physiological state and collision risk. When the user has both a high motion sickness index and a high collision risk, the safety warnings and view adjustments related to the collision risk are preferentially executed, and then the rendering parameter adjustments related to motion sickness are performed to ensure that physical safety is the primary goal; when the user's motion sickness index is at the critical value (60 < MSI ≤ 70) and there is a mild collision risk (0.3 < P collision ≤ 0.7), appropriately reduce the field of view angle and trigger a slight visual warning to simultaneously achieve physiological comfort protection and mild safety protection.

[0058] In summary, the extended reality content adjustment method based on physiological and environmental perception of this application sequentially executes the above four steps, and takes the changes in the user's physiological state, collision risk changes, and the user's interaction behavior data (such as ignoring warnings and actively adjusting XR content) after each adjustment as feedback data, storing them in the user data warehouse on the local or cloud side. Regularly (such as every 24 hours), use the reinforcement learning algorithm to optimize and update the various parameters in the method (such as the online calibrated weight coefficients, the thresholds of the physiological state recognition model, the algorithm parameters of the collision risk probability, and the adjustment amplitude of the rendering parameters), so that the execution strategy of the method continuously adapts to the user's personalized physiological characteristics, movement habits, and XR experience preferences, realizing the upgrade from general adjustment to personalized adjustment, and truly achieving a people-centered XR adaptive experience.

[0059] This application provides a method and apparatus for adjusting augmented reality content based on physiological and environmental perception. The method includes: acquiring the user's physiological data and environmental perception data of the current physical environment; processing the physiological data and environmental perception data separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data; determining the user's current physiological state based on the physiological state data, and determining the collision risk based on the real-time environmental data and the user's pose information; and adjusting the rendering parameters of the virtual reality or augmented reality content presented to the user according to the current physiological state and / or collision risk. This method achieves accurate perception and fusion processing of multimodal physiological and environmental data of XR users, can predict motion sickness and discomfort in advance and dynamically adjust rendering parameters, effectively alleviate physiological discomfort, avoid physical collisions, and balance immersive experience with user safety.

[0060] Corresponding to the above method, embodiments of this application also provide an extended reality content adjustment device based on physiological and environmental perception, such as... Figure 4 As shown, the device includes: The acquisition unit 410 is used to acquire the user's physiological data and the environmental perception data of the current physical environment; Processing unit 420 is used to process the physiological data and the environmental perception data respectively to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The determining unit 430 is used to determine the user's current physiological state based on the physiological state data, and to determine the collision risk based on the real-time environmental data and the user's pose information; The adjustment unit 440 is used to adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

[0061] The functions of each unit in the extended reality content adjustment device based on physiological and environmental perception provided in the above embodiments of this application can be implemented through the above-described method steps. Therefore, the specific working process and beneficial effects of each unit in the extended reality content adjustment device based on physiological and environmental perception provided in the embodiments of this application will not be repeated here.

[0062] This application also provides an electronic device, such as... Figure 5 As shown, it includes a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540.

[0063] Memory 530 is used to store computer programs; When the processor 510 executes the program stored in the memory 530, it performs the following steps: Acquire user physiological data and environmental perception data of the current physical environment; The physiological data and the environmental perception data are processed separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The user's current physiological state is determined based on the physiological state data, and the collision risk is determined based on the real-time environmental data and the user's pose information. Adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

[0064] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0065] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0066] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0067] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0068] The implementation methods and beneficial effects of the various components of the electronic device in the above embodiments for solving the problem can be found in [reference needed]. Figure 1 The steps in the illustrated embodiments are used to implement the electronic device. Therefore, the specific working process and beneficial effects of the electronic device provided in this application will not be repeated here.

[0069] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the above embodiments of an extended reality content adjustment method based on physiological and environmental perception.

[0070] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the above embodiments of an extended reality content adjustment method based on physiological and environmental perception.

[0071] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This application describes embodiments of methods, apparatus (systems), and computer program products according to embodiments of this application with reference to flowchart illustrations and / or block diagrams. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected," "coupled," or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0076] Although preferred embodiments have been described in this application, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the embodiments in this application are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments in this application.

[0077] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the embodiments of this application and their equivalents, then these modifications and variations are also intended to be included in the embodiments of this application.

Claims

1. A method for adjusting extended reality content based on physiological and environmental perception, characterized in that, The method includes: Acquire user physiological data and environmental perception data of the current physical environment; The physiological data and the environmental perception data are processed separately to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The user's current physiological state is determined based on the physiological state data, and the collision risk is determined based on the real-time environmental data and the user's pose information. Adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

2. The method as described in claim 1, characterized in that, The physiological data is processed to obtain the corresponding physiological state data, including: The baseline physiological parameters are extracted from the acquired contact-type physiological sensing signals, and the observed physiological parameters extracted from the acquired non-contact environmental sensing signals are calibrated online to obtain the physiological state data.

3. The method as described in claim 1, characterized in that, The environmental perception data includes at least one of visual image data, millimeter-wave radar point cloud data, and inertial measurement data; The physiological data includes at least one of heart rate data, blood oxygen data, heart rate variability data, electroencephalogram (EEG) data, and skin conductance response data.

4. The method as described in claim 1, characterized in that, Determining the user's current physiological state based on the aforementioned physiological state data includes: The physiological state data is processed using a trained physiological state recognition model, and the current physiological state is output.

5. The method as described in claim 4, characterized in that, Based on the real-time environmental data and the user's pose information, collision risk is determined, including: Using a configured collision risk probability algorithm, the real-time distance between the user and the obstacles identified in the real-time environmental data is calculated to obtain the collision risk probability that characterizes the collision risk.

6. The method as described in claim 5, characterized in that, The current physiological state includes: motion sickness index, cognitive load index, and emotional arousal level; Based on the current physiological state and / or the collision risk, adjust the rendering parameters of the virtual reality or augmented reality content presented to the user, including: When the motion sickness index exceeds a configured first preset threshold, perform at least one of the following operations: reduce the field of view of the virtual reality or augmented reality content, or reduce the movement speed of the virtual camera; When the cognitive load index exceeds the configured second preset threshold, the image information density is reduced. When the emotional excitement level is lower than the configured third preset threshold and the current application is entertainment, the intensity of content stimulation is increased; And / or, when the probability of collision risk exceeds the risk thresholds configured for different levels, trigger visual warnings, auditory warnings, tactile warnings, or switch to a see-through reality video view.

7. The method as described in claim 6, characterized in that, Tiered triggering of visual, auditory, and tactile alerts includes: When the probability of collision risk exceeds the configured first risk threshold, the visual warning is triggered; When the collision risk probability exceeds the configured second risk threshold, the auditory warning and the tactile warning are triggered.

8. An extended reality content adjustment device based on physiological and environmental perception, characterized in that, The device includes: The acquisition unit is used to acquire the user's physiological data and environmental perception data of the current physical environment; The processing unit is used to process the physiological data and the environmental perception data respectively to obtain physiological state data corresponding to the physiological data and real-time environmental data containing obstacle semantic information corresponding to the environmental perception data. The determining unit is used to determine the user's current physiological state based on the physiological state data, and to determine the collision risk based on the real-time environmental data and the user's pose information; An adjustment unit is used to adjust the rendering parameters of the virtual reality or augmented reality content presented to the user based on the current physiological state and / or the collision risk.

9. An electronic device, characterized in that, The electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

Images