Control method and device of augmented reality system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a control method and device of an augmented reality system, electronic equipment and a storage medium, and belongs to the technical field of augmented reality, and the control method of the augmented reality system comprises the steps: obtaining physiological data and motion data of a user; establishing a parameter set of the physiological data and the motion data; according to the parameter set, a first control signal is determined, and the first control signal comprises a visual adjustment parameter, an auditory adjustment parameter and a tactile adjustment parameter; and controlling visual feedback, auditory feedback and tactile feedback of the augmented reality system according to the first control signal.

Description

Technical Field

[0001] The present application belongs to the field of extended reality technology, and specifically relates to a control method, device, electronic device and storage medium of an extended reality system. Background Art

[0002] In related technologies, the Extended Reality (XR) system mainly relies on hardware devices such as head-mounted displays (HMDs) and handheld controllers to enable user interaction with the virtual world. However, the virtual environment simulated by the XR system is the same for different wearers and cannot provide personalized virtual environments for different wearers, thus affecting the quality and immersion of the virtual environment experience. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a control method, device, electronic device and storage medium for an extended reality system, which can effectively solve the technical problem that the extended reality system cannot provide a personalized virtual environment.

[0004] In a first aspect, an embodiment of the present application provides a method for controlling an augmented reality system, comprising:

[0005] Obtain users' physiological and motion data;

[0006] Establish parameter sets for physiological and motion data;

[0007] Determining a first control signal according to the parameter set, the first control signal including a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter;

[0008] According to the first control signal, visual feedback, auditory feedback and tactile feedback of the augmented reality system are controlled.

[0009] In a second aspect, an embodiment of the present application provides a control device for an augmented reality system, comprising:

[0010] An acquisition module is used to obtain the user's physiological data and motion data;

[0011] Establishing a module for establishing parameter sets for physiological data and motion data;

[0012] a determination module, configured to determine a first control signal according to the parameter set, the first control signal including a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter;

[0013] The control module is configured to control the visual feedback, auditory feedback, and tactile feedback of the extended reality system according to the first control signal.

[0014] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the control method of the extended reality system provided in the first aspect are implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a storage medium storing a program or instruction, which, when executed by a processor, implements the steps of the control method of the extended reality system provided in the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the control method of the extended reality system provided in the first aspect.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the control method of the extended reality system provided in the first aspect.

[0018] In an embodiment of the present application, when a user uses an extended reality system, the user's physiological data and motion data are obtained, and a parameter set of the physiological data and motion data is established. Based on the physiological data and motion data in the parameter set, a first control signal is determined. The first control signal includes visual adjustment parameters, auditory adjustment parameters, and tactile adjustment parameters. Thus, according to the first control signal, the visual feedback, auditory feedback, and tactile feedback of the extended reality system are controlled. That is, the virtual environment generated by the extended reality system will adjust the visual feedback, auditory feedback, and tactile feedback based on the user's physiological data and motion data, thereby providing the user with a personalized virtual environment and improving the quality and immersion of the virtual environment experience.

[0019] In addition, the unified management of physiological data and motion data based on parameter sets can improve the data processing speed and reduce the latency of the extended reality system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 One of the flow charts of the control method of the extended reality system according to an embodiment of the present application is shown;

[0021] Figure 2 A second flowchart of a method for controlling an extended reality system according to an embodiment of the present application is shown;

[0022] Figure 3 A structural block diagram of a control device for an extended reality system according to an embodiment of the present application is shown;

[0023] Figure 4shows a structural block diagram of an electronic device according to an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0026] The terms "first," "second," and the like in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in an order other than that illustrated or described herein, and that the objects distinguished by "first," "second," and the like are generally of the same type, and do not limit the number of objects; for example, the first object can be one or more. In addition, the term "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0027] In conjunction with the accompanying drawings, the following describes a control method, device, electronic device, and storage medium for an extended reality system provided by the embodiments of the present application through specific embodiments and their application scenarios.

[0028] Figure 1 FIG1 shows one of the flow charts of the control method of the extended reality system according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0029] Step 102: Acquire the user's physiological data and motion data.

[0030] Specifically, the user's physiological data and motion data are obtained through sensors, wherein the physiological data includes but is not limited to skin electrical activity data D EDA (t), ECG signal data D ECG (t) and blood oxygen saturation data D SpO2 (t), motion data includes but is not limited to posture change data D in three-dimensional space pose (t), gait analysis data D gait (t) and eye tracking data D eye (t).

[0031] Among them, the skin electrical activity data can be obtained through the skin electrical activity (Electrodermal Activity, EDA) sensor S EDA The electrocardiogram signal data can be obtained by the electrocardiogram (ECG) sensor S ECG The blood oxygen saturation data can be obtained through the blood oxygen saturation (Oxygen Saturation, SpO2) sensor S SpO2 Acquisition, the attitude change data in three-dimensional space can be obtained by the accelerometer S ACC Gait analysis data can be obtained through gyroscope S GYRO Eye tracking data can be obtained through the eye tracking system S ET Get.

[0032] Various sensors can be set in wearable devices.

[0033] Step 104: Establish parameter sets of physiological data and motion data.

[0034] Specifically, a parameter set of physiological data and motion data is established, that is, skin electrode activity data, electrocardiogram signal data, blood oxygen saturation data, posture change data in three-dimensional space, gait analysis data and eye tracking data are integrated into a parameter set DataSet(t).

[0035] DataSet(t)={D EDA (t), D ECG (t), D SpO2 (t), D pose (t), D gait (t), D eye

[0036] (t)}. t represents the time point, that is, the data value in the parameter set. The data collected by the sensor at each time point, that is, each data collection, is recorded once, and all the data are integrated into the parameter set.

[0037] Step 106: Determine a first control signal according to the parameter set, where the first control signal includes a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter.

[0038] Specifically, a first control signal is determined by analyzing the data in the parameter set, and the first control signal includes a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter.

[0039] Step 108: Control the visual feedback, auditory feedback, and tactile feedback of the augmented reality system according to the first control signal.

[0040] Specifically, according to the first control signal, the visual feedback, auditory feedback and tactile feedback of the extended reality system are controlled to generate a personalized virtual environment.

[0041] In an embodiment of the present application, when a user uses an extended reality system, the user's physiological data and motion data are obtained, and a parameter set of the physiological data and motion data is established. Based on the physiological data and motion data in the parameter set, a first control signal is determined. The first control signal includes visual adjustment parameters, auditory adjustment parameters, and tactile adjustment parameters. Thus, according to the first control signal, the visual feedback, auditory feedback, and tactile feedback of the extended reality system are controlled. That is, the virtual environment generated by the extended reality system will adjust the visual feedback, auditory feedback, and tactile feedback based on the user's physiological data and motion data, thereby providing the user with a personalized virtual environment and improving the quality and immersion of the virtual environment experience.

[0042] In addition, the unified management of physiological data and motion data based on parameter sets can improve the data processing speed and reduce the latency of the extended reality system.

[0043] Among them, extended reality systems include augmented reality (AR) systems, virtual reality (VR) systems and mixed reality (MR) systems.

[0044] The control method of the extended reality system based on multi-sensor fusion of wearable devices proposed in this application can adjust the virtual environment in real time to adapt to the user's specific needs and reactions by collecting and analyzing the user's physiological data and motion data.

[0045] As a possible implementation method, determining the first control signal based on the parameter set includes: performing a quantum Fourier transform on the data in the parameter set to obtain frequency domain data; filtering the frequency domain data; performing an inverse quantum Fourier transform on the filtered frequency domain data to obtain time domain data; matching weights for the time domain data and determining a eigenvector in combination with a bias vector; and determining the first control signal based on the eigenvector.

[0046] Specifically, according to the parameter set, the first control signal is determined, including: since the data in the parameter set is stored based on the time domain, the time domain data is first transformed into frequency domain data through quantum Fourier transform, and the frequency domain data is filtered to remove non-typical frequency components and noise. Then, the filtered frequency domain data is transformed into time domain data through quantum Fourier inverse transform. At this time, the time domain data retains the characteristics of the original data and removes the noise. Then, the time domain data is matched with weights and combined with the bias vector to obtain a feature vector. Then, based on the feature vector, the first control signal is determined. Through the above sensor fusion and quantum computing, the first control signal can be obtained quickly and accurately, thereby improving the response speed of the extended reality system.

[0047] That is, for each sensor data D in the parameter set type , D type Can take the value D EDA (t), D ECG (t), D SpO2 (t), D pose (t), D gait (t) and D eye One of (t) performs frequency domain analysis through quantum Fourier transform, integrating the parallelism and superposition state characteristics of quantum computing.

[0048] As a possible implementation, performing a quantum Fourier transform on the data in the parameter set to obtain frequency domain data includes: inputting the data in the parameter set into a first transformation formula; receiving the frequency domain data output by the first transformation formula; the first transformation formula is:

[0049]

[0050] QFT(D type (k)) represents frequency domain data, D type (k) represents the data value of a specific sensor type at time point k, D type (y) represents the data value of the final state of the quantum state, q represents the number of quantum bits, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, Represents the transformation process of data from time domain to frequency domain, i represents the imaginary unit;

[0051] Performing an inverse quantum Fourier transform on the filtered frequency domain data to obtain time domain data includes: inputting the filtered frequency domain data into a second transformation formula; receiving the time domain data output by the second transformation formula; the second transformation formula is:

[0052]

[0053] QIFT(D filtered(k)) represents the time domain data, q represents the number of quantum bits, D filtered (y) represents the frequency domain data after filtering, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit.

[0054] Specifically, the data in the parameter set is subjected to quantum Fourier transform to obtain frequency domain data, including: inputting the data in the parameter set into a first transformation formula, outputting the frequency domain data after the first transformation formula is calculated, and receiving the frequency domain data output by the first transformation formula.

[0055] The filtered frequency domain data is subjected to an inverse quantum Fourier transform to obtain time domain data, including: inputting the filtered frequency domain data into a second transformation formula, outputting the time domain data after the second transformation formula is calculated, and receiving the time domain data output by the second transformation formula.

[0056] Specifically, the first transformation formula is:

[0057]

[0058] Among them, QFT(D type (k)) represents the frequency domain data obtained after applying quantum Fourier transform, D type (k) represents the data value of a specific sensor type at time point k, D type (y) represents the data value of the final state of the quantum state, and q represents the number of quantum bits, that is, the number of quantum bits used in quantum computing. The higher the number of quantum bits, the more states the system can represent. q The more, 2 q Indicates the number of states of a quantum computing system, which is determined by the number of quantum bits q. In quantum computing, each quantum bit is in a superposition state of 0 and 1. Therefore, q quantum bits represent 2 q There are different states, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, and the above states interact with each other in the quantum Fourier transform. It is the core part of the transformation, which represents the transformation process of data from time domain to frequency domain. i represents the imaginary unit.

[0059] Through the quantum filter QFT (D type (k)) is filtered to remove non-typical frequency components and noise to obtain D filtered (k), D filtered (k) represents the frequency domain data after filtering, that is, D filtered (k) = F × QFT(D type(k)); where F represents a quantum filtering operation matrix, which is designed based on the superposition and entanglement characteristics of quantum states to remove noise while maintaining signal characteristics.

[0060] The data D processed by quantum filtering filtered (k) Converting back to time domain data through quantum inverse Fourier transform, that is, the process of quantum inverse Fourier transform, remaps the frequency domain data to time domain data and combines all possible quantum states. The second transformation formula is:

[0061]

[0062] Among them, QIFT(D filtered (k)) represents the time domain data obtained after the frequency domain data is transformed by the inverse quantum Fourier transform, and q represents the number of quantum bits, that is, the number of quantum bits used in quantum computing. The higher the number of quantum bits, the more states the system can represent. q The more, 2 q Indicates the number of states of a quantum computing system, which is determined by the number of quantum bits q. In quantum computing, each quantum bit is in a superposition state of 0 and 1. Therefore, q quantum bits represent 2 q Different states, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, the above states interact in the quantum Fourier transform, D filtered (y) represents the frequency domain data after filtering, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit.

[0063] Then, feature extraction is performed on the set of time domain data obtained by quantum inverse Fourier transform, and the data set DataSet at each time point t is obtained by using the quantum machine learning model. pre (t) Extract the feature vector V features The quantum machine learning model is a model for parsing and understanding the data structure generated by quantum computing. The feature vector comprehensively reflects the user's physiological and motion status, including but not limited to frequency domain characteristics, time series analysis results, and the relationship between physiological data and motion data.

[0064] That is, V features (t) = W features ×QIFT(D filtered (k))+b features , where V features (t) represents the feature vector at time point t, W features represents the feature extraction weight matrix, b features represents the bias vector, QIFT(D filtered (k)) represents the data set after quantum inverse Fourier transform.

[0065] As a possible implementation, determining the first control signal according to the feature vector includes: inputting the feature vector into a decision algorithm; receiving the first control signal output by the decision algorithm; wherein the decision algorithm is: C XR (t) = W decision ×V features (t)+b decision , C XR (t) represents the first control signal at time t, W decision represents the weight matrix, b decision Represents the bias vector.

[0066] In this embodiment, determining the first control signal based on the feature vector includes: inputting the feature vector into a decision algorithm, obtaining the first control signal through calculation by the decision algorithm, and receiving the first control signal output by the decision algorithm for subsequent control.

[0067] Specifically, the feature vector V features (t) is input into the decision algorithm. The decision algorithm can be a quantum decision algorithm. The quantum decision algorithm uses the super position and entanglement characteristics of quantum computing to solve complex decision problems and generate the first control signal C required for adjusting the virtual environment. XR (t), the first control signal indicates how to adjust the virtual environment to provide a personalized and highly realistic user experience, including specific adjustment parameters of visual feedback (visual scene changes), audio feedback (audio response), and tactile feedback.

[0068] That is, C XR (t) = W decision ×V features (t)+b decision Among them, C XR (t) represents the first control signal at time t, W decision represents the weight matrix of the decision layer in the decision algorithm, b decision Represents the bias vector.

[0069] The first control signal C XR (t) Specifically indicates how to adjust the visual feedback, auditory feedback, and tactile feedback in the virtual environment so that the adjustment of each parameter is based on the user's current physiological state, movement state, and the user's personalized needs and preferences. The virtual environment can respond to the user's behavior and physiological reactions in real time, thereby enhancing the immersion and personalization of the extended reality experience.

[0070] As a possible implementation method, the visual feedback, auditory feedback and tactile feedback of the extended reality system are controlled according to the first control signal, including: decomposing the first control signal to obtain visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters; determining an update strategy for the virtual environment according to the feature vector, visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters; and controlling the visual feedback, auditory feedback and tactile feedback of the extended reality system according to the update strategy.

[0071] Specifically, according to the first control signal, the visual feedback, auditory feedback and tactile feedback of the extended reality system are controlled, including: decomposing the first control signal to obtain visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters in the first control signal, determining the update strategy of the virtual environment according to the feature vector, visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters, and then controlling the visual feedback, auditory feedback and tactile feedback of the extended reality system according to the update strategy to enhance the immersion of the virtual environment.

[0072] That is, upon receiving the first control signal C XR (t) After that, the first control signal C XR (t) is decomposed into three main dimensions of adjustment parameters, namely, the visual adjustment parameter V param , auditory adjustment parameter A param and the tactile adjustment parameter T param .

[0073] C XR (t)={V param (t), A param (t), T param (t)}; where V param (t) represents the time point t, the visual adjustment parameter of the virtual environment, A param (t) represents the time point t, the auditory adjustment parameter of the virtual environment, T param (t) represents the tactile adjustment parameters of the virtual environment at time point t.

[0074] Adjust the parameter V by visually param Dynamically adjust the lighting, color, texture, and object position in the virtual environment to enhance the immersion and realism of the visual experience.

[0075] Among them, V param (t) Time point t, the visual adjustment parameters of the virtual environment, represents the component of the first control signal for visual feedback, α represents the adjustment coefficient, V base Represents basic visual parameters to ensure that the adjustment amount is within an acceptable range.

[0076] Adjust parameter A by hearingparam Dynamically adjust the volume, pitch, and spatial positioning of sound effects, music, and voice commands in the virtual environment to enhance the immersive and realistic auditory experience.

[0077] Among them, A param (t) represents the auditory adjustment parameters of the virtual environment at time point t, represents the component of the first control signal for auditory feedback, β represents the adjustment coefficient, n represents the attenuation factor, Δt represents the time interval, N represents the time window size for considering future adjustments, and A base Represents the basic auditory parameters, thus ensuring that the adjustment amount is within the acceptable range, e -γn Represents the attenuation parameter.

[0078] Adjust parameter T by haptics param Dynamically adjust wearable devices, such as haptic gloves or haptic clothing, to adjust their vibration intensity, frequency, and pattern to simulate real-world tactile sensations such as touch, grasping, collision, and ambient temperature changes.

[0079] Among them, T param (t) represents the tactile adjustment parameters of the virtual environment at time point t, represents the component of the first control signal for tactile feedback, σ represents the conversion coefficient, which is used to convert the integral value of the first control signal into the adjustment parameter of the tactile intensity, t0 represents the starting time, T base is the basic tactile parameter to ensure that the adjustment amount is within an acceptable range, and t represents the time point.

[0080] As mentioned above, by comprehensively considering the adjustments of visual feedback, auditory feedback, and tactile feedback, the extended reality system will generate a multi-dimensional immersive virtual environment that can be adjusted according to the user's personalized needs and real-time physiological data and motion data.

[0081] Specifically, Among them, E param (t) represents the adjustment parameters at time point t, which includes time point t, the visual adjustment parameters V of the virtual environment param (t), time point t, the auditory adjustment parameter A of the virtual environment param (t) and time point t, the tactile adjustment parameter T of the virtual environment param (t).

[0082] The E param (t) contains all necessary virtual environment adjustment parameters, thereby achieving real-time update of the virtual environment.

[0083] Based on the feature vector V features (t) and the adjustment parameter Eparam (t),Determine the content adjustment strategy in the virtual environment, including the dynamic generation of content, real-time changes of scenes, and updates of interactive elements.

[0084] As a possible implementation, determining an update strategy for the virtual environment based on the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter includes: inputting the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter into an update strategy algorithm; and receiving an update strategy output by the update strategy algorithm; wherein the update strategy algorithm is: Indicates the update strategy, Func strategy represents the function that calculates the content update strategy based on the input parameters, ω1 and ω2 represent the adjustment weights, V features (t) represents the feature vector, t represents the time point, E param (t) represents the adjustment parameter at time point t, ψ(t') is the time-dependent adjustment function, and t' represents the time variable.

[0085] Specifically, the update strategy of the virtual environment is determined based on the feature vector, visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters, including: inputting the feature vector, visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters into the update strategy algorithm, and the update strategy algorithm outputs the update strategy after calculation, thereby obtaining the update strategy.

[0086] The update strategy algorithm is:

[0087] Among them, Strategy content (t) represents the update strategy, Func strategy represents the function that calculates the content update strategy based on the input parameters. ω1 and ω2 are adjustment weights used to balance the influence of the feature vector and the environment adjustment parameters on the update strategy. ψ(t') is a time-dependent adjustment function that considers the cumulative effect from the past to the present moment.

[0088] Combined with E param The calculation method of (t) can determine the update strategy.

[0089] As a possible implementation, the update strategy includes a visual update strategy, an auditory update strategy, and a tactile update strategy; controlling the visual feedback, auditory feedback, and tactile feedback of the extended reality system according to the update strategy includes: inputting the visual update strategy and the auditory update strategy into a rendering algorithm; receiving a rendering result output by the rendering algorithm; and controlling the extended reality system to perform visual rendering and auditory rendering according to the rendering result; the rendering algorithm is: XR rendered(t) represents the rendering result, n represents the number of types of rendering parameters, φ i is the rendering function of the i-th parameter, represents the update strategy of the i-th parameter, α i and β i Represent the adjustment coefficients of visual and auditory parameters respectively, represents the visual adjustment parameter of the i-th parameter, represents the auditory adjustment parameter of the i-th parameter, Γ(t) represents the dynamic rendering adjustment item based on user response and demand; the tactile update strategy is input into the tactile simulation algorithm; the tactile simulation result output by the tactile simulation algorithm is received; according to the tactile simulation result, the extended reality system is controlled to perform tactile simulation; the tactile simulation algorithm is:

[0090]

[0091] represents the tactile simulation result, m represents the number of tactile feedback types, ξ j represents the adjustment function of the j-th tactile feedback, represents the tactile adjustment parameter of the jth parameter, represents the update strategy of the jth parameter, δ is a factor that considers the impact of the content strategy change rate on the tactile output and increases the sense of dynamic change. represents the derivative of the update policy.

[0092] Specifically, according to the update strategy, the visual feedback, auditory feedback and tactile feedback of the extended reality system are controlled, including: according to the visual update strategy, the extended reality system is controlled to perform visual rendering to generate the visual effect of the virtual environment; according to the auditory update strategy, the extended reality system is controlled to perform auditory rendering to generate the auditory effect of the virtual environment; according to the tactile update strategy, the extended reality system is controlled to perform tactile simulation to generate the tactile effect of the virtual environment.

[0093] Using rendering technology, the update strategy Strategy content (t) Transformed into specific visual and auditory effects, including scene rendering, lighting and shadow effects, sound processing, and character animation to adapt to user needs and reactions.

[0094] The rendering structure may be determined by a rendering algorithm, and the tactile simulation structure may be determined by a tactile simulation algorithm.

[0095] The rendering algorithm is:

[0096] Among them, XR rendered (t) represents the rendering result, n represents the number of types of rendering parameters, φ i is the rendering function of the i-th parameter, represents the update strategy of the i-th parameter, α i and β i Represent the adjustment coefficients of visual and auditory parameters respectively, represents the visual adjustment parameter of the i-th parameter, represents the auditory adjustment parameter of the i-th parameter, and Γ(t) represents the dynamic rendering adjustment item based on user response and demand.

[0097] Based on the time point t, the output of the tactile feedback device is updated in real time, and the parameter T is adjusted according to the tactile feedback. param (t) and adjustment strategies to regulate the output intensity and pattern of the tactile feedback device to simulate physical contact and environmental effects in the virtual environment, including wind, temperature changes, etc.:

[0098] The tactile simulation algorithm is:

[0099] Among them, Tactile output (t) represents the tactile simulation result, m represents the number of tactile feedback types, ξ j represents the adjustment function of the j-th tactile feedback, represents the tactile adjustment parameter of the jth parameter, represents the update strategy of the jth parameter, δ is a factor that considers the impact of the content strategy change rate on the tactile output and increases the sense of dynamic change. represents the derivative of the update policy.

[0100] Render the result to XR rendered (t) and tactile simulation results Tactile output (t)Synchronization is applied to the user's VR experience for consistency and coherence of visual feedback, auditory feedback, and tactile feedback.

[0101]

[0102] Sync output (t) represents the final output of the user experience by integrating visual feedback, auditory feedback, and tactile feedback. rendered (t') represents the rendering result, Tactile output (t') represents the tactile simulation result, η represents the synchronization coefficient, Strategy content (t) represents the update strategy.

[0103] As a possible implementation method, obtaining the user's physiological data and motion data includes: determining the interaction intensity between the extended reality system and the user; determining the target frequency for collecting the physiological data and motion data based on the interaction intensity; and collecting the user's physiological data and motion data based on the target frequency.

[0104] Specifically, obtaining the user's physiological data and motion data includes: determining the interaction intensity between the extended reality system and the user, and determining the target frequency for collecting physiological data and motion data based on the interaction intensity, and then collecting the user's physiological data and motion data based on the target frequency, thereby reducing latency, reducing motion sickness, etc.

[0105] As a possible implementation, determining a target frequency for collecting physiological data and motion data based on interaction intensity includes: inputting the interaction intensity into a frequency algorithm; receiving a target frequency for collecting physiological data and motion data; wherein the frequency algorithm is: F adaptive (t) represents the target frequency, F base represents the basic frequency of data collection, λ represents the adjustment coefficient, P interaction (t) represents the interaction strength, P max Indicates the maximum value of interaction strength.

[0106] Specifically, the target frequency for collecting physiological data and motion data is determined based on the interaction intensity, including: inputting the interaction intensity into the frequency algorithm, the frequency algorithm calculates and outputs the target frequency, and receiving the target frequency for collecting physiological data and motion data for subsequent data collection.

[0107] A real-time feedback mechanism is used to adjust the data collection frequency in response to changes in system load and user interaction intensity, thereby optimizing system performance and reducing latency. The mechanism dynamically adjusts data collection and processing parameters based on the system's current processing capacity and real-time data of user interaction.

[0108]

[0109] Among them, F adaptive (t) represents the target frequency of adaptive data collection, F base represents the basic frequency of data collection, λ represents the adjustment coefficient, P interaction (t) represents the interaction intensity between the augmented reality system and the user at the current moment, P max Indicates the maximum value of interaction strength.

[0110] Furthermore, combined with quantum algorithms, this application reduces the overall response time from data acquisition to content adjustment through data processing and fusion optimization algorithms, and enables sensor data to be processed and fused by optimizing the data flow path and processing steps. Where, ΔToptimized represents the processing time, which is the total time after optimization. is the time delay of the kth data processing step, K is the total number of data processing steps, and min represents the minimum value.

[0111] Apply acceleration algorithms based on quantum computing to improve data processing speed. During the data fusion and feature extraction stages, quantum algorithms use the superposition and entanglement characteristics of quantum bits to achieve parallel execution of data processing tasks.

[0112] T quantum =T classical ÷2 q Among them, T quantum represents the processing time after quantum computing, T classical It represents the time required by traditional computing methods, and q represents the number of quantum bits, which is an indicator of parallel processing capabilities.

[0113] Through the above calculations, the extended reality system can process and fuse sensor data from the user's wearable device with a shorter response time, significantly reducing latency and motion sickness in the virtual reality experience, thereby improving user comfort and satisfaction.

[0114] Figure 2 FIG2 shows a second flow chart of a control method of an extended reality system according to an embodiment of the present application. Figure 2 As shown, the method includes:

[0115] Step 202: Initialize the extended reality system;

[0116] Step 204: Collect data, including physiological data and motion data;

[0117] Step 206: Perform data fusion using quantum computing technology and fusion algorithms;

[0118] Step 208: Adjusting the feedback of the extended reality system in real time, the feedback including visual feedback, auditory feedback, and tactile feedback;

[0119] Step 210: Adjusting personalized content through virtual environment generation technology;

[0120] Step 212: Optimize data collection and processing to improve comfort and reduce latency.

[0121] The embodiments of the present application utilize multi-sensor fusion technology to respond to the user's status, including subtle physiological and movement changes, with high precision, thereby providing visual, auditory, and tactile feedback. By optimizing the processing and fusion process of sensor data, and in particular introducing an acceleration algorithm based on quantum computing, the response time from data acquisition to content adjustment can be significantly reduced. The rapid response effectively reduces the delay problems and motion sickness that users may encounter in interactive scenarios, thereby improving the user experience comfort.

[0122] By collecting and analyzing the user's physiological data and motion data, the embodiments of the present application can adjust the virtual environment in real time to adapt to the user's specific needs and reactions, thereby achieving a high degree of personalization of the virtual environment, allowing each user to find unique content that suits their preferences in the virtual environment.

[0123] The control method of the extended reality system provided in the embodiment of the present application can be executed by the control device of the extended reality system. In the embodiment of the present application, the control method of the extended reality system provided in the embodiment of the present application is described by taking the control device of the extended reality system executing the control method of the extended reality system as an example.

[0124] like Figure 3 As shown, the present application provides a control device 300 for an extended reality system, including: an acquisition module 302 for acquiring physiological data and motion data of a user; an establishment module 304 for establishing a parameter set of physiological data and motion data; a determination module 306 for determining a first control signal based on the parameter set, the first control signal including visual adjustment parameters, auditory adjustment parameters and tactile adjustment parameters; a control module 308 for controlling visual feedback, auditory feedback and tactile feedback of the extended reality system based on the first control signal.

[0125] In an embodiment of the present application, when a user uses an extended reality system, the user's physiological data and motion data are obtained, and a parameter set of the physiological data and motion data is established. Based on the physiological data and motion data in the parameter set, a first control signal is determined. The first control signal includes visual adjustment parameters, auditory adjustment parameters, and tactile adjustment parameters. Thus, according to the first control signal, the visual feedback, auditory feedback, and tactile feedback of the extended reality system are controlled. That is, the virtual environment generated by the extended reality system will adjust the visual feedback, auditory feedback, and tactile feedback based on the user's physiological data and motion data, thereby providing the user with a personalized virtual environment and improving the quality and immersion of the virtual environment experience.

[0126] In addition, the unified management of physiological data and motion data based on parameter sets can improve the data processing speed and reduce the latency of the extended reality system.

[0127] As a possible implementation, the determination module includes: a first transformation submodule, used to perform quantum Fourier transform on the data in the parameter set to obtain frequency domain data; a filtering submodule, used to filter the frequency domain data; a second transformation submodule, used to perform quantum Fourier transform on the filtered frequency domain data to obtain time domain data; a first determination submodule, used to match weights for the time domain data, and determine the eigenvector in combination with the bias vector; and a second determination submodule, used to determine the first control signal based on the eigenvector.

[0128] As a possible implementation, the first transformation submodule includes: a first input unit, configured to input data in a parameter set into a first transformation formula; a first receiving unit, configured to receive frequency domain data output by the first transformation formula; the first transformation formula is:

[0129]

[0130] QFT(D type (k)) represents frequency domain data, D type (k) represents the data value of a specific sensor type at time point k, D type (y) represents the data value of the final state of the quantum state, q represents the number of quantum bits, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, Represents the transformation process of data from time domain to frequency domain, i represents the imaginary unit;

[0131] The second transformation submodule includes: a second input unit for inputting the filtered frequency domain data into a second transformation formula; a second receiving unit for receiving the time domain data output by the second transformation formula; the second transformation formula is:

[0132]

[0133] QIFT(D filtered (k)) represents the time domain data, q represents the number of quantum bits, D filtered (y) represents the frequency domain data after filtering, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit.

[0134] As a possible implementation, the control module includes: a decomposition submodule for decomposing the first control signal to obtain visual adjustment parameters, auditory adjustment parameters, and tactile adjustment parameters; a third determination submodule for determining an update strategy for the virtual environment based on the feature vector, the visual adjustment parameters, the auditory adjustment parameters, and the tactile adjustment parameters; and a control submodule for controlling the visual feedback, auditory feedback, and tactile feedback of the extended reality system according to the update strategy.

[0135] As a possible implementation, the third determination submodule includes: a third input unit, configured to input the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter into an update strategy algorithm; and a third receiving unit, configured to receive an update strategy output by the update strategy algorithm; wherein the update strategy algorithm is: Strategy content (t) represents the update strategy, Func strategy represents the function that calculates the content update strategy based on the input parameters, ω1 and ω2 represent the adjustment weights, V features (t) represents the feature vector, t represents the time point, E param (t) represents the adjustment parameter at time point t, ψ(t') is the time-dependent adjustment function, and t' represents the time variable.

[0136] As a possible implementation, the update strategy includes a visual update strategy, an auditory update strategy, and a tactile update strategy; according to the update strategy, the control submodule includes: a fourth input unit for inputting the visual update strategy and the auditory update strategy into the rendering algorithm; a fourth receiving unit for receiving the rendering result output by the rendering algorithm; a first control unit for controlling the extended reality system to perform visual rendering and auditory rendering based on the rendering result; the rendering algorithm is: Indicates the rendering result, n indicates the number of types of rendering parameters, φ i is the rendering function of the i-th parameter, represents the update strategy of the i-th parameter, α i and β i Represent the adjustment coefficients of visual and auditory parameters respectively, represents the visual adjustment parameter of the i-th parameter, represents the auditory adjustment parameter of the i-th parameter, Γ(t) represents the dynamic rendering adjustment item based on user response and demand; a fifth input unit is used to input the tactile update strategy into the tactile simulation algorithm; a fifth receiving unit is used to receive the tactile simulation result output by the tactile simulation algorithm; a second control unit is used to control the extended reality system to perform tactile simulation based on the tactile simulation result; the tactile simulation algorithm is:

[0137]

[0138] represents the tactile simulation result, m represents the number of tactile feedback types, ξ j represents the adjustment function of the j-th tactile feedback, represents the tactile adjustment parameter of the jth parameter, represents the update strategy of the jth parameter, δ is a factor that considers the impact of the content strategy change rate on the tactile output and increases the sense of dynamic change. represents the derivative of the update policy.

[0139] As a possible implementation, the acquisition module includes: a fourth determination submodule, used to determine the interaction intensity between the extended reality system and the user; a fifth determination submodule, used to determine the target frequency for collecting physiological data and motion data based on the interaction intensity; and a collection submodule, used to collect the user's physiological data and motion data based on the target frequency.

[0140] As a possible implementation, the collection submodule includes: a sixth input unit for inputting the interaction intensity into a frequency algorithm; a sixth receiving unit for receiving a target frequency for collecting physiological data and motion data; wherein the frequency algorithm is:

[0141] F adaptive (t) represents the target frequency, F base represents the basic frequency of data collection, λ represents the adjustment coefficient, P interaction (t) represents the interaction strength, P max Indicates the maximum value of interaction strength.

[0142] The control device of the extended reality system in the embodiment of the present application can be an electronic device or a component in the electronic device, such as an integrated circuit or chip. The electronic device can be a terminal or other device other than an electronic device. For example, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a private network communication terminal device (such as a walkie-talkie), a mobile Internet device (Mobile Internet Device, MID), an augmented reality / virtual reality / mixed reality device, a robot, a wearable device, an ultra-mobile personal computer (Ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), an ATM or a self-service machine, etc., and the embodiment of the present application does not specifically limit it.

[0143] The control device of the extended reality system in the embodiment of the present application can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0144] The control device of the extended reality system provided in the embodiment of the present application can implement each process implemented in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be described here.

[0145] The embodiment of the present application also provides an electronic device, Figure 4 The structural block diagram of the electronic device according to the embodiment of the present application is shown as follows: Figure 4 As shown, the electronic device 400 includes a processor 402 and a memory 404, and the memory 404 stores programs or instructions that can be run on the processor 402. When the program or instruction is executed by the processor 402, the various steps of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0146] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0147] Figure 5 A schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present application.

[0148] The electronic device 500 includes but is not limited to components such as a radio frequency unit 501 , a network module 502 , an audio output unit 503 , an input unit 504 , a sensor 505 , a display unit 506 , a user input unit 507 , an interface unit 508 , a memory 509 , and a processor 510 .

[0149] Those skilled in the art will understand that the electronic device 500 may also include a power source (such as a battery) to power each component, and the power source may be logically connected to the processor 510 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system. Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be repeated here.

[0150] The processor 510 is used to obtain the user's physiological data and motion data;

[0151] The processor 510 is used to establish parameter sets of physiological data and motion data;

[0152] The processor 510 is configured to determine a first control signal according to the parameter set, where the first control signal includes a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter;

[0153] The processor 510 is configured to control the visual feedback, auditory feedback, and tactile feedback of the augmented reality system according to the first control signal.

[0154] In some embodiments, optionally, the processor 510 is configured to determine the first control signal according to the parameter set, including:

[0155] The processor 510 is configured to perform quantum Fourier transform on the data in the parameter set to obtain frequency domain data;

[0156] The processor 510 is used to perform filtering processing on the frequency domain data;

[0157] The processor 510 is configured to perform a quantum Fourier transform on the filtered frequency domain data to obtain time domain data;

[0158] The processor 510 is configured to match weights for the time domain data and determine a feature vector in combination with the bias vector;

[0159] The processor 510 is configured to determine a first control signal according to the feature vector.

[0160] In some embodiments, optionally, the processor 510 is configured to perform a quantum Fourier transform on the data in the parameter set to obtain frequency domain data, including:

[0161] The processor 510 is configured to input the data in the parameter set into the first transformation formula;

[0162] The processor 510 is configured to receive frequency domain data output by the first transformation formula;

[0163] The first transformation formula is:

[0164]

[0165] QFT(D type (k)) represents frequency domain data, D type (k) represents the data value of a specific sensor type at time point k, D type (y) represents the data value of the final state of the quantum state, q represents the number of quantum bits, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, Represents the transformation process of data from time domain to frequency domain, i represents the imaginary unit;

[0166] The processor 510 is configured to perform an inverse quantum Fourier transform on the filtered frequency domain data to obtain time domain data, including:

[0167] The processor 510 is configured to input the filtered frequency domain data into a second transformation formula;

[0168] The processor 510 is configured to receive the time domain data output by the second transformation formula;

[0169] The second transformation formula is:

[0170]

[0171] QIFT(D filtered (k)) represents the time domain data, q represents the number of quantum bits, D filtered (y) represents the frequency domain data after filtering, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit.

[0172] In some embodiments, optionally, the processor 510 is configured to control visual feedback, auditory feedback, and tactile feedback of the augmented reality system according to the first control signal, including:

[0173] The processor 510 is configured to decompose the first control signal to obtain a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter;

[0174] The processor 510 is configured to determine an update strategy for the virtual environment based on the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter;

[0175] The processor 510 is configured to control the visual feedback, auditory feedback, and tactile feedback of the augmented reality system according to the update strategy.

[0176] In some embodiments, optionally, the processor 510 is configured to determine an update strategy for the virtual environment based on the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter, including:

[0177] The processor 510 is configured to input the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter into an update strategy algorithm;

[0178] The processor 510 is configured to receive an update strategy output by an update strategy algorithm;

[0179] Among them, the update strategy algorithm is:

[0180]

[0181] Strategy content (t) represents the update strategy, Func strategy represents a function that calculates the content update strategy based on the input parameters, ω1 and ω2 represent the adjustment weights, t represents the time point, and Eparam (t) represents the adjustment parameter at time point t, ψ(t') is the time-dependent adjustment function, and t' represents the time variable.

[0182] In some embodiments, the update strategy optionally includes a visual update strategy, an auditory update strategy, and a tactile update strategy; the processor 510 is configured to control the visual feedback, auditory feedback, and tactile feedback of the extended reality system according to the update strategy, including:

[0183] The processor 510 is used to input the visual update strategy and the auditory update strategy into the rendering algorithm;

[0184] The processor 510 is used to receive the rendering result output by the rendering algorithm;

[0185] The processor 510 is configured to control the extended reality system to perform visual rendering and auditory rendering according to the rendering result;

[0186] The rendering algorithm is:

[0187]

[0188] XR rendered (t) represents the rendering result, n represents the number of types of rendering parameters, φ i is the rendering function of the i-th parameter, represents the update strategy of the i-th parameter, α i and β i Represent the adjustment coefficients of visual and auditory parameters respectively, represents the visual adjustment parameter of the i-th parameter, represents the auditory adjustment parameter of the i-th parameter, Γ(t) represents the dynamic rendering adjustment item based on user response and demand;

[0189] The processor 510 is configured to input the haptic update strategy into the haptic simulation algorithm;

[0190] The processor 510 is configured to receive a tactile simulation result output by a tactile simulation algorithm;

[0191] The processor 510 is configured to control the extended reality system to perform tactile simulation according to the tactile simulation result;

[0192] The tactile simulation algorithm is:

[0193]

[0194] Tactile output (t) represents the tactile simulation result, m represents the number of tactile feedback types, ξ j represents the adjustment function of the j-th tactile feedback, represents the tactile adjustment parameter of the jth parameter, represents the update strategy of the jth parameter, δ is a factor that considers the impact of the content strategy change rate on the tactile output and increases the sense of dynamic change. represents the derivative of the update policy.

[0195] The processor 510 is configured to control the extended reality system to perform visual rendering and auditory rendering according to the visual update strategy and the auditory update strategy;

[0196] The processor 510 is configured to control the extended reality system to perform tactile simulation according to the tactile update strategy.

[0197] In some embodiments, the processor 510 is optionally configured to obtain physiological data and motion data of the user, including:

[0198] The processor 510 is configured to determine the interaction intensity between the augmented reality system and the user;

[0199] The processor 510 is configured to determine a target frequency for collecting physiological data and motion data based on the interaction intensity;

[0200] The processor 510 is configured to collect physiological data and motion data of the user according to the target frequency.

[0201] In some embodiments, optionally, the processor 510 is configured to determine a target frequency for collecting physiological data and motion data based on the interaction intensity, including:

[0202] The processor 510 is used to input the interaction intensity into the frequency algorithm;

[0203] The processor 510 is used to receive a target frequency for collecting physiological data and motion data;

[0204] Among them, the frequency algorithm is:

[0205] F adaptive (t) represents the target frequency, F base represents the basic frequency of data collection, λ represents the adjustment coefficient, P interaction (t) represents the interaction strength, P max Indicates the maximum value of interaction strength.

[0206] In an embodiment of the present application, when a user uses an extended reality system, the user's physiological data and motion data are obtained, and a parameter set of the physiological data and motion data is established. Based on the physiological data and motion data in the parameter set, a first control signal is determined. The first control signal includes visual adjustment parameters, auditory adjustment parameters, and tactile adjustment parameters. Thus, according to the first control signal, the visual feedback, auditory feedback, and tactile feedback of the extended reality system are controlled. That is, the virtual environment generated by the extended reality system will adjust the visual feedback, auditory feedback, and tactile feedback based on the user's physiological data and motion data, thereby providing the user with a personalized virtual environment and improving the quality and immersion of the virtual environment experience.

[0207] In addition, the unified management of physiological data and motion data based on parameter sets can improve the data processing speed and reduce the latency of the extended reality system.

[0208] It should be understood that in an embodiment of the present application, the input unit 504 may include a graphics processor 5041 and a microphone 5042, and the graphics processor 5041 processes the image file of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 507 includes a touch panel 5071 and at least one of the other input devices 5072. The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts: a touch detection device and a touch controller. Other input devices 5072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0209] The memory 509 can be used to store software programs and various files. The memory 509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing files, wherein the first storage area may store an operating system, applications or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 509 may include a volatile memory or a non-volatile memory, or the memory 509 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 509 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0210] Processor 510 may include one or more processing units. Optionally, processor 510 integrates an application processor and a modem processor. The application processor primarily handles operations related to the operating system, user interface, and application programs, while the modem processor primarily processes wireless communication signals, such as a baseband processor. It is understood that the modem processor may not be integrated into processor 510.

[0211] An embodiment of the present application also provides a storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the control method embodiment of the above-mentioned extended reality system are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0212] The storage medium may be a readable storage medium.

[0213] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0214] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned control method embodiment of the extended reality system, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0215] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0216] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the control method embodiment of the above-mentioned extended reality system, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0217] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0218] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling an electronic device (which can be a mobile phone, computer, server or network device, etc.) to execute the methods of each embodiment of the present application.

[0219] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A control method for an extended reality system, characterized in that: include: Obtain the user's physiological data and motion data; Establishing parameter sets of the physiological data and the motion data; Determining a first control signal according to the parameter set, the first control signal comprising a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter; According to the first control signal, visual feedback, auditory feedback and tactile feedback of the extended reality system are controlled.

2. The control method of the extended reality system according to claim 1, characterized in that: Determining a first control signal according to the parameter set includes: Performing quantum Fourier transformation on the data in the parameter set to obtain frequency domain data; Performing filtering processing on the frequency domain data; Performing an inverse quantum Fourier transform on the filtered frequency domain data to obtain time domain data; Matching weights for the time domain data and determining a feature vector in combination with a bias vector; The first control signal is determined according to the feature vector.

3. The control method of the extended reality system according to claim 2, characterized in that: The data in the parameter set is subjected to quantum Fourier transformation to obtain frequency domain data, including: inputting the data in the parameter set into a first transformation formula; Receiving frequency domain data output by the first transformation formula; The first transformation formula is: QFT(D type (k)) represents the frequency domain data, D type (k) represents the data value of a specific sensor type at time point k, D type (y) represents the data value of the final state of the quantum state, q represents the number of quantum bits, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit; The filtered frequency domain data is subjected to inverse quantum Fourier transform to obtain time domain data, including: Inputting the filtered frequency domain data into a second transformation formula; Receiving the time domain data output by the second transformation formula; The second transformation formula is: QIFT(D filtered (k)) represents the time domain data, q represents the number of quantum bits, D filtered (y) represents the frequency domain data after filtering, x represents the index of the initial state of the quantum state, y represents the index of the final state of the quantum state, It represents the transformation process of data from time domain to frequency domain, and i represents the imaginary unit.

4. The control method of the extended reality system according to claim 2, characterized in that: According to the first control signal, controlling visual feedback, auditory feedback and tactile feedback of the extended reality system includes: Decomposing the first control signal to obtain the visual adjustment parameter, the auditory adjustment parameter and the tactile adjustment parameter; Determining an update strategy for the virtual environment according to the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter; According to the update strategy, the visual feedback, the auditory feedback and the tactile feedback of the extended reality system are controlled.

5. The control method of the extended reality system according to claim 4, characterized in that: Determining an update strategy for a virtual environment according to the feature vector, the visual adjustment parameter, the auditory adjustment parameter, and the tactile adjustment parameter includes: Inputting the feature vector, the visual adjustment parameter, the auditory adjustment parameter and the tactile adjustment parameter into an update strategy algorithm; Receiving the update strategy output by the update strategy algorithm; Wherein, the update strategy algorithm is: Strategy content (t) represents the update strategy, Func strategy represents the function that calculates the content update strategy based on the input parameters, ω1 and ω2 represent the adjustment weights, V features (t) represents the feature vector, t represents the time point, E param (t) represents the adjustment parameter at time point t, ψ(t') is the time-dependent adjustment function, and t' represents the time variable.

6. The control method of the extended reality system according to claim 5, characterized in that: The updating strategy includes a visual updating strategy, an auditory updating strategy and a tactile updating strategy; According to the update strategy, controlling the visual feedback, the auditory feedback, and the tactile feedback of the extended reality system includes: inputting the visual update strategy and the auditory update strategy into a rendering algorithm; Receiving a rendering result output by the rendering algorithm; According to the rendering result, controlling the extended reality system to perform visual rendering and auditory rendering; The rendering algorithm is: XR rendered (t) represents the rendering result, n represents the number of types of rendering parameters, φ i is the rendering function of the i-th parameter, represents the update strategy of the i-th parameter, α i and β i Represent the adjustment coefficients of visual and auditory parameters, represents the visual adjustment parameter of the i-th parameter, represents the auditory adjustment parameter of the i-th parameter, Γ(t) represents a dynamic rendering adjustment item based on user response and demand; inputting the haptic update strategy into a haptic simulation algorithm; Receiving a tactile simulation result output by the tactile simulation algorithm; According to the tactile simulation result, controlling the extended reality system to perform tactile simulation; The tactile simulation algorithm is: Tactile output (t) represents the tactile simulation result, m represents the number of tactile feedback types, ξ j represents the adjustment function of the jth tactile feedback, represents the tactile adjustment parameter of the j-th parameter, represents the update strategy of the jth parameter, δ is a factor that takes into account the impact of the content strategy change rate on the tactile output and increases the sense of dynamic change. represents the derivative of the update strategy.

7. The control method of the extended reality system according to any one of claims 1 to 6, characterized in that: Obtain the user's physiological and motion data, including: Determining the interaction strength between the extended reality system and the user; Determining a target frequency for collecting the physiological data and the motion data according to the interaction intensity; The physiological data and the motion data of the user are collected according to the target frequency.

8. The control method of the extended reality system according to claim 7, characterized in that: Determining a target frequency for collecting the physiological data and the motion data according to the interaction intensity includes: inputting said interaction strength into a frequency algorithm; Receiving the target frequency for collecting the physiological data and the motion data; Wherein, the frequency algorithm is: F adaptive (t) represents the target frequency, F base represents the basic frequency of data collection, λ represents the adjustment coefficient, P interaction (t) represents the interaction strength, P max Indicates the maximum value of the interaction strength.

9. A control device for an extended reality system, characterized in that: include: An acquisition module, used to acquire the user's physiological data and motion data; An establishing module, used for establishing a parameter set of the physiological data and the motion data; A determination module, configured to determine a first control signal according to the parameter set, wherein the first control signal includes a visual adjustment parameter, an auditory adjustment parameter, and a tactile adjustment parameter; A control module is used to control the visual feedback, auditory feedback and tactile feedback of the extended reality system according to the first control signal.

10. An electronic device, characterized in that: It includes a processor and a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the control method of the extended reality system as described in any one of claims 1 to 8 are implemented.

Images