A method and system for enhancing virtual reality interaction

Abstract

The present invention relates to the field of data processing technology, and in particular to an enhanced virtual reality interaction method and system. The method comprises the steps of: obtaining a first noise index or a second noise index of a data point by the data deviation degree between a data point and a reference data segment and the data deviation degree between a data segment and a reference data segment in a user motion data time series sequence; obtaining a third noise index of a data point by the Pearson correlation coefficient between acceleration data and angular velocity data in the data point; calculating the first noise degree of the data point; taking the mean of the first noise degree of all data points in the data segment as the second noise degree of the data segment, and taking the mean of the second noise degree of a group consisting of a data segment and left and right adjacent data segments as the third noise degree of the group; filtering and denoising by the third noise degree to enhance the interaction of virtual reality, and effectively improving the accuracy of the interaction between the user and the virtual reality.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an enhanced virtual reality interaction method and system. Background Art

[0002] Virtual Reality (VR) is a technology that uses computer technology to generate a three-dimensional environment. Users can enter the virtual world and interact with the virtual environment by wearing specific equipment. Sensors play a key role in VR interaction. By capturing the user's motion data or physiological data, they help the VR system update the user's virtual perspective in real time to achieve a more natural interactive experience.

[0003] There is a lot of research content on virtual reality technology in the prior art. For example, the patent application document with publication number CN106383578A discloses a virtual reality system, a virtual reality interaction device and a method. The application determines the image acquisition area through position information, uses the image acquisition module to start the image acquisition component, receives the image information of the image acquisition area acquired by the image acquisition component, and overlays the image information on the virtual reality screen. The virtual reality interaction device of the application can enable the user to interact with the preset objects in the real scene without interrupting the virtual reality experience.

[0004] Most of the research in the existing technology tends to be about how to realize the interaction between users and virtual reality. However, when collecting the user's motion data through the sensors in the VR, noise data may be generated due to interference in the environment, resulting in the VR system being unable to accurately realize the scene conversion when the virtual perspective scene is switched, affecting the user's interactive experience.

[0005] Based on this, how to effectively enhance the accuracy of user interaction with virtual reality is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the invention

[0006] In order to solve the technical problem of how to effectively enhance the accuracy of user interaction with virtual reality, the present invention provides a method and system for enhancing virtual reality interaction.

[0007] In a first aspect, the present invention provides an enhanced virtual reality interaction method, which adopts the following technical solution:

[0008] An enhanced virtual reality interaction method comprises the steps of:

[0009] Obtain a preset reference data segment for each data point in the data segment collected every second in the user motion data time series; obtain a first noise index or a second noise index for the data point by the data deviation degree between the data point and the reference data segment and the data deviation degree between the data segment and the reference data segment; obtain the Pearson correlation coefficient between the acceleration data and the angular velocity data in the data point, and use the negative of the sum of the Pearson correlation coefficient and 1 as the exponent of the exponential function with e as the base to obtain a third noise index for the data point:

[0010] ;

[0011] In the formula, represents the first noise level of the ith data point, , , Respectively represent the first noise index, the second noise index, and the third noise index of the i-th data point, , Respectively represent the weights of the first noise index and the second noise index of the i-th data point, represents a linear normalization function; taking the mean value of the first noise levels of all data points in the data segment as the second noise level of the data segment, and taking the mean value of the second noise levels of a group consisting of the data segment and the left and right adjacent data segments as the third noise level of the group; filtering and denoising are performed through the third noise level to enhance the interaction of virtual reality.

[0012] The present invention obtains the user's motion pattern by analyzing the changes in the user's acceleration and angular velocity, thereby realizing the interaction between the user and AR. In this process, the present invention takes into account that noise data will affect the recognition accuracy of the motion pattern. Therefore, the noise degree of the data point is obtained by respectively analyzing the changes in the acceleration and angular velocity and the correlation between the acceleration and angular velocity, thereby realizing accurate denoising. On this basis, the present invention also takes into account that the noise is highly random and relatively frequent, and directly filtering and denoising a single data point may affect the continuity and integrity of the data. Therefore, by further obtaining the noise degree of the group, accurate denoising is realized while the original characteristics of the data can be relatively completely retained, effectively improving the accuracy of the interaction between the user and virtual reality.

[0013] According to an augmented virtual reality interaction method provided by the present invention, the method of obtaining a preset reference data segment for each data point in a data segment collected every second in a time series of user motion data includes: taking a point composed of acceleration data and angular velocity data collected at the same time as a data point, arranging the data points collected every second in time series to obtain a data segment; determining the length T of the reference data segment by the Fisher optimal segmentation method, taking the data point as the center, and obtaining an equal amount of reference data segments on both sides of the data point. data points constitute a reference data segment for the data points.

[0014] The present invention determines the length of the reference data segment by the Fisher optimal segmentation method, and can find the optimal data segment segmentation method based on the principle of minimizing the loss function, so that the data points in each data segment finally obtained have high similarity and the data points between different data segments have high differences. And by obtaining the data points on both sides of the data point, each data point has sufficient context information, thereby improving the accuracy of data analysis.

[0015] According to an augmented virtual reality interaction method provided by the present invention, the first noise index or the second noise index of the data point is obtained, including: recording the absolute value of the difference between the acceleration in the data point and the average acceleration of all data points in a reference data segment as a first difference; recording the absolute value of the difference between the average acceleration of all data points in the data segment where the data point is located and the average acceleration of all data points in the reference data segment as a second difference; using the normalized data of the sum of the first difference and the second difference as the first noise index of the data point; recording the absolute value of the difference between the angular velocity in the data point and the average angular velocity of all data points in the reference data segment as a third difference; recording the absolute value of the difference between the average angular velocity of all data points in the data segment where the data point is located and the average angular velocity of all data points in the reference data segment as a fourth difference; and obtaining the second noise index of the data point after normalizing the sum of the third difference and the fourth difference.

[0016] According to an augmented virtual reality interaction method provided by the present invention, the obtaining of the Pearson correlation coefficient between the acceleration data and the angular velocity data in the data point includes: forming a correlation group of the data point with the data point and the left and right adjacent data points, and using the Pearson correlation coefficient between the acceleration and the angular velocity of all the data points in the correlation group as the Pearson correlation coefficient between the acceleration and the angular velocity in the data point.

[0017] According to an augmented virtual reality interaction method provided by the present invention, the filtering and denoising by the third noise level includes: obtaining the normalized third noise level of each group, taking the group whose normalized third noise level is greater than a preset noise threshold as the target group, and filtering and denoising the target group.

[0018] The present invention takes into account that denoising directly based on the first noise level will affect the continuity and integrity of the motion data. Therefore, by further obtaining the noise level of the group, accurate denoising can be achieved while relatively completely retaining the original characteristics of the data, thereby effectively improving the accuracy of user interaction with virtual reality.

[0019] According to an enhanced virtual reality interaction method provided by the present invention, the filtering and denoising by the third noise level is performed to enhance the virtual reality interaction, including: inputting the acceleration and angular velocity corresponding to the data points in each denoised data segment into a pre-trained motion pattern recognition model to obtain the user's motion pattern; the virtual reality system adjusts the virtual environment through the matching result of the preset standard motion pattern and the user's motion pattern.

[0020] The present invention can accurately and quickly obtain the user's motion pattern after collecting new acceleration and angular velocity data by pre-training a neural network model.

[0021] According to an augmented virtual reality interaction method provided by the present invention, the step of obtaining the user's motion pattern further includes: monitoring the user's motion pattern for abnormalities.

[0022] The present invention takes into account that some of the user's motion patterns may be risky, and therefore abnormal motion patterns of the user can be acquired and processed in a timely manner through abnormal monitoring.

[0023] In a second aspect, the present invention provides an augmented virtual reality interactive system, which adopts the following technical solution:

[0024] An augmented virtual reality interaction system includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the augmented virtual reality interaction method described above is implemented.

[0025] By adopting the above technical solution, the above-mentioned augmented virtual reality interaction method is generated into a computer program and stored in a memory so as to be loaded and executed by a processor, thereby making a terminal device based on the memory and the processor for easy use.

[0026] The present invention has the following technical effects:

[0027] Based on the above technical scheme, when realizing virtual reality interaction, the present invention obtains the user's motion pattern by analyzing the changes in the user's acceleration and angular velocity, thereby realizing the interaction between the user and AR. In this process, the present invention takes into account that noise data will affect the recognition accuracy of the motion pattern, and therefore obtains the noise degree of the data point by respectively analyzing the changes in acceleration and angular velocity and the correlation between acceleration and angular velocity, thereby realizing accurate denoising; on this basis, the present invention also takes into account that the noise is highly random and relatively frequent, and directly filtering and denoising a single data point may affect the continuity and integrity of the data. Therefore, by further obtaining the noise degree of the group, accurate denoising can be realized while retaining the original characteristics of the data more completely, effectively improving the accuracy of the interaction between the user and virtual reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0029] Figure 1 A flowchart of an enhanced virtual reality interaction method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0032] It should be noted that when realizing the interaction between users and AR, the corresponding AR interaction mode can be designed in advance according to the user's motion state, such as controlling the movement in the AR scene by the user's walking, controlling the rotation of the AR object by the swing of the arm, etc. By placing sensors on the user's body, the user's acceleration and angular velocity values ​​can be collected, and then the user's current motion state can be obtained by analyzing the changes in the user's acceleration and angular velocity data, thereby realizing the interaction between users and AR.

[0033] However, due to electromagnetic interference in the environment, temperature changes, and heating of electronic components inside the sensor, the collected acceleration data and angular velocity data will contain noise data. Such noise data will affect the accuracy of determining the user's motion state, resulting in the VR system being unable to accurately achieve scene conversion when the perspective scene changes, affecting the user's interactive experience.

[0034] Based on this, the embodiment of the present invention discloses an enhanced virtual reality interaction method, which accurately obtains data points with high noise levels by analyzing the local prominence of each data point in a data segment and the data fluctuation between the data segment as a whole, and accurately denoises the data points with high noise levels, effectively improving the overall quality of the data, thereby effectively enhancing the accuracy of virtual reality interaction and user experience. For details, please refer to Figure 1 As shown, Figure 1 A flowchart of an augmented virtual reality interaction method provided by an embodiment of the present invention includes the following steps S1 to S5.

[0035] S1: Obtain a preset reference data segment for each data point in a data segment collected every second in a time series of user motion data.

[0036] It should be noted that in a piece of data, normal data will show certain regular changes, while noise data generally exhibits randomness and uncertainty. Through the different manifestations of noise data and normal data, noise data can be filtered out from the data and filtered to remove noise.

[0037] Based on this, the noise performance of each data point can be obtained by obtaining the fluctuation of the data point in a piece of data to distinguish normal data from noise data.

[0038] For example, in an embodiment of the present invention, obtaining a preset reference data segment for each data point in a data segment collected every second in a time series of user motion data includes: taking a point composed of acceleration data and angular velocity data collected at the same time as a data point, arranging the data points collected every second in time series to obtain a data segment; determining the length T of the reference data segment by the Fisher optimal segmentation method, taking the data point as the center, and obtaining an equal amount of reference data segments on both sides of the data point. data points constitute the reference data segment of the data points.

[0039] The specific steps of determining the length of the reference data segment by the Fisher optimal segmentation method can be implemented by the prior art, and the embodiments of the present invention are not described in detail here. In order to facilitate data processing, the length T of the reference data segment can be set to an odd number, which can be set according to actual needs.

[0040] Specifically, when collecting data, the collection frequency can be preset, and the single collection time is 1 second. The accelerometer and gyroscope are fixed on the user (such as the head, waist, etc.) through a wearable device to detect the user's motion state. At the same time, the acceleration is collected by the accelerometer, and the angular velocity is collected by the gyroscope to form data points. The data points collected each time are stored by timestamps. The standard fraction method is used to identify and remove acceleration data and angular velocity data that exceed the reasonable range, and the missing data values ​​are filled with the adjacent values ​​of the missing points to obtain the motion data time series sequence. The motion data time series sequence is divided into multiple data segments by the single collection time. The collection time of each data segment is 1 second, and each data point corresponds to a unique acceleration and angular velocity.

[0041] In this way, the embodiment of the present invention determines the length of the reference data segment through the Fisher optimal segmentation method, and can find the optimal data segment segmentation method based on the principle of minimizing the loss function (such as the sum of squared deviations), so that the data points in each data segment finally obtained have high similarity and the data points between different data segments have high differences. And by obtaining the data points on both sides of the data point, each data point has sufficient context information, thereby improving the accuracy of data analysis.

[0042] For example, the acquisition frequency can be set to 100 Hz, that is, 100 data points are collected per second; the acquisition frequency can be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0043] Among them, the specific steps of using the standard fraction method to identify and remove acceleration data and angular velocity data that exceed a reasonable range, and filling the missing data values ​​with neighboring values ​​at the missing locations can be achieved through existing methods, and the embodiments of the present invention are not described in detail here.

[0044] For example, when obtaining a reference data segment of a data point in a data segment, you can also: preset the length of the reference data segment; and use the data point as the end, and obtain several data points in the historical data of the data point to form the reference data segment of the data point.

[0045] The length of the reference data segment may be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0046] It can be understood that the reference data segment of the data point finally obtained includes the data point itself; the length of the reference data segment is the number of data points included in the reference data segment.

[0047] In this way, the embodiment of the present invention can effectively reduce the amount of data processing by presetting the length of the reference data segment according to actual conditions, and can effectively reduce the data waiting time by using historical data as the reference data segment, thereby quickly responding to changes in acceleration and angular velocity.

[0048] It can be understood that based on any of the above embodiments, a reference data segment of a data point can be obtained, and the noise performance of the data point can be obtained by analyzing the data fluctuation of the data point in the reference data segment and the fluctuation difference between the reference data segment and the data segment as a whole, that is, continue to execute the following steps.

[0049] S2: Obtaining a first noise index or a second noise index of the data point according to the data deviation degree between the data point and the reference data segment and the data deviation degree between the data segment and the reference data segment.

[0050] It should be noted that in the data segment obtained every second, each data point has a corresponding acceleration and angular velocity. If one of the data points is significantly different from the overall fluctuation of the data points, the noise performance of the data point will also be greater.

[0051] Based on this, an embodiment of the present invention obtains a first noise index of a data point through the fluctuation of acceleration corresponding to the data point, obtains a second noise index of the data point through the fluctuation of angular velocity corresponding to the data point, and then describes the noise performance of the data point through the first noise index and the second noise index of the data point.

[0052] By way of example, in an embodiment of the present invention, obtaining a first noise index or a second noise index of a data point includes: recording the absolute value of the difference between the acceleration in the data point and the average acceleration of all data points in a reference data segment as a first difference; recording the absolute value of the difference between the average acceleration of all data points in the data segment where the data point is located and the average acceleration of all data points in the reference data segment as a second difference; using the normalized data of the sum of the first difference and the second difference as the first noise index of the data point; recording the absolute value of the difference between the angular velocity in the data point and the average angular velocity of all data points in the reference data segment as a third difference; recording the absolute value of the difference between the average angular velocity of all data points in the data segment where the data point is located and the average angular velocity of all data points in the reference data segment as a fourth difference; and obtaining the second noise index of the data point after normalizing the sum of the third difference and the fourth difference.

[0053] For example, the first noise index of a data point is determined, and the details can be referred to the following relationship:

[0054] ;

[0055] In the formula, represents the first noise index of the ith data point, represents the acceleration of the ith data point, represents the mean acceleration of all data points in the reference data segment of the i-th data point, represents the mean acceleration of all data points in the data segment where the i-th data point is located, represents the linear normalization function, Represents the absolute value symbol.

[0056] In the above formula, It represents the first difference of the i-th data point. The larger the value, the more abrupt the acceleration of the i-th data point. It represents the second difference of the ith data point. The larger the value is, the greater the difference between the local acceleration change of the ith data point and the overall acceleration change of the data segment is. That is, the more abrupt the acceleration of the ith data point is, the greater the possibility that the ith data point belongs to noise data.

[0057] After the first noise index of each data point is obtained based on the above formula, the second noise index of the data point can be obtained by the following formula.

[0058] For example, the second noise index of the data point is determined, and the details can be referred to the following relationship:

[0059] ;

[0060] In the formula, represents the second noise index of the ith data point, represents the angular velocity of the ith data point, represents the mean angular velocity of all data points in the reference data segment of the i-th data point, represents the mean angular velocity of all data points in the data segment where the i-th data point is located, represents the linear normalization function, Represents the absolute value symbol.

[0061] In the above formula, It represents the third difference of the i-th data point. The larger the value, the more abrupt the angular velocity of the i-th data point. It represents the fourth difference of the ith data point. The larger the value is, the greater the difference between the local angular velocity change of the ith data point and the overall angular velocity change of the data segment is. That is, the more abrupt the angular velocity of the ith data point is, the greater the possibility that the ith data point belongs to noise data.

[0062] After obtaining the first noise index and the second noise index of each data point based on the above steps, continue to perform the following steps.

[0063] S3: Obtain the Pearson correlation coefficient between the acceleration data and the angular velocity data in the data point, and use the negative of the sum of the Pearson correlation coefficient and 1 as the exponent of an exponential function with e as the base to obtain a third noise index of the data point.

[0064] It should be noted that the first noise index and the second noise index of the data point can be accurately obtained by analyzing the acceleration and angular velocity changes of the data point through the above steps. However, the first noise index and the second noise index can only reflect the acceleration and angular velocity changes respectively. When there is no noise, the acceleration and angular velocity corresponding to the data point are interrelated, while the noise data is randomly generated. When a data point has noise, the acceleration and angular velocity corresponding to the data point have different change trends.

[0065] Based on this, such correlation can be reflected by the Pearson correlation coefficient between acceleration and angular velocity, thereby obtaining the third noise level of the data point.

[0066] It should be further explained that the Pearson correlation coefficient can only describe the degree of linear correlation between two variables. Obviously, the acceleration and angular velocity corresponding to a single data point cannot calculate the Pearson correlation coefficient, but the data changes between normal data points in adjacent time are relatively small.

[0067] Based on this, the left and right adjacent data points of the current data point can be combined with the data point to form a correlation group, so as to accurately obtain the Pearson correlation coefficient between the acceleration and angular velocity of the data point.

[0068] By way of example, in an embodiment of the present invention, the Pearson correlation coefficient between the acceleration data and the angular velocity data in a data point is obtained, including: the data point and the left and right adjacent data points form a correlation group of data points, and the Pearson correlation coefficient between the acceleration and angular velocity of all data points in the correlation group is used as the Pearson correlation coefficient between the acceleration and angular velocity in the data point.

[0069] For example, the Pearson correlation coefficient between the acceleration and the angular velocity in the data point is determined, and the specific relationship can be seen in the following relation:

[0070] ;

[0071] In the formula, represents the Pearson correlation coefficient between the acceleration and angular velocity corresponding to the i-th data point, represents the first data point in the relevant group The acceleration of the data point, represents the first data point in the relevant group The angular velocity of each data point.

[0072] In the above formula, represents the mean acceleration of the relevant group of the ith data point, represents the mean angular velocity of the associated group of the ith data point.

[0073] After the Pearson correlation coefficient between the acceleration and the angular velocity corresponding to the data point is obtained by the above formula, the third noise index of the data point can be obtained by the Pearson correlation coefficient between the acceleration and the angular velocity corresponding to the data point.

[0074] For example, in the embodiment of the present invention, the third noise index of the data point is determined, and specifically, the following relationship can be referred to:

[0075] ;

[0076] In the formula, represents the third noise index of the ith data point, represents the Pearson correlation coefficient between the acceleration and angular velocity corresponding to the i-th data point, Represents an exponential function with base e.

[0077] In the above formula, The smaller it is, the closer the Pearson correlation coefficient between the acceleration and angular velocity corresponding to the ith data point is to -1, the smaller the correlation between the acceleration and angular velocity corresponding to the ith data point is, the greater the possibility that the data point is noise data, and the larger the third noise index of the corresponding data point is, and the two are negatively correlated.

[0078] After respectively obtaining the first noise index, the second noise index and the third noise index of the data point based on the above steps, the actual noise level of the data point can be accurately obtained through the first noise index, the second noise index and the third noise index of the data point, that is, continue to execute the following steps.

[0079] S4: Calculate the first noise level of the data points, take the average of the first noise levels of all data points in the data segment as the second noise level of the data segment, and take the average of the second noise levels of the group consisting of the data segment and the left and right adjacent data segments as the third noise level of the group.

[0080] It should be noted that, based on the above steps, the data changes of acceleration and angular velocity and the correlation between acceleration and angular velocity are analyzed respectively, and the first noise index, the second noise index and the third noise index of the data point can be obtained, thereby determining the first noise degree of the data point.

[0081] For example, in the embodiment of the present invention, the first noise level of the data point is determined, and specifically, the following relationship can be referred to:

[0082] ;

[0083] In the formula, represents the first noise level of the ith data point, , , Respectively represent the first noise index, the second noise index, and the third noise index of the i-th data point, Respectively represent the weights of the first noise index and the second noise index of the i-th data point, represents the linear normalization function.

[0084] In the above formula, ; For example, the weight value of the first noise index can be 0.4, and the weight value of the second noise index can be 0.6; the specific settings can be made according to actual needs.

[0085] It should be further explained that the first noise level of each data point can be obtained based on the above formula. When the noise data is determined directly based on the first noise level, if the noise randomness is high and relatively frequent, the noise data may persist for a period of time, which may cause a large number of data points to be removed as noise data, thereby affecting the continuity and integrity of the motion data time series sequence.

[0086] Based on this, each data segment and its left and right adjacent data segments can be judged as a filtering unit, the first noise level average of all data points in the data segment can be used as the second noise level of the data segment, and the second noise level average of the group formed by the data segment and the left and right adjacent data segments can be used as the third noise level of the group, so as to accurately obtain the noise performance of each group and continue to execute the following steps.

[0087] S5: Filtering and denoising through the third noise level to enhance the interaction of virtual reality.

[0088] It should be noted that, based on the above steps, the third noise level of each group can be accurately obtained, and the noise data group that needs to be filtered and denoised can be accurately obtained through the third noise level of each group.

[0089] For example, in an embodiment of the present invention, filtering and denoising is performed using a third noise level, including: obtaining the normalized third noise level of each group, taking the group whose normalized third noise level is greater than a preset noise threshold as the target group, and filtering and denoising the target group.

[0090] The preset noise threshold may be 0.6, which may be set according to actual needs, and the embodiment of the present invention does not impose too many limitations on this.

[0091] It can be understood that the group whose third noise level after normalization is greater than the preset noise threshold is the group with a higher noise level; the group whose third noise level after normalization is not greater than the preset noise threshold is the group with a lower noise level. After obtaining the target group with a higher noise level, the target group with a higher noise level can be filtered and denoised.

[0092] For example, when filtering and denoising the target group, the group whose normalized third noise level is not greater than the preset noise threshold can be taken as the normal group, and the acceleration mean and angular velocity mean of the normal group closest to the target group are obtained. The data points in the target group are smoothed using the acceleration mean and angular velocity mean of the normal group to achieve data denoising.

[0093] In this way, the embodiment of the present invention can filter the group with a higher noise level more accurately by screening out the target group with a higher noise level as the noise data group that needs to be filtered and denoised.

[0094] For example, in an embodiment of the present invention, filtering and denoising are performed through a third noise level to enhance the interaction of virtual reality, including: inputting the acceleration and angular velocity corresponding to the data points in each denoised data segment into a pre-trained motion pattern recognition model to obtain the user's motion pattern; the virtual reality system adjusts the virtual environment through the matching result of the preset standard motion pattern and the user's motion pattern.

[0095] Among them, the movement mode may include raising hands, waving hands, turning around, etc., which can be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0096] Take the interaction process as an example: when the user turns around, AR quickly matches the perspective change of the user's posture, and then controls the perspective change of the surrounding scene in the virtual environment.

[0097] For example, when pre-training the motion pattern recognition model, the acceleration and angular velocity of various motion patterns can be manually labeled to obtain a data set of annotated historical collection. The labeled data set is used to complete the training of the long short-term memory network to obtain a trained motion pattern recognition model. The currently collected angular velocity and acceleration are input into the motion pattern recognition model to obtain the corresponding motion pattern to realize the interaction between the user and AR.

[0098] It should be noted that when interacting with AR, users may maintain the same posture for a long time, causing fatigue or discomfort, or the system may detect that the user's posture may cause inconvenience or errors in interaction. Based on this, abnormal monitoring can be used to promptly remind users to adjust their posture to avoid health problems, or to provide early warnings and adjustment suggestions.

[0099] For example, in an embodiment of the present invention, the user's motion pattern is obtained, and then the method further includes: performing abnormality monitoring on the user's motion pattern.

[0100] It can be seen that in the embodiments of the present invention, when the accuracy of the interaction between the user and the virtual reality is enhanced, a preset reference data segment of each data point in the data segment collected every second in the user motion data time series sequence can be obtained; the first noise index or the second noise index of the data point is obtained by the data deviation degree between the data point and the reference data segment and the data deviation degree between the data segment and the reference data segment; the Pearson correlation coefficient between the acceleration data and the angular velocity data in the data point is obtained, and the negative of the sum of the Pearson correlation coefficient and 1 is used as the exponent of the exponential function with e as the base to obtain the third noise index of the data point; the first noise degree of the data point is calculated; the mean of the first noise degrees of all data points in the data segment is used as the second noise degree of the data segment, and the mean of the second noise degrees of the group formed by the data segment and the left and right adjacent data segments is used as the third noise degree of the group; filtering and denoising are performed through the third noise degree to enhance the interaction of the virtual reality.

[0101] In this way, the embodiment of the present invention obtains the user's motion pattern through the changes in the acceleration and angular velocity of the data points, thereby realizing the interaction between the user and AR. In this process, the embodiment of the present invention takes into account the influence of noise data on the recognition of the motion pattern, and therefore obtains the noise degree of the data point by respectively analyzing the changes in the acceleration and angular velocity and the correlation between the acceleration and angular velocity, thereby realizing accurate denoising; on this basis, the embodiment of the present invention also takes into account that the noise is highly random and relatively frequent, and directly filtering and denoising a single data point may affect the continuity and integrity of the data. Therefore, by further obtaining the noise degree of the group, accurate denoising can be realized while retaining the original characteristics of the data, effectively improving the accuracy of the interaction between the user and virtual reality.

[0102] An embodiment of the present invention further discloses an augmented virtual reality interaction system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an augmented virtual reality interaction method provided by the present invention is implemented.

[0103] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0104] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM, a dynamic random access memory DRAM, a static random access memory SRAM, an enhanced dynamic random access memory EDRAM, a high bandwidth memory HBM, a hybrid memory cube HMC, etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0105] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0106] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An enhanced virtual reality interaction method, characterized in that: include: Obtaining a preset reference data segment for each data point in a data segment collected every second in a time series of user motion data; ； is the first noise index of the ith data point, is the acceleration of the ith data point, is the mean acceleration of all data points in the reference data segment of the i-th data point, is the mean acceleration of all data points in the data segment where the i-th data point is located, is the absolute value symbol; ； is the second noise index of the ith data point, is the angular velocity of the ith data point, is the mean angular velocity of all data points in the reference data segment of the i-th data point, is the mean angular velocity of all data points in the data segment where the i-th data point is located; The Pearson correlation coefficient between the acceleration and angular velocity of the data point and the left and right adjacent data points is used as the Pearson correlation coefficient between the acceleration and angular velocity of the data point; the negative of the sum of the Pearson correlation coefficient and 1 is used as the exponent of the exponential function with e as the base to obtain the third noise index of the data point; ； is the first noise level of the ith data point, , , are the first noise index, the second noise index, and the third noise index of the i-th data point, respectively. , are the weights of the first noise index and the second noise index of the i-th data point, respectively. is a linear normalization function; the mean value of the first noise level of all data points in the data segment is taken as the second noise level of the data segment, and the mean value of the second noise level of the group formed by the data segment and the left and right adjacent data segments is taken as the third noise level of the group; filtering and denoising are performed through the third noise level to enhance the interaction of virtual reality.

2. The method for enhancing virtual reality interaction according to claim 1, characterized in that: The step of obtaining a preset reference data segment for each data point in a data segment collected every second in a time series of user motion data includes: The points formed by the acceleration data and angular velocity data collected at the same time are taken as data points, and the data points collected every second are arranged in time series to obtain data segments; The length T of the reference data segment is determined by the Fisher optimal segmentation method, and the data point is taken as the center and equal amounts of data are obtained on both sides of the data point. data points constitute a reference data segment for the data points.

3. The method for augmented virtual reality interaction according to claim 1, characterized in that: The filtering and denoising by the third noise level comprises: The normalized third noise level of each group is obtained, the group whose normalized third noise level is greater than a preset noise threshold is taken as a target group, and filtering and denoising is performed on the target group.

4. The method for augmented virtual reality interaction according to claim 1, characterized in that: The filtering and denoising by the third noise level to enhance the interaction of virtual reality includes: The acceleration and angular velocity corresponding to the data points in each denoised data segment are input into a pre-trained motion pattern recognition model to obtain the user's motion pattern; the virtual reality system adjusts the virtual environment through the matching result of the preset standard motion pattern with the user's motion pattern.

5. The method for enhancing virtual reality interaction according to claim 4, characterized in that: The step of obtaining the user's motion pattern further includes: Monitor the user's movement patterns for abnormalities.

6. An augmented virtual reality interactive system, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, an augmented virtual reality interaction method according to any one of claims 1 to 5 is implemented.

Images