User interaction method and system based on virtual reality

Abstract

The invention relates to the technical field of data processing, and discloses a user interaction method and system based on virtual reality, which are used for improving the efficiency and accuracy of user interaction based on virtual reality. Comprising the steps of performing virtual interaction environment construction on an initial intelligent interaction scheme to obtain a corresponding initial virtual interaction environment, and collecting real-time feedback data and real-time image data of a target user in the initial virtual interaction environment in real time; based on the real-time image data, interaction part motion trail recognition is carried out on the target user to obtain a real-time motion trail, and posture deviation analysis is carried out on the real-time feedback data based on the real-time motion trail to obtain posture deviation data; and correcting the initial intelligent interaction scheme through the posture deviation data to obtain a target intelligent interaction scheme, and switching the initial virtual interaction environment through the target intelligent interaction scheme.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a user interaction method and system based on virtual reality. Background Art

[0002] In recent years, with the rapid development of sensor technology, artificial intelligence, virtual reality and other fields, human-computer interaction systems have played an increasingly important role in health monitoring, smart device control, etc. In response to user personalization and real-time needs, an important research direction is to combine physiological indicator data, sensor technology and virtual interactive environment to achieve intelligent user experience.

[0003] However, some systems lose precision in complex or fast motion situations, resulting in inaccurate user movements and gestures in the virtual environment, which reduces the realism of the interaction and the user experience. Latency is a common problem in VR systems, especially when there is a certain time difference between the user performing an action and the system generating corresponding feedback. This delay causes the movement and operation in the virtual environment to feel unnatural, reducing the real-time and smoothness of user interaction. Summary of the invention

[0004] The present invention provides a user interaction method and system based on virtual reality, which are used to improve the efficiency and accuracy of user interaction based on virtual reality.

[0005] A first aspect of the present invention provides a user interaction method based on virtual reality, the user interaction method based on virtual reality comprising: collecting physiological indicator data of a target user, and performing interaction part analysis on the target user according to the physiological indicator data to obtain at least one interaction part of the target user; performing sensor wearing position analysis on the target user through at least one of the interaction parts to obtain a sensor wearing position set, and performing sensor wearing on the target user through the sensor wearing position set; acquiring historical interaction data of the target user, and performing intelligent interaction scheme matching on the target user through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction scheme; constructing a virtual interaction environment for the initial intelligent interaction scheme to obtain a corresponding initial virtual interaction environment, and collecting real-time feedback data and real-time image data of the target user in the initial virtual interaction environment in real time; performing interaction part motion trajectory recognition on the target user based on the real-time image data to obtain a real-time motion trajectory, and performing posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data; correcting the initial intelligent interaction scheme through the posture deviation data to obtain a target intelligent interaction scheme, and switching the initial virtual interaction environment through the target intelligent interaction scheme.

[0006] In combination with the first aspect, in a first implementation of the first aspect of the present invention, the sensor wearing position analysis of the target user is performed through at least one of the interaction parts to obtain a sensor wearing position set, and the sensor is worn on the target user through the sensor wearing position set, including: performing a sensor wearing position analysis on the target user through at least one of the interaction parts to obtain an inertial measurement unit wearing position set and an electric force sensor wearing position set; wearing a preset plurality of inertial measurement units to the target user through the inertial measurement unit wearing position set; wearing a preset plurality of electric force sensors to the target user through the electric force sensor wearing position set.

[0007] In combination with the first implementation manner of the first aspect, in the second implementation manner of the first aspect of the present invention, the historical interaction data of the target user is obtained, and an intelligent interaction plan is matched for the target user through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction plan, including: obtaining the historical interaction data of the target user, standardizing the historical interaction data to obtain standardized interaction information; performing an interaction type analysis on the standardized interaction information to obtain corresponding interaction type data; performing a joint motion range analysis on the target user through the physiological indicator data to obtain corresponding joint motion range data; and matching the intelligent interaction plan for the target user through the interaction type data and the joint motion range data to obtain an initial intelligent interaction plan.

[0008] In combination with the second implementation manner of the first aspect, in the third implementation manner of the first aspect of the present invention, the virtual interaction environment is constructed for the initial intelligent interaction scheme to obtain the corresponding initial virtual interaction environment, and the real-time feedback data and real-time image data of the target user in the initial virtual interaction environment are collected in real time, including: extracting the scheme scene for the initial intelligent interaction scheme to obtain the scheme scene; analyzing the training duration of the initial intelligent interaction scheme to obtain the training duration; matching the environmental elements of the scheme scene based on the training duration to obtain multiple virtual environment element data; constructing interactive elements for the multiple virtual environment element data to obtain multiple interactive element data; constructing the virtual environment for the initial intelligent interaction scheme based on the multiple virtual environment element data and the multiple interactive element data to obtain the initial virtual interaction environment; and collecting the real-time feedback data and real-time image data of the target user in the initial virtual interaction environment in real time.

[0009] In combination with the first implementation manner of the first aspect, in the fourth implementation manner of the first aspect of the present invention, the interaction part motion trajectory of the target user is identified based on the real-time image data to obtain a real-time motion trajectory, and the posture deviation analysis of the real-time feedback data is performed based on the real-time motion trajectory to obtain posture deviation data, including: based on at least one of the interaction parts, the real-time image data is segmented into user interaction part images to obtain multiple segmented images; continuous edge segment extraction is performed on each of the segmented images to obtain continuous edge segment data corresponding to each of the segmented images; elliptical arc segment extraction is performed on the continuous edge segment data corresponding to each of the segmented images to obtain an elliptical arc segment set corresponding to each of the segmented images; the elliptical arc segment set corresponding to each of the segmented images is fitted respectively to obtain fitted elliptical arc segment data corresponding to each of the segmented images; the interaction part motion trajectory of the target user is identified through the fitted elliptical arc segment data corresponding to each of the segmented images to obtain a real-time motion trajectory; and the real-time feedback data is analyzed based on the real-time motion trajectory to obtain posture deviation data.

[0010] In combination with the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect of the present invention, the posture deviation analysis of the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data includes: data classification of the real-time feedback data to obtain joint angle data of the target user collected by multiple inertial measurement units and muscle strength distribution data of the target user collected by multiple electric force sensors; matching the joint angle standard data of the target user through the real-time motion trajectory to obtain a corresponding joint angle standard data set; matching the muscle strength standard data of the target user through the real-time motion trajectory to obtain a corresponding muscle strength standard data set; performing data difference comparison between the joint angle data and the joint angle standard data set to obtain angle difference data; performing data difference comparison between the muscle strength distribution data and the joint angle standard data set to obtain angle difference data; performing data difference comparison between the muscle strength distribution data and the muscle strength standard data set to obtain strength difference data; performing user bad posture analysis on the target user through the angle difference data and the strength difference data to obtain user bad posture data; performing posture deviation analysis on the target user through the user bad posture data to obtain posture deviation data.

[0011] In combination with the first aspect, in a sixth implementation of the first aspect of the present invention, the initial intelligent interaction scheme is corrected by the posture deviation data to obtain a target intelligent interaction scheme, and the initial virtual interaction environment is switched by the target intelligent interaction scheme, including: performing a posture deviation intensity analysis on the posture deviation data to obtain posture deviation intensity data; correcting the initial intelligent interaction scheme by the posture deviation intensity to obtain a target intelligent interaction scheme; performing an interaction environment analysis on the target intelligent interaction scheme to obtain a switching interaction environment; and switching the initial virtual interaction environment by the switching interaction environment.

[0012] A second aspect of the present invention provides a user interaction system based on virtual reality, the user interaction system based on virtual reality comprising:

[0013] A collection module, used to collect physiological indicator data of a target user, and perform interaction part analysis on the target user according to the physiological indicator data to obtain at least one interaction part of the target user;

[0014] An analysis module, configured to analyze the sensor wearing position of the target user through at least one of the interaction parts to obtain a sensor wearing position set, and wear the sensor on the target user through the sensor wearing position set;

[0015] A matching module, used to obtain the historical interaction data of the target user, and match the target user with an intelligent interaction plan through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction plan;

[0016] A construction module is used to construct a virtual interaction environment for the initial intelligent interaction scheme, obtain a corresponding initial virtual interaction environment, and collect real-time feedback data and real-time image data of the target user in the initial virtual interaction environment in real time;

[0017] an identification module, configured to identify the motion trajectory of the target user's interaction part based on the real-time image data to obtain the real-time motion trajectory, and to perform posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data;

[0018] The switching module is used to correct the initial intelligent interaction scheme according to the posture deviation data to obtain a target intelligent interaction scheme, and switch the initial virtual interaction environment according to the target intelligent interaction scheme.

[0019] In the technical solution provided by the present invention, by collecting the physiological index data and historical interaction data of the target user, a personalized intelligent interaction scheme can be generated for each user, taking into account the physiological state and personalized needs of the user, and improving the user experience. Through the analysis of the interaction part and the sensor wearing position, the sensor wearing position that is most suitable for the user can be determined, the accuracy of the sensor and the comfort of the wearable device can be improved, so as to more effectively obtain the physiological index data. In the virtual interactive environment, the feedback data and image data of the target user are collected in real time, and through the motion trajectory recognition and posture deviation analysis, the user's actions and postures can be more comprehensively understood, so as to more accurately reflect the real-time state of the user. Based on real-time data analysis, the interaction effect between the user and the virtual environment can be identified, and the initial intelligent interaction scheme can be corrected in real time through the posture deviation data. This helps to improve the adaptability, so that the interaction scheme is more in line with the actual needs and status of the user. According to the target intelligent interaction scheme, the switching of the virtual interactive environment can be realized to better meet the needs of the user. This dynamic switching can enable users to obtain a more suitable interactive experience in different situations, and improve flexibility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of an embodiment of a user interaction method based on virtual reality in an embodiment of the present application;

[0021] Figure 2 This is a schematic diagram of an embodiment of a user interaction system based on virtual reality in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The embodiments of the present invention provide a user interaction method and system based on virtual reality, which are used to improve the efficiency and accuracy of user interaction based on virtual reality.

[0023] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1, an embodiment of the user interaction method based on virtual reality in an embodiment of the present invention includes:

[0025] S101, collecting physiological indicator data of a target user, and performing interaction part analysis on the target user according to the physiological indicator data to obtain at least one interaction part of the target user;

[0026] It is understandable that the execution subject of the present invention may be a user interaction system based on virtual reality, or may be a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0027] Specifically, a comprehensive and reliable physiological index collection system is established. Physiological indicators are objective reflections of the physiological state of the human body, such as heart rate, skin electrode activity, etc. This system includes various sensors that can monitor and record the user's physiological parameters in real time. On the basis of physiological index data collection, in order to more deeply understand the interaction needs of the target user, the system performs interaction site analysis. Physiological indicators are linked to specific interaction actions to reveal the physiological changes of users in different situations. In this way, the system identifies the interaction sites that users often use in specific states, which helps to more accurately understand the user's interaction habits and preferences. For example, when the target user is in an anxious state, the heart rate and skin electrode activity increase significantly. By analyzing these physiological index data, the system infers that the user is more inclined to use hand movements for interaction, because the range of movement of the hands is usually larger, which helps to release tension. On the contrary, if the physiological indicators show that the user is relaxed, he is more inclined to use head movements or eye contact as a means of interaction. By combining the physiological index data and the results of the interaction site analysis, the system can obtain at least one interaction site of the target user. This interaction site is a specific part of the body such as the hand, head, and eyes, depending on the user's physiological state and interaction preferences in different situations. Therefore, the system is not just collecting physiological data, but mining deep information from it to gain a more comprehensive understanding of the user's interactive behavior.

[0028] S102, analyzing the sensor wearing position of the target user through at least one interaction part to obtain a sensor wearing position set, and wearing the sensor on the target user through the sensor wearing position set;

[0029] Specifically, the sensor wearing position analysis of the target user is performed through at least one interaction part. Selecting an appropriate interaction part involves a specific area of ​​the user's body, such as a hand, head, or other specific body part. This selection needs to be reasonably determined according to the actual user interaction needs and application scenarios. After analysis, an inertial measurement unit wearing position set and an electric force sensor wearing position set are obtained. Using an inertial measurement unit and an electric force sensor, these two types of sensors can provide information about the user's body movement and applied force, respectively. An inertial measurement unit is usually used to measure acceleration and angular velocity, while an electric force sensor is used to sense the force applied to the sensor. By obtaining the inertial measurement unit wearing position set, multiple preset inertial measurement units can be worn to the target user. Multiple inertial measurement units are placed on the selected interaction part to ensure that comprehensive and accurate motion information is obtained. In this way, the system can track the user's body dynamics in real time and provide highly flexible data support for interaction in a virtual reality environment. At the same time, by obtaining the electric force sensor wearing position set, multiple preset electric force sensors can be worn to the target user. The purpose is to install multiple electric force sensors on the selected interaction part to accurately measure the force applied by the user. This is very important for some virtual interaction scenarios that require sensing the user's physical strength, such as simulating the manipulation of objects or recognizing body movements.

[0030] S103, obtaining historical interaction data of the target user, and matching the target user with an intelligent interaction plan through physiological indicator data and historical interaction data to obtain an initial intelligent interaction plan;

[0031] Specifically, historical interaction data of the target user is obtained. This includes information such as the user's actions, interaction types, and interaction parts used in the virtual environment. In order to better understand and analyze this data, the system standardizes the historical interaction data. Standardization can eliminate differences between different interaction devices or systems, making the data more comparable. After obtaining the standardized interaction information, interaction type analysis is performed. The system identifies the types of different actions in the historical interaction data, such as gestures, head movements, eye gaze, etc. Through this step, the system can understand which specific interaction methods the user has adopted in past interactions. At the same time, by performing interaction type analysis on the standardized interaction information, the corresponding interaction type data can be obtained. These data describe the types of interactions and their frequencies that the user has adopted in the virtual environment in the past. This is an important input for subsequent intelligent interaction solution matching, because the user's historical interaction behavior is a key factor in understanding their habits and preferences. Further, the target user's joint range of motion is analyzed through physiological indicator data. This can measure the user's joint movement through devices such as sensors. Joint range of motion data is an important basis for understanding the user's physical flexibility and motor ability, because these data reflect the limitations of the interaction postures adopted by the user in the virtual environment. By combining the interaction type data and the joint range of motion data, the system can match intelligent interaction solutions. For example, if the user frequently uses gestures in historical interactions and the range of joint motion indicates high hand flexibility, the system will recommend a virtual interaction method based on gestures. This matching is based on the user's individual characteristics and past interaction experience, so it is more likely to provide users with interesting and adaptable interaction methods. Through this process, the system obtains an initial intelligent interaction solution. This solution combines the user's historical interaction behavior, interaction type preference, and physiological indicator data, so that the virtual reality system can more intelligently meet the user's interaction needs. The advantage of this method is personalization and adaptability, because it takes into account the individual differences and changes of users and provides an interactive experience that is closer to user expectations.

[0032] S104, constructing a virtual interactive environment for the initial intelligent interactive solution to obtain a corresponding initial virtual interactive environment, and collecting real-time feedback data and real-time image data of the target user in the initial virtual interactive environment in real time;

[0033] Specifically, the scenario extraction is performed on the initial intelligent interaction scenario. This includes analyzing the scenario elements in the interaction scenario, such as virtual objects, backgrounds, and interactive interfaces. By identifying these elements, the system can understand the basic structure of the interaction scenario and the virtual situations encountered by the user. The training time of the initial intelligent interaction scenario is analyzed to understand the time the user spends in a specific interaction scenario. Different scenario elements and interaction methods result in different time for users to stay in the virtual environment, so analyzing the training time helps to optimize the design of the virtual environment to improve user engagement and experience. Based on the analysis results of the training time, environmental element matching is performed. The virtual environment elements are matched with the scenarios encountered by the user during the training time to ensure that the user's experience in the virtual environment is more coherent and natural. For example, if the user spends more time in a certain scenario, the system increases the complexity of the scenario or adds eye-catching virtual elements accordingly. Interactive element construction is performed on multiple virtual environment element data. This includes introducing interactive elements such as clickable objects, animation effects, etc. in the virtual environment to increase the user's sense of engagement and fun. The design of these interactive elements should be based on the user's historical interaction data and physiological indicators to ensure that they match the user's personalized needs. The system obtains an initial virtual interactive environment. This environment takes into account the user's historical interaction behavior, training time, personalized needs, and actual scene elements, making it more adaptable and attractive. In this virtual environment, users can interact with virtual elements, and the system collects real-time feedback data and real-time image data from users in this initial virtual interaction environment.

[0034] S105, identifying the motion trajectory of the target user's interactive part based on the real-time image data to obtain the real-time motion trajectory, and performing posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data;

[0035] Specifically, based on at least one interactive part, the real-time image data is segmented into images of the user interactive parts. Image processing technology is used to identify the user's body parts, such as hands, head, etc., and the image is segmented into multiple regions. The purpose is to extract information related to the interactive parts from complex images. Continuous edge segments are extracted for each segmented image to obtain continuous edge segment data corresponding to each segmented image. Continuous edge segments are edge features in an image. By identifying continuous edges in an image, the system can more accurately capture the shape and motion trajectory of the user interactive parts. This step usually uses edge detection algorithms, such as the Canny algorithm. Elliptical arc segments are extracted for the continuous edge segment data corresponding to each segmented image to obtain an elliptical arc segment set corresponding to each segmented image. Elliptical arc segments are a natural way to describe the movement of the user interactive parts, because the elliptical shape can better fit the motion trajectory of human joints. By extracting elliptical arc segments, the system captures the motion characteristics of the user interactive parts more finely. Further, the elliptical arc segment set corresponding to each segmented image is fitted respectively to obtain the fitted elliptical arc segment data corresponding to each segmented image. The shape of the elliptical arc segment is fitted using a mathematical model to obtain more accurate motion trajectory information. Commonly used fitting methods include the least square method, etc., to ensure that the elliptical arc segment has a high degree of fit with the actual motion trajectory. The motion trajectory of the interactive part is identified through these fitted elliptical arc segment data. The system can track the motion trajectory of the user's interactive part in real time, thereby obtaining the real-time motion trajectory of the target user. The real-time feedback data is analyzed for posture deviation based on the real-time motion trajectory to obtain posture deviation data. By comparing the difference between the real-time motion trajectory and the expected motion trajectory, the system can identify the user's posture deviation. This analysis helps to understand whether the user is acting in accordance with the predetermined interaction method, thereby further optimizing the virtual interaction experience. For example, the system analyzes the posture deviation of the user's hand, such as whether the angle and position of the hand are as expected. This provides the system with feedback on the quality and accuracy of user interaction, which helps to further improve the virtual interaction solution.

[0036] The real-time feedback data is classified, and the joint angle data and muscle strength distribution data are extracted from the mixed real-time feedback data. Specifically, the system uses multiple inertial measurement units and electric force sensors to collect these data, so the classification step needs to separate the corresponding data for subsequent analysis. The target user's joint angle standard data is matched through the real-time motion trajectory to obtain the corresponding joint angle standard data set. The real-time motion trajectory is matched with the predetermined joint angle standard to establish a benchmark data set for evaluating the user's posture deviation. Similarly, the target user's muscle strength standard data is matched through the real-time motion trajectory to obtain the corresponding muscle strength standard data set. The purpose is to establish a standard data set related to the user's muscle strength for subsequent comparison and analysis. The joint angle data is compared with the joint angle standard data set for data difference to obtain angle difference data. The system analyzes the difference between the real-time collected joint angle data and the standard data to determine whether the user has abnormal joint angles. At the same time, the muscle strength distribution data is compared with the joint angle standard data set for data difference to obtain angle difference data. The purpose is to detect whether the user's muscle strength distribution in different parts is consistent with the expected standard. In addition, the muscle strength distribution data and the muscle strength standard data set are compared for data differences to obtain strength difference data. This helps analyze whether there are abnormalities in the user's muscle strength in different parts of the body, and further refines the assessment of posture deviation. The target user is analyzed for poor posture through angle difference data and strength difference data to obtain user poor posture data. This step comprehensively considers information on joint angles and muscle strength to identify whether the user has bad posture habits or abnormalities. Finally, the target user is analyzed for posture deviation through the user's poor posture data to obtain posture deviation data. This step combines various data to comprehensively evaluate the user's posture, providing a deeper understanding of the user's posture state.

[0037] S106. Correct the initial intelligent interaction solution through the posture deviation data to obtain a target intelligent interaction solution, and switch the initial virtual interaction environment through the target intelligent interaction solution.

[0038] Specifically, the posture deviation data is subjected to posture deviation intensity analysis. The degree of posture deviation is analyzed, that is, whether the user's posture deviates from the expected range. By measuring the intensity of posture deviation, the system quantifies the degree of bad posture of the user, providing a basis for subsequent correction. The initial intelligent interaction scheme is corrected by the posture deviation intensity to obtain the target intelligent interaction scheme. According to the analysis results of the posture deviation intensity, the system adjusts the initial intelligent interaction scheme to better adapt to the user's posture deviation. This involves adjusting the position of the interactive elements in the virtual environment, the posture range, or changing the interaction method. The target intelligent interaction scheme is subjected to interactive environment analysis to obtain information for switching the interactive environment. This step considers the user's posture correction and determines whether it is necessary to switch to another virtual interactive environment to provide a better interactive experience. The analysis may include elements, scenes, interaction methods, etc. of the virtual environment to ensure that the user can better complete the task or experience in the new environment. The initial virtual interactive environment is switched by switching the interactive environment. If the interactive environment analysis results show that switching is necessary, the system will automatically switch to the virtual environment corresponding to the target intelligent interaction scheme. This includes adjusting the virtual scene, changing the position or shape of the virtual object, and resetting the interactive elements.

[0039] In the embodiment of the present invention, by collecting the physiological index data and historical interaction data of the target user, a personalized intelligent interaction scheme can be generated for each user, taking into account the physiological state and personalized needs of the user, and improving the user experience. Through the analysis of the interaction part and the sensor wearing position, the sensor wearing position that is most suitable for the user can be determined, the accuracy of the sensor and the comfort of the wearable device can be improved, so as to more effectively obtain the physiological index data. In the virtual interactive environment, the feedback data and image data of the target user are collected in real time, and the user's actions and postures can be more comprehensively understood through motion trajectory recognition and posture deviation analysis, so as to more accurately reflect the real-time state of the user. Based on real-time data analysis, the interaction effect between the user and the virtual environment can be identified, and the initial intelligent interaction scheme can be corrected in real time through the posture deviation data. This helps to improve the adaptability, so that the interaction scheme is more in line with the actual needs and status of the user. According to the target intelligent interaction scheme, the switching of the virtual interaction environment can be realized to better meet the needs of the user. This dynamic switching can enable users to obtain a more suitable interactive experience in different situations, and improve flexibility and adaptability.

[0040] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0041] (1) Performing posture deviation intensity analysis on the posture deviation data to obtain posture deviation intensity data;

[0042] (2) Correct the initial intelligent interaction plan by the posture deviation strength to obtain the target intelligent interaction plan;

[0043] (3) Analyze the interaction environment of the target intelligent interaction solution and obtain the switching interaction environment;

[0044] (4) Switch the initial virtual interaction environment by switching the interaction environment.

[0045] Specifically, posture deviation data is subjected to posture deviation intensity analysis to obtain posture deviation intensity data. These data are usually derived from wearable devices or sensors, which record the user's movements and postures in the virtual environment. These data will be converted into numerical or graphical information for easy analysis. During the analysis process, the system uses machine learning and artificial intelligence technology to identify and quantify the degree of posture deviation. For example, if the user's movements are significantly different from the preset ideal movement mode when performing a virtual reality environment, this difference will be detected and quantified by the algorithm as posture deviation intensity data. These posture deviation intensity data will be used to correct the initial intelligent interaction plan. The system adjusts the interaction plan according to the degree of posture deviation to better adapt to the user's actual situation. For example, if the user shows a high posture deviation intensity in a specific action, the system will reduce the difficulty of the action or provide more intuitive guidance. This process involves not only the adjustment of existing interaction plans, but also the introduction of new interaction elements or mechanisms to better adapt to the needs and capabilities of users. The purpose of this personalized adjustment is to provide a more comfortable and effective user experience while ensuring the effectiveness and safety of the interaction. The system will conduct an interactive environment analysis on the adjusted target intelligent interaction plan. The purpose is to ensure that the new interaction plan can be effectively implemented in the virtual environment. In this process, multiple aspects need to be considered, such as the layout of the virtual environment, the arrangement of interactive elements, and the design of the user interface. For example, if the new interaction scheme requires users to move more widely in space, the layout of the virtual environment needs to be adjusted to provide more space for activities. In addition, interface elements and interaction prompts also need to be optimized according to the new interaction scheme to ensure that users can interact smoothly. The initial virtual interaction environment is switched by switching the interaction environment. All previous analyses and adjustments will be concretized into visible and experiential environmental changes. This switch is not only visual, but also involves fundamental changes in the interaction logic and user experience. For example, if the original virtual environment is designed for standing use, but analysis finds that users interact better in a sitting state, then the environment will be adjusted to a design suitable for sitting use, including lowering the viewing angle, adjusting the position of interactive elements, etc. This adjustment can not only improve the user's comfort, but also improve the accuracy and efficiency of the interaction.

[0046] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0047] (1) Analyzing the sensor wearing position of the target user through at least one interaction part to obtain a wearing position set of the inertial measurement unit and a wearing position set of the electric force sensor;

[0048] (2) Wearing a plurality of preset inertial measurement units on a target user through an inertial measurement unit wearing position set;

[0049] (3) By using the electric force sensor wearing position set, multiple preset electric force sensors are worn on the target user.

[0050] Specifically, the sensor wearing position of the target user is analyzed through at least one interactive part to obtain an inertial measurement unit wearing position set and an electric force sensor wearing position set. Select a suitable sensor. The inertial measurement unit usually includes an accelerometer and a gyroscope to measure the user's movement and posture. The electric force sensor is used to measure the force applied by the user on the object. The selection of the sensor should take into account factors such as its accuracy, sensitivity, size and weight to ensure an accurate and comfortable interactive experience in the virtual reality environment. The sensor wearing position is analyzed for the target user through at least one interactive part. The system analyzes the physiological characteristics and interaction needs of the target user to determine the wearing position of the sensor. For example, if the user wants to perform body movement control in the virtual reality environment, the system needs to wear the inertial measurement unit on the user's wrist, ankle or head to capture the user's movement trajectory and posture. The wearing position of the electric force sensor depends on the interaction scenario that the user needs to simulate. For example, if the user needs to pull a virtual rod with force, the sensor needs to be placed on the user's hand. Once the wearing position of the sensor is determined, the system wears a plurality of preset inertial measurement units and electric force sensors to the target user. The sensor devices are fixed to the user's body parts, usually using straps, clips or other fixing devices. These sensors need to ensure a close fit to the user's body in order to accurately capture the user's movement and force data.

[0051] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0052] (1) Obtain the historical interaction data of the target user, standardize the historical interaction data, and obtain standardized interaction information;

[0053] (2) Analyze the interaction type of the standardized interaction information to obtain the corresponding interaction type data;

[0054] (3) Analyze the joint range of motion of the target user through physiological index data to obtain the corresponding joint range of motion data;

[0055] (4) Intelligent interaction plans are matched to target users through interaction type data and joint range of motion data to obtain an initial intelligent interaction plan.

[0056] Specifically, historical interaction data of the target user is obtained. These historical interaction data can come from various interactive activities of the user in the virtual reality environment, such as games, training, simulation, etc. These data may include user actions, clicks, gestures, voice input, etc. The collection of historical interaction data can be completed through various sensors and devices such as virtual reality headsets, handles, cameras, etc. Standardize it. Standardization includes operations such as data cleaning, denoising, and unified units to ensure the consistency and comparability of the data. For example, if the historical interaction data includes different types of gesture inputs, they need to be converted into unified gesture encoding or description for subsequent analysis. After standardization, standardized interaction information can be obtained. This is a data set containing various standardized interaction data points, describing the user's interactive activities in the virtual reality environment. These data points may include information such as the timestamp, type, location, and intensity of the interaction. Interaction type analysis is performed on the standardized interaction information. Using machine learning or data mining technology, the standardized interaction data is classified into different interaction types according to its patterns and features. For example, the system identifies the user's gestures as different gesture types such as "fist", "nod" or "wave". At the same time, the target user's joint range of motion is analyzed through physiological indicator data. This can be achieved by measuring the user's physiological parameters such as joint angles and muscle tension through sensors. The range of motion data describes the range of motion and limitations of each joint of the user. The interaction type data and joint range of motion data are combined to perform intelligent interaction solution matching. An algorithm is used to match the user's interaction preferences and physiological conditions to determine the interaction method and virtual environment settings that best suit the user. For example, if the user has a limited range of motion, the system selects a suitable virtual interaction method to ensure that the user can comfortably participate in the virtual reality experience. Finally, an initial intelligent interaction solution is obtained based on the matching results. This solution includes information such as the user's interaction type, virtual environment settings, and interaction method to meet the user's needs and limitations.

[0057] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0058] (1) Extracting scenario scenarios from the initial intelligent interaction solution to obtain scenario scenarios;

[0059] (2) Analyze the training duration of the initial intelligent interaction scheme to obtain the training duration;

[0060] (3) Based on the training duration, the environmental elements of the scenario are matched to obtain multiple virtual environment element data;

[0061] (4) constructing interactive elements for multiple virtual environment element data to obtain multiple interactive element data;

[0062] (5) Based on the multiple virtual environment element data and the multiple interactive element data, construct a virtual environment for the initial intelligent interaction scheme to obtain an initial virtual interactive environment;

[0063] (6) Collect real-time feedback data and real-time image data of the target user in the initial virtual interactive environment.

[0064] Specifically, scenario extraction is performed to extract key scenario elements and information from the initial intelligent interaction scenario. This can include the nature of the task, the environment of the scenario, the user's goals, etc. The goal of scenario extraction is to clarify the elements in the virtual environment in order to create a meaningful context for the user. Training duration analysis is performed to estimate the time required for the user to complete the interaction scenario in the virtual environment. This analysis can be based on the user's historical data, the complexity of the task, and the user's ability level. Training duration analysis helps determine the duration of activities in the virtual environment to ensure that the user completes the task within a reasonable time. Environmental element matching is performed, which includes selecting and configuring various elements in the virtual environment, such as scenes, objects, props, etc., to adapt to the user's training duration and scenario scenario. The goal of matching is to ensure that the virtual environment is consistent with the user's interaction goals and time requirements. Interactive element construction is performed to add interactive properties to virtual environment elements to respond to user operations and behaviors. This can include user interaction with virtual objects, dynamic behavior of objects, sound effects, etc. The construction of interactive elements enables users to interact with the virtual environment to achieve their goals. Based on the data of virtual environment elements and interactive elements, the virtual environment is constructed. This includes placing virtual environment elements in the virtual scene to create a virtual world in which users will participate. The goal of virtual environment construction is to provide users with an immersive, challenging and interesting virtual scene to promote their participation and learning. Real-time feedback data and real-time image data of the target user in the initial virtual interactive environment are collected in real time. This includes monitoring the user's movement, posture, reaction time, and real-time visual and auditory feedback. This data is used to evaluate the user's performance, reaction and needs, so as to dynamically adjust the virtual environment and interaction scheme based on real-time feedback to provide a better user experience.

[0065] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0066] (1) Based on at least one interaction part, performing user interaction part image segmentation on the real-time image data to obtain a plurality of segmented images;

[0067] (2) Extracting continuous edge segments from each segmented image to obtain continuous edge segment data corresponding to each segmented image;

[0068] (3) Extracting elliptical arc segments from the continuous edge segment data corresponding to each segmented image to obtain an elliptical arc segment set corresponding to each segmented image;

[0069] (4) Fitting the elliptical arc segment set corresponding to each segmented image respectively to obtain the fitted elliptical arc segment data corresponding to each segmented image;

[0070] (5) Identify the motion trajectory of the target user's interactive part through the fitted elliptical arc segment data corresponding to each segmented image to obtain the real-time motion trajectory;

[0071] (6) Perform posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data.

[0072] Specifically, based on at least one interaction part, the real-time image data is segmented into user interaction parts. The purpose is to separate the user's interaction parts from the image, which usually involves image segmentation algorithms, such as semantic segmentation or instance segmentation. For example, if the user uses hands to interact in virtual reality, image segmentation will help to separate the hand area. Continuous edge segment extraction is performed on each segmented image. The goal of continuous edge segment extraction is to detect the edges of the interaction parts in the image and segment them into continuous line segments. This can be achieved by edge detection algorithms (such as Canny edge detection) to identify the position and shape of the edges in the image. Elliptical arc segment extraction is performed on the continuous edge segments of each segmented image. The system further analyzes the shape of the edge segment to determine whether it meets the standard of the elliptical arc shape. This usually involves a curve fitting algorithm, such as the least squares method, to detect the elliptical arc segment. The elliptical arc segments of each segmented image are fitted respectively to obtain the fitted elliptical arc segment data corresponding to each segmented image. Using the fitting algorithm, the parameters of the detected elliptical arc segment (such as the center, axis length, axis ratio, etc.) are estimated for subsequent analysis. Through the fitted elliptical arc segment data of each segmented image, the motion trajectory of the target user's interaction part can be identified. Analyze the changes and trajectories of the elliptical arc segments to infer the user's movements and interactive behaviors. For example, if the user uses gestures to control a virtual object, the system tracks the motion trajectory of the elliptical arc segment to identify the user's gestures. Based on the real-time motion trajectory, the real-time feedback data is analyzed for posture deviation to obtain posture deviation data. The purpose is to analyze whether the user's interactive posture is consistent with expectations. For example, if the user tries to grab a virtual object, the system determines whether the user's gesture is accurate or whether there is posture deviation based on the motion trajectory of the elliptical arc segment and the user's actual feedback data.

[0073] In a specific embodiment, the process of performing the posture deviation analysis step on the real-time feedback data may specifically include the following steps:

[0074] (1) Classifying the real-time feedback data to obtain the target user's joint angle data collected by multiple inertial measurement units and the target user's muscle force distribution data collected by multiple electric force sensors;

[0075] (2) Match the target user’s joint angle standard data through the real-time motion trajectory to obtain the corresponding joint angle standard data set;

[0076] (3) Match the target user’s muscle strength standard data through real-time motion trajectory to obtain the corresponding muscle strength standard data set;

[0077] (4) Compare the joint angle data with the joint angle standard data set to obtain angle difference data;

[0078] (5) Compare the muscle force distribution data with the joint angle standard data set to obtain angle difference data;

[0079] (6) Compare the muscle strength distribution data with the muscle strength standard data set to obtain strength difference data;

[0080] (7) Analyze the bad posture of the target user through the angle difference data and the force difference data to obtain the bad posture data of the user;

[0081] (8) Perform posture deviation analysis on the target user based on the user’s bad posture data to obtain posture deviation data.

[0082] Specifically, the real-time feedback data is classified to obtain the joint angle data of the target user collected by multiple inertial measurement units and the muscle strength distribution data of the target user collected by multiple electric force sensors. These data can be obtained by placing sensors on the user's body. For example, the user wears a sensor kit, including an inertial measurement unit and an electric force sensor, for monitoring the user's movement and muscle strength. The target user's joint angle standard data is matched through the real-time motion trajectory. The actual measured joint angle data is compared with the pre-set standard data. The standard data can be a normal physiological joint angle range. Through matching, it can be determined whether the user's joint angle is within the normal range. At the same time, the target user's muscle strength standard data is matched through the real-time motion trajectory. This includes comparing the actual measured muscle strength distribution data with the pre-set standard data to determine whether the user's muscle strength meets the normal standard. The joint angle data is compared with the joint angle standard data set for data difference to obtain angle difference data. This step is used to quantify the deviation of the user's joint angle from the standard range. If the angle difference is large, it indicates that the user's posture is abnormal. At the same time, the muscle strength distribution data is compared with the joint angle standard data set for data difference to obtain angle difference data. This helps to evaluate whether the muscle strength distribution matches the normal joint angle. If there is a mismatch, it will lead to poor posture. The muscle strength distribution data and the muscle strength standard data set are compared for data difference to obtain strength difference data. This step is used to analyze whether the user's muscle strength distribution meets the normal standard. The target user is analyzed for user poor posture through angle difference data and strength difference data to obtain user poor posture data. This helps to identify whether the user is in an abnormal posture or muscle strength distribution state, causing discomfort or injury. The target user is analyzed for posture deviation through user poor posture data to obtain posture deviation data, further evaluate whether the user's posture deviates from the normal physiological range, and provide feedback or suggestions to improve the posture.

[0083] The user interaction method based on virtual reality in the embodiment of the present invention is described above. The user interaction system based on virtual reality in the embodiment of the present invention is described below. Figure 2 In one embodiment of the present invention, a user interaction system based on virtual reality includes:

[0084] The collection module 201 is used to collect physiological indicator data of a target user, and analyze the interaction part of the target user according to the physiological indicator data to obtain at least one interaction part of the target user;

[0085] An analysis module 202 is used to analyze the sensor wearing position of the target user through at least one of the interaction parts to obtain a sensor wearing position set, and wear a sensor on the target user through the sensor wearing position set;

[0086] A matching module 203 is used to obtain the historical interaction data of the target user, and to match the target user with an intelligent interaction solution through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction solution;

[0087] A construction module 204 is used to construct a virtual interaction environment for the initial intelligent interaction scheme, obtain a corresponding initial virtual interaction environment, and collect real-time feedback data and real-time image data of the target user in the initial virtual interaction environment in real time;

[0088] The recognition module 205 is used to recognize the motion trajectory of the interaction part of the target user based on the real-time image data to obtain the real-time motion trajectory, and to perform posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data;

[0089] The switching module 206 is used to modify the initial intelligent interaction solution according to the posture deviation data to obtain a target intelligent interaction solution, and switch the initial virtual interaction environment according to the target intelligent interaction solution.

[0090] Through the collaboration of the above components, by collecting the physiological index data and historical interaction data of the target user, a personalized intelligent interaction plan can be generated for each user, taking into account the user's physiological state and personalized needs, and improving the user experience. Through the analysis of the interaction part and the sensor wearing position, the sensor wearing position that best suits the user can be determined, the accuracy of the sensor and the comfort of the wearable device can be improved, so as to obtain physiological index data more effectively. In the virtual interactive environment, the feedback data and image data of the target user are collected in real time, and through motion trajectory recognition and posture deviation analysis, the user's actions and postures can be more comprehensively understood, so as to more accurately reflect the user's real-time status. Based on real-time data analysis, the interaction effect between the user and the virtual environment can be identified, and the initial intelligent interaction plan can be corrected in real time through posture deviation data. This helps to improve the adaptability, so that the interaction plan is more in line with the user's actual needs and status. According to the target intelligent interaction plan, the switching of the virtual interaction environment can be realized to better meet the needs of the user. This dynamic switching can enable users to obtain a more suitable interactive experience in different situations, and improve flexibility and adaptability.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0093] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A user interaction method based on virtual reality, characterized in that: The user interaction method based on virtual reality comprises: Collecting physiological indicator data of a target user, and performing interaction part analysis on the target user according to the physiological indicator data to obtain at least one interaction part of the target user; Performing sensor wearing position analysis on the target user through at least one of the interaction parts to obtain a sensor wearing position set, and performing sensor wearing on the target user through the sensor wearing position set; Acquire historical interaction data of the target user, and perform intelligent interaction scheme matching on the target user through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction scheme; Constructing a virtual interactive environment for the initial intelligent interactive solution to obtain a corresponding initial virtual interactive environment, and collecting real-time feedback data and real-time image data of the target user in the initial virtual interactive environment in real time; Based on the real-time image data, the target user's interactive part motion trajectory is identified to obtain a real-time motion trajectory, and based on the real-time motion trajectory, the real-time feedback data is analyzed for posture deviation to obtain posture deviation data; The initial intelligent interaction scheme is corrected by using the posture deviation data to obtain a target intelligent interaction scheme, and the initial virtual interaction environment is switched by using the target intelligent interaction scheme.

2. The user interaction method based on virtual reality according to claim 1, characterized in that: The performing sensor wearing position analysis on the target user through at least one of the interaction parts to obtain a sensor wearing position set, and performing sensor wearing on the target user through the sensor wearing position set, comprises: Performing sensor wearing position analysis on the target user through at least one of the interaction parts to obtain an inertial measurement unit wearing position set and an electric force sensor wearing position set; Wearing a plurality of preset inertial measurement units to the target user through the inertial measurement unit wearing position set; Through the electric force sensor wearing position set, a plurality of preset electric force sensors are worn on the target user.

3. The user interaction method based on virtual reality according to claim 2, characterized in that: The acquiring the historical interaction data of the target user, and matching the target user with an intelligent interaction plan through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction plan, includes: Acquire historical interaction data of the target user, and perform standardization processing on the historical interaction data to obtain standardized interaction information; Performing interaction type analysis on the standardized interaction information to obtain corresponding interaction type data; Performing joint motion range analysis on the target user through the physiological index data to obtain corresponding joint motion range data; An intelligent interaction plan is matched with the target user through the interaction type data and the joint motion range data to obtain an initial intelligent interaction plan.

4. The user interaction method based on virtual reality according to claim 3, characterized in that: The constructing of a virtual interactive environment for the initial intelligent interactive solution to obtain a corresponding initial virtual interactive environment, and collecting real-time feedback data and real-time image data of the target user in the initial virtual interactive environment in real time, includes: Extracting a scenario from the initial intelligent interaction scenario to obtain a scenario; Performing training duration analysis on the initial intelligent interaction scheme to obtain training duration; Based on the training duration, matching the environment elements of the scenario to obtain a plurality of virtual environment element data; Constructing interactive elements for the plurality of virtual environment element data to obtain a plurality of interactive element data; Based on the plurality of virtual environment element data and the plurality of interactive element data, constructing a virtual environment for the initial intelligent interaction scheme to obtain the initial virtual interaction environment; Real-time feedback data and real-time image data of the target user in the initial virtual interactive environment are collected in real time.

5. The user interaction method based on virtual reality according to claim 2, characterized in that: The step of identifying the motion trajectory of the target user's interaction part based on the real-time image data to obtain the real-time motion trajectory, and performing posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data includes: Based on at least one of the interaction parts, performing user interaction part image segmentation on the real-time image data to obtain a plurality of segmented images; Extracting continuous edge segments from each of the segmented images to obtain continuous edge segment data corresponding to each of the segmented images; Extracting elliptical arc segments from the continuous edge segment data corresponding to each segmented image to obtain an elliptical arc segment set corresponding to each segmented image; Fitting the ellipse arc segment set corresponding to each segmented image respectively to obtain the fitted ellipse arc segment data corresponding to each segmented image; Identify the motion trajectory of the target user's interaction part through the fitted ellipse arc segment data corresponding to each segmented image to obtain a real-time motion trajectory; Performing posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data.

6. The user interaction method based on virtual reality according to claim 5, characterized in that: The performing posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data includes: Classifying the real-time feedback data to obtain the joint angle data of the target user collected by the plurality of inertial measurement units and the muscle strength distribution data of the target user collected by the plurality of electric force sensors; Matching the target user's joint angle standard data with the real-time motion trajectory to obtain a corresponding joint angle standard data set; Matching the muscle strength standard data of the target user through the real-time motion trajectory to obtain a corresponding muscle strength standard data set; Comparing the joint angle data with the joint angle standard data set to obtain angle difference data; Performing data difference comparison between the muscle strength distribution data and the joint angle standard data set to obtain angle difference data; Performing data difference comparison on the muscle strength distribution data and the muscle strength standard data set to obtain strength difference data; Performing a bad posture analysis on the target user by using the angle difference data and the force difference data to obtain bad posture data of the user; The target user is subjected to posture deviation analysis based on the user's bad posture data to obtain posture deviation data.

7. The user interaction method based on virtual reality according to claim 1, characterized in that: The method of correcting the initial intelligent interaction scheme by using the posture deviation data to obtain a target intelligent interaction scheme, and switching the initial virtual interaction environment by using the target intelligent interaction scheme includes: Performing posture deviation intensity analysis on the posture deviation data to obtain posture deviation intensity data; The initial intelligent interaction scheme is corrected according to the posture deviation strength to obtain a target intelligent interaction scheme; Performing an interactive environment analysis on the target intelligent interactive solution to obtain a switching interactive environment; The initial virtual interactive environment is switched by switching the interactive environment.

8. A user interaction system based on virtual reality, characterized in that: The user interaction system based on virtual reality includes: A collection module, used to collect physiological indicator data of a target user, and perform interaction part analysis on the target user according to the physiological indicator data to obtain at least one interaction part of the target user; An analysis module, configured to analyze the sensor wearing position of the target user through at least one of the interaction parts to obtain a sensor wearing position set, and wear the sensor on the target user through the sensor wearing position set; A matching module, used to obtain the historical interaction data of the target user, and match the target user with an intelligent interaction plan through the physiological indicator data and the historical interaction data to obtain an initial intelligent interaction plan; A construction module is used to construct a virtual interaction environment for the initial intelligent interaction scheme, obtain a corresponding initial virtual interaction environment, and collect real-time feedback data and real-time image data of the target user in the initial virtual interaction environment in real time; an identification module, configured to identify the motion trajectory of the target user's interaction part based on the real-time image data to obtain the real-time motion trajectory, and to perform posture deviation analysis on the real-time feedback data based on the real-time motion trajectory to obtain posture deviation data; The switching module is used to correct the initial intelligent interaction scheme according to the posture deviation data to obtain a target intelligent interaction scheme, and switch the initial virtual interaction environment according to the target intelligent interaction scheme.

Images