Human-computer interaction system based on virtual reality

Abstract

The invention discloses a virtual reality-based man-machine interaction system, which comprises a data management module, an interaction management module and a fatigue management module, and is characterized in that the data management module is used for collecting eye data, position data of a virtual article in a virtual environment, eye movement data and user data, preprocessing the collected data and storing the preprocessed data in a database; the interaction management module is used for judging whether a user carries out interaction or not through the eye data, determining a virtual article for interaction of the user through calculation and carrying out interaction by analyzing the eye movement data of the user and using an interaction behavior corresponding to the eye movement data, and the fatigue management module is used for calculating a user fatigue index by calling the user data and sending the user fatigue index to the interaction management module. Whether the user is in a fatigue state or not is judged, fatigue data are collected, a fatigue prediction model is obtained through historical fatigue data, the time period in which the user is prone to fatigue is predicted, warning is given to the user, and the method has the advantages of improving user experience and predicting fatigue.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to a human-computer interaction system based on virtual reality. Background Art

[0002] Virtual reality technology is a technology that uses computer technology to generate a virtual three-dimensional environment, which users enter and interact with by wearing special equipment. The goal of virtual reality technology is to create an immersive experience for users, making people feel as if they are in a virtual world and interact with virtual objects and environments.

[0003] In recent years, with the rapid development of virtual reality technology, virtual reality human-computer interaction technologies have become diverse. Existing virtual reality technologies mainly perform human-computer interaction through gesture recognition, eye movement recognition, and voice recognition technologies. Eye movement recognition technology achieves human-computer interaction by capturing the user's line of sight to determine the user's focus in the virtual environment. However, existing eye movement interactions can only simply lock virtual objects and obtain corresponding object information, and cannot perform other interactive behaviors, which affects the user experience. In addition, since users are prone to fatigue after performing interactive behaviors in a virtual environment for a long time, which affects their physical health, it is very necessary to design a virtual reality-based human-computer interaction system that improves user experience and predicts fatigue. Summary of the invention

[0004] The purpose of the present invention is to provide a human-computer interaction system based on virtual reality to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: a human-computer interaction system based on virtual reality, the system specifically comprising:

[0006] Data management module, used to manage the data generated in the human-computer interaction system;

[0007] An interaction management module, used to manage the interaction between users and virtual items in the virtual space;

[0008] The fatigue management module is used to perform corresponding management when the user is in a fatigue state.

[0009] According to the above technical solution, the data management module includes:

[0010] A data acquisition unit, used to collect and store eye data, location data of virtual objects in the virtual environment, eye movement data, and user data;

[0011] The data processing unit is used to pre-process the collected eye data, the position data of virtual objects in the virtual environment, the eye movement data, and the user data.

[0012] According to the above technical solution, the interaction management module includes:

[0013] An interaction judgment unit, used to judge whether human-computer interaction is performed by comparing the time difference between eyeball rotations;

[0014] An interaction determination unit, used to determine the virtual object with which the user interacts by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera;

[0015] The interaction execution unit is used to analyze the eye movement data and use the interaction behavior corresponding to the eye movement data to perform human-computer interaction.

[0016] According to the above technical solution, the fatigue management module includes:

[0017] A fatigue judgment unit, used to judge the user's physical condition by calling the user data;

[0018] A fatigue prediction unit, used to predict the time period when the user is prone to fatigue based on historical fatigue data;

[0019] The warning unit is used to issue a warning when the user is in a fatigue state.

[0020] According to the above technical solution, the data acquisition unit includes:

[0021] A virtual helmet with a built-in camera to capture eye movement, collect eye and eye movement data, and collect data on virtual objects in the virtual space;

[0022] Sensors to collect the user's heart rate data.

[0023] According to the above technical solution, the operation method of the human-computer interaction system based on virtual reality comprises the following specific operation steps:

[0024] Step S1: Establish a database, collect user eye data, location data of virtual objects in the virtual environment, eye movement data, and user data in real time, pre-process the data, and store the pre-processed data in the corresponding database;

[0025] Step S2: Determine whether the user is interacting. When the user is interacting with the computer, the virtual object that the user is interacting with is determined by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera, and the user's eye movement data is analyzed to determine the interactive behavior on the virtual object.

[0026] Step S3: Call the user data in the user database, calculate the user's fatigue index, and warn the user, collect the timestamps when the user is in a fatigue state, predict the time period when the user is prone to fatigue, and remind the user.

[0027] According to the above technical solution, an eye database is established to collect eye data in real time, wherein the eye data includes an eyeball pupil image, eyeball pupil coordinates, an initial coordinate of the eyeball pupil, and a distance between the eyeball and the display. An infrared camera is arranged to capture an eyeball image in real time, a coordinate system is established for the eyeball image, and the coordinates corresponding to the eyeball pupil are obtained, and a timestamp corresponding to the eyeball pupil coordinate data is obtained, and the eyeball pupil coordinate data and the corresponding timestamp data are stored in the eye database;

[0028] Establish a virtual environment database, obtain the position coordinates of virtual objects in the virtual environment in real time, store the coordinate data of the virtual objects in the virtual environment database, establish an eye movement database, and store the eye movement data and corresponding interactive behaviors established by the system in the eye movement database;

[0029] A user database is established to collect user data in real time, and the user data is pre-processed and stored in the user database. The user data includes heart rate data, blinking frequency, running time, and pupil contraction data.

[0030] According to the above technical solution, step S2 further includes the following steps:

[0031] Step S21: Call the coordinate data of the pupil of the eye in the eye database within the system setting period X (x t ,y t ), the initial coordinates of the pupil (x0, y0), the distance L between the eye and the display, and calculate the rotation angle of the eye Where t = 1, 2, ..., m, store the eye rotation angle in the database, and calculate the difference Δθ between the eye rotation angles of adjacent pupil images within the period X y =|θ t+1 -θ t |, where y = 1, 2, ..., n, call the threshold Δθ0 in the database, and calculate the difference Δθ of all eye rotation angles within the period X y Marking is performed and the number of markings is counted by a counter. y ≥Δθ0, it is marked as the first eye movement result, the value of counter 1 is increased, and the number M of consecutively marked first eye movement results in counter 1 is obtained. When Δθ yWhen Δθ < 0, it is marked as the second eye movement result, the value of counter two is incremented, the number N of consecutive markings of the second eye movement result in counter two is obtained, the thresholds N0 and M0 set in the database are called. When M ≥ M0, it is determined that the user is quickly browsing the virtual item. When N < M0 or M < M0, it is determined that the user is performing normal eye activities. When N ≥ N0, it is determined that the user is performing an interaction behavior with the virtual item;

[0032] Step S22: When the system determines that the user is performing an interaction behavior, the corresponding eye rotation angle θ in the database is called t , and the corresponding direction is V = x t + y t + L, to obtain the corresponding direction feature D = (θ t , V) of the eyeball. The coordinate data (x i , y i , z i ) of the virtual item in the virtual environment database is called, where i = 1, 2,..., x is the number of virtual items in the database, and the included angle between the virtual item and the position of the camera in the virtual environment is calculated where (x C , y C , z C ) is the coordinate of the camera in the virtual space, and the direction corresponding to the virtual item is represented as V i = (x i - x C ) + (y i - y C ) + (z i - z C ), to obtain the direction feature corresponding to the virtual item as D i = (θ i , V i ), and the direction feature corresponding to the virtual item is stored in the database;

[0033] Step S23: Call the direction feature D corresponding to the virtual item in the database i , and calculate the difference between the direction feature of the eyeball and the direction feature of the virtual item Select the virtual item with the smallest difference between the direction feature of the eyeball and the direction feature of the virtual item as the virtual item for interaction with the user, and display the simple information of the corresponding item on the virtual item.

[0034] According to the above technical solution, the step S23 further includes:

[0035] After the user selects a virtual item to interact with through eye selection, the system generates a set interaction interface including different eye movement data and corresponding interaction behaviors. The system obtains the timestamp of the interaction, and calls the pictures of the eyeball, the corresponding pupil coordinates (x t , y t ) and the initial coordinates (x0, y0) of the eyeball pupil in the eye database for a period of time after the corresponding timestamp, and extracts the motion feature K of the eyeball, including the rotation direction, blink count, and eye closure count of the eyeball after selecting the virtual item to interact with;

[0036] The rotation direction of the eyeball is obtained by comparing the magnitudes of the pupil coordinates of the eyeball. When x t > x0, it is determined that the eyeball rotates to the right. When x t < x0, it is determined that the eyeball rotates to the left. When y t > y0, it is determined that the eyeball rotates upward. When y t < y0, it is determined that the eyeball rotates downward, and the obtained rotation direction of the eyeball is recorded into the motion feature K of the eyeball. The eye movement data K x in the eye movement database and the corresponding interaction behaviors are called. Among them, x is the type of interaction behavior in the database. By comparing the rotation direction of the eyeball in the motion feature of the eyeball with the rotation direction of the eyeball in the eye movement database, the interaction behavior corresponding to the eye movement data with the same rotation direction of the eyeball is selected to interact with the virtual item;

[0037] The blink count is obtained by screening the pictures of the eyeball without pupils, obtaining the corresponding timestamp, and calculating the timestamp difference ΔT between the picture without pupils and the next picture with pupils. The threshold ΔT0 set in the database is called. When ΔT < ΔT0, it is determined that the user blinks, and the blink count of the user is counted. The obtained blink count is recorded into the motion feature K of the eyeball. The eye movement data K x in the eye movement database and the corresponding interaction behaviors are called. By comparing the blink count in the motion feature of the eyeball with the blink count in the eye movement database, the interaction behavior corresponding to the eye movement data with the same blink count is selected to interact with the virtual item;

[0038] The eye closure count is obtained by screening the pictures of the eyeball without pupils, obtaining the corresponding timestamp, and calculating the timestamp difference ΔT between the picture without pupils and the next picture with pupils. The threshold ΔT0 set in the database is called. When ΔT ≥ ΔT0, it is determined that the user closes the eyes, and the eye closure count is recorded. The obtained eye closure count is recorded into the motion feature K of the eyeball. The eye movement data K x in the eye movement database and the corresponding interaction behaviors are called. By comparing the eye closure count in the motion feature of the eyeball with the eye closure count in the eye movement database, the interaction behavior corresponding to the eye movement data with the same eye closure count is selected to interact with the virtual item.

[0039] According to the above technical solution, step S3 further includes the following steps:

[0040] Step S31: Call the heart rate data α, blink frequency β, running time γ, pupil contraction data in the user database Calculate the user's fatigue index Among them, ω α ,ω β ,ω γ , is the weight parameter established in the database, and the threshold ε0 established in the database is called. When ε<ε0, the user is judged to be in a normal physical state. When ε≥ε0, the user is judged to be in a fatigue state. The system prompts the user that the body is in a fatigue state, collects the corresponding timestamp when the user is in a fatigue state, and stores the corresponding timestamp in the database;

[0041] Step S32: call the timestamp data of the historical fatigue state in the database, call the corresponding model algorithm in the database, use the timestamp data of the historical fatigue state as sample data to train the model, obtain the fatigue time prediction model, predict the time period when the user is prone to enter the fatigue state, and when the user approaches the predicted fatigue time period, the system will remind the user in advance to avoid excessive fatigue.

[0042] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention collects eye data, position data of virtual objects in a virtual environment, eye movement data, and user data, stores the collected data in a database after pre-processing, determines whether the user interacts through the eye data, and determines the virtual objects that the user interacts with by calculating the similarity between the eye rotation angle and the angle between the virtual object and the camera, analyzes the user's eye movement data, uses the interactive behavior corresponding to the eye movement data to interact and calculates the user's fatigue index by calling the user data, determines whether the user is in a fatigue state, collects fatigue data, derives a fatigue prediction model through historical fatigue data, predicts the time period when the user is prone to fatigue, and warns the user, which has the characteristics of improving user experience and predicting fatigue. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0044] Figure 1 A schematic diagram of the module composition of a human-computer interaction system based on virtual reality provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Embodiment 1: Embodiment 1 of the present invention provides a human-computer interaction system based on virtual reality. Figure 1 A schematic diagram of the module composition of a human-computer interaction system based on virtual reality provided in the first embodiment of the present invention, such as Figure 1 As shown, the system includes:

[0047] Data management module, used to manage the data generated in the human-computer interaction system;

[0048] An interaction management module, used to manage the interaction between users and virtual items in the virtual space;

[0049] The fatigue management module is used to perform corresponding management when the user is in a fatigue state.

[0050] In some embodiments of the present invention, comprising:

[0051] A data acquisition unit, used to collect and store eye data, location data of virtual objects in the virtual environment, eye movement data, and user data;

[0052] The data processing unit is used to pre-process the collected eye data, the position data of virtual objects in the virtual environment, the eye movement data, and the user data.

[0053] In some embodiments of the present invention, comprising:

[0054] An interaction judgment unit, used to judge whether human-computer interaction is performed by comparing the time difference between eyeball rotations;

[0055] An interaction determination unit, used to determine the virtual object with which the user interacts by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera;

[0056] The interaction execution unit is used to analyze the eye movement data and use the interaction behavior corresponding to the eye movement data to perform human-computer interaction.

[0057] In some embodiments of the present invention, comprising:

[0058] A fatigue judgment unit, used to judge the user's physical condition by calling the user data;

[0059] A fatigue prediction unit, used to predict the time period when the user is prone to fatigue based on historical fatigue data;

[0060] The warning unit is used to issue a warning when the user is in a fatigue state.

[0061] In some embodiments of the present invention, comprising:

[0062] A virtual helmet with a built-in camera to capture eye movement, collect eye and eye movement data, and collect data on virtual objects in the virtual space;

[0063] Sensors to collect the user's heart rate data.

[0064] Embodiment 2: This embodiment can be applied to a human-computer interaction scenario. The method can be executed by a human-computer interaction system based on virtual reality provided in Embodiment 1. The method specifically includes the following steps:

[0065] Step S1: Establish a database, collect user eye data, location data of virtual objects in the virtual environment, eye movement data, and user data in real time, pre-process the data, and store the pre-processed data in the corresponding database;

[0066] Step S2: Determine whether the user is interacting. When the user is interacting with the computer, the virtual object that the user is interacting with is determined by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera, and the user's eye movement data is analyzed to determine the interactive behavior on the virtual object.

[0067] Step S3: Call the user data in the user database, calculate the user's fatigue index, and warn the user, collect the timestamps when the user is in a fatigue state, predict the time period when the user is prone to fatigue, and remind the user.

[0068] In the embodiment of the present invention, an eye database is established to collect eye data in real time, wherein the eye data includes an eyeball pupil image, eyeball pupil coordinates, initial coordinates of the eyeball pupil, and the distance between the eyeball and the display. An infrared camera is arranged to capture an eyeball image in real time, a coordinate system is established for the eyeball image, and the coordinates corresponding to the eyeball pupil are obtained, and the timestamp corresponding to the eyeball pupil coordinate data is obtained, and the eyeball pupil coordinate data and the corresponding timestamp data are stored in the eye database;

[0069] Establish a virtual environment database, obtain the position coordinates of virtual objects in the virtual environment in real time, and store the coordinate data of virtual objects in the virtual environment database; establish an eye movement database, and store the eye movement data and corresponding interactive behaviors established by the system in the eye movement database;

[0070] A user database is established to collect user data in real time. After preprocessing the user data, it is stored in the user database. The user data includes heart rate data, blink frequency, running duration, and pupil constriction data.

[0071] In an embodiment of the present invention, step S2 further includes the following steps:

[0072] Step S21: Call the coordinate data (x t , y t ) of the eyeball pupil, the initial coordinates (x0, y0) of the eyeball pupil, and the distance L between the eyeball and the display in the eye database within the system-set period X, and calculate the rotation angle of the eyeball where t = 1, 2,..., m, store the rotation angle of the eyeball in the database, and calculate the difference Δθ in the rotation angle of the eyeball between adjacent pupil pictures within the period X y = |θ t+1 - θ t |, where y = 1, 2,..., n, call the threshold Δθ0 in the database, mark all the differences Δθ in the rotation angle of the eyeball within the period X y , and count the number of marks through a counter. When Δθ y ≥Δθ0, mark it as the first eye movement result, increase the value of counter one by one, and obtain the number M of consecutive marks of the first eye movement result in counter one. When Δθ y <Δθ0, mark it as the second eye movement result, increase the value of counter two by one, and obtain the number N of consecutive marks of the second eye movement result in counter two. Call the thresholds N0 and M0 set in the database. When M≥M0, it is determined that the user is quickly browsing virtual items. When N<N0 or M<M0, it is determined that the user is performing normal eye activities. When N≥N0, it is determined that the user is interacting with virtual items;

[0073] Step S22: When the system determines that the user is performing an interaction behavior, call the corresponding rotation angle θ of the eyeball in the database t , and the corresponding direction is V = x t + y t + L, to obtain the direction feature D = (θ t , V) of the eyeball. Call the coordinate data (x i , y i , z i ) of the virtual item in the virtual environment database, where i = 1, 2,..., x is the number of virtual items in the database, and calculate the included angle between the virtual item and the position of the camera in the virtual environment where (x C , y C , z C ) are the coordinates of the camera in the virtual space, and the direction corresponding to the virtual item is represented as Vi = (x i - x C ) + (y i - y C ) + (z i - z C ), the obtained direction feature corresponding to the virtual item is D i = (θ i , V i ), and the direction feature corresponding to the virtual item is stored in the database;

[0074] Step S23: Call the direction feature D corresponding to the virtual item in the database i , and calculate the difference between the direction feature of the eyeball and the direction feature of the virtual item Select the virtual item with the smallest difference between the direction feature of the eyeball and the direction feature of the virtual item as the virtual item for interaction with the user, and display the simple information of the corresponding item on the virtual item.

[0075] In the embodiment of the present invention, the step S23 further includes:

[0076] When the user selects the virtual item to be interacted with through the eyes, the system generates a set interaction interface including different eye movement data and corresponding interaction behaviors. The system obtains the time stamp of the interaction, calls the pictures of the eyeball, the corresponding pupil coordinates (x t , y t ) and the initial coordinates (x0, y0) of the pupil of the eyeball in the eye database for a period of time after the corresponding time stamp, and extracts the movement feature K of the eyeball, including the rotation direction, the number of blinks, and the number of eye closures of the eyeball after selecting the virtual item to be interacted with;

[0077] The rotation direction of the eyeball is obtained by comparing the sizes of the pupil coordinates of the eyeball. When x t > x0, it is determined that the eyeball rotates to the right. When x t < x0, it is determined that the eyeball rotates to the left. When y t > y0, it is determined that the eyeball rotates upward. When y t < y0, it is determined that the eyeball rotates downward, and the obtained rotation direction of the eyeball is recorded into the movement feature K of the eyeball. Call the eye movement data K x in the eye movement database and the corresponding interaction behavior. Among them, x is the type of interaction behavior in the database. By comparing the rotation direction of the eyeball in the movement feature of the eyeball with the rotation direction of the eyeball in the eye movement database, select the interaction behavior corresponding to the eye movement data with the same rotation direction of the eyeball to interact with the virtual item;

[0078] The number of blinks is obtained by screening the pictures without pupils in the eye pictures, obtaining the corresponding timestamps, and calculating the timestamp difference ΔT between the pictures without pupils and the corresponding next picture with pupils, calling the threshold ΔT0 set in the database, and when ΔT<ΔT0, judging that the user blinks, and counting the number of times the user blinks, and entering the obtained number of blinks into the eye movement feature K, calling the eye movement data K in the eye movement database x By comparing the number of blinks in the eye movement characteristics with the number of blinks in the eye movement database, the corresponding interactive behaviors of the eye movement data with the same number of blinks are selected to interact with the virtual objects;

[0079] The number of eye closures is obtained by screening the pictures without pupils in the eye pictures, obtaining the corresponding timestamps, and calculating the timestamp difference ΔT between the pictures without pupils and the corresponding next picture with pupils, calling the threshold ΔT0 set in the database, and when ΔT≥ΔT0, judging that the user has closed his eyes, and recording the number of eye closures, entering the obtained number of eye closures into the eye movement feature K, calling the eye movement data K in the eye movement database x By comparing the number of eye closures in the eye movement characteristics with the number of eye closures in the eye movement database, the interaction behavior corresponding to the eye movement data with the same number of eye closures is selected to interact with the virtual objects; when the user decides to perform an interaction, the user's eye movement behavior is analyzed and the corresponding interaction behavior is performed on the virtual objects. For example, blinking twice corresponds to enlarging the virtual object, rotating the eyeballs to the left corresponds to rotating the virtual object to the left, and closing the eyes corresponds to ending the interaction behavior, thereby improving the user's experience of virtual reality human-computer interaction.

[0080] In an embodiment of the present invention, step S3 further includes the following steps:

[0081] Step S31: Call the heart rate data α, blink frequency β, running time γ, pupil contraction data in the user database Calculate the user's fatigue index Among them, ω α ,ω β ,ω γ , is the weight parameter established in the database, and the threshold ε0 established in the database is called. When ε<ε0, the user is judged to be in a normal physical state. When ε≥ε0, the user is judged to be in a fatigue state. The system prompts the user that the body is in a fatigue state, collects the corresponding timestamp when the user is in a fatigue state, and stores the corresponding timestamp in the database;

[0082] Step S32: Call the timestamp data of the historical fatigue state in the database, call the corresponding model algorithm in the database, use the timestamp data of the historical fatigue state as sample data to train the model, and obtain the fatigue time prediction model to predict the time period when the user is prone to enter the fatigue state. When the user approaches the predicted fatigue time period, the system will remind the user in advance to avoid excessive fatigue.

[0083] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0084] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A human-computer interaction system based on virtual reality, characterized in that: The system specifically comprises: Data management module, used to manage the data generated in the human-computer interaction system; An interaction management module, used to manage the interaction between users and virtual items in the virtual space; The fatigue management module is used to perform corresponding management when the user is in a fatigue state.

2. A human-computer interaction system based on virtual reality according to claim 1, characterized in that: The data management module comprises: A data acquisition unit, used to collect and store eye data, location data of virtual objects in the virtual environment, eye movement data, and user data; The data processing unit is used to pre-process the collected eye data, the position data of virtual objects in the virtual environment, the eye movement data, and the user data.

3. A human-computer interaction system based on virtual reality according to claim 2, characterized in that: The interaction management module includes: An interaction judgment unit, used to judge whether human-computer interaction is performed by comparing the time difference between eyeball rotations; An interaction determination unit, used to determine the virtual object with which the user interacts by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera; The interaction execution unit is used to analyze the eye movement data and use the interaction behavior corresponding to the eye movement data to perform human-computer interaction.

4. A human-computer interaction system based on virtual reality according to claim 3, characterized in that: The fatigue management module comprises: A fatigue judgment unit, used to judge the user's physical condition by calling the user data; A fatigue prediction unit, used to predict the time period when the user is prone to fatigue based on historical fatigue data; The warning unit is used to issue a warning when the user is in a fatigue state.

5. A human-computer interaction system based on virtual reality according to claim 4, characterized in that: The data acquisition unit comprises: A virtual helmet with a built-in camera to capture eye movement, collect eye and eye movement data, and collect data on virtual objects in the virtual space; Sensors to collect the user's heart rate data.

6. A method for operating a human-computer interaction system based on virtual reality, characterized in that: The specific operation steps are: Step S1: Establish a database, collect user eye data, location data of virtual objects in the virtual environment, eye movement data, and user data in real time, pre-process the data, and store the pre-processed data in the corresponding database; Step S2: Determine whether the user is interacting. When the user is interacting with the computer, the virtual object that the user is interacting with is determined by calculating the rotation angle of the eyeball and the angle between the virtual object and the corresponding camera, and the user's eye movement data is analyzed to determine the interactive behavior on the virtual object. Step S3: Call the user data in the user database, calculate the user's fatigue index, and warn the user, collect the timestamps when the user is in a fatigue state, predict the time period when the user is prone to fatigue, and remind the user.

7. The method for operating a human-computer interaction system based on virtual reality according to claim 6, characterized in that: The eye database is established to collect eye data in real time, wherein the eye data includes an eyeball pupil image, eyeball pupil coordinates, an initial coordinate of the eyeball pupil, and a distance between the eyeball and the display. An infrared camera is arranged to capture an eyeball image in real time, a coordinate system is established for the eyeball image, and coordinates corresponding to the eyeball pupil are obtained, and a timestamp corresponding to the eyeball pupil coordinate data is obtained, and the eyeball pupil coordinate data and the corresponding timestamp data are stored in the eye database; Establish a virtual environment database, obtain the position coordinates of virtual objects in the virtual environment in real time, store the coordinate data of the virtual objects in the virtual environment database, establish an eye movement database, and store the eye movement data and corresponding interactive behaviors established by the system in the eye movement database; A user database is established to collect user data in real time, and the user data is pre-processed and stored in the user database. The user data includes heart rate data, blinking frequency, running time, and pupil contraction data.

8. The method for operating a human-computer interaction system based on virtual reality according to claim 7, characterized in that: The step S2 further comprises the following steps: Step S21: Call the coordinate data (x t , y t ) of the eyeball pupil, the initial coordinates (x0, y0) of the eyeball pupil, and the distance L between the eyeball and the display in the eye database within the system-set period X, and calculate the rotation angle of the eyeball where t = 1, 2,..., m, store the rotation angle of the eyeball in the database, and calculate the difference Δθ in the rotation angle of the eyeball between adjacent pupil pictures within the period X y = |θ t+1 - θ t |, where y = 1, 2,..., n, call the threshold Δθ0 in the database, mark all the differences Δθ in the rotation angle of the eyeball within the period X y , and count the number of marks through a counter. When Δθ y ≥ Δθ0, mark it as the first eye movement result, increase the value of the first counter by one, and obtain the number M of consecutive marks of the first eye movement result in the first counter. When Δθ y < Δθ0, mark it as the second eye movement result, increase the value of the second counter by one, and obtain the number N of consecutive marks of the second eye movement result in the second counter. Call the thresholds N0 and M0 set in the database. When M ≥ M0, it is judged that the user quickly browses the virtual item. When N < N0 or M < M0, it is judged that the user is performing normal eye activities. When N ≥ N0, it is judged that the user is performing an interaction behavior with the virtual item; Step S22: When the system determines that the user is performing an interactive behavior, the corresponding eye rotation angle θ in the database is called t , the corresponding direction is V = x t +y t + L, and obtain the direction feature D corresponding to the eyeball = (θ t ,V), calling the coordinate data of the virtual item in the virtual environment database (x i ,y i ,z i ), where i = 1, 2, ..., x is the number of virtual objects in the database, and the angle between the virtual object and the camera position in the virtual environment is calculated. Where (x C ,y C ,z C ) is the coordinate of the camera in the virtual space, and the direction corresponding to the virtual object is represented by V i =(x i -x C )+(y i -y C )+(z i -z C ), the directional feature corresponding to the virtual item is obtained as D i =(θ i ,V i ), the directional features corresponding to the virtual items are stored in the database; Step S23: Calling the direction feature D corresponding to the virtual item in the database i , calculate the difference between the direction characteristics of the eyeball and the direction characteristics of the virtual object A virtual item with the smallest difference between the directional feature of the eyeball and the directional feature of the virtual item is selected as the virtual item for interacting with the user, and simple information of the corresponding item is displayed on the virtual item.

9. The method for operating a human-computer interaction system based on virtual reality according to claim 8, characterized in that: The step S23 further comprises: When the user selects the virtual object to be interacted with through the eyes, the system generates a preset interactive interface including different eye movement data and corresponding interactive behaviors. The system obtains the timestamp of the interaction and calls the eye database for the image of the eyeball and the corresponding pupil coordinates (x t ,y t ), the initial coordinates of the pupil of the eyeball (x0, y0), extracting the eyeball movement features K including the rotation direction of the eyeball, the number of blinks, and the number of eye closures after selecting the virtual item to be interacted with; The rotation direction of the eyeball is obtained by comparing the magnitudes of the pupil coordinates of the eyeball. When x t > x0, it is determined that the eyeball rotates to the right. When x t < x0, it is determined that the eyeball rotates to the left. When y t > y0, it is determined that the eyeball rotates upward. When y t < y0, it is determined that the eyeball rotates downward, and the obtained rotation direction of the eyeball is recorded in the eye movement feature K, and the eye movement data K x in the eye movement database corresponding to the interaction behavior is called. Among them, x is the type of interaction behavior in the database. By comparing the rotation direction of the eyeball in the eye movement feature with the rotation direction of the eyeball in the eye movement database, the interaction behavior corresponding to the eye movement data with the same eyeball rotation direction is selected to interact with the virtual item; The number of blinks is obtained by screening the pictures without pupils in the eye pictures, obtaining the corresponding timestamps, and calculating the timestamp difference ΔT between the pictures without pupils and the corresponding next picture with pupils, calling the threshold ΔT0 set in the database, and when ΔT<ΔT0, judging that the user blinks, and counting the number of times the user blinks, and entering the obtained number of blinks into the eye movement feature K, calling the eye movement data K in the eye movement database x By comparing the number of blinks in the eye movement characteristics with the number of blinks in the eye movement database, the corresponding interactive behaviors of the eye movement data with the same number of blinks are selected to interact with the virtual objects; The number of eye closures is obtained by screening the pictures without pupils in the eye pictures, obtaining the corresponding timestamps, and calculating the timestamp difference ΔT between the pictures without pupils and the corresponding next picture with pupils, calling the threshold ΔT0 set in the database, and when ΔT≥ΔT0, judging that the user has closed his eyes, and recording the number of eye closures, entering the obtained number of eye closures into the eye movement feature K, calling the eye movement data K in the eye movement database x By comparing the number of eye closures in the eye movement characteristics with the number of eye closures in the eye movement database, the corresponding interaction behavior of the eye movement data with the same number of eye closures is selected to interact with the virtual object.

10. The method for operating a human-computer interaction system based on virtual reality according to claim 9, characterized in that: The step S3 further comprises the following steps: Step S31: Call the heart rate data α, blink frequency β, running time γ, pupil contraction data in the user database Calculate the user's fatigue index Among them, ω α ,ω β ,ω γ , is the weight parameter established in the database, and the threshold ε0 established in the database is called. When ε<ε0, the user is judged to be in a normal physical state. When ε≥ε0, the user is judged to be in a fatigue state. The system prompts the user that the body is in a fatigue state, collects the corresponding timestamp when the user is in a fatigue state, and stores the corresponding timestamp in the database; Step S32: call the timestamp data of the historical fatigue state in the database, call the corresponding model algorithm in the database, use the timestamp data of the historical fatigue state as sample data to train the model, obtain the fatigue time prediction model, predict the time period when the user is prone to enter the fatigue state, and when the user approaches the predicted fatigue time period, the system will remind the user in advance to avoid excessive fatigue.

Images