Virtual reality system based on AR chip and intelligent contact lens and implementation method

Abstract

The invention provides a virtual reality system based on an AR chip and intelligent contact lenses and an implementation method, and relates to the technical field of virtual reality, the system comprises a parameter acquisition module, a first data processing module, a second data processing module and an AR display module, the first data processing module is connected with the parameter acquisition module, and the second data processing module is connected with the AR display module. The first data processing module is connected with the parameter acquisition module, the second data processing module is connected with the parameter acquisition module, the AR display module is arranged on the intelligent contact lens, and the AR display module is connected with the first data processing module and the second data processing module; according to the invention, the latest AR technology and the intelligent contact lens technology are combined, virtual reality display is realized through the intelligent contact lens, and the overall processing efficiency of the system is improved.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and more particularly to a virtual reality system based on AR chips and smart contact lenses and an implementation method thereof. Background Art

[0002] At present, with the development of Internet of Things technology, contact lenses are not only limited to vision correction, but have become an emerging field of smart wearable devices, and augmented reality (AR) technology is becoming increasingly important.

[0003] However, the existing technology lacks a virtual system that combines smart contact lenses with AR. At the same time, the virtual display through the virtual system does not take into account the user's intention, so the user's ideas cannot be reflected, the information they really need is missed, and the system efficiency is reduced.

[0004] Therefore, how to provide a virtual reality system that can solve the above problems is an issue that those skilled in the art need to solve urgently. Summary of the invention

[0005] In view of this, the present invention provides a virtual reality system and implementation method based on AR chip and smart contact lenses, which combines the latest AR technology and smart contact lens technology, realizes virtual reality display through smart contact lenses, and improves the overall processing efficiency of the system.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A virtual reality system based on an AR chip and a smart contact lens, implemented based on the smart contact lens, includes:

[0008] Parameter collection module, used to collect the user's physiological parameters and environmental parameters;

[0009] A first data processing module, which is connected to the parameter acquisition module and is used to process the physiological parameters and environmental parameters to obtain corresponding user display intention recognition results;

[0010] A second data processing module, which is connected to the parameter acquisition module and is used to process the physiological parameters to obtain the user's pupil position;

[0011] An AR display module, wherein the AR display module is disposed on the smart contact lens and is connected to the first data processing module and the second data processing module, and is used for performing AR virtual display according to the pupil position and the user display intention recognition result.

[0012] Preferably, the parameter acquisition module includes:

[0013] An eye movement data collection unit, used to collect the user's eye movement data;

[0014] An EEG data acquisition unit, used to acquire EEG signal data of the user;

[0015] A voice data collection unit, used to collect voice data of users;

[0016] The environmental parameter collection unit is used to collect the user's environmental parameters.

[0017] Preferably, the first data processing module includes:

[0018] A preprocessing unit, the preprocessing unit is connected to the eye movement data acquisition unit, the EEG data acquisition unit, the voice data acquisition unit and the environmental parameter acquisition unit, and is used to determine whether there is voice data, and if so, preprocess the received eye movement data, voice data, EEG signal data, and environmental parameters; if not, preprocess the received eye movement data, EEG signal data, and environmental parameters;

[0019] A first model building unit, the first model building unit is connected to the preprocessing unit and is used to build an intention recognition decision model when there is voice data;

[0020] A first processing unit, connected to the first model building unit, for inputting the pre-processed eye movement data, voice data, EEG signal data, and environmental parameters into the intention recognition decision model for processing to obtain a corresponding first user display intention recognition result;

[0021] A second model building unit, connected to the preprocessing unit, for building an intention recognition model when there is no voice data;

[0022] The second processing unit is connected to the second model building unit, and is used to input the pre-processed eye movement data, EEG signal data, and environmental parameters into the intention recognition decision model for processing to obtain the corresponding second user display intention recognition result.

[0023] Preferably, the second data processing module includes:

[0024] A data acquisition unit, used to acquire structural parameters of the smart contact lens;

[0025] A processing unit, the processing unit is connected to the data acquisition unit and the eye movement data acquisition unit, and is used to process the eye movement data and the structural parameters to obtain a corresponding pupil position.

[0026] Preferably, the AR display module includes:

[0027] A holographic display unit, the holographic display unit is connected to the processing unit, the first processing unit and the second processing unit, and is used to perform AR virtual projection according to the first user display intention recognition result, or the second user display intention recognition result and the pupil position;

[0028] An enhancement unit, wherein the enhancement unit is connected to the holographic display unit and is used to enhance the AR virtual projection.

[0029] The present invention also provides a method for realizing virtual reality based on an AR chip and a smart contact lens, comprising the following steps:

[0030] Acquiring physiological parameters and environmental parameters of the user, and preprocessing the physiological parameters and environmental parameters;

[0031] Determining whether the pre-processed physiological parameters include voice data of the user;

[0032] If included, constructing an intention recognition decision model, and inputting the pre-processed physiological parameters and environmental parameters into the intention recognition decision model for processing to obtain a corresponding first user display intention recognition result;

[0033] Perform AR virtual display according to the first user display intention recognition result.

[0034] Preferably, the specific process of determining whether the pre-processed physiological parameters include the user's voice data further includes:

[0035] If not, constructing an intention recognition model, and inputting the pre-processed physiological parameters and environmental parameters into the intention recognition model for processing, to obtain a corresponding second user display intention recognition result;

[0036] Perform AR virtual display according to the second user display intention recognition result.

[0037] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a virtual reality system and implementation method based on AR chip and smart contact lenses, which has the following beneficial effects:

[0038] 1. The system can collect the user's physiological parameters (such as eye movement data, EEG signals, voice data) and environmental parameters in real time, accurately understand the user's display intentions, and provide a highly personalized virtual reality experience;

[0039] 2. The system design takes into account the presence or absence of voice data and constructs an intent recognition decision model and an intent recognition model respectively. This design increases the flexibility of the system and enables it to adapt to user needs in different scenarios;

[0040] 3. The system uses holographic display technology to perform virtual projection through the AR display module, and enhances the projection with the enhancement unit to bring users an immersive virtual reality experience; precise tracking of the pupil position ensures that the virtual image can always be consistent with the user's line of sight, enhancing the accuracy and comfort of the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0042] Figure 1 A structural principle block diagram of a virtual reality system based on an AR chip and smart contact lenses provided by the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 As shown, the embodiment of the present invention discloses a virtual reality system based on an AR chip and a smart contact lens, which is implemented based on the smart contact lens and includes:

[0045] Parameter collection module 1, used to collect the user's physiological parameters and environmental parameters;

[0046] The first data processing module 2 and the first data processing module 3 are connected to the parameter acquisition module 1 and are used to process the physiological parameters and the environmental parameters to obtain the corresponding user display intention recognition results;

[0047] A second data processing module 3, which is connected to the parameter acquisition module 1 and is used to process the physiological parameters to obtain the user's pupil position;

[0048] AR display module 4, AR display module 4 is arranged on the smart contact lens, and AR display module 4 is connected to the first data processing module 2 and the second data processing module 3, and is used for performing AR virtual display according to the pupil position and the user display intention recognition result.

[0049] In a specific embodiment, the parameter acquisition module 1 includes:

[0050] An eye movement data collection unit 11, used to collect the user's eye movement data;

[0051] An EEG data acquisition unit 12 is used to acquire EEG signal data of the user;

[0052] A voice data collection unit 13, used to collect the user's voice data;

[0053] The environment parameter collection unit 14 is used to collect the user's environment parameters.

[0054] In a specific embodiment, the first data processing module 2 includes:

[0055] A preprocessing unit 21, which is connected to the eye movement data acquisition unit 11, the EEG data acquisition unit 12, the voice data acquisition unit 13 and the environmental parameter acquisition unit 14, and is used to determine whether there is voice data, and if so, preprocess the received eye movement data, voice data, EEG signal data and environmental parameters; if not, preprocess the received eye movement data, EEG signal data and environmental parameters;

[0056] A first model building unit 22, the first model building unit 22 is connected to the preprocessing unit 21, and is used to build an intention recognition decision model when there is voice data;

[0057] The first processing unit 23 is connected to the first model building unit 22, and is used to input the pre-processed eye movement data, voice data, EEG signal data, and environmental parameters into the intention recognition decision model for processing to obtain a corresponding first user display intention recognition result;

[0058] A second model building unit 24, the second model building unit 24 is connected to the preprocessing unit 21, and is used to build an intention recognition model when there is no voice data;

[0059] The second processing unit 25 is connected to the second model building unit 24, and is used to input the pre-processed eye movement data, EEG signal data, and environmental parameters into the intention recognition decision model for processing to obtain the corresponding second user display intention recognition result.

[0060] Specifically, the intention recognition decision model can be a model combining a neural network and a decision tree. The neural network is used to extract features, reduce dimensions, and fuse eye movement data, EEG signal data, environmental parameters, and voice data to obtain corresponding feature vectors. The feature vectors are processed by the decision tree to obtain the final recognition result. The intention recognition model can be a long short-term memory neural network.

[0061] In a specific embodiment, the second data processing module 3 includes:

[0062] A data acquisition unit 31 is used to acquire structural parameters of the smart contact lens;

[0063] The processing unit 32 is connected to the data acquisition unit 31 and the eye movement data acquisition unit 11, and is used for processing the eye movement data and the structural parameters to obtain the corresponding pupil position.

[0064] In a specific embodiment, the AR display module 4 includes:

[0065] A holographic display unit 41, which is connected to the processing unit 32, the first processing unit 23 and the second processing unit 25, and is used to perform AR virtual projection according to the first user display intention recognition result, or the second user display intention recognition result and the pupil position;

[0066] The enhancement unit 42 is connected to the holographic display unit 41 and is used to enhance the AR virtual projection.

[0067] The embodiment of the present invention further provides a method for implementing a virtual reality system based on an AR chip and a smart contact lens using any one of the above embodiments, comprising the following steps:

[0068] Acquire the user's physiological parameters and environmental parameters through the parameter acquisition module 1, and pre-process the physiological parameters and environmental parameters;

[0069] Determining, by the preprocessing unit 21, whether the preprocessed physiological parameters include the user's voice data;

[0070] If included, the intention recognition decision model is constructed by the first model construction unit 22, and the pre-processed physiological parameters and environmental parameters are input into the intention recognition decision model for processing by the first processing unit 23 to obtain the corresponding first user display intention recognition result;

[0071] Perform AR virtual display according to the first user display intention recognition result.

[0072] In a specific embodiment, the specific process of determining whether the preprocessed physiological parameters include the user's voice data further includes:

[0073] If not included, the intention recognition model is constructed by the second model construction unit 24, and the pre-processed physiological parameters and environmental parameters are input into the intention recognition model for processing by the second processing unit 25 to obtain the corresponding second user display intention recognition result;

[0074] The AR display module 4 performs AR virtual display according to the second user display intention recognition result.

[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0076] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A virtual reality system based on AR chip and smart contact lens, implemented based on smart contact lens, characterized in that: include: A parameter collection module (1), used for collecting physiological parameters of the user and environmental parameters; A first data processing module (2), the first data processing module (3) being connected to the parameter acquisition module (1) and being used for processing the physiological parameters and the environmental parameters to obtain a corresponding user display intention recognition result; A second data processing module (3), the second data processing module (3) being connected to the parameter acquisition module (1) and being used for processing the physiological parameters to obtain the pupil position of the user; An AR display module (4), wherein the AR display module (4) is arranged on the smart contact lens, and the AR display module (4) is connected to the first data processing module (2) and the second data processing module (3), and is used to perform AR virtual display according to the pupil position and the user display intention recognition result.

2. A virtual reality system based on AR chip and smart contact lenses according to claim 1, characterized in that: The parameter acquisition module (1) comprises: An eye movement data collection unit (11), used for collecting eye movement data of a user; An electroencephalogram data acquisition unit (12), used for acquiring electroencephalogram signal data of a user; A voice data collection unit (13), used for collecting voice data of a user; The environmental parameter collection unit (14) is used to collect the environmental parameters of the user.

3. A virtual reality system based on AR chip and smart contact lenses according to claim 2, characterized in that: The first data processing module (2) comprises: A preprocessing unit (21), the preprocessing unit (21) is connected to the eye movement data acquisition unit (11), the electroencephalogram data acquisition unit (12), the voice data acquisition unit (13) and the environmental parameter acquisition unit (14), and is used to determine whether voice data exists, and if so, preprocess the received eye movement data, voice data, electroencephalogram signal data and environmental parameters; if not, preprocess the received eye movement data, electroencephalogram signal data and environmental parameters; A first model building unit (22), the first model building unit (22) being connected to the preprocessing unit (21) and being used to build an intention recognition decision model when there is voice data; A first processing unit (23), the first processing unit (23) being connected to the first model building unit (22), and being used for inputting the pre-processed eye movement data, speech data, electroencephalogram signal data, and environmental parameters into the intention recognition decision model for processing, so as to obtain a corresponding first user display intention recognition result; A second model building unit (24), the second model building unit (24) being connected to the preprocessing unit (21) and being used for building an intention recognition model when no speech data exists; A second processing unit (25), the second processing unit (25) is connected to the second model building unit (24), and is used to input the pre-processed eye movement data, electroencephalogram signal data, and environmental parameters into the intention recognition decision model for processing to obtain a corresponding second user display intention recognition result.

4. A virtual reality system based on AR chip and smart contact lenses according to claim 3, characterized in that: The second data processing module (3) comprises: A data acquisition unit (31), used for acquiring structural parameters of the smart contact lens; A processing unit (32), the processing unit (32) is connected to the data acquisition unit (31) and the eye movement data acquisition unit (11), and is used to process the eye movement data and the structural parameters to obtain a corresponding pupil position.

5. A virtual reality system based on AR chip and smart contact lenses according to claim 4, characterized in that: The AR display module (4) comprises: A holographic display unit (41), the holographic display unit (41) being connected to the processing unit (32), the first processing unit (23) and the second processing unit (25), and being used for performing AR virtual projection according to a first user display intention recognition result, or a second user display intention recognition result and a pupil position; An enhancement unit (42), the enhancement unit (42) being connected to the holographic display unit (41) and being used to enhance the AR virtual projection.

6. A method for implementing a virtual reality system based on an AR chip and a smart contact lens using any one of claims 1 to 5, characterized in that: The following steps are involved: Acquiring physiological parameters and environmental parameters of the user, and preprocessing the physiological parameters and environmental parameters; Determining whether the pre-processed physiological parameters include voice data of the user; If included, constructing an intention recognition decision model, and inputting the pre-processed physiological parameters and environmental parameters into the intention recognition decision model for processing to obtain a corresponding first user display intention recognition result; Perform AR virtual display according to the first user display intention recognition result.

7. The implementation method according to claim 6, characterized in that: The specific process of determining whether the pre-processed physiological parameters include the user's voice data also includes: If not, constructing an intention recognition model, and inputting the pre-processed physiological parameters and environmental parameters into the intention recognition model for processing, to obtain a corresponding second user display intention recognition result; Perform AR virtual display according to the second user display intention recognition result.

Images