Multimodal physiological data acquisition device based on augmented reality and use method thereof

Abstract

The invention relates to a multi-modal physiological data acquisition device based on augmented reality and a use method thereof. The multi-modal physiological data acquisition device comprises a display assembly used for displaying a virtual scene; the acquisition assembly comprises a dry electrode module and a sensor integration module and is used for acquiring multiple physiological data of a user; the processing component is used for determining a scene display strategy of a virtual scene according to the multiple physiological data of the user and / or the multiple physiological data of other users in the team to which the user belongs, and determining target display content according to the scene display strategy; and the control component is used for adjusting the display content of the virtual scene of the display component according to the target display content and / or a confirmation instruction sent by the user based on the target display content. Therefore, the multi-modal physiological data acquisition device based on the augmented reality adopts the dry electrode, can acquire multiple items of physiological data, solves the problems that a wet electrode is relatively short in use time and can only acquire EEG signals in one aspect in related technologies, and improves the operation flexibility.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a multimodal physiological data acquisition device based on extended reality and a use method thereof. Background Art

[0002] In the related art, VR-EEG (Virtual Reality-Electroencephalogram) equipment is generally implemented using independent VR and EEG hardware. For example, an EEG electrode cap is fixed in place and then a VR headset is worn on the EEG electrode cap.

[0003] However, since the EEG electrode cap uses Ag-AgCl wet electrodes, there are problems such as long installation time and difficulty in cleaning. The wet electrodes will dehydrate over time and cannot be used for a long time. In addition, the VR-EEG device only collects EEG signals unilaterally. When other signals need to be collected, cumbersome equipment needs to be worn, which limits the user's flexibility when wearing VR work, and this problem needs to be solved urgently. Summary of the invention

[0004] The present invention provides a multimodal physiological data acquisition device based on extended reality and a method for using the same, so as to solve the problems in the related art that the wet electrode has a short usage time and can only collect EEG signals unilaterally, etc. The device is easy to use and operate, and the operational flexibility of the multimodal physiological data acquisition device based on extended reality is improved.

[0005] A first aspect of the present invention provides a multimodal physiological data acquisition device based on augmented reality, comprising:

[0006] A display component for displaying a virtual scene;

[0007] A collection component, comprising a dry electrode module and a sensor integration module, for collecting multiple physiological data of the user;

[0008] a processing component, configured to determine a scene display strategy of the virtual scene according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, and determine target display content according to the scene display strategy;

[0009] The control component is used to adjust the current display content of the virtual scene of the display component according to the target display content and / or a confirmation instruction sent by the user based on the target display content.

[0010] Furthermore, in some embodiments, the dry electrode module includes a plurality of dry electrodes arranged according to a preset arrangement rule. When the user wears the multimodal physiological data acquisition device based on augmented reality, the plurality of dry electrodes are in contact with the forehead scalp area of ​​the user, so as to collect the user's electroencephalogram signal through the plurality of dry electrodes.

[0011] Further, in some embodiments, the processing component includes:

[0012] A first analyzing unit, configured to determine a current emotional state of the user based on a plurality of physiological data of the user;

[0013] a matching unit, configured to match a virtual scene with the highest matching degree with the current emotional state from a preset scene library as the target display content, or to determine the target display content based on the current display content of the virtual scene and the current emotional state;

[0014] Furthermore, in some embodiments, the display content of the virtual scene includes at least one of display content corresponding to a group training scene, display content corresponding to a difficulty level and a rhythm level of a game, and display content corresponding to a relaxation scene under different emotions.

[0015] Furthermore, in some embodiments, the processing component is also used for:

[0016] The second analyzing unit is used to analyze the multiple physiological data of the user to generate a health report and / or health advice for the user.

[0017] Furthermore, in some embodiments, the control component is also used to:

[0018] Controlling the display component to display multiple physiological data and / or health reports and / or health suggestions of the user;

[0019] And / or, performing physiological interaction based on the user's interactive actions.

[0020] Furthermore, in some embodiments, the multimodal physiological data acquisition device based on extended reality further includes:

[0021] The amplifying component is used to amplify the multiple physiological data and then input them into the processing component.

[0022] Furthermore, in some embodiments, the multimodal physiological data acquisition device based on extended reality further includes:

[0023] A storage component, the storage component is used to store multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs offline based on a preset storage duration;

[0024] The encryption component is used to encrypt multiple physiological data of the user.

[0025] Further, in some embodiments, the multiple physiological data include: at least two of: EEG, fNIRS, EDA, PPG, RESP, FSR and SKT.

[0026] According to the multimodal physiological data acquisition device based on extended reality provided by the embodiment of the present invention, the target display content can be determined according to the multiple physiological data of the user and / or the multiple physiological data of other users in the team to which the user belongs, so as to adjust the current display content of the virtual scene of the display component according to the target display content and / or the confirmation instruction sent by the user based on the target display content. Therefore, the multimodal physiological data acquisition device based on extended reality adopts dry electrodes and can collect multiple physiological data, which solves the problems of the short use time of wet electrodes in the related art and the unilateral collection of EEG signals. It is easy to use and operate, and improves the operational flexibility of the multimodal physiological data acquisition device based on extended reality.

[0027] A second aspect of the present invention provides a method for using a multimodal physiological data acquisition device based on extended reality, using the multimodal physiological data acquisition device based on extended reality, wherein the method comprises the following steps:

[0028] Collect multiple physiological data of users;

[0029] Determining a scene display strategy of the virtual scene according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, and determining target display content according to the scene display strategy;

[0030] According to the target display content and / or a confirmation instruction sent by the user based on the target display content, the current display content of the virtual scene of the display component is adjusted.

[0031] Furthermore, in some embodiments, after collecting multiple physiological data of the user, the method further includes:

[0032] Analyze the multiple physiological data of the user to generate a health report and / or health suggestions for the user.

[0033] According to the method for using the multimodal physiological data acquisition device based on extended reality provided by the embodiment of the present invention, the target display content can be determined based on multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, so as to adjust the current display content of the virtual scene of the display component according to the target display content and / or the confirmation instruction sent by the user based on the target display content. Therefore, the multimodal physiological data acquisition device based on extended reality adopts dry electrodes and can collect multiple physiological data, which solves the problems of the short use time of wet electrodes in related technologies and can only collect EEG signals unilaterally. It is easy to use and operate, and improves the operational flexibility of the multimodal physiological data acquisition device based on extended reality.

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0036] Figure 1 A block diagram of a multimodal physiological data acquisition device based on extended reality according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the structure of a dry electrode module provided according to a specific embodiment of the present invention;

[0038] Figure 3 A schematic diagram of the structure of a multimodal physiological data acquisition device based on extended reality according to a specific embodiment of the present invention;

[0039] Figure 4 The present invention is a flowchart of a method for using a multimodal physiological data acquisition device based on extended reality according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0041] The following describes the multimodal physiological data acquisition device based on extended reality and the method of using the same in the embodiment of the present invention with reference to the accompanying drawings. In view of the problems mentioned in the above background technology that the use time of the wet electrode in the related art is short and the EEG signal can only be collected unilaterally, the present invention provides a multimodal physiological data acquisition device based on extended reality, which can determine the target display content according to the multiple physiological data of the user and / or the multiple physiological data of other users in the team to which the user belongs, so as to adjust the current display content of the virtual scene of the display component according to the target display content and / or the confirmation instruction sent by the user based on the target display content. The multimodal physiological data acquisition device based on extended reality adopts dry electrodes and can collect multiple physiological data, which solves the problems in the related art that the wet electrode has a short use time and can only collect EEG signals unilaterally. It is easy to use and operate, and improves the operational flexibility of the multimodal physiological data acquisition device based on extended reality.

[0042] Specifically, Figure 1 It is a block diagram of a multimodal physiological data acquisition device based on extended reality according to an embodiment of the present invention.

[0043] like Figure 1 As shown, the multimodal physiological data acquisition device 10 based on extended reality includes: a display component 100, a collection component 200, a processing component 300 and a control component 400.

[0044] Among them, the display component 100 is used to display the virtual scene; the acquisition component 200 includes a dry electrode module and a sensor integration module, which are used to collect multiple physiological data of the user; the processing component 300 is used to determine the scene display strategy of the virtual scene according to the multiple physiological data of the user and / or the multiple physiological data of other users in the team to which the user belongs, and determine the target display content according to the scene display strategy; the control component 400 is used to adjust the current display content of the virtual scene of the display component 100 according to the target display content and / or the confirmation instruction sent by the user based on the target display content.

[0045] In some embodiments, the multiple physiological data include at least two of EEG, fNIRS, EDA, PPG, RESP, FSR and SKT.

[0046] It is understandable that the display component 100 of the embodiment of the present invention can adopt the display component in the related art, which can be a liquid crystal display screen or a holographic display, so as to provide an immersive experience through the display component 100; the acquisition component 200 can be used to collect multiple physiological data of the user, for example, the multiple physiological data can be at least two of EEG, fNIRS, EDA, PPG, RESP, FSR and SKT. After collecting the multiple physiological data of the user, it can be processed by the processing component 300, for example, the scene display strategy of the virtual scene can be determined according to the multiple physiological data of the user, or according to the multiple physiological data of other users in the team to which the user belongs. Determine the scene display strategy of the virtual scene, or determine the scene display strategy of the virtual scene according to multiple physiological data of the user and multiple physiological data of other users in the team to which the user belongs, so as to determine the target display content according to the pre-set scene display strategy. The embodiment of the present invention can directly adjust the current display content of the virtual scene of the display component 100 according to the target display content, for example, automatically adjust the difficulty and rhythm of the VR scene according to real-time physiological feedback, and for example, provide a relaxation scene in an anxious state; the embodiment of the present invention can first send it to the user for confirmation, and after the user agrees, adjust the current display content of the virtual scene of the display component 100 to the target display content again.

[0047] The following is a specific example to illustrate how to determine the target display content.

[0048] As a possible implementation method, in some embodiments, the processing component 300 includes: a first analysis unit and a matching unit, wherein the first analysis unit is used to determine the user's current emotional state based on multiple physiological data of the user; the matching unit is used to match the virtual scene with the highest matching degree with the current emotional state from a preset scene library as the target display content, or, based on the current display content of the virtual scene and the current emotional state, determine the target display content.

[0049] In some embodiments, the display content of the virtual scene includes at least one of display content corresponding to a group training scene, display content corresponding to a difficulty level and a rhythm level of the game, and display content corresponding to a relaxation scene under different emotions.

[0050] It is understandable that the embodiments of the present invention can determine the user's current emotional state based on multiple physiological data of the user, such as the user's emotional state (such as sadness, anxiety, or depression), stress level, etc. The embodiments of the present invention can treat the user through some pre-set treatment plans. For example, the embodiments of the present invention can also adjust the user's gaming experience. For example, when the user is playing a game, due to the high difficulty of the game and the difficulty of passing the level, it is monitored that the user has a certain degree of sadness. The embodiments of the present invention can adjust the difficulty of the game, that is, reduce the intensity of the game displayed in the virtual scene. For another example, the embodiments of the present invention monitor the physiological state of the athlete in real time to optimize the training effect and safety.

[0051] Furthermore, the embodiment of the present invention can also realize group therapy and training scenarios of a team. For example, if multiple team members of a team to which a user belongs are all in the same local area network, the embodiment of the present invention can determine the optimal training plan or group therapy plan of the team to which the user belongs based on multiple physiological data of the user and multiple physiological data of other users in the team to which the user belongs in the same local area network through a pre-embedded algorithm, thereby using the display content corresponding to the optimal training plan or group therapy plan as the target display content.

[0052] The dry electrode module and the sensor integration module of the collection component 200 are described below with reference to a specific embodiment.

[0053] As a possible implementation method, in some embodiments, the dry electrode module includes a plurality of dry electrodes arranged according to a preset arrangement rule. When the user wears the multimodal physiological data acquisition device based on extended reality, the plurality of dry electrodes contact the forehead scalp area of ​​the user to collect the user's EEG signals through the plurality of dry electrodes.

[0054] The preset arrangement rule may be an arrangement rule preset by a user, or an arrangement rule obtained by a user through a limited number of experiments, or an arrangement rule obtained through a limited number of computer simulations.

[0055] Specifically, Figure 2 As shown, Figure 2 Schematic diagram of the structure of a dry electrode module according to a specific embodiment of the present invention, the dry electrode module is a forehead 8-lead EEG + 1GND integrated dry electrode module, that is, the dry electrode module has 8 EEG signal acquisition channels (such as Figure 2 F3, FZ, F4, AF7, FP1, FPZ, FP2, and AF8) and ground channels (such as Figure 2The dry electrode can be made of Ag / Agcl solid powder electrode, which is the best electrode material using international leading technology. The dry electrode module can be detachable. When there is a need to collect EEG signals, it can be directly installed in the multimodal physiological data acquisition device based on extended reality. It has the advantages of easy and quick installation, and there is no dehydration problem. It is suitable for long-term use and application scenarios outside the laboratory. It should be noted that Figure 2 The sizes (such as 30, 128, 15, etc.) are only exemplary and are not intended to limit the application.

[0056] Therefore, the unique design module of the forehead and scalp contact enables the dry electrode to form a stable and close contact interface with the skin, thereby enhancing the stability of the EEG signal and improving the motion tolerance of the signal. In addition, the dry electrode module of the embodiment of the present invention can be seamlessly integrated into a VR headset, thereby improving the comfort of the multimodal physiological data acquisition device based on extended reality.

[0057] Furthermore, the sensor integration module of the embodiment of the present invention can be integrated according to the physiological signals that need to be collected in practice. For example, if the embodiment of the present invention needs to collect fNIRS (functional near-infrared spectroscopy), EDA (Electrodermal activity), PPG (Photo PlethysmoGraphy), RESP (respiration, respiratory rate), FSR (Fractional Synthetic Rate, local muscle synthesis rate) and SKT (Skin Temperature), etc., then the sensor integration module can be integrated through the sensors corresponding to fNIRS, EDA, PPG, RESP, FSR and SKT. Therefore, by integrating multiple physiological sensors, multi-dimensional analysis of data is achieved, the richness and accuracy of information is improved, and the problem of overly cumbersome wearing and collecting physiological equipment in related technologies is solved.

[0058] In addition, when the acquisition component 200 transmits data to the processing component 300, the embodiment of the present invention can ensure that the data transmission delay is within an acceptable range (such as less than 100ms) by optimizing the communication protocol and data processing flow; and the embodiment of the present invention can integrate the physiological response of the user when experiencing VR content to be monitored in real time. For example, through eye tracking technology, the user's concentration and visual interest points can be analyzed, and then the virtual content can be adjusted to improve the user's participation and experience quality.

[0059] It should be noted that the display component 100 and the acquisition component 200 of the embodiment of the present invention can be integrated, and an integrated and lightweight design is performed. The equipment is integrated on a multimodal physiological data acquisition device based on extended reality (such as a VR helmet), so that the user can collect physiological signals while wearing VR and can move flexibly. It can also be set separately, and the specific setting can be made by technical personnel in this field according to actual conditions, and no specific limitation is made here.

[0060] Therefore, the display component 100 of the embodiment of the present invention can provide high-resolution display and immersive experience. By combining virtual reality and dry electrode EEG, and combining a sensor integrated module to collect multiple physiological indicators, it has the advantages of being easy and quick to use, and there is no dehydration problem. It is suitable for long-term use and application scenarios outside the laboratory.

[0061] Furthermore, in some embodiments, the processing component 300 is also used for: a second analysis unit, configured to analyze a plurality of physiological data of a user to generate a health report and / or health advice for the user.

[0062] The embodiment of the present invention can record the physiological data of the user every day, generate a health report, and help the user understand the changes in their health. At the same time, data mining technology is used to analyze the long-term trend of the user's physiological data and provide personalized health suggestions.

[0063] That is to say, the embodiments of the present invention can monitor multiple physiological data of the user, such as heart rate, brain waves and galvanic skin response, in real time. Among them, heart rate monitoring can help evaluate the user's emotional state and stress level, and provide data support for psychotherapy and relaxation training. The monitoring of galvanic skin response (EDA / GSR) can reveal the user's excitement or tension in a specific virtual situation, provide a basis for psychological assessment and emotional regulation, and provide users with instant feedback to enhance the user experience.

[0064] Furthermore, in some embodiments, the multimodal physiological data acquisition device 10 based on augmented reality further includes: an amplification component, wherein the amplification component is used to amplify the multiple physiological data and then input them into the processing component 300 .

[0065] Wherein, the amplifying component may be a modular power amplifier,

[0066] Specifically, the embodiment of the present invention may also be provided with a detachable amplifying component, which amplifies the signal through the amplifying component to improve the accuracy of the processing component 300 in processing multiple physiological data.

[0067] Therefore, the amplifying component of the embodiment of the present invention can be quickly installed and transmit signals via Bluetooth with a transmission distance of more than ten meters, and can be suitable for VR-EEG application research in complex scenarios.

[0068] Furthermore, in some embodiments, the above-mentioned multimodal physiological data acquisition device 10 based on extended reality further includes: a storage component and an encryption component. The storage component is used to store multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs offline based on a preset storage duration; the encryption component is used to encrypt multiple physiological data of the user.

[0069] The storage component may be a hardware storage device, such as a removable storage device such as an SD card.

[0070] Specifically, in order to more comprehensively monitor the physiological data of the user, the embodiment of the present invention can store multiple physiological data of the user offline, and help users and professionals better understand and manage health conditions by storing and analyzing the physiological data of the user. It should be noted that in normal use, the storage capacity of the hardware storage device under normal working conditions can reach more than 300 days.

[0071] In addition, for the sake of user security, the embodiment of the present invention uses encryption algorithms and related security protocols to encrypt multiple physiological data of the user to ensure the security of the user's physiological data during transmission and storage.

[0072] Furthermore, in some embodiments, the control component 400 is also used to: control the display component 100 to display multiple physiological data and / or health reports and / or health suggestions of the user; and / or perform physiological interactions based on the user's interactive actions.

[0073] That is, the embodiments of the present invention can provide real-time feedback on the user's physiological state, for example, through graphic or sound prompts, and can also allow the user to interact with the physiological feedback through gestures or controllers. Thus, in a VR environment, real-time feedback on the user's physiological state can be provided through graphic displays or sound signals.

[0074] In addition, the processing component 300 also provides a software algorithm interface to support users in secondary development and continuously optimize data processing and analysis algorithms according to actual needs to improve accuracy and response speed.

[0075] In order to enable relevant practitioners in the field to better understand the multimodal physiological data acquisition device based on extended reality according to the embodiment of the present invention, it will be explained in conjunction with specific embodiments below.

[0076] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of a multimodal physiological data acquisition device based on extended reality according to a specific embodiment of the present invention. Figure 3(a) is a rendering of a multimodal physiological data acquisition device based on augmented reality according to a specific embodiment of the present invention, Figure 3 (b) is a line diagram of a multimodal physiological data acquisition device based on augmented reality according to a specific embodiment of the present invention.

[0077] The multimodal physiological data acquisition device 10 based on extended reality includes an acquisition component 200, a processing component 300, a power supply component, a display component 100 and headphones. The acquisition component 200 and other inherited sensors collect multiple physiological data and transmit them to the data processing module, determine the target display content and / or target game difficulty of the virtual scene according to the user's emotional state, and provide visual output through the VR head display module and sound output through the headphones; the above components can all be powered by the power supply component integrated in the device.

[0078] Therefore, the multimodal physiological data acquisition device based on extended reality (such as VR-EEG equipment) can track the user's EEG activity in virtual interaction in real time, which is of great significance for studying human cognitive activities in virtual environments. It can also use EEG signals as direct, immediate and quantitative feedback, providing users with a new perception dimension and enriching the multi-sensor array of modern VR systems.

[0079] According to the multimodal physiological data acquisition device based on extended reality provided by the embodiment of the present invention, the target display content can be determined according to the multiple physiological data of the user and / or the multiple physiological data of other users in the team to which the user belongs, so as to adjust the current display content of the virtual scene of the display component according to the target display content and / or the confirmation instruction sent by the user based on the target display content. Therefore, the multimodal physiological data acquisition device based on extended reality adopts dry electrodes and can collect multiple physiological data, which solves the problems of the short use time of wet electrodes in the related art and the unilateral collection of EEG signals. It is easy to use and operate, and improves the operational flexibility of the multimodal physiological data acquisition device based on extended reality.

[0080] Next, a method for using the multimodal physiological data acquisition device based on augmented reality according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0081] Figure 4 The present invention is a flowchart of a method for using a multimodal physiological data acquisition device based on extended reality according to an embodiment of the present invention.

[0082] In this embodiment, the method for using the multimodal physiological data acquisition device based on extended reality adopts Figure 1 The multimodal physiological data acquisition device based on extended reality shown in the embodiment.

[0083] Specifically, if Figure 4 As shown, the method for using the multimodal physiological data acquisition device based on extended reality includes the following steps:

[0084] In step S401, multiple physiological data of the user are collected;

[0085] In step S402, a scene display strategy of a virtual scene is determined according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, and a target display content is determined according to the scene display strategy;

[0086] In step S403, the current display content of the virtual scene of the display component is adjusted according to the target display content and / or a confirmation instruction sent by the user based on the target display content.

[0087] Furthermore, in some embodiments, after collecting multiple physiological data of the user, the method further includes:

[0088] Analyze multiple physiological data of the user to generate a health report and / or health suggestions for the user.

[0089] It should be noted that the aforementioned explanation of the embodiment of the multimodal physiological data acquisition device based on extended reality is also applicable to the method for using the multimodal physiological data acquisition device based on extended reality in this embodiment, which will not be elaborated here.

[0090] According to the method for using the multimodal physiological data acquisition device based on extended reality provided by the embodiment of the present invention, the target display content can be determined according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, so as to adjust the current display content of the virtual scene of the display component according to the target display content and / or the confirmation instruction sent by the user based on the target display content. Thus, the problems in the related art that the wet electrode has a short use time and can only collect EEG signals unilaterally are solved, and the operation is simple and convenient, which improves the operational flexibility of the multimodal physiological data acquisition device based on extended reality.

[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0092] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0093] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0094] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0095] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. A multimodal physiological data acquisition device based on extended reality, characterized in that: include: A display component for displaying a virtual scene; A collection component, comprising a dry electrode module and a sensor integration module, for collecting multiple physiological data of the user; a processing component, configured to determine a scene display strategy of the virtual scene according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, and determine target display content according to the scene display strategy; The control component is used to adjust the current display content of the virtual scene of the display component according to the target display content and / or a confirmation instruction sent by the user based on the target display content.

2. The multimodal physiological data acquisition device based on extended reality according to claim 1, characterized in that: The dry electrode module includes a plurality of dry electrodes arranged according to a preset arrangement rule. When the user wears the multimodal physiological data acquisition device based on extended reality, the plurality of dry electrodes are in contact with the forehead scalp area of ​​the user to collect the user's electroencephalogram signals through the plurality of dry electrodes.

3. The multimodal physiological data acquisition device based on extended reality according to claim 1, characterized in that: Processing components, including: A first analyzing unit, configured to determine a current emotional state of the user based on a plurality of physiological data of the user; A matching unit is used to match the virtual scene with the highest matching degree with the current emotional state from a preset scene library as the target display content, or to determine the target display content based on the current display content of the virtual scene and the current emotional state.

4. The multimodal physiological data acquisition device based on extended reality according to any one of claims 1 to 3, characterized in that: The display content of the virtual scene includes at least one of display content corresponding to a group training scene, display content corresponding to a difficulty level and a rhythm level of a game, and display content corresponding to a relaxation scene under different emotions.

5. The multimodal physiological data acquisition device based on extended reality according to claim 1 or 3, characterized in that: The processing component is also used to: The second analyzing unit is used to analyze the multiple physiological data of the user to generate a health report and / or health advice for the user.

6. The multimodal physiological data acquisition device based on extended reality according to claim 1, characterized in that: The control component is also used for: Controlling the display component to display multiple physiological data and / or health reports and / or health suggestions of the user; And / or, performing physiological interaction based on the user's interactive actions.

7. The multimodal physiological data acquisition device based on extended reality according to claim 1, characterized in that: Also includes: an amplifying component, the amplifying component is used to amplify the multiple physiological data and then input them into the processing component; A storage component, the storage component is used to store multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs offline based on a preset storage duration; The encryption component is used to encrypt multiple physiological data of the user.

8. The multimodal physiological data acquisition device based on extended reality according to claim 1, characterized in that: The multiple physiological data include: at least two of EEG, fNIRS, EDA, PPG, RESP, FSR and SKT.

9. A method for using a multimodal physiological data acquisition device based on extended reality, characterized in that: The method adopts the multimodal physiological data acquisition device based on extended reality as described in any one of claims 1 to 8, wherein the method comprises the following steps: Collect multiple physiological data of users; Determining a scene display strategy of the virtual scene according to multiple physiological data of the user and / or multiple physiological data of other users in the team to which the user belongs, and determining target display content according to the scene display strategy; According to the target display content and / or a confirmation instruction sent by the user based on the target display content, the current display content of the virtual scene of the display component is adjusted.

10. The method according to claim 9, characterized in that After collecting multiple physiological data of the user, it also includes: Analyze the multiple physiological data of the user to generate a health report and / or health suggestions for the user.

Images