Intelligent cabin system based on multimode interaction and virtual reality technology

Abstract

The invention relates to the technical field of intelligent cabins, and discloses an intelligent cabin system based on multimode interaction and virtual reality technology, which comprises a data acquisition unit, the output end of the data acquisition unit is in signal connection with a data analysis unit, and the output end of the data analysis unit is in signal connection with a data control center. The output end of the data control center is in signal connection with an environment data correction unit, the output end of the data control center is in signal connection with a health monitoring unit, the output end of the data control center is in signal connection with an AR processing unit, and the output end of the AR processing unit is in signal connection with a cabin optimization unit. The radar module is used for acquiring data such as the peripheral position of a vehicle, the navigation recording module is used for recording data of the using position and the common position of the vehicle, the analysis and recognition module is used for recognizing and analyzing gestures, expressions and the like of a user, and the interaction module is used for carrying out serial interaction on recognized information and the display module and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of smart cockpits, and in particular to a smart cockpit system based on multi-mode interaction and virtual reality technology. Background Art

[0002] With the increasing popularity of automobiles, the application of related human-computer interaction technologies such as artificial intelligence is becoming more and more widespread. However, the daily long-distance commuting and self-driving ratio continues to rise, and new technologies are urgently needed to improve the driving experience while ensuring driving safety. The combination of multi-modal interaction technologies such as voice, vision, and acoustics has become the information sensing and processing method of the current mainstream interactive system, and the intelligent cockpit system has also emerged as a result.

[0003] In the prior art, with the increasing development of social networks, the performance of in-vehicle entertainment system chips has been continuously improved, and more and more applications and streaming media can be seamlessly applied to the cockpit. The rich and diverse pan-entertainment system requires efficient and mobile safe storage and effective use of the explosive user data. At this time, the smart cockpit has evolved into a data generator and processing center, and has also become an important interface for other operating terminals. The driver status monitoring system (DMS) based on the prior art can solve the safety problems of some drivers during driving, such as fatigue monitoring (yawning, eye closing times and duration, etc.), dangerous behavior monitoring (eating, calling, smoking, chatting in the back seat, etc.) At the same time, the passenger monitoring system (OMS) can also detect the private status of passengers, such as helping to determine whether the passengers in the car are safely seated, whether there are children or pets left alone, and whether the passengers are wearing seat belts. However, these technologies are relatively traditional and one-sided, and they only consider regularity and do not make more improvements in the driving experience. Therefore, the entire smart cockpit system still has room for improvement and improvement. Therefore, it is necessary to invent a smart cockpit system based on multi-modal interaction and virtual reality technology to solve the above problems. Summary of the invention

[0004] The purpose of the present invention is to provide an intelligent cockpit system based on multi-modal interaction and virtual reality technology to solve the problems raised in the above background technology.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent cockpit system based on multi-modal interaction and virtual reality technology, comprising a data acquisition unit, the output end signal of the data acquisition unit is connected to a data analysis unit, the output end signal of the data analysis unit is connected to a data control center, the output end signal of the data control center is connected to an environmental data correction unit, the output end signal of the data control center is connected to a health monitoring unit, the output end signal of the data control center is connected to an AR processing unit, and the output end signal of the AR processing unit is connected to a cockpit optimization unit; The output signal of the environmental data correction unit is connected to the radar module, the output signal of the radar module is connected to the navigation recording module, the output signal of the navigation recording module is connected to the data acquisition module, and the output signal of the data acquisition module is connected to the data upload module.

[0006] Preferably, the data acquisition unit includes a detector monitoring module, the output signal of the detector monitoring module is connected to a microphone monitoring module, the output signal of the microphone monitoring module is connected to an image acquisition module, the output signal of the image acquisition module is connected to an information recognition module, and the output signal of the information recognition module is connected to a data preprocessing module, so as to facilitate real-time monitoring of the interior of the vehicle.

[0007] Preferably, the data analysis unit includes a data receiving module, the output end signal of the data receiving module is connected to the data analysis module, the output end signal of the data analysis module is connected to the data filtering module, and the output end signal of the data filtering module is connected to the data comparison module, so as to facilitate the analysis of the received data.

[0008] Preferably, the output end signal of the data comparison module is connected to a data control center, and the data control center is connected to a cloud database to facilitate real-time transmission of data.

[0009] Preferably, the health monitoring unit includes a heartbeat monitoring module, the output signal of the heartbeat monitoring module is connected to the respiratory monitoring module, the output signal of the respiratory monitoring module is connected to the data conversion module, and the output signal of the data conversion module is connected to the emergency docking module, so as to facilitate the real-time monitoring of the user.

[0010] Preferably, the output end of the emergency docking module is connected to the data control center signal to facilitate data conversion.

[0011] Preferably, the AR processing unit includes an analysis and identification module, the output signal of the analysis and identification module is connected to an interaction module, the output signal of the interaction module is connected to a game control module, and the output signal of the game control module is connected to an AR enhancement module, so as to facilitate passengers' entertainment use.

[0012] Preferably, the cockpit optimization unit includes a display module, the output end signal of the display module is connected to the adjustment module, the output end signal of the adjustment module is connected to the vibration feedback module, and the output end signal of the vibration feedback module is connected to the RGB atmosphere light module, so as to enhance the user experience.

[0013] Compared with the prior art, the present invention provides an intelligent cockpit system based on multi-modal interaction and virtual reality technology, which has the following beneficial effects: The intelligent cockpit system based on multi-mode interaction and virtual reality technology monitors the real-time situation inside the car through the set detector monitoring module, monitors the instructions issued by users at different positions inside the car at all times through the set microphone monitoring module and image acquisition module, cooperates with the information recognition module to recognize the collected data, and pre-processes the recognized data through the set data preprocessing module, receives the stored data through the set data receiving module, analyzes the received data through the set data analysis module, filters the interference data such as noise in the received data through the set data filtering module, and compares the real-time data of the user with the internal data of the data control center through the set data comparison module, so as to realize the real-time uploading and comparison of the data in the intelligent cockpit system to meet the user's use needs while storing and memorizing the user's commonly used functions.

[0014] The intelligent cockpit system based on multi-mode interaction and virtual reality technology acquires data such as the vehicle's surrounding location through a radar module, records the vehicle's usage location and common location data through a navigation recording module, uploads the screened data through a data acquisition module in conjunction with a data upload module, reduces false alarms and the like, and increases monitoring accuracy. The heartbeat monitoring module is used in conjunction with a breathing monitoring module to monitor and upload the user's usage process at all times, compares the data with the data control center through a data conversion module, and cooperates with an emergency docking module to achieve real-time transmission and monitoring of data in the intelligent cockpit system, thereby increasing user safety protection.

[0015] The intelligent cockpit system based on multi-modal interaction and virtual reality technology recognizes and analyzes the user's gestures and expressions through the set analysis and recognition module, interacts in series with the recognized information and the display module through the set interaction module, controls the recognized information and entertainment content through the set game control module, integrates the computer-generated virtual environment with the real environment around the user through the set AR enhancement module with the help of optoelectronic display technology, interaction technology, multiple sensor technology and computer graphics and multimedia technology to increase the user's actual sensory effect, adjusts the display module and the cockpit body to meet the needs of users of different body shapes through the set adjustment module, interacts with the internal entertainment content and music through the set vibration feedback module, and makes the cockpit body vibrate to meet the needs of the user, and changes the color of the atmosphere light as a whole through the set RGB atmosphere light module to adapt to different modes and usage scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative labor: Figure 1 It is a schematic diagram of the overall system of the present invention; Figure 2 It is a schematic diagram of a data acquisition unit of the present invention; Figure 3 It is a schematic diagram of a data analysis unit of the present invention; Figure 4 This is a schematic diagram of the environmental data correction unit of the present invention; Figure 5 This is a schematic diagram of a health monitoring unit of the present invention; Figure 6 It is a schematic diagram of the AR processing unit of the present invention; Figure 7 It is a schematic diagram of the cockpit optimization unit of the present invention.

[0017] In the figure: 1. Data acquisition unit; 11. Detector monitoring module; 12. Microphone monitoring module; 13. Image acquisition module; 14. Information recognition module; 15. Data preprocessing module; 2. Data analysis unit; 21. Data receiving module; 22. Data analysis module; 23. Data filtering module; 24. Data comparison module; 3. Data control center; 4. Environmental data correction unit; 41. Radar module; 42. Navigation recording module; 43. Data acquisition module; 44. Data upload module; 5. Health monitoring unit; 51. Heartbeat monitoring module; 52. Respiration monitoring module; 53. Data conversion module; 54. Emergency docking module; 6. AR processing unit; 61. Analysis and recognition module; 62. Interaction module; 63. Game control module; 64. AR enhancement module; 7. Cockpit optimization unit; 71. Display module; 72. Adjustment module; 73. Vibration feedback module; 74. RGB atmosphere light module. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] See also Figure 1-7 The present invention provides a technical solution: an intelligent cockpit system based on multi-mode interaction and virtual reality technology, comprising a data acquisition unit 1, the output end signal of the data acquisition unit 1 is connected to a data analysis unit 2, the output end signal of the data analysis unit 2 is connected to a data control center 3, the output end signal of the data control center 3 is connected to an environmental data correction unit 4, the output end signal of the data control center 3 is connected to a health monitoring unit 5, the output end signal of the data control center 3 is connected to an AR processing unit 6, and the output end signal of the AR processing unit 6 is connected to a cockpit optimization unit 7; The output signal of the environmental data correction unit 4 is connected to the radar module 41, the output signal of the radar module 41 is connected to the navigation recording module 42, the output signal of the navigation recording module 42 is connected to the data acquisition module 43, and the output signal of the data acquisition module 43 is connected to the data upload module 44.

[0021] Furthermore, the data acquisition unit 1 includes a detector monitoring module 11, the output signal of the detector monitoring module 11 is connected to the microphone monitoring module 12, the output signal of the microphone monitoring module 12 is connected to the image acquisition module 13, the output signal of the image acquisition module 13 is connected to the information recognition module 14, and the output signal of the information recognition module 14 is connected to the data preprocessing module 15, which is convenient for real-time monitoring of the interior of the vehicle.

[0022] Furthermore, the data analysis unit 2 includes a data receiving module 21, the output end signal of the data receiving module 21 is connected to the data analysis module 22, the output end signal of the data analysis module 22 is connected to the data filtering module 23, and the output end signal of the data filtering module 23 is connected to the data comparison module 24, so as to facilitate the analysis of the received data.

[0023] Furthermore, the output signal of the data comparison module 24 is connected to the data control center 3, and the data control center 3 is connected to the cloud database to facilitate real-time data transmission.

[0024] Furthermore, the health monitoring unit 5 includes a heartbeat monitoring module 51, the output signal of the heartbeat monitoring module 51 is connected to the respiratory monitoring module 52, the output signal of the respiratory monitoring module 52 is connected to the data conversion module 53, and the output signal of the data conversion module 53 is connected to the emergency docking module 54, so as to facilitate the real-time monitoring of the user.

[0025] Furthermore, the output end of the emergency docking module 54 is connected to the data control center 3 for signal conversion.

[0026] Furthermore, the AR processing unit 6 includes an analysis and identification module 61, the output signal of the analysis and identification module 61 is connected to the interaction module 62, the output signal of the interaction module 62 is connected to the game control module 63, and the output signal of the game control module 63 is connected to the AR enhancement module 64, so as to facilitate passengers' entertainment use.

[0027] Furthermore, the cockpit optimization unit 7 includes a display module 71, the output end signal of the display module 71 is connected to the adjustment module 72, the output end signal of the adjustment module 72 is connected to the vibration feedback module 73, and the output end signal of the vibration feedback module 73 is connected to the RGB atmosphere light module 74, so as to enhance the user experience.

[0028] In actual operation, when this device is used, the real-time situation inside the car is first monitored by the detector monitoring module 11, and the instructions issued by users at different positions inside the car are constantly monitored by the microphone monitoring module 12 and the image acquisition module 13. The information recognition module 14 is used to recognize the collected data, and the recognized data is preprocessed by the data preprocessing module 15. The stored data is received by the data receiving module 21, and the received data is analyzed by the data analysis module 22. The noise and other interference data in the received data are filtered out by the data filtering module 23. The real-time data of the user is compared with the internal data of the data control center 3 by the data comparison module 24, so as to realize the real-time uploading and comparison of the data in the smart cockpit system to meet the user's use needs and store and remember the user's commonly used functions. The radar module 41 is used to obtain data such as the vehicle's surrounding location. The navigation recording module 42 is used to record the vehicle's usage location and common location data. The data acquisition module 43 is used to upload the filtered data in conjunction with the data upload module 44 to reduce false alarms and increase monitoring accuracy. The heartbeat monitoring module 51 is used in conjunction with the breathing monitoring module 52 to monitor and upload the user's usage process at all times. The data is compared with the data control center 3 in conjunction with the data conversion module 53. In conjunction with the emergency docking module 54, real-time transmission and monitoring of data in the smart cockpit system is achieved to increase user safety protection. When the user needs to use it for entertainment, the analysis and recognition module 61 is set to recognize and analyze the user's gestures and expressions, etc., and the interactive module 62 is set to interact in series with the recognized information and the display module 71, etc., and the game control module 63 is set to control the recognized information and entertainment content. The AR enhancement module 64 is set to integrate the computer-generated virtual environment with the real environment around the user with the help of optoelectronic display technology, interactive technology, multiple sensor technology, computer graphics and multimedia technology to increase the user's actual sensory effect, and the display module 71 and the cockpit body are adjusted to meet the needs of users of different body shapes through the adjustment module 72. The internal entertainment content and music are interacted through the vibration feedback module 73, and the cockpit body vibrates to meet the needs of the user. The RGB atmosphere light module 74 is set to change the color of the atmosphere light as a whole, which can adapt to different modes and usage scenarios.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. An intelligent cockpit system based on multi-modal interaction and virtual reality technology, comprising a data acquisition unit (1), characterized in that: The output end signal of the data acquisition unit (1) is connected to the data analysis unit (2), the output end signal of the data analysis unit (2) is connected to the data control center (3), the output end signal of the data control center (3) is connected to the environmental data correction unit (4), the output end signal of the data control center (3) is connected to the health monitoring unit (5), the output end signal of the data control center (3) is connected to the AR processing unit (6), and the output end signal of the AR processing unit (6) is connected to the cabin optimization unit (7); The output end signal of the environmental data correction unit (4) is connected to a radar module (41), the output end signal of the radar module (41) is connected to a navigation recording module (42), the output end signal of the navigation recording module (42) is connected to a data acquisition module (43), and the output end signal of the data acquisition module (43) is connected to a data upload module (44).

2. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 1, characterized in that: The data acquisition unit (1) comprises a detector monitoring module (11), the output end signal of the detector monitoring module (11) is connected to a microphone monitoring module (12), the output end signal of the microphone monitoring module (12) is connected to an image acquisition module (13), the output end signal of the image acquisition module (13) is connected to an information recognition module (14), and the output end signal of the information recognition module (14) is connected to a data preprocessing module (15).

3. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 1, characterized in that: The data analysis unit (2) comprises a data receiving module (21), the output end signal of the data receiving module (21) is connected to a data analysis module (22), the output end signal of the data analysis module (22) is connected to a data filtering module (23), and the output end signal of the data filtering module (23) is connected to a data comparison module (24).

4. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 3 is characterized in that: The output end signal of the data comparison module (24) is connected to a data control center (3), and the data control center (3) is connected to a cloud database.

5. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 1, characterized in that: The health monitoring unit (5) comprises a heartbeat monitoring module (51), the output end signal of the heartbeat monitoring module (51) is connected to the respiratory monitoring module (52), the output end signal of the respiratory monitoring module (52) is connected to the data conversion module (53), and the output end signal of the data conversion module (53) is connected to the emergency docking module (54).

6. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 5, characterized in that: The output end of the emergency docking module (54) is signal-connected to the data control center (3).

7. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 1, characterized in that: The AR processing unit (6) comprises an analysis and identification module (61), the output end signal of the analysis and identification module (61) is connected to an interaction module (62), the output end signal of the interaction module (62) is connected to a game control module (63), and the output end signal of the game control module (63) is connected to an AR enhancement module (64).

8. The intelligent cockpit system based on multi-modal interaction and virtual reality technology according to claim 1, characterized in that: The cockpit optimization unit (7) comprises a display module (71), the output end signal of the display module (71) is connected to a regulating module (72), the output end signal of the regulating module (72) is connected to a vibration feedback module (73), and the output end signal of the vibration feedback module (73) is connected to an RGB ambient light module (74).

Images