Artificial intelligence interaction device

Abstract

The invention relates to the technical field of artificial intelligence interaction, and discloses an artificial intelligence interaction device, which comprises an information acquisition unit, a data processing unit, a somatosensory feedback unit, a visual feedback unit and an audio feedback unit, and is characterized in that the information acquisition unit comprises a high-precision motion capture sensor, a voice recognition sensor and an environment sensor; the information collection unit is an information inlet of the device and is responsible for collecting multi-dimensional information of a user in the interaction process, and the multi-dimensional information of the user in the interaction process is obtained through the arranged somatosensory feedback unit and the touch sense by means of the synergistic effect of a micro vibration motor array, a flexible pressure sensor and a temperature adjusting element. The touch sense of various materials, the counter-acting force of an object, the temperature change and the touch sense evolution in continuous actions can be accurately simulated, and in combination with an environment sensor, a user can really sense the physical characteristics of the object in the virtual world in the interaction process, so that the interaction experience is more real, fine and natural.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence interaction technology, and in particular to an artificial intelligence interaction device. Background Art

[0002] Artificial intelligence interaction breaks through the limitations of traditional interaction technology through multimodal information fusion and precise feedback mechanism, greatly enhancing the user's real experience in various interactive scenarios. It is widely used in virtual reality (VR), augmented reality (AR), mixed reality (MR), smart games, immersive education, telemedicine, industrial simulation and other fields.

[0003] At present, traditional interactive interfaces mainly rely on screens (ordinary screens and VR screens) and voice commands to achieve interactive operations. Taking VR applications as an example, when simulating daily activities such as holding an umbrella and picking up objects, the user experience has obvious defects. When simulating holding an umbrella, it is impossible to accurately perceive the changes in impact force corresponding to the size, speed, and density of raindrops. The texture of the umbrella handle material (such as the rough texture of wood and the cold and smooth feeling of metal), the grip friction, and the changes in the opening and closing tension of the umbrella surface are difficult to convey truthfully. When simulating picking up objects, the key tactile information such as the shape details of the object (such as the curvature of a sphere, the edges and corners of a cube), surface roughness (from sandpaper-like roughness to silky smoothness), weight distribution (difference in center of gravity position), elastic deformation and reaction force at the moment of picking up, etc. are seriously missing. This lack of somatosensory feedback greatly reduces the user's immersion and sense of reality in the virtual environment.

[0004] In order to solve the above problems, this application proposes an artificial intelligence interaction device. Summary of the invention

[0005] The purpose of the present invention is to provide an artificial intelligence interaction device to solve the shortcomings of the somatosensory feedback in the prior art proposed in the above background technology, thereby greatly reducing the problems of user immersion and sense of reality in the virtual environment.

[0006] To achieve the above-mentioned object, the present invention provides the following technical solutions: an artificial intelligence interaction device, comprising an information collection unit, a data processing unit, a somatosensory feedback unit, a visual feedback unit and an audio feedback unit;

[0007] The information collection unit includes a high-precision motion capture sensor, a voice recognition sensor and an environmental sensor. The information collection unit is the information entrance of the device and is responsible for collecting multi-dimensional information of the user during the interaction process;

[0008] The data processing unit is closely connected with the information collection unit to receive and process the massive amount of information collected by it;

[0009] The somatosensory feedback unit includes a high-density distributed micro-vibration motor array, a flexible pressure sensor and a temperature adjustment element. The micro-vibration motor array adopts a high-density distribution design and is distributed throughout the parts where the user may interact with the device. The flexible pressure sensor is based on the piezoresistive or capacitive sensing principle and has high sensitivity and fast response characteristics. It is integrated on the contact surface of the interactive device and can sense the pressure applied by the user, the pressure distribution and the pressure change rate in real time. The temperature adjustment element adopts a semiconductor cooling and heating module based on the Peltier effect principle, which can quickly achieve temperature adjustment.

[0010] The visual feedback unit includes an ultra-high-definition resolution display screen, a rendering module and a stereoscopic display module. The display screen adopts micro-light-emitting diode display technology. The rendering module uses physically based rendering technology and ray tracing algorithms to realistically simulate the optical phenomena of reflection, refraction and scattering of light on the surfaces of objects of different materials, as well as the spatial relationship of shadows and occlusions of objects. The stereoscopic display module adopts time-division stereoscopic display combined with head tracking technology to provide users with an immersive stereoscopic visual experience.

[0011] The audio feedback unit includes a 7.1-channel audio system and an audio synthesis and special effects engine. It uses audio spatialization technology to calculate the sound propagation path, attenuation effect and acoustic characteristics of reflected echoes in real time according to the user's position in the virtual scene and the positional relationship of the interactive objects. The audio synthesis and special effects engine can generate various realistic environmental sound effects and interactive sound effects according to the requirements of the interactive scene.

[0012] Furthermore, the high-precision motion capture sensor adopts advanced optical tracking and inertial measurement unit fusion technology, which can accurately capture the subtle movements of various parts of the user's body. The sampling frequency of the high-precision motion capture sensor is 1000-2500 Hz, ensuring the real-time and accuracy of the motion information;

[0013] The speech recognition sensor uses a deep learning speech recognition algorithm and has a strong ability to resist environmental noise interference. It can accurately recognize the user's voice commands under different environmental noise backgrounds. The speech recognition accuracy rate exceeds 97%, and it can accurately analyze the user's semantic intentions.

[0014] The environmental sensors include light sensors, temperature sensors, humidity sensors and spatial positioning sensors. The light sensor can accurately measure the intensity of ambient light and color temperature parameters, providing a basis for scene lighting adaptation for the visual feedback unit; the temperature sensor and humidity sensor can monitor the temperature and humidity changes in the user's environment in real time, and provide key data when simulating specific climate scenes or interactive experiences related to temperature and humidity; the spatial positioning sensor uses a combination of satellite positioning and indoor positioning technology to accurately determine the user's location information globally and in specific indoor spaces, providing basic data support for building personalized interactive scenes.

[0015] Furthermore, with respect to motion information, the data processing unit performs real-time analysis on the continuous motion data stream transmitted from the motion capture sensor through a well-trained neural network model. With respect to voice information, the data processing unit converts the voice signal into text information through a speech recognition algorithm, and further performs semantic understanding and intent analysis through natural language processing technology. Combined with environmental information, the data processing unit can dynamically adjust the rendering parameters of the interactive scene, the intensity of somatosensory feedback, and the voice interaction logic according to factors such as light, temperature, humidity, and spatial position.

[0016] Furthermore, the micro vibration motors are all equipped with independent control circuits, which can accurately simulate vibration modes of different frequencies, amplitudes and waveforms according to the instructions transmitted by the data processing unit. When simulating the touch of the surface of objects of different materials, the vibration parameters of the vibration motors are adjusted so that the user's hands or body parts can feel the corresponding texture differences.

[0017] When the user performs a grabbing action in the virtual environment, the flexible pressure sensor feeds back pressure data to the data processing unit, and the data processing unit controls the micro vibration motor array to generate corresponding reaction force vibration according to the weight and hardness properties of the virtual object;

[0018] When the temperature regulating element simulates an interactive scenario related to temperature, the temperature regulating element adjusts the temperature of the part in contact with the user to the set value in a very short time according to the instruction of the data processing unit; at the same time, in order to avoid discomfort or injury to the user caused by excessively high or low temperature, the temperature regulating element also includes a temperature monitoring and safety protection mechanism to ensure that the temperature is always controlled within a safe and comfortable range for the human body.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention provides a somatosensory feedback unit, which, with the help of the synergistic effect of a micro-vibration motor array, a flexible pressure sensor and a temperature regulating element, can accurately simulate the touch of various materials, the reaction force of objects, temperature changes and the evolution of touch in continuous actions. In addition, combined with environmental sensors, users can truly feel the physical properties of objects in the virtual world during the interaction process, making the interactive experience more realistic, delicate and natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of the architecture of an artificial intelligence interaction device of the present invention;

[0022] Figure 2 This is a somatosensory feedback flow chart of an artificial intelligence interaction device of the present invention;

[0023] Figure 3 A schematic diagram of the installation of a micro-vibration motor of an artificial intelligence interaction device of the present invention;

[0024] In the figure:

[0025] 1. Information collection unit; 11. High-precision motion capture sensor; 12. Voice recognition sensor; 13. Environmental sensor; 131. Light sensor; 132. Temperature sensor; 133. Humidity sensor; 134. Spatial positioning sensor;

[0026] 2. Data processing unit;

[0027] 3. Somatosensory feedback unit; 31. Micro vibration motor; 32. Flexible pressure sensor; 33. Temperature regulating element;

[0028] 4. Visual feedback unit;

[0029] 5. Audio feedback unit;

[0030] 6. Handle. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0032] The standard parts used in this application can all be purchased from the market, and special-shaped parts can be customized according to the instructions and drawings. The specific connection methods of each part adopt conventional means such as mature bolts, rivets, welding, and bonding in the existing technology, and the components used for circuit connection are all conventional models in the existing technology.

[0033] At the same time, in order to clearly express the connection relationship and working principle between the components and highlight the key points, the drawings in the specification are organized and drawn in the form of simple diagrams. One simple diagram can correspond to a variety of materials and actual external structural shapes.

[0034] See also Figure 1-Figure 3 , the present invention provides a technical solution: an artificial intelligence interaction device, comprising an information collection unit 1, a data processing unit 2, a somatosensory feedback unit 3, a visual feedback unit 4 and an audio feedback unit 5;

[0035] The information collection unit 1 includes a high-precision motion capture sensor 11, a voice recognition sensor 12 and an environmental sensor 13. The information collection unit 1 is the information entry of the device and is responsible for collecting multi-dimensional information of the user during the interaction process;

[0036] The data processing unit 2 is closely connected to the information collection unit 1 to receive and process the massive amount of information collected by it;

[0037] The somatosensory feedback unit 3 includes a high-density distributed array of micro-vibration motors 31, a flexible pressure sensor 32 and a temperature adjustment element 33. The micro-vibration motor 31 array adopts a high-density distribution design and is distributed throughout the parts where the user may interact with the device. The flexible pressure sensor 32 is based on the piezoresistive or capacitive sensing principle and has high sensitivity and fast response characteristics. It is integrated on the contact surface of the interactive device and can sense the pressure applied by the user, the pressure distribution and the pressure change rate in real time. The temperature adjustment element 33 adopts a semiconductor cooling and heating module based on the Peltier effect principle, which can quickly achieve temperature adjustment.

[0038] The visual feedback unit 4 includes an ultra-high-definition resolution display screen, a rendering module and a stereoscopic display module. The display screen adopts micro-light-emitting diode display technology. The rendering module uses physically based rendering technology and ray tracing algorithm to realistically simulate the optical phenomena of reflection, refraction and scattering of light on the surfaces of objects of different materials, as well as the spatial relationship of shadows and occlusions of objects. The stereoscopic display module adopts time-division stereoscopic display combined with head tracking technology to provide users with an immersive stereoscopic visual experience.

[0039] The audio feedback unit 5 includes a 7.1-channel audio system and an audio synthesis and special effects engine. It uses audio spatialization technology to calculate the sound propagation path, attenuation effect and acoustic characteristics of reflected echoes in real time according to the user's position in the virtual scene and the positional relationship of the interactive objects. The audio synthesis and special effects engine can generate various realistic environmental sound effects and interactive sound effects according to the requirements of the interactive scene.

[0040] The high-precision motion capture sensor 11 adopts advanced optical tracking and inertial measurement unit fusion technology, which can accurately capture the subtle movements of various parts of the user's body. The sampling frequency of the high-precision motion capture sensor 11 is 1000-2500 Hz, ensuring the real-time and accuracy of the motion information;

[0041] The speech recognition sensor 12 uses a deep learning speech recognition algorithm and has a strong ability to resist environmental noise interference. It can accurately recognize the user's voice commands under different environmental noise backgrounds. The speech recognition accuracy rate exceeds 97%, and it can accurately analyze the user's semantic intentions.

[0042] The environmental sensor 13 includes a light sensor 131, a temperature sensor 132, a humidity sensor 133 and a spatial positioning sensor 134. The light sensor 131 can accurately measure the intensity of the ambient light and the color temperature parameters, and provide a scene lighting adaptation basis for the visual feedback unit 4. The light sensor 131 adopts high dynamic range imaging and spectral analysis technology to accurately measure the ambient light intensity (ranging from 0.001 lux to 200,000 lux), color temperature (accuracy within 5K) and spectral distribution and other parameters; the temperature sensor 132 and the humidity sensor 133 can monitor the temperature and humidity changes of the user's environment in real time, and provide a certain climate scene or interactive experience related to temperature and humidity. Provide key data; the spatial positioning sensor 134 uses a combination of satellite positioning and indoor positioning technology to accurately determine the user's location information globally and in specific indoor spaces, providing basic data support for building personalized interaction scenarios. The spatial positioning sensor 134 uses satellite positioning (multi-star system integration such as GPS, Beidou, GLONASS, etc.) and indoor positioning (Wi-Fi fingerprint positioning, Bluetooth low energy positioning, ultra-wideband high-precision positioning, geomagnetic positioning and other multi-technology collaboration) to achieve accurate positioning of user positions in complex global and indoor spatial environments. The outdoor positioning accuracy can reach within 0.5 meters, and the indoor positioning accuracy can reach within 0.05 meters, providing basic data basis for building personalized interaction scenarios.

[0043] It should be understood that: the environmental sensor 13 also includes a gas sensor and a biosensor to build a comprehensive environmental perception system. The gas sensor can detect various harmful gas components and odor concentration information in the air, and plays an important role in specific interactive scenarios (such as fire escape simulation, chemical experiment simulation, etc.); the biosensor can monitor the user's physiological indicators such as heart rate, blood pressure, skin conductivity, etc., and provide data reference for personalized adjustment of the interactive experience (such as adjusting the interaction difficulty or feedback intensity according to the user's tension level).

[0044] The data processing unit 2 targets the motion information. The data processing unit 2 performs real-time analysis on the continuous motion data stream transmitted by the motion capture sensor through a well-trained neural network model. For the voice information, the data processing unit 2 converts the voice signal into text information using a voice recognition algorithm, and then further performs semantic understanding and intent analysis through natural language processing technology. Combined with the environmental information, the data processing unit 2 uses a reinforcement learning algorithm to dynamically adjust the rendering parameters, somatosensory feedback intensity and voice interaction logic of the interactive scene according to factors such as light, temperature, humidity and spatial position. When the light is dim, the brightness and contrast of the display screen of the visual feedback module are automatically increased, and the picture color is adjusted according to the light spectrum to ensure clear and comfortable vision. In the cold environment simulation, the temperature drop perception effect of the somatosensory feedback module is enhanced, and the timbre and speech speed of the voice interaction are adjusted to create a cold atmosphere.

[0045] It should be noted that the data processing unit 2 includes a high-performance graphics processor, a tensor processor and / or a field programmable gate array chip optimized for deep learning, and runs a neural network algorithm architecture based on deep learning, including a convolutional neural network, a recurrent neural network, a long short-term memory network, an attention mechanism, and a generative adversarial network architecture that integrates and optimizes multiple models.

[0046] The micro vibration motors 31 are all equipped with independent control circuits. The micro vibration motors 31 are carefully arranged in key areas where the user may come into contact with the device, such as the handle 6, gloves, seat backs and cushions, headbands of head-mounted devices and face contact areas, and can accurately simulate vibration modes of different frequencies (1Hz-2000Hz), amplitudes (0.005mm-20mm) and waveforms (sine waves, square waves, triangle waves, sawtooth waves and custom complex waveforms) according to the instructions of the data processing unit 2; when simulating the touch of the surfaces of objects of different materials, the vibration parameters of the vibration motor are adjusted so that the user's hands or body parts can feel the corresponding texture differences.

[0047] When the user performs a grasping action in the virtual environment, the flexible pressure sensor 32 feeds back the pressure data to the data processing unit 2. The data processing unit 2 controls the micro vibration motor 31 array to generate corresponding reaction force vibration according to the weight and hardness properties of the virtual object. The flexible pressure sensor 32 collects dust on the contact surface of the interactive device, such as the handle grip, glove fingertips, seat surface, etc., and accurately senses the pressure size, pressure distribution, pressure change rate and other information applied by the user in real time. When the user performs a grasping action in the virtual environment, the flexible pressure sensor 32 feeds back the pressure data at a high speed. The data is fed to the data processing unit 2, which controls the micro-vibration motor 31 array to generate corresponding reaction force vibration according to the physical properties of the virtual object, such as weight, hardness, elasticity, and viscosity, and plays realistic object collision and squeezing sound effects through the audio feedback module, so that the user can feel the real grasping experience. For example, when simulating the grasping of viscous colloid, as the user's hand pressure changes, the flexible pressure sensor 32 feeds back data, and the data processing unit 2 instructs the micro-vibration motor 31 array to generate corresponding viscous resistance vibration, and the audio feedback module plays the sound of colloid stretching and separation, and the sound frequency and volume are dynamically adjusted with the pressure change.

[0048] When the temperature regulating element 33 simulates an interactive scenario related to temperature, the temperature regulating element 33 adjusts the temperature of the part in contact with the user to the set value in a very short time according to the instruction of the data processing unit 2; at the same time, in order to avoid discomfort or injury to the user caused by excessively high or low temperature, the temperature regulating element 33 also includes a temperature monitoring and safety protection mechanism, including a redundant temperature sensor 132 design, an over-temperature protection circuit, an intelligent temperature early warning system and emergency cooling measures, to ensure that the temperature is always controlled within a safe and comfortable range for the human body, and to take effective protection measures in time when the temperature is abnormal, such as automatically cutting off the power supply of the temperature regulating element 33, starting an emergency cooling program, and issuing a high-decibel alarm.

Claims

1. An artificial intelligence interactive device, characterized in that: It includes an information collection unit, a data processing unit, a somatosensory feedback unit, a visual feedback unit and an audio feedback unit; The information collection unit includes a high-precision motion capture sensor, a voice recognition sensor and an environmental sensor. The information collection unit is the information entrance of the device and is responsible for collecting multi-dimensional information of the user during the interaction process; The data processing unit is closely connected with the information collection unit to receive and process the massive amount of information collected by it; The somatosensory feedback unit includes a high-density distributed micro-vibration motor array, a flexible pressure sensor and a temperature adjustment element. The micro-vibration motor array adopts a high-density distribution design and is distributed throughout the parts where the user may interact with the device. The flexible pressure sensor is based on the piezoresistive or capacitive sensing principle and has high sensitivity and fast response characteristics. It is integrated on the contact surface of the interactive device and can sense the pressure applied by the user, the pressure distribution and the pressure change rate in real time. The temperature adjustment element adopts a semiconductor cooling and heating module based on the Peltier effect principle, which can quickly achieve temperature adjustment. The visual feedback unit includes an ultra-high-definition resolution display screen, a rendering module and a stereoscopic display module. The display screen adopts micro-light-emitting diode display technology. The rendering module uses physically based rendering technology and ray tracing algorithms to realistically simulate the optical phenomena of reflection, refraction and scattering of light on the surfaces of objects of different materials, as well as the spatial relationship of shadows and occlusions of objects. The stereoscopic display module adopts time-division stereoscopic display combined with head tracking technology. The audio feedback unit includes a 7.1-channel audio system and an audio synthesis and special effects engine. It uses audio spatialization technology to calculate the sound propagation path, attenuation effect and acoustic characteristics of reflected echoes in real time according to the user's position in the virtual scene and the positional relationship of the interactive objects. The audio synthesis and special effects engine can generate various realistic environmental sound effects and interactive sound effects according to the requirements of the interactive scene.

2. An artificial intelligence interactive device according to claim 1, characterized in that: The high-precision motion capture sensor adopts advanced optical tracking and inertial measurement unit fusion technology, which can accurately capture the subtle movements of various parts of the user's body. The sampling frequency of the high-precision motion capture sensor is 1000-2500 Hz; The speech recognition sensor uses a deep learning speech recognition algorithm; The environmental sensors include light sensors, temperature sensors, humidity sensors and spatial positioning sensors. The light sensor can accurately measure the intensity of ambient light and color temperature parameters, providing a basis for scene lighting adaptation for the visual feedback unit; the temperature sensor and humidity sensor can monitor the temperature and humidity changes in the user's environment in real time, and provide key data when simulating specific climate scenes or interactive experiences related to temperature and humidity; the spatial positioning sensor uses a combination of satellite positioning and indoor positioning technology to accurately determine the user's location information globally and in specific indoor spaces.

3. An artificial intelligence interaction device according to claim 2, characterized in that: With respect to motion information, the data processing unit performs real-time analysis on the continuous motion data stream transmitted by the motion capture sensor through a well-trained neural network model. With respect to voice information, the data processing unit converts the voice signal into text information through a voice recognition algorithm, and further performs semantic understanding and intent analysis through natural language processing technology. Combined with environmental information, the data processing unit can dynamically adjust the rendering parameters of the interactive scene, the intensity of somatosensory feedback, and the voice interaction logic according to factors such as light, temperature, humidity, and spatial position.

4. An artificial intelligence interaction device according to claim 3, characterized in that: The micro vibration motors are all equipped with independent control circuits, which can accurately simulate vibration modes of different frequencies, amplitudes and waveforms according to the instructions sent by the data processing unit. When simulating the touch of the surface of objects of different materials, the vibration parameters of the vibration motors are adjusted so that the user's hands or body parts can feel the corresponding texture differences. When the user performs a grabbing action in the virtual environment, the flexible pressure sensor feeds back pressure data to the data processing unit, and the data processing unit controls the micro vibration motor array to generate corresponding reaction force vibration according to the weight and hardness properties of the virtual object; When the temperature regulating element simulates an interactive scenario related to temperature, the temperature regulating element adjusts the temperature of the part in contact with the user to the set value in a very short time according to the instructions of the data processing unit; at the same time, in order to avoid discomfort or injury to the user due to excessively high or low temperature, the temperature regulating element also includes a temperature monitoring and safety protection mechanism.

Images