Three-dimensional display emotion management device based on multi-mode emotion perception

Abstract

The invention relates to a three-dimensional display emotion management device based on multi-mode emotion perception, and solves the problems of single display dimension and weak perception ability of existing emotion feedback equipment. According to the technical scheme, a multi-mode sensing layer is integrated with a millimeter wave radar (penetrating clothes to detect breath / heartbeat), a camera (analyzing micro expressions), a microphone (recognizing voice and intonation) and a wearable equipment communication interface, and non-contact physiological signal fusion collection is achieved; the three-dimensional display core adopts a double-layer LED array or a spherical flexible OLED screen to generate more than ten 3D emotion particle flow maps (such as anger-lava eddy current red light and calm-deep sea ripple blue light), and is also suitable for geometries such as spheres, cubes, cuboids, cylinders, cones, prisms and pyramids; the emotion management closed loop enhances cognition through a real-time emotion mirror image, a'forest breathing 'guide mode is automatically triggered when anxiety continues for 10 minutes, and a built-in memory records emotion data at an interval of 5 minutes and outputs a time sequence analysis report.

Description

Technical Field

[0001] The present invention relates to the field of intelligent display devices, and in particular to an emotion management device that integrates biological signal perception and stereoscopic visualization technology. Background Art

[0002] Technical flaws of existing emotion regulation devices. Analyzing emotion-related patents over the past three years, we found three common bottlenecks in current technology: (1) Single perception dimension. CN120000914A (Guqin healing system) relies on millimeter-wave radar + camera, but does not integrate wearable device data, and physiological signal coverage is incomplete; CN119541493A (voice soothing lamp) only recognizes emotions through voice, and the accuracy rate drops by more than 40% under environmental noise interference; CN115422992A (lighting control method) requires users to actively express emotions and cannot achieve passive perception; (2) Feedback is disconnected from emotions. Although CN120018354A (intelligent light adjustment system) combines ambient light adjustment, it only outputs two-dimensional color temperature / brightness changes (e.g., cool white light → warm yellow light), which cannot express the complexity of emotions. CN119450875A (ambient light management system) uses CNN to identify office emotions, but the feedback is simplified to "focus mode / rest mode" switch instructions, lacking emotional intensity gradient mapping. Although CN120000914A's guqin system integrates sound / light / smell, the lighting module only simulates natural light (morning light / moonlight) and is not dynamically coupled with emotions. (3) Lack of functional closed loop. None of the cited patents have achieved the “perception-feedback-intervention-tracking” closed loop: CN119450875A can monitor concentration but has no active intervention strategy; CN120000914A’s guqin system records physical and mental data but does not establish a time series analysis model; CN115422992A relies on historical data to train the network and cannot optimize the control strategy in real time.

[0003] Market application bottlenecks: (1) The contradiction between medical-grade precision and consumer-grade cost. The CN120000914A system is difficult to popularize in home scenarios because it contains a simulated guqin speaker (unit price > ￥3000); (2) The dilemma of balancing privacy and efficiency. CN119450875A requires continuous camera monitoring, which has caused privacy disputes, while the CN119541493A voice-only solution has a misjudgment rate of up to 35%.

[0004] Innovative breakthroughs of this patent (Table 1) Table 1. Multimodal perception fusion architecture Comparative Patents Perceptual Dimension Enhanced points of the present invention CN120000914A Millimeter-wave radar + camera New wearable device interface to obtain HRV / blood oxygen in real time CN119541493A Single voice input Fusion of voice + micro-expression + physiological signal triple cross-validation CN115422992A User active expression Support active + passive dual-mode perception .

[0005] Technical effect: The accuracy of emotion recognition of the present invention is greatly improved, and it is compatible with devices such as Apple Watch through the Bluetooth 104 interface, and medical-grade data is directly transmitted to consumer terminals.

[0006] This invention uses an emotional stereoscopic visualization engine, and its innovations are as follows: (1) Innovation of display architecture. Solution A (double-layer LED array): The inner layer 201a uses Fibonacci spiral arrangement of 512 Micro-LEDs to generate basic textures, and the outer layer 201b uses 1024 RGB LEDs to superimpose dynamic light effects, realizing 10 types of 3D particle flows such as "anger-lava vortex"; Solution B (flexible OLED spherical screen): resolution ≥300PPI, supporting magic ball special effects rendering; compared with the flat dimming technology of CN120018354A, it realizes the three-dimensional mapping of emotional complexity for the first time; (2) Cross-morphology compatible design. The shell supports seven geometric shapes, including sphere, cube, and cylinder. Summary of the Invention

[0007] The present invention relates to a stereoscopic display emotion management device based on multimodal emotion perception, which solves the problems of existing emotion feedback devices with single display dimensions and weak perception capabilities. The technical solution includes: (1) Multimodal perception layer: Integrates millimeter-wave radar (to detect breathing / heartbeat through clothing), camera (to analyze micro-expressions), microphone (to recognize voice intonation), and wearable device communication interface to achieve non-contact physiological signal fusion acquisition; (2) Stereoscopic display core: using a double-layer LED array (512 Micro-LEDs in the inner layer form the Fibonacci spiral principle, and 1024 RGB LEDs in the outer layer superimpose dynamic light effects) or a spherical flexible OLED screen (resolution ≥ 300PPI) to generate more than 10 3D emotional particle flow maps (such as anger → lava vortex red light, calm → deep sea ripple blue light); the present invention uses a sphere as an example to illustrate the function and application. In fact, in addition to the sphere, common geometric bodies such as cubes, cuboids, cylinders, cones, prisms and pyramids are also geometric body types covered by the stereoscopic display emotion management device of the present invention; (3) Emotional management closed loop: Strengthen cognition through real-time emotional mirroring. Continuous anxiety for 10 minutes will automatically trigger the "Forest Breathing" guidance mode. The built-in memory will record emotional data at 5-minute intervals and output a time series analysis report.

[0008] Figure 1 The complete system architecture of the stereoscopic display emotion management device is shown. The structure of this stereoscopic luminous ball is divided into five parts: perception layer, display layer, processing layer, storage layer, and interaction layer: (1) Perception layer (101-104): millimeter wave radar (101) detects breathing / heartbeat, camera (102) captures facial expressions, microphone (103) analyzes voice intonation, and Bluetooth / WiFi interface (104) connects to wearable devices to obtain physiological data; (2) Processing layer (300): After signal fusion, it is input into the emotion analysis engine (300) and the emotion map is generated through the CNN+LSTM model; (3) Display layer (201a / b / 202): drives the double-layer LED array (201a / b) or the flexible OLED screen (202) to display 3D emotional particle flow; (4) Storage layer (400-401): The data storage (400) records the emotion data with a timestamp, and the analysis report generation module (401) outputs the time series analysis; (5) Interaction layer: The user (N) interacts with the device, and the mobile phone APP (O) accesses historical data through the WiFi module (302).

[0009] Key connection instructions for the 3D display emotion management device: (1) Multimodal perception fusion: 101 / 102 / 103 / 104 → signal fusion module → 300; (2) Emotion display path: 300 → 201a / b (Scheme A) or 202 (Scheme B) → three-dimensional emotion particle flow; (3) Data analysis closed loop: 300 → 400 (storage) → 401 (analysis) → time series report; (4) Mobile interaction: mobile app → WiFi302 → 104 (communication interface) → access 400 / 401.

[0010] Beneficial effects of the stereoscopic display emotion management device: (1) Cognitive enhancement: Angry emotions trigger the red light of "lava vortex", and calm state displays the blue light of "deep sea ripple"; (2) Misjudgment avoidance: When the camera and heart rate data conflict, the arbitration algorithm is activated (e.g., smiling face + high heart rate → marked as "excited / nervous"); (3) Behavioral intervention: After 10 minutes of anxiety, the "forest breathing" guidance mode (gradual green light pulsation) is automatically activated. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1: System module connection diagram. Showing the multimodal perception layer composed of millimeter wave radar (101), camera (102), microphone (103) and Bluetooth / WiFi module (104). After signal fusion, it is input into the processor (300), which drives the stereo display core (Solution A: 201a / b double-layer LED; Solution B: 202 flexible screen), synchronously stores data in a time-stamped memory (400) and generates an analysis report (401).

[0012] Figure 2 : Double-layer LED cross-section structure. The cross-section of the sphere shows that the outer LED array (201b) surrounds the inner LED array (201a), with the inner layer diameter ≤ 50% of the outer layer. The two layers form iris-like three-dimensional light and shadow through independent control circuits, and the substrate adopts a Fibonacci spiral layout to optimize the light path.

[0013] Figure 3 : Schematic diagram of spherical unfolding of a flexible screen. The flexible OLED screen (202) displays longitude / latitude partitions after UV unfolding. The driver chip supports spherical 3D rendering with a resolution of ≥300PPI and an edge splicing error of <0.1mm.

[0014] Figure 4 A sample time series analysis report diagram. This line chart shows 24-hour mood swings, with timestamps on the horizontal axis and emotional intensity on the vertical axis. Peak anxiety events and their duration are marked, along with a table of triggering events at the bottom (e.g., "2:30 PM meeting → anxiety index +40%)."

[0015] Figure 5 Diagram of the surface light source emotion mapping system. The emotion-light effect mapping library includes at least six basic modes, which visualize emotions through a combination of overall color and brightness changes (e.g., anger triggers 0.5Hz red breathing).

[0016] Figure 6 Supported geometric forms and display schemes. Geometric forms and display schemes supported by this invention include: Sphere (left), using a double-layer LED array (201a / b) to create an iris effect; Cube (center), using six independent LED arrays to synchronize emotional states; Cylinder (right), using circular LEDs on the side and a radial array on the bottom to generate a rotating light pattern. DETAILED DESCRIPTION

[0017] Example 1: Social scene emotional feedback (fireworks mode); (1) Product form: Double-layer LED array solution (Scheme A), acrylic sphere with a shell diameter of 150mm; (2) Deployment method: Placed in the center of the coffee table in the living room, the radar monitors a sector area with a radius of 3 meters; (3) Vital signs: millimeter-wave radar 101 detects waving (frequency > 2 Hz); microphone 103 recognizes laughter (sound intensity > 70 dB and fundamental frequency fluctuation > 50 Hz); camera 102 captures the upward movement of the mouth corners (displacement of facial landmarks ≥ 15 pixels); (4) The light display triggers the "fireworks bloom" mode. The inner LED 201a (Fibonacci spiral) emits a golden particle stream (color temperature 2200K). The outer LED 201b generates an explosive diffusion halo every 0.2 seconds (peak brightness 1000nit). After 5 seconds, it turns into a warm yellow breathing light (frequency 0.8Hz).

[0018] Example 2: Nighttime health monitoring (sleep anxiety warning): (1) Product form: Flexible OLED screen solution (Scheme B), outer shell with a diameter of 120mm and a silicone sphere (transmittance 92%); (2) Deployment method: Place on the bedside table and connect to Apple Watch Series 8 via Bluetooth interface 104; (3) Vital signs: wearable device data, blood oxygen saturation ≤ 93% and lasting > 10 minutes; millimeter wave radar 101, respiratory rate > 22 times / minute (normal value 12-20); microphone 103 detects teeth grinding sound (energy surge in the 3-6kHz frequency band); (4) Luminous display activates the sleep anxiety protocol: the flexible screen 202 displays a pulsed purple ripple (RGB 128, 0, 128); the ripple spreads from the bottom of the ball to the top (speed 1.5 cycles / second); the memory 400 marks the "sleep anxiety event" and records the start and end time and the lowest blood oxygen value.

[0019] Example 3: Surface mood light source mapping system. Technical features: (1) The surface of the sphere shell is covered with a single layer of RGB LED light strip (301), and the light strip density is ≥ 24 per circumference; (2) The emotion-light effect mapping library contains at least 6 basic modes, which realize emotion visualization through the combination of overall color + brightness changes (such as anger triggering 0.5Hz red breathing); (3) Support custom color mapping logic through mobile APP (new module 302); (4) The surface emotion light source mapping system adopts 2D color change, and the product production cost is low, which is suitable for children's education and smart home.

[0020] Example 4: Emotional History Analysis (Mobile Data Interaction): (1) Product form: Basic version with LED light strip (claim 5), 100mm diameter ABS engineering plastic sphere; (2) Deployment method: Place on a desk and connect to a mobile phone APP (iOS / Android) via WiFi module 302; (3) Vital signs: Camera 102, time spent facing the screen during work > 6 hours / day; Millimeter-wave radar 101, shoulder and neck muscle tension (micro-movement frequency < 0.1 Hz); Wearable device, HRV heart rate variability ≤ 35 ms (normal > 50 ms) (4) Luminous display, data request and response process: the user clicks the "weekly report" button on the APP → the device uploads 7 days of emotional data; the analysis module 401 generates a three-dimensional chart, where the X-axis is time (divided into 30-minute segments), the Y-axis is the emotional intensity value (0-100), and the Z-axis is the emotional type code (red = anxious / blue = calm); the LED 301 on the surface of the sphere displays the data transmission status, and a green marquee (rotating speed 2 revolutions / second) is displayed during transmission, and the blue light is always on for 5 seconds when the transmission is completed.

Claims

1. An emotion-sensing stereoscopic display device, characterized in that Include: a. Shell: having a three-dimensional geometric shape, including but not limited to a sphere, cube, cuboid, cylinder, cone, prism or pyramid; b. a multimodal sensing module: at least one of a millimeter-wave radar (101), a camera (102), a microphone (103), and a communication interface (104); c. 3D display core: Solution A, double-layer LED array inside and outside (201a / 201b); Solution B, fully covered with flexible display screen (202); d. Emotion analysis engine (300): generates ≥10 dynamic emotion maps; e. Data memory unit: emotion memory with time stamp (400) and analysis report module (401).

2. The device according to claim 1, characterized in that: When the shell is a sphere, the diameter is 100-200 mm; when it is a cube / cuboid, the length of one side is 80-180 mm; when it is a cylinder / cone, the bottom diameter is 70-150 mm and the height-to-diameter ratio is 0.8-1.

5.

3. The device according to claim 1, characterized in that: When the housing is a prism / pyramid, the number of planes on the side surface of the housing on which the LED array (201b) is mounted is ≥4, and the arc of the top angle is ≥90°.

4. The device according to claim 1, characterized in that: The adaptation method of the stereo display core in the aspherical geometry is: (1) Cube: six independently controlled LED arrays; (2) Cylinder: side ring LED array + top and bottom radial array; (3) Cone: Conical spiral LED layout + bottom concentric circle array.

5. The device according to claim 1, characterized in that: The millimeter wave radar (101) operates at a frequency of 60-64 GHz and can penetrate clothing to detect breathing / heartbeat.

6. The device according to claim 1, characterized in that: The emotion map includes a three-dimensional dynamic particle flow, and the particle movement speed is positively correlated with the emotion intensity.

7. The device according to claim 1, characterized in that: The analysis report module (401) outputs a table of association between emotion change curves and triggering events, with a data sampling interval of ≤5 minutes.

8. The device according to claim 1, characterized in that: The three-dimensional display core can be replaced by a surface annular LED light strip (301), comprising at least three groups of independently controlled RGB LED modules.

9. The device according to claim 8, characterized in that: The emotion analysis engine (300) has a built-in simplified light effect library, which includes six basic emotion-color mapping relationships: (1) Joy → warm yellow (color temperature 2700K ± 10%); (2) Sadness → Dark blue (RGB 0,0,255); (3) Angry → Red (brightness pulsation frequency 0.5-2Hz); (4) Calm → light blue and steady; (5) Anxiety → Orange breathing (frequency 1 Hz); (6) Focus → Green light gradually becomes stronger.

10. The device according to claim 8, characterized in that: A wireless communication module (302) is added to receive custom light effect instructions and data request instructions sent by a mobile terminal.

Images