Emotion perception and expression system and method based on multi-modal interaction of Chinese character elements

Abstract

The invention discloses an emotion perception and expression system and method based on multi-modal interaction of Chinese character elements, which realize semantic expression of Chinese characters of emotional states through double-source fusion of equipment perception data and active input of a user in combination with multi-screen dynamic adaptation and social design. The core innovation comprises: 1) an AI dynamic generation and user coverage mechanism of an emotion label; 2) cross-screen collaborative emotion artistic presentation; and 3) emotional propagation ecology driven by social interaction. The system is suitable for families, offices, old-age care and other scenes, and promotes emotional health management from data monitoring to emotional integration.

Description

Technical Field

[0001] The present invention belongs to the cross - technical field of intelligent human - computer interaction and emotion computing, and particularly relates to an emotion perception and expression system and method based on multi - modal interaction of Chinese character elements. Through a dual - source fusion mechanism of device - sensed data (physiological signs, behavior habits) and user - active expression (subjective input of Chinese character elements), combined with multi - screen collaborative interaction and socialized design, it realizes the dynamic mapping and dissemination of emotional states, and is applicable to the emotional health management in family, office, elderly care, education and social scenarios. Background Art

[0002] The existing emotion interaction technologies have the following problems: 1) Data simplification: relying on a single sensor (such as a heart rate belt) or subjective questionnaires, lacking multi - modal data fusion; 2) Interaction fragmentation: the results of emotion analysis are presented in numerical values / graphs, which do not conform to the cognitive habits of Chinese users; 3) Weak social connection: emotion data is stored in isolation, lacking cross - screen sharing and social interaction functions.

[0003] Citation of Existing Technologies: 1) Patent Application No. CN2025106599753 (the same applicant) proposed the dynamic mapping of Chinese character elements and lighting parameters, but did not involve emotion analysis; 2) Existing emotion monitoring bracelets (such as Huawei / Apple) only provide stress indices. Huawei uses anime patterns to express emotional states, but cannot achieve semantic expression of Chinese characters; 3) The "mood daily sign" function of social platforms is limited to user - active input, lacking dynamic supplementation of device - sensed data. Summary of the Invention

[0004] The present invention discloses an emotion perception and expression system and method based on multi - modal interaction of Chinese character elements. Through the dual - source fusion of device - sensed data and user - active input, combined with multi - screen dynamic adaptation and socialized design, it realizes the semantic expression of Chinese characters for emotional states.

[0005] The core innovation points of the present invention include dual - source emotion data fusion, multi - screen dynamic presentation, AI dynamic matching and coverage logic.

[0006] Dual - source emotion data fusion: 1) Device - sensed data: millimeter - wave radar (breathing / heart rate), camera (micro - expressions), smart home behavior logs (frequency of turning on / off lights, duration of air - conditioner use); 2) User - active expression: custom Chinese character elements (such as "forge ahead bravely", "calm waters run deep"), supporting font, color, and animation design.

[0007] Multi - screen dynamic presentation: 1) The emotional state is adaptively displayed on terminals such as the wall screen, mobile phone screensaver, and smartwatch dial in the form of Chinese character elements; 2) The social design supports the like, forward, and cross-screen push of emotional elements.

[0008] AI Dynamic Matching and Coverage Logic: 1) At 06:00 every day, a benchmark emotional label (such as "confused") is generated based on the perceived data; 2) After the user actively modifies it, the benchmark label is overwritten, and the historical records are archived in the form of logs. Brief Description of the Drawings

[0009] Figure 1 : System architecture diagram. This diagram shows the core architecture of the system, which is divided into four major modules: data collection, AI analysis, multi-screen interaction, and social network; 1) Data collection module: Obtain the user's physiological data (breathing, heart rate, micro-expression) through devices such as millimeter-wave radar and cameras, and at the same time receive the Chinese character emotional label (such as "forge ahead") actively input by the user; 2) AI emotion analysis engine: Integrate device data and the user's historical behavior (such as air conditioner usage frequency, night activity log), and calculate the benchmark emotional label (such as "anxious") through weighted calculation. In the algorithm, the breathing rate weight accounts for 40%, the micro-expression accounts for 30%, and the behavior log accounts for 30%; 3) Multi-screen adaptation module: Dynamically render the emotional label into an adapted interface for different terminals. For example, the wall screen is displayed full screen in calligraphy font, the mobile phone screensaver adopts a simple card design, and the smartwatch dial overlays an animation effect; 4) Social platform: Support users to like the emotional label, forward it to the circle of friends, and obtain integral rewards. The integral can be exchanged for personalized fonts or unlock advanced animation effects; Technical effects: Achieve a closed loop from data collection to social dissemination, and the user's emotion is upgraded from "passive monitoring" to "active expression"; The multi-screen adaptation ensures the visual experience consistency of different devices (large screen / small screen), such as the semantic unity of the official script "quiet" on the wall screen and the Song typeface "quiet" on the mobile phone screensaver.

[0010] Figure 2 : Dual-source data fusion logic flow chart. This diagram reveals the fusion logic of device perception data and user input: 1) Device perception data: The millimeter-wave radar monitors the breathing rate, the camera captures micro-expressions (such as the frequency of frowning), and the smart home log analyzes the behavior pattern (such as frequent light on and off late at night). The AI calculates the emotional intensity through a weighted model (breathing rate 40% + micro-expression 30% + behavior 30%) and generates a benchmark label (such as "anxious"); 2) User initiative input: Users can manually input Chinese character tags (such as "forge ahead"), and the system analyzes their semantic intensity (positive / negative) and overrides the AI tags. For example, if the AI generates "confused" and the user modifies it to "endeavor", the system will display the latter preferentially; 3) Dynamic override mechanism: If the user does not modify, the AI tag is retained; if modified, it is overridden and a log is recorded. The overridden tag is pushed to multiple terminals in real time. For example, the wall screen displays "endeavor" and triggers an inspiring animation, and the mobile phone is updated synchronously; Technical effect: The complementary dual-source data improves the accuracy of emotion analysis, and the device data fills the blind spot of users' subjective expressions; the override mechanism gives users the right of dominance and avoids negative experiences caused by AI misjudgment. <无数个

[0011] Figure 3 : Schematic diagram of dynamic override and log recording of emotion tags. This figure focuses on the dynamic override and historical traceability of emotion tags: 1) AI-generated tag: At 06:00 every day, the system generates a benchmark tag based on the previous day's data. For example, detecting the user's nocturnal heart rate fluctuation + low activity level, it generates "confused" (Song typeface, gray); 2) User override operation: Users can manually modify the tag content, font, and color. For example, change "confused" to "forge ahead" in boldface and gold, and after clicking save, it immediately overrides the screen display; 3) Log recording and traceability: All modification records are stored in the cloud in a structured format (timestamp + original tag + new tag). Users can view the historical log through the sub-menu, such as "2023-10-01 07: thirty confused → forge ahead"; Technical effect: The log function provides a visual trace for emotion evolution, assisting users in identifying long-term psychological changes; cloud storage supports multi-device synchronization, and users can review the modification records of the home wall screen on the mobile phone.

[0012] Figure 4 : Example of multi-screen adaptation interface. This figure shows the multi-terminal interface adaptation strategy: 1) Wall touch screen: The emotion tag is displayed full screen in a calligraphy font (clerical script / regular script), with dynamic effects superimposed. For example, the "serenity" tag is accompanied by a water ripple animation, and clicking on it can view detailed physiological data; 2) Mobile phone screensaver: Adopts a simple design, the tag is displayed in Song typeface, and the background color changes gradually according to the emotion intensity (such as "endeavor" is a gold gradient). Tapping on the tag on the lock screen can quickly jump to the social platform; 3) Smartwatch: Round font is adapted for small screen display, and the pulse animation maps the real-time emotion fluctuation (such as the "focus" tag flashes with the acceleration of the heart rate); Technical effects: Calligraphy fonts enhance cultural identity, and dynamic effects improve emotional perception and immersion; small-screen devices (watches) reduce power consumption and extend battery life through streamlined animations.

[0013] Figure 5 :Flowchart of social interaction functions. This diagram analyzes the core process of social interaction: 1) Like and Retweet: Users can like a friend's emotional tag, with the number of likes displayed in real time in the upper right corner of the tag. When forwarding, select a target device (such as a home wall screen). The system pushes and records the forwarding path (e.g., "User B forwarded 'Warm Sun Afternoon' to the living room screen"). 2) Points Rewards: You can earn points by liking or forwarding. When you reach 100 points, you can unlock exclusive fonts (such as Chinese running script) or advanced animations (such as 3D particle effects). 3) Cross-screen collaboration: Forwarded content is adapted to the target device. For example, when the "Neon Walk" tag on a mobile phone is forwarded to a large screen in a shopping mall, it automatically switches to a full-size cursive script dynamic effect. Technical effect: The points mechanism encourages users to participate in emotional social interaction and improves system activity; cross-screen push enables the physical space extension of emotional expression and enhances social influence. DETAILED DESCRIPTION

[0014] Example 1: Family scenario (parent-child emotional care). Hardware configuration: 4-inch touch screen on the wall of the child's bedroom (displaying emotional tags), millimeter-wave radar (monitoring sleep and breathing); the parent's mobile app simultaneously displays the child's emotional status. Interaction process: 1) The system detects increased frequency of nighttime tossing and turning in children and rapid breathing, generating the label "anxiety" (written in red in official script); 2) After checking the app, parents manually change the message to "Rest assured sleep" (in regular script blue) and push it to the child's bedroom screen; 3) The screen displays "Sleep Peacefully" and plays sleep-inducing white noise. The screen will reset to the new AI label at 06:00 the next day.

[0015] Example 2: Office scenario (stress relief and team interaction). Hardware configuration: 7-inch touch screen on the office wall, smart ID card (to monitor heart rate variability); employees can access the emotional social platform on their mobile phones. Interaction process: 1) The system detects that an employee has not left their seat for three consecutive hours and has a fluctuating heart rate, and generates a label of "tense" (in bold). 2) Employees manually change the post to "Calm" and synchronize it to the team's emotional wall, where colleagues can like or forward it; 3) Click the "Calm" tag to trigger the office lights to become warmer and play light music.

[0016] Example 3: Nursing home scenario (emotional companionship and social motivation). Hardware configuration: wall screen in the elderly room (displaying emotion tags), smart wristband (monitoring activity); large screen in the public area displays the entire hospital's emotional calendar. Interaction process: 1) The system detects that the elderly person's total number of steps per day is less than 500 and generates a label of "loneliness" (Song Dynasty gray); 2) The caregiver manually adds "Warm Afternoon" and pushes it to the elderly person's screen and synchronizes it to the public calendar; Other elderly people can click on "Warm Afternoon" to send voice greetings to create social incentives.

[0017] Example 4: Educational scenario (student emotion management and classroom interaction). Hardware configuration: classroom wall screen, student smartwatch (to monitor classroom concentration); teacher's pad to view class emotion heat map in real time. Interaction process: 1) The system detects that a student frequently lowers their head and has a slow heart rate, and generates a label of "distracted" (written in yellow cursive script); 2) The teacher pushes the "Focus Sprint" tag to the student's watch, and a motivational animation is simultaneously displayed on the wall screen; 3) Students change the label to "Focused" and receive points.

[0018] Example 5: Social scene (artistic expression of emotions in public spaces). Hardware configuration: interactive large screen in the mall atrium, Wi-Fi probe (to measure crowd density); users scan the QR code on their mobile phones to participate in emotional co-creation. Interaction process: 1) The system generates a group emotion label of "noisy" (dynamic cursive script effects) based on crowd density; 2) Users scan the QR code and submit their personal tags (e.g., "Neon Walk"), which are then integrated to create an artistic Chinese character waterfall; 3) When a tag receives more than 100 likes, a light show and customized sound effects will be triggered.

Claims

1. An emotion perception and expression system based on multimodal interaction of Chinese character elements, characterized by include: a. Multi-source data acquisition module: millimeter-wave radar, camera, and smart home log; b. AI emotion analysis engine: This engine integrates physiological data and behavioral habits to generate baseline emotion labels. c. User-defined module: supports Chinese characters, short sentences, artistic font editing and multi-screen adaptation; d. Social interaction features: likes, reposts, cross-screen push notifications, and point rewards for emotional tags; e. Daily reset and log archiving mechanism: Baseline tags are updated daily, and user modification records are traceable.

2. The system according to claim 1, characterized in that: 1) The font, color, and animation of the emotion label reflect the intensity of the emotion in real time (e.g., the flashing frequency of the red "anxiety" symbol = the heart rate variability coefficient); 2) Social platforms support the "emotional calendar" function, which displays the emotional evolution of individuals and friends on a timeline.

3. The system according to claim 1, wherein the multi-screen adaptation logic includes: 1) The wall screen displays the emotion label in the form of a screen saver, and switches to the functional interface when the user approaches; 2) The mobile phone supports dual entrances: lock screen emotion cards and social interaction within the app.

4. The system according to claim 1, wherein the AI ​​analysis engine comprises the following algorithms: 1) Physiological data weighting model: respiratory rate weight 40%, micro-expression weight 30%, behavioral log weight 30%; 2) User history preference correction: Frequently modified tag types are given priority (e.g., "positive" tag weight +20%).

Images