An interactive art healing device with emotion recognition and triggering

Abstract

This invention discloses an interactive art therapy device that recognizes and triggers emotions, belonging to the fields of mental health and human-computer interaction. It includes: an image acquisition module for acquiring user facial images; an expression recognition module for identifying whether an emotion is positive; a control module for generating control signals when a positive emotion is detected; a mechanical drive module for generating mechanical motion; an animation display module, including a grid sheet and a base frame capable of relative reciprocating motion, driven by the mechanical motion to produce dynamic animation effects; and a power supply module. This invention transforms the user's positive emotions in real time into physical motion driving the physical grid animation, constructing a non-judgmental, instant positive feedback loop, avoiding the psychological pressure brought by emotion labels, and transforming abstract animation principles into a tangible art interactive experience. The device is integrated and portable, and can be flexibly deployed in public spaces, effectively enhancing users' positive emotions and sense of participation.

Description

Technical Field

[0001] This invention relates to the field of mental health promotion and human-computer interaction technology, and more specifically, to an interactive art therapy device that recognizes and triggers emotions. Background Technology

[0002] Studies have shown that art therapy, as an effective adjunct, can relieve stress and release emotions. However, traditional art therapy methods rely on professional instructors, are relatively simple in form, and lack immediate interaction, making it difficult to deploy on a large scale and at low cost in public spaces such as campuses and communities.

[0003] Currently, existing technologies include emotion monitoring systems based on facial expression recognition. These systems typically include a camera and a data processing unit, using deep learning models to identify the user's emotions (such as happiness or sadness) and displaying the results on the screen in the form of text, charts, or colors. This type of technology has the following drawbacks:

[0004] 1. Lack of effective application scenarios: The recognition results are mostly statistical reports or screen displays, which are disconnected from the user's subsequent emotion regulation behavior and lack a continuous interaction mechanism.

[0005] 2. Risk of “labeling”: Directly displaying negative emotion labels such as “sadness” and “anger” may cause psychological pressure to users, which deviates from the core concept of “non-judgment” emotion regulation.

[0006] 3. Abstract interactive forms: Even when combined with animation elements, current interactions are mostly limited to digital animation on the screen, lacking physical, tangible interactive experiences, making it difficult to generate deep emotional resonance.

[0007] Furthermore, as a traditional tool for demonstrating animation principles, raster animation has a limited form of presentation, mostly consisting of static explanations of principles, lacking a tangible and concrete interactive feedback mechanism, thus limiting its application scenarios. Summary of the Invention

[0008] The present invention aims to provide an interactive art therapy device that recognizes and triggers emotions, in order to solve the technical problems in the prior art, such as the disconnect between emotion recognition and application scenarios, the lack of continuity and physicality in the interaction mechanism, and the risk of negative psychological labeling.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] An interactive art therapy device that recognizes and triggers emotions is characterized by comprising:

[0011] The image acquisition module is used to acquire the user's facial images in real time;

[0012] An expression recognition module, connected to the image acquisition module, is used to identify whether the user's expression in the facial image is a preset positive emotional expression and to calculate the intensity of a smile;

[0013] A control module, connected to the expression recognition module, is used to generate and output a drive control signal when the positive emotional expression is recognized. The control module converts the expression recognition result into a mechanical drive signal. Specifically, let P_t be the confidence score of the smile output by the model for the t-th frame image, with a value range of 0 to 1. The control module calculates the stable smile intensity as follows: S_t = α × P_t + (1 - α) × S_{t-1}, where α is a smoothing coefficient, preferably between 0.2 and 0.6.

[0014] When S_t is greater than the trigger threshold T_on for N consecutive frames, the system enters the animation trigger state; when S_t is lower than the release threshold T_off for M consecutive frames, the system exits the animation trigger state; T_on is greater than T_off, thus forming a hysteresis judgment interval to avoid the device repeatedly starting and stopping when the child is in a smiling boundary state;

[0015] In the animation-triggered state, the control module further maps the stable smile intensity to motor drive parameters; for example, the motor drive frequency can be generated according to the following formula: F_t = F_min + K × max(0, S_t - T_on); where F_min is the minimum motion frequency of the animation, and K is the speed mapping coefficient; the higher the smile intensity, the higher the pulse frequency or PWM duty cycle output by the control module, the faster the relative motion speed between the grid and the background frame, and the more obvious the animation feedback observed by the user; the control module outputs pulse signals, direction signals, or PWM signals to the motor drive board through the GPIO port, and the drive board then controls the motor rotation direction, speed, number of steps, or duration;

[0016] A mechanical drive module, connected to the control module, is used to receive the drive control signal and generate mechanical motion. The mechanical drive module includes a motor, a drive board, and a transmission mechanism. The motor uses a stepper motor. The control signal output by the control module is amplified and converted by the drive board to drive the motor. The transmission mechanism uses a driving gear and a driven gear to convert the motor's rotational motion into a relative reciprocating motion between the grid sheet and the base plate frame.

[0017] The animation display module, connected to the mechanical drive module, includes a grid sheet capable of relative reciprocating motion and a base frame, with the base frame bearing an artistic pattern. The mechanical motion drives the grid sheet and the base frame to reciprocate relative to each other, producing a dynamic animation effect. The animation display module consists of a grid sheet, a base frame, and a moving frame. The grid sheet has periodically arranged occlusion stripes, and the base frame displays multi-frame decomposed artistic patterns. When the grid sheet and the base frame move back and forth at a set distance, different pattern areas are sequentially revealed through the grid sheet, allowing the user to observe a continuous dynamic animation effect from the front. This animation is not a video played on a screen, but a mechanical lenticular animation formed by the movement of a physical structure. The base frame is designed to be detachable or replaceable, allowing children to place their self-drawn patterns into the device. Thus, the user's smiling behavior not only triggers the mechanical animation but also activates the patterns they created, creating a closed loop between emotion recognition and individual artistic expression.

[0018] Power module, used to provide power.

[0019] The preset positive emotional expression is "smiling".

[0020] The power module is a portable power bank.

[0021] In this embodiment, the control module uses an embedded computer, the expression recognition module runs on the embedded computer, and uses a lightweight deep learning framework suitable for edge deployment for real-time inference; the expression recognition model includes a face region input layer, a quantized feature extraction layer, and a positive emotion judgment output layer; the quantized feature extraction layer uses model quantization to lightweight the expression recognition model, converting the floating-point calculation parameters in the model into low-bit-width fixed-point parameters, so as to reduce the model storage occupation, the number of parameters, and the amount of inference computation, so that the expression recognition process can be completed locally on the embedded computer without relying on a cloud server;

[0022] The system prioritizes the largest face or the face located in the center of the image from the camera as the target face, and only performs positive emotion recognition on the target face area, thereby reducing false triggers caused by multiple people passing by, background interference, or non-target faces; the expression recognition module can also adopt an interval inference method, which judges the expression according to a preset time interval after detecting a valid face, instead of performing complete deep learning inference on every frame of the image, in order to further reduce system power consumption, processing latency and heat generation;

[0023] Furthermore, the expression recognition module performs confidence smoothing or majority voting on several consecutive recognition results to stably confirm positive emotion results. When the confidence level of positive emotion is higher than a set threshold for a consecutive preset number of times, the control module generates control commands to drive the smile grid animation mechanism to move and control the expression display module to display positive feedback expressions. When the confidence level of positive emotion is lower than the set threshold, no valid face is detected, or the recognition result is unstable, the control module controls the drive mechanism to stop or enter standby mode. Thus, this embodiment can achieve stable recognition and mechanical feedback for smile interaction scenarios under low computing power, low power consumption, and localized operation conditions, and is particularly suitable for children's emotional interaction, campus artificial intelligence popularization, and public display scenarios.

[0024] Furthermore, the device is encapsulated in an integrated, independent housing, and the power module is a portable power source, allowing the device to be flexibly deployed without the need for an external fixed power source.

[0025] Furthermore, the control module is an embedded computer, preferably a Raspberry Pi 4B; it has a USB interface and WiFi functionality, and serves as the "brain" of the device, responsible for running the facial recognition program and control logic.

[0026] The image acquisition module uses a camera with a resolution of ≥1080P, which is connected to the Raspberry Pi via a USB or CSI interface, serving as the device's "eyes" for capturing facial images.

[0027] The facial expression recognition module runs on the embedded computer and uses a lightweight deep learning framework (such as TensorFlow Lite) for real-time inference.

[0028] Furthermore, the mechanical drive module includes a stepper motor and a gear transmission mechanism; the control module outputs pulse signals to the stepper motor drive board through GPIO pins to control the rotation direction and number of steps of the stepper motor; the gear transmission mechanism transmits the rotation of the stepper motor to the animation display module, driving the grid sheet and the base frame to produce a set relative reciprocating motion.

[0029] Furthermore, the device also includes a data recording module for anonymously recording users' emotion recognition events and the corresponding device trigger frequency and duration, forming a group mental health data reference.

[0030] Furthermore, the device does not include any display screen or indicator light for showing text, icons, or color labels for displaying emotion recognition results, thereby completely avoiding the output of negative emotion judgment information to the user.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. A materialized, closed-loop positive feedback system is constructed: This invention directly and in real-time transforms the positive emotion recognition result of "smiling" into a physical behavior that drives the movement of a physical grid animation, forming a positive feedback loop of "smiling → animation start → continuous smiling → animation movement → emotion enhancement → more likely to smile". Users can obtain an immediate and perceptible sense of accomplishment and pleasure from the physical movement of the animation, enhancing the sustainability and effectiveness of emotional healing.

[0033] 2. Avoids the risk of emotional "labeling": The device does not display any emotional judgment results (such as text or icons) to the user. The user triggers the animation by spontaneously smiling without being aware of it. The whole process is natural, non-judgmental, and stress-free, which is more in line with the psychological safety principles of emotion regulation.

[0034] 3. Expanded application scenarios and achieved productization and portability: It adopts an embedded platform and is powered by a mobile power supply. It is integrated into an independent, lightweight device (<2kg) that does not require an external computer or complex wiring. It can be flexibly deployed in public spaces such as campus corridors, classrooms, psychological counseling rooms, and community centers, achieving plug-and-play functionality and significantly reducing the application threshold.

[0035] 4. Transforming abstract animation principles into embodied interactive experiences: Users can not only see the animation effects, but also draw their own raster animation content, infusing it with their creativity and emotions. The movement of the installation is controlled in real time by its own emotional behavior, transforming abstract visual principles into embodied artistic creation and interactive experiences closely related to personal emotions, greatly enhancing participation and creativity. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the structure of the present invention. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to specific embodiments. The following embodiments are only used to explain the present invention and are not intended to limit its scope.

[0038] Example 1:

[0039] This embodiment provides an interactive art therapy device that recognizes and triggers emotions. For example... Figure 1 As shown, the device consists of the following six modules: image acquisition module, expression recognition module, control module, mechanical drive module, animation display module, and power supply module.

[0040] (I) Hardware composition and connection relationship

[0041] Control module: It uses a Raspberry Pi 4B as the main control unit and has a built-in microSD card of 32GB or more for storing the operating system and programs.

[0042] Image acquisition module: A 1080P resolution USB camera, connected to the Raspberry Pi via a USB interface.

[0043] Facial expression recognition module: This is not a standalone hardware component, but a software program running on the Raspberry Pi. It runs a pre-trained deep learning model for smile detection based on the TensorFlow Lite framework.

[0044] Mechanical drive module: Includes a stepper motor and its driver board. The driver board receives pulse signals from the Raspberry Pi via GPIO pins to control the rotation of the stepper motor.

[0045] The animation display module includes a 3D-printed outer frame, a hand-drawn pattern frame fixed to the base plate, and a grid masking plate that can move horizontally back and forth relative to the base plate. A stepper motor transmits power to the grid plate through a set of gear transmission mechanisms (including a driving gear and four driven gears that rotate in opposite directions in pairs), causing it to produce precise relative reciprocating motion with the base plate.

[0046] Power module: A 5V / 3A power bank that powers both the Raspberry Pi and the stepper motor simultaneously via USB. The entire device is encapsulated in a 3D-printed housing, forming a standalone, all-in-one unit.

[0047] (II) Work Process

[0048] In this embodiment, the device's workflow is as follows: S1: Standby monitoring. After the device is powered on, the control module starts the camera module, expression recognition module, and drive control program, while the mechanical drive module remains in standby mode. The control module continuously monitors the working status of the camera module and performs real-time monitoring of the area in front of the device.

[0049] S2: Image Acquisition. The camera module continuously acquires image frames in front of the device at a preset frame rate and transmits the acquired image frames to the control module. The image frames are used to determine whether there is a valid user face area in front of the device.

[0050] S3: Face localization and target selection. The control module performs face detection on the image frame. When no valid face is detected in the image, the system returns to step S1 to continue monitoring; when one or more faces are detected in the image, the control module prioritizes the face with the largest area or the face located in the center of the image as the target face, and only performs subsequent recognition processing on the target face to reduce false triggers caused by multiple people passing by, background faces, or non-target users.

[0051] S4: Image Preprocessing. The control module crops the target face region and adjusts the cropped face image to the input size required by the expression recognition model. Subsequently, the face image undergoes grayscale conversion, normalization, or brightness equalization to reduce the impact of different lighting conditions, camera distance variations, and changes in children's facial poses on the recognition results. The preprocessed face image is then input to the expression recognition module.

[0052] S5: Smile Confidence Output. The facial expression recognition module runs on an embedded computer and uses a lightweight deep learning framework suitable for edge deployment for local real-time inference. The module targets positive emotions, determines the smile state of preprocessed facial images, and outputs the smile confidence score P_t corresponding to the t-th frame. The value of P_t ranges from 0 to 1. The smile confidence score P_t represents the probability that the current target user is in a smiling or positive emotional state.

[0053] S6: Calculation of Stable Smile Intensity. The control module performs time smoothing on the smile confidence P_t output in consecutive frames to obtain the stable smile intensity S_t. The stable smile intensity S_t is calculated according to the following formula: S_t = α × P_t + (1 - α) × S_{t-1}, where α is the smoothing coefficient, preferably between 0.2 and 0.6; and S_{t-1} is the stable smile intensity of the previous frame or the previous moment. Through the above time smoothing process, the impact of single-frame recognition fluctuations on the mechanical drive state can be reduced, avoiding frequent starts or stops of the raster animation mechanism due to instantaneous misidentification.

[0054] S7: Trigger State Determination. The control module determines the state based on the stable smile intensity S_t, trigger threshold T_on, release threshold T_off, and the number of consecutive frames. When the stable smile intensity S_t is greater than the trigger threshold T_on for N consecutive frames, the system enters the animation trigger state; when the stable smile intensity S_t is lower than the release threshold T_off for M consecutive frames, the system exits the animation trigger state. T_on is greater than T_off, thus forming a hysteresis judgment interval to prevent the device from repeatedly starting and stopping when the child is in a smiling boundary state. When no valid face is detected, the target face leaves the acquisition area, or the recognition result is unstable, the system enters a non-trigger state or standby state.

[0055] S8: Drive Parameter Generation. After the system enters the animation trigger state, the control module maps the stable smile intensity S_t to mechanical drive control parameters. These mechanical drive control parameters include one or more of the stepper motor's drive frequency, rotation direction, number of steps, or duration. For example, the control module can generate the stepper motor drive frequency according to the following formula:

[0056] F_t = F_min + K × max(0, S_t - T_on)

[0057] Where F_min is the minimum motion frequency of the animation, and K is the speed mapping coefficient. The higher the stable smile intensity S_t, the higher the step pulse frequency output by the control module, the faster the relative motion speed between the grid and the background frame, and the more obvious the animation feedback observed by the user.

[0058] S9: Mechanical Animation Driven. The control module outputs stepping pulse signals and direction control signals to the motor driver board via GPIO ports. The motor driver board controls the rotation direction, speed, number of steps, or duration of the stepper motor according to these signals. The stepper motor drives the grid plate to reciprocate relative to the base frame through a gear transmission mechanism, causing the child's drawn or preset artistic patterns to visually change continuously, thus creating a grid animation effect.

[0059] S10: Positive Feedback Display. While driving the grid animation mechanism, the control module can also control the expression display module to display positive feedback expressions, such as smiling expressions, encouraging graphics, or interactive prompt images, so that users can obtain interactive feedback combining mechanical movement and on-screen expressions.

[0060] S11: Continuous Driving and Dynamic Adjustment. When the expression recognition module continuously detects a smiling state, and the stable smile intensity S_t remains above the trigger threshold T_on or the maintenance threshold, the control module continuously outputs driving pulses to keep the grid animation mechanism in motion. Furthermore, the control module can dynamically adjust the stepper motor's driving frequency, number of steps, or animation duration based on changes in the stable smile intensity S_t, creating a correspondence between the smile intensity and the animation feedback intensity.

[0061] S12: Stop Animation. When the stable smile intensity S_t is lower than the release threshold T_off for M consecutive frames, or when the target face leaves the acquisition area, no valid face is detected, or the recognition result is unstable, the control module stops outputting stepping pulse signals to the motor drive board, the stepper motor stops rotating, and the grid animation mechanism stops moving or returns to its initial position. Subsequently, the device re-enters the standby monitoring state.

[0062] S13: Data Recording. The system can anonymously record the trigger time, duration, number of triggers, stable smile intensity S_t, or animation runtime of each smile trigger event in the local storage unit of the embedded computer. This data does not contain user identification information and can be used for subsequent statistical analysis of group interactions, data analysis of campus activity displays, or evaluation of device operating status.

[0063] This program has undergone a prototype trial with 36 primary school students in a school environment. During the test, after learning the basic interaction methods, the children approached the device and triggered raster animations by smiling; some children participated in drawing or selecting animated patterns. Objectively, the device recorded the number of interactions, the number of smile triggers, the duration of smiles, and the average interaction time. Based on the current interim records, after using the device, participants' average smile frequency increased by approximately 40%, average interaction time increased by approximately 1 minute, and smile rate increased by approximately 30%. Subjectively, participating children stated that the device "moves when it sees me smile, which is fun and makes me want to smile"; some children also reported that "when I'm in a bad mood, it interacts with me and makes me happy." Participants were generally willing to draw their own animation content and place it into the device. These results indicate that this device, through a non-judgmental, real-time, and tangible positive feedback mechanism, can increase children's willingness to participate and promote the activation of positive emotions.

Claims

1. An interactive art therapy device for emotion recognition and triggering, characterized in that, include: The image acquisition module is used to acquire the user's facial images in real time; An expression recognition module, connected to the image acquisition module, is used to identify whether the user's expression in the facial image is a preset positive emotional expression and to calculate the intensity of a smile; A control module, connected to the expression recognition module, is used to generate and output a drive control signal when the positive emotional expression is recognized. The control module converts the expression recognition result into a mechanical drive signal. Specifically, let P_t be the confidence score of the smile output by the model for the t-th frame image, with a value ranging from 0 to 1. The control module calculates the stable smile intensity as follows: S_t = α × P_t + (1 - α) × S_{t-1}, where α is a smoothing coefficient, ranging from 0.2 to 0.

6. When S_t is greater than the trigger threshold T_on for N consecutive frames, the system enters the animation trigger state; when S_t is lower than the release threshold T_off for M consecutive frames, the system exits the animation trigger state; T_on is greater than T_off, thus forming a hysteresis judgment interval to avoid the device repeatedly starting and stopping when the child is in a smiling boundary state; In the animation-triggered state, the control module further maps the stable smile intensity to motor drive parameters; A mechanical drive module, connected to the control module, is used to receive the drive control signal and generate mechanical motion. The mechanical drive module includes a motor, a drive board, and a transmission mechanism. The motor uses a stepper motor. The control signal output by the control module is amplified and converted by the drive board to drive the motor. The transmission mechanism uses a driving gear and a driven gear to convert the motor's rotational motion into a relative reciprocating motion between the grid sheet and the base plate frame. An animation display module, connected to the mechanical drive module, includes a grid sheet capable of relative reciprocating motion and a base frame, the base frame bearing artistic patterns; the mechanical motion drives the grid sheet and the base frame to reciprocate relative to each other to produce a dynamic animation effect; the animation display module consists of a grid sheet, a base frame, and a moving frame; the grid sheet has periodically arranged masking stripes, and the base frame is set with multi-frame decomposed artistic patterns; when the grid sheet and the base frame reciprocate at a set distance, different pattern areas are sequentially revealed through the grid sheet, allowing the user to observe a continuous dynamic animation effect from the front; Power module, used to provide power.

2. The interactive art healing device of claim 1, wherein, The preset positive emotional expression is "smiling".

3. The interactive art therapy device for emotion recognition and triggering according to claim 1, characterized in that, The power module is a portable power bank.

4. The interactive art therapy device for emotion recognition and triggering according to claim 1, characterized in that, The control module is an embedded computer, and the expression recognition module runs on the embedded computer and uses a lightweight deep learning framework for real-time inference; the expression recognition model includes a face region input layer, a quantized feature extraction layer, and a positive emotion judgment output layer. The quantization feature extraction layer uses model quantization to lightweight the expression recognition model, converting the floating-point calculation parameters in the model into fixed-point parameters with low bit width. The facial expression recognition module performs confidence smoothing or majority voting on several consecutive recognition results to stably confirm positive emotion results. When the confidence of positive emotion is higher than the set threshold for a consecutive preset number of times, the control module generates control commands to drive the smile grid animation mechanism to move and control the facial expression display module to display positive feedback expressions. When the confidence of positive emotion is lower than the set threshold, no valid face is detected, or the recognition result is unstable, the control module controls the drive mechanism to stop or enter standby mode.

5. An interactive art therapy device for emotion recognition and triggering according to claim 4, characterized in that, The embedded computer is a Raspberry Pi series computer.

6. The interactive art therapy device for emotion recognition and triggering according to claim 1, characterized in that, The mechanical drive module includes a stepper motor and a gear transmission mechanism; the control module outputs pulse signals to the stepper motor drive board through GPIO pins to control the rotation of the stepper motor; the gear transmission mechanism converts the rotation of the stepper motor into power to drive the grid sheet and the base plate frame to reciprocate relative to each other.

7. The interactive art therapy device for emotion recognition and triggering according to claim 1, characterized in that, It also includes a data recording module, which is used to anonymously record the frequency and duration of emotion recognition events to form group data.

Images