An emotional interaction intervention system fusing multi-sensory feedback

Abstract

This invention discloses an emotional intervention system integrating multi-sensory feedback, relating to the field of emotional intervention technology, and includes the following steps: S1, scene library construction; S2, scene-based preference collection; S3, personalized intervention scene generation; S4, real-time dynamic adjustment. This invention realizes the transformation of intervention content from standardized output to personalized customization. Based on a preference adaptation mechanism of children's autonomous feedback, it can accurately capture each child's unique interests, making the intervention content more aligned with their cognitive habits and emotional needs. This helps to stimulate children's active participation in the intervention process, reducing resistance. Furthermore, it achieves dynamic adaptation between the intervention content and the child's real-time state, enabling timely responses to fluctuations in children's attention and emotional changes during the intervention process. This avoids guidance failure due to rigid content, ensuring the intervention content remains attractive to children, thereby helping to maintain their focused attention.

Description

Technical Field

[0001] This invention relates to the field of emotion intervention technology, specifically to an emotional interaction and intervention system that integrates multi-sensory feedback. Background Technology

[0002] Existing child emotion intervention techniques mainly cover the following categories: one-on-one or group conversations between professional psychological counselors and children, using non-verbal communication methods such as sandplay therapy and art therapy to help children express emotions and guide them to recognize and manage negative emotions; designing specialized emotion management courses, using classroom teaching or gamified learning to teach children to identify emotions, understand the causes of emotions, and learn emotion regulation skills; and using virtual reality (VR) and smart wearable devices for intervention. VR technology can create immersive scenarios for children, simulating situations that trigger emotions, allowing children to learn coping methods in a safe environment; smart bracelets can monitor children's physiological indicators (such as heart rate and skin conductance), and when significant emotional fluctuations are detected, push relaxation training voice messages to guide them to calm down. Achieving emotional interaction through virtual reality technology can effectively achieve emotion intervention results.

[0003] Existing child emotion intervention techniques, especially those using virtual reality technology, often fail to capture individual child preferences in their scenario simulations (e.g., they do not differentiate between children's preferences for "cartoon" versus "natural" visuals, or their acceptance of "nursery rhymes" versus "white noise"), resulting in insufficient content appeal and low child cooperation. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an emotional intervention system that integrates multi-sensory feedback and emotional interaction, thus solving the problems mentioned in the background section.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an emotional interaction and intervention system integrating multi-sensory feedback, comprising the following steps: S1. Scene library construction: Deploy data acquisition equipment, multi-sensory output equipment, and core control units to build an intervention material scene library that includes three dimensions: scene theme, image type, and sound style; S2. Contextualized Preference Collection: The system combines different options such as scene theme, image type, and sound style, and sequentially displays the corresponding intervention materials to the user. During this process, it collects data on user gaze duration, number of times they actively point, and the proportion of pleasant expressions. By analyzing this data, a three-dimensional preference vector is generated. S3, Personalized Intervention Scenario Generation: The optimal combination is selected based on preference vectors and weight allocation algorithms to generate the initial intervention scenario; S4. Real-time dynamic adjustment: Real-time collection of attention and emotion feedback indicators, calculation of scene suitability, and adjustment of intervention content based on the suitability results.

[0006] Furthermore, in step S1, the scene library includes eight categories in the scene theme dimension: forest, ocean, space, castle, family, amusement park, classroom, and zoo; four categories in the image type dimension: cartoon animation, realistic image, simple line drawing, and interactive picture book; and four categories in the sound style dimension: natural sound effects, instrumental sounds, nursery rhymes, and white noise.

[0007] Furthermore, in step S2, the three-dimensional preference vector The calculation method is as follows: in, , , For the gaze duration of the corresponding material, , , For the number of times the pointer is actively pointed to, , , The percentage of expressions indicating pleasure.

[0008] Furthermore, in step S3, the initial intervention scenario The screening formula is: in, , , These are the weighting coefficients.

[0009] Furthermore, in step S4, the attention metrics collected in real time include the duration of eye movement away from the screen. Duration of gaze Emotional feedback indicators include the percentage of pleasant expressions. The proportion of irritable expressions .

[0010] Furthermore, scene adaptability The calculation formula is: in, The total duration of the intervention period is considered as the total duration of the intervention period. If the fit is less than 0.8, it is considered as an ineffective fit.

[0011] Furthermore, adaptability When the value is greater than or equal to 0.8, the adaptation is considered effective. At this time, the core elements of the scene remain unchanged, and the sound volume and screen brightness are gradually adjusted upwards or downwards in a gradient. Combined with the changes in user sensory feedback measured in real time, the volume and brightness are adjusted to the optimal value. The specific volume adjustment algorithm is as follows: Initial volume is Volume step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time To achieve the optimal volume; in, This represents the emotional feedback intensity function. This is the current set of basic parameters for the system, which includes scene features and user preference information; The brightness adjustment algorithm is as follows: Initial brightness is Brightness step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time For optimal brightness; Its existence, These represent visual scene parameters, encompassing the scene's environmental features and content type. For the first The brightness value at the next iteration.

[0012] Furthermore, when the adaptation fails, in the new scenario... The formula for generating it is: in, For the feature matrix of the new scene, The scene complexity coefficient. This is the scene preference weighting coefficient, used to measure the degree of user preference for different scene features.

[0013] This invention provides an emotional intervention system that integrates multi-sensory feedback and has the following beneficial effects: 1. This multi-sensory feedback-integrated emotional interaction intervention system constructs an intervention material library encompassing three dimensions: scene theme, visual type, and sound style. Employing standardized preference collection steps, it sequentially displays different combinations of intervention materials while collecting children's gaze behavior, active pointing actions, and facial emotional feedback. A preference vector is generated using a comprehensive calculation method combining multi-dimensional feedback data. A weighted allocation algorithm then filters out personalized intervention scenes that highly match the child's individual preferences. This achieves a shift from standardized output to personalized customization of intervention content, effectively solving the problem of low child cooperation caused by insufficient content appeal in traditional emotional interventions. This preference adaptation mechanism based on children's autonomous feedback accurately captures each child's unique interests, making the intervention content more aligned with their cognitive habits and emotional needs. This helps stimulate children's active participation in the intervention process, reduces resistance, creates a relaxed and pleasant atmosphere for emotional guidance, ensures the smooth and continuous implementation of intervention activities, and lays a solid foundation for improving subsequent intervention effects.

[0014] 2. This multi-sensory feedback-integrated emotional interaction intervention system collects real-time sensory feedback data on children's attention span and facial emotion changes during the intervention process. It uses an adaptation assessment formula to quantify the effectiveness of the current intervention scenario and flexibly adjusts the intervention strategy based on the assessment results. When the scenario is well-adapted, it fine-tunes output parameters to optimize the experience; when adaptation fails, it replaces the intervention content through a scenario switching algorithm. Simultaneously, it iteratively updates the preference vector based on the feedback data after each adjustment, achieving dynamic adaptation between the intervention content and the child's real-time state. This system can promptly respond to children's attention fluctuations and emotional changes during the intervention process, avoiding guidance failure due to rigid content. It ensures the intervention content remains attractive to children, thus helping to maintain their attention span, improve the targeting and effectiveness of emotional guidance, and help children gradually improve their emotional regulation abilities in a comfortable experience, achieving long-term stable intervention effects. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the steps of an emotional interaction and emotion intervention system that integrates multi-sensory feedback according to the present invention. Detailed Implementation

[0016] The embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and should not be construed as limiting the scope of the invention.

[0017] like Figure 1 As shown, the present invention provides a technical solution: an emotional interaction and emotion intervention system integrating multi-sensory feedback, comprising the following: Hardware module deployment: Data acquisition equipment: camera for data acquisition (capturing facial expressions and eye movements), touch interaction panel (recording active selection behavior), microphone (collecting sound feedback); Multi-sensory output devices: high-definition display terminal (playing scene images), adjustable speaker (outputting sound materials); Data Processing Center: Used for data processing, preference calculation, and content adjustment; The specific implementation steps are as follows: S1. Scene library construction: It covers 3 major dimensions and 24 categories of expandable intervention materials, as detailed below: Scene theme dimension: Includes 8 scene themes: forest, ocean, space, castle, family, amusement park, classroom, and zoo; Visual Types: There are four visual types: cartoon animation, realistic images, simple line drawings, and interactive picture books. Sound style dimensions: divided into four categories: natural sound effects (such as birdsong, flowing water, and wind), instrumental sounds (such as piano and violin), nursery rhymes, and white noise. S2. Contextualized Preference Collection: This step is used to determine the child's preferred scenes, sounds, and images. The preference test process is as follows: The system displays 24 categories of materials in a combination of "scene theme, image type, and sound style" (such as "space theme, cartoon image, bird song sound effect" and "forest theme, realistic image, flowing water sound effect"). Real-time collection of three types of feedback indicators from children: Attention metric: Duration of gaze on footage captured by the camera. This reflects the level of attention; Active preference indicator: Number of times children actively point at objects recorded by the touch panel. This reflects the degree of liking; Emotional feedback metric: The percentage of pleasant expressions extracted by facial recognition algorithms This reflects the level of acceptance; Based on the above data, a three-dimensional preference vector is calculated and generated. (Values ​​range from 0 to 1, with larger values ​​indicating stronger preferences): in, The theme preference coefficient (e.g., space theme scores the highest) is used for scene theme preference coefficient. Approaching 1); This is a preference coefficient for image type (e.g., cartoon images score the highest). Approaching 1); This is a sound style preference coefficient (e.g., the highest score for natural sound effects is...). Approaching 1); For example, if a child's gaze duration for "space theme, cartoon image, and bird song sound effect" is 30 seconds, they actively point at it twice, and their happy expression accounts for 80%, then the overall score for this combination is 30×0.4+2×0.8+80%×0.8=12+0.6+0.24=12.84. The system calculates the preference coefficient based on the relative value of the scores for all combinations. S3, Personalized Intervention Scenario Generation: Based on preference vector The optimal combination is selected from the scenario library to generate the initial intervention scenario. , As a scene theme, As for the type of image, For sound style; The selection formula is as follows: As for scene theme weight, Image type weighting Weighting for sound style; S4. Real-time dynamic adjustment: The system outputs the initial intervention scenario. Simultaneously, four types of sensory feedback indicators are collected in real time: Attention metric: Duration of eye movement away from screen Duration of gaze ; Emotional feedback indicator: Percentage of pleasant expressions The proportion of irritable expressions ; Calculate the fit of the current scene. (Values ​​range from 0 to 1; values ​​below 0.8 are considered adaptation failures). The formula is as follows: in, The total duration of the intervention period. For the percentage of attention focused, The percentage of emotional acceptance; when (i.e., effective adaptation): Keeping the core elements of the scene unchanged, the sound volume and screen brightness are gradually adjusted upwards or downwards in a gradient. Combined with real-time measured changes in user sensory feedback, the volume and brightness are adjusted to the optimal value. The specific volume adjustment algorithm is as follows: Initial volume is Volume step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time To achieve the optimal volume; in, This represents the emotional feedback intensity function. This is the current set of basic parameters for the system, which includes scene features and user preference information; The brightness adjustment algorithm is as follows: Initial brightness is Brightness step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time For optimal brightness; Its existence, These represent visual scene parameters, encompassing the scene's environmental features and content type. For the first Brightness value at the next iteration; when (i.e., adaptation failure): Triggers scene element switching, generating a new scene. The switching formula is: in, For the feature matrix of the new scene, The scene complexity coefficient is set as follows (dynamic scene = 0.8, static scene = 0.2). When adaptation fails, the lower complexity scene is prioritized. This is the scene preference weighting coefficient, used to measure the degree of user preference for different scene features.

[0018] Example: The above procedure was used to intervene in a 6-year-old child (who was emotionally agitated); The system displays materials in combinations based on "scene theme, visual type, and sound style," and collects three types of feedback data from target children (user gaze duration, number of times they actively point, and percentage of expressions of pleasure), as well as a three-dimensional preference vector. The calculation (based on the scene theme preference coefficient) For example, the rest are similar): Final preference vector: ; Calculate the initial intervention scenario according to the screening formula: ; , , ; Target combination score calculation: The highest score indicates the initial intervention scenario. Space theme + cartoon spaceship animation + bird song sound effects; Output initial parameters: brightness cd / m², volume dB; Thirty seconds after the intervention, feedback data is collected, and the intervention effect is evaluated according to preset judgment rules. Attention metrics: Recording time spent with attention elsewhere Concentrated time Total duration ; Emotional feedback metrics: Among the percentages of users' emotional states, the percentage of pleasant emotions... The proportion of irritability According to the fit calculation formula Substituting the data, we get: 0.8 indicates that the current emotional interaction intervention strategy is highly compatible with the user's state. The system will continue to execute the current intervention plan. The system will immediately terminate the current plan and regenerate the intervention strategy; Introducing an emotional feedback intensity function (Quantitative indicators: gaze duration + percentage of pleasant expressions, unit: minutes), initial state point; Volume adjustment iteration ( ): First iteration: dB At this point, adjust the direction and reverse. Second iteration: dB At this point, continue adjusting in the opposite direction; 3rd iteration: dB At this point, maintain that direction; 4th iteration: dB At this point, the iteration stops. Optimal volume: dB, satisfying and ; Brightness adjustment ( =50cd / m²): First iteration: cd / m² point, At this point, adjust the direction to reverse; Second iteration: cd / m² At this point, continue adjusting in the opposite direction; 3rd iteration: cd / m² At this point, maintain that direction; 4th iteration: cd / m² At this point, the iteration stops. Optimal brightness: ,satisfy and ; in , =50cd / m² is the optimal solution obtained from experiments on children's auditory and luminous sensitivity; Intervention results: Children's attention span increased by 40% (from 15 minutes / session to 21 minutes / session), and the percentage of happy expressions remained stable at over 85%.

[0019] In summary, this multi-sensory feedback-integrated emotional interaction intervention system constructs an intervention material library encompassing three dimensions: scene theme, image type, and sound style. Employing standardized preference collection steps, it sequentially displays different combinations of intervention materials while collecting children's gaze behavior, active pointing actions, and facial emotional feedback. A preference vector is generated using a comprehensive calculation method combining multi-dimensional feedback data. A weighted allocation algorithm then filters out personalized intervention scenarios that highly match the child's individual preferences. This achieves a shift from standardized output to personalized customization of intervention content, effectively solving the problem of low child cooperation caused by insufficient content appeal in traditional emotional interventions. This preference adaptation mechanism based on children's autonomous feedback accurately captures each child's unique interests, making the intervention content more aligned with their cognitive habits and emotional needs. This, in turn, helps stimulate children's active participation in the intervention process, reduces resistance, creates a relaxed and pleasant atmosphere for emotional guidance, ensures the smooth and continuous implementation of intervention activities, and lays a solid foundation for improving subsequent intervention effects. Furthermore, by collecting sensory feedback data such as children's attention span and facial emotion changes in real time during the intervention, and using an adaptation assessment formula to quantify the effectiveness of the current intervention scenario, the intervention strategy is flexibly adjusted based on the assessment results. When the scenario is well-adapted, the output parameters are fine-tuned to optimize the experience; when the scenario fails, the intervention content is replaced through a scenario switching algorithm. At the same time, the preference vector is iteratively updated based on the feedback data after each adjustment. This achieves dynamic adaptation between the intervention content and the child's real-time state, enabling timely responses to children's attention fluctuations and emotional changes during the intervention process. It avoids guidance failure caused by rigid content and ensures that the intervention content remains attractive to children. This helps to maintain children's attention span, improve the pertinence and effectiveness of emotional guidance, and help children gradually improve their emotional regulation ability in a comfortable experience, achieving long-term and stable intervention effects.

[0020] The embodiments of the present invention are given for illustrative and descriptive purposes only, and are not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention, and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. An emotional intervention system integrating multi-sensory feedback, characterized in that: Includes the following steps: S1. Scene library construction: Deploy data acquisition equipment, multi-sensory output equipment, and core control units to build an intervention material scene library that includes three dimensions: scene theme, image type, and sound style; S2. Contextualized Preference Collection: The system combines different options such as scene theme, image type, and sound style, and sequentially displays the corresponding intervention materials to the user. During this process, it collects data on user gaze duration, number of times they actively point, and the proportion of pleasant expressions. By analyzing this data, a three-dimensional preference vector is generated. S3, Personalized Intervention Scenario Generation: The optimal combination is selected based on preference vectors and weight allocation algorithms to generate the initial intervention scenario; S4. Real-time dynamic adjustment: Real-time collection of attention and emotion feedback indicators, calculation of scene suitability, and adjustment of intervention content based on the suitability results.

2. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 1, characterized in that: In step S1, the scene library includes four categories: natural, humanistic, and fantasy in the scene theme dimension; four categories in the image type dimension: cartoon animation, realistic image, simple line drawing, and interactive picture book; and four categories in the sound style dimension: natural sound effects, instrumental sound, children's song, and white noise.

3. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 1, characterized in that: In step S2, the three-dimensional preference vector The calculation method is as follows: in, , , For the gaze duration of the corresponding material, , , For the number of times the pointer is actively pointed to, , , The percentage of expressions indicating pleasure.

4. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 1, characterized in that: In step S3, the initial intervention scenario The screening formula is: in, , , These are the weighting coefficients.

5. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 1, characterized in that: In step S4, the attention metrics collected in real time include the duration of eye movement away from the screen. Duration of gaze Emotional feedback indicators include the percentage of pleasant expressions. The proportion of irritable expressions .

6. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 5, characterized in that: Scene adaptability The calculation formula is: in, The total duration of the intervention period is considered as the total duration of the intervention period. If the fit is less than 0.8, it is considered as an ineffective fit.

7. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 6, characterized in that: compatibility When the value is greater than or equal to 0.8, the adaptation is considered effective. At this time, the core elements of the scene remain unchanged, and the sound volume and screen brightness are gradually adjusted upwards or downwards in a gradient. Combined with the changes in user sensory feedback measured in real time, the volume and brightness are adjusted to the optimal value. The specific volume adjustment algorithm is as follows: Initial volume is Volume step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time To achieve the optimal volume; in, This represents the emotional feedback intensity function. This is the current set of basic parameters for the system, which includes scene features and user preference information; The brightness adjustment algorithm is as follows: Initial brightness is Brightness step size is Perform the following cyclical adjustments: when hour, ; when hour, ; Until and ,at this time For optimal brightness; Its existence, These represent visual scene parameters, encompassing the scene's environmental features and content type. For the first The brightness value at the next iteration.

8. The emotional interaction and emotion intervention system integrating multi-sensory feedback according to claim 6, characterized in that: When adaptation fails, new scene The formula for generating it is: in, For the feature matrix of the new scene, The scene complexity coefficient. This is the scene preference weighting coefficient, used to measure the degree of user preference for different scene features.

Images