Emotion healing system based on active sound and picture fusion

Abstract

The invention discloses an emotion healing system based on active sound and picture fusion, and belongs to the field of emotion regulation technologies and digital healing. According to the system, an active sound and picture fusion emotion healing scheme is provided, a user performs drawing creation on a touch interface, the system recognizes the shape and position of drawing pen touch in real time, corresponding timbre, pitch and visual feedback are matched according to a preset sound and picture mapping rule, and music synchronously corresponding to the drawing creation content of the user is dynamically generated. Multiple emotion theme modes are built in the system, each theme corresponds to different musical scales, tone combinations and background color matching, and a user can select or customize themes according to the current emotion state. According to the invention, through multi-sensory stimulation, the art creation process of guiding the user to participate actively, the cognitive participation of the user is enhanced, the emotion expression is promoted, a brand-new active emotion healing approach is provided for the user, the user is effectively helped to express and recognize, the negative emotion of the user is effectively adjusted, and the mental health level is improved.

Description

Technical Field

[0001] The present invention relates to the fields of emotion regulation technology and digital therapy, and in particular to an emotion therapy system based on active audio-visual fusion. Technical Background

[0002] With the accelerating pace of life and increasing pressure in modern society, mental health issues are becoming increasingly prominent, especially among young people, where depression, anxiety, and other emotional distress are prevalent. Some emotion regulation tools or apps currently on the market focus on providing relaxing meditation content, with users primarily passively receiving the content. Others may incorporate single artistic elements, such as digital painting or a metronome, but lack mechanisms for deep integration and real-time interaction across diverse sensory inputs. These tools often lack active user engagement, the depth of personalized emotional expression, and the immersive experience brought about by multi-sensory synergy. Art therapy theory suggests that through creative activities, individuals can externalize internal emotions, thereby achieving emotional relief and self-awareness. However, how to effectively combine the artistic creation process with immediate, emotionally relevant sensory feedback to enhance the effectiveness and enjoyment of emotion regulation remains a pressing technical challenge.

[0003] Art therapy is a non-drug intervention that uses artistic activities as a medium to regulate emotions and promote mental health through artistic creation, expression, and experience. It encompasses a variety of modalities, including music therapy, art therapy, dance therapy, and drama therapy. Its application in psychotherapy and health intervention has become increasingly widespread in recent years. Research has shown that art therapy not only provides a safe avenue for emotional expression but also helps individuals recover from negative emotions by stimulating their creativity and attentional diversion mechanisms. Among the many art forms, music and art therapy are the most widely used due to their accessibility and practicality. Audio-visual fusion therapy combines painting and music to regulate emotions, creating a multimodal emotional experience. At the same time, with the widespread use of smart devices and mobile applications, the digitization of emotional healing tools is becoming a trend. Through the design and implementation of digital systems, art therapy can be integrated into users' daily lives, providing them with a safe and convenient tool for emotional regulation. Summary of the Invention

[0004] The purpose of this invention is to build an emotional healing system based on active audio-visual fusion, aiming to overcome the limitations of existing emotional regulation tools in terms of user engagement, personalized feedback and multi-sensory interaction. By generating matching music in real time based on the user's active painting creation behavior, it provides users with a novel, immersive and effective way to express and regulate emotions.

[0005] The technical solution adopted in the present invention is as follows:

[0006] An emotional healing system based on active audio-visual fusion, including:

[0007] The creative canvas module receives user drawing strokes and generates corresponding musical feedback in real time. Specifically, it receives user drawing strokes in real time, automatically classifies the strokes through shape recognition technology, and converts them into notes of specific timbre. The pitch is determined by the vertical position of the strokes on the canvas. As the user draws, a melody sequence is dynamically generated from left to right according to the horizontal position, supporting instant playback and looping playback.

[0008] The local works module is used to store and manage the works generated by users in the creation canvas module;

[0009] Theme definition module allows users to customize the color scheme and theme name of each emotional theme;

[0010] The inspiration gallery module provides a gallery of images categorized by emotional valence and arousal, allowing users to select images as the background for their creative canvases.

[0011] The Relaxation Space module provides a passive relaxation function combining meditation background videos and white noise;

[0012] The system settings module is used to adjust brush stroke parameters, playback speed and canvas display properties.

[0013] As a preferred embodiment of the present invention, the creative canvas module integrates multiple sets of emotional theme systems, each theme is independently configured with scale rules, timbre mapping relationships and visual color schemes, and users can switch themes instantly to change their creative style.

[0014] As a preferred embodiment of the present invention, the creation canvas module includes:

[0015] A stroke recognition unit that calls a shape recognition algorithm when the user completes a single stroke and classifies the stroke as a line, circle, or polygon;

[0016] The timbre mapping unit maps the shape of the brush stroke to a specific timbre according to preset rules, where circles, lines, and polygons are associated with different categories of timbre respectively;

[0017] The pitch mapping unit divides the vertical area of ​​the canvas into multiple pitch zones, each corresponding to a note in the musical scale. The vertical center position of the brush stroke determines the pitch of the generated note;

[0018] The melody generation unit sorts the notes generated by multiple strokes from left to right according to their horizontal positions on the canvas to form a melody sequence that can be played in a loop;

[0019] The theme switching unit provides at least four preset emotional themes, each of which is associated with an independent scale, timbre mapping rules and visual color scheme.

[0020] As a preferred embodiment of the present invention, the preset rules of the timbre mapping unit include:

[0021] Circular brushstrokes map to a timbre in trumpet, flute, vocal choir, or vibraphone;

[0022] Line strokes are mapped to a sound from fingerpicked bass, marimba, organ, or glockenspiel;

[0023] The polygonal brushstrokes are mapped to a sound from a drum kit, percussion, or piano.

[0024] As a preferred embodiment of the present invention, the theme switching unit includes four preset themes:

[0025] High-arousal, high-valence themes are mapped to the C major scale;

[0026] The high-arousal, low-valence theme maps to the F major scale;

[0027] The low-arousal, high-valence theme maps to the A minor scale;

[0028] The low-arousal, low-valence theme maps to the D minor scale.

[0029] As a preferred embodiment of the present invention, the working logic of the melody generation unit includes:

[0030] Play the musical note corresponding to each brush stroke in real time while the user is drawing;

[0031] When the user actively triggers playback or completes drawing, a continuous melody is generated from left to right according to the horizontal position of the brush strokes;

[0032] Supports looping melody sequences to form complete music clips.

[0033] As a preferred embodiment of the present invention, the color customization function of the theme definition module includes:

[0034] Adjust the canvas background color;

[0035] Set the display colors of line, circle and polygon strokes respectively;

[0036] The color scheme is associated with the emotional theme but does not change the timbre and scale mapping rules.

[0037] As a preferred embodiment of the present invention, the images in the inspiration gallery module are classified based on the arousal and valence of emotions, where valence refers to the degree of positivity or negativity of an emotion and arousal refers to the intensity of an emotion;

[0038] The types include a high-titer high-awakening category, a high-titer low-awakening category, a low-titer high-awakening category and a low-titer low-awakening category, and each category contains multiple pictures matched with emotional features.

[0039] As a preferred embodiment of the application, the relaxation space module comprises multiple meditation theme combinations, each theme being associated with a specific scene video and matching white noise, and theme interface switching is achieved by left and right sliding to realize quick selection.

[0040] As a preferred embodiment of the application, the system setting module comprises a brush stroke thickness adjustment function, a melody playing speed adjustment function, a canvas grid display switch and an automatic saving function switch.

[0041] The application discloses an emotion healing system based on active sound-picture fusion, which can generate notes in real time through user's painting creation, and output corresponding music in combination with different music styles and emotional dimensions, so as to realize emotional regulation of the user. The application builds a closed-loop "behavior-perception-feedback" system through the collaborative work of a creation canvas module, a local work module, a theme definition module, an inspiration gallery module, a relaxation space module and a system setting module. Each painting behavior of the user can be immediately converted into unique auditory and visual feedback. This active and creative interactive process can not only effectively attract the attention of the user and divert bad emotions, but also stimulate the desire of the user to express emotions, so that the effect of emotional relief and psychological healing is achieved through the "externalization" process of artistic creation. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 It is a system function architecture diagram of the application;

[0043] Figure 2 It is a basic interface layout of the system of the application;

[0044] Figure 3 It is a system use flowchart of the application. DETAILED DESCRIPTION

[0045] The application will be further described and explained in conjunction with the specific embodiments. The technical features of each embodiment in the application can be combined accordingly without conflict.

[0046] The core design concept of the "emotion healing system based on active sound-picture fusion" is to realize the expression, regulation and healing of emotions by combining the active painting creation of the user with the immediate and personalized sound-picture feedback. The system closely combines art therapy and emotion regulation theory, and aims to help the user relieve negative emotions and improve the level of mental health through creative artistic experience.

[0047] The function architecture of the emotion healing system based on active sound-picture fusion is as follows: Figure 1The system mainly includes a creation canvas module, a local work module, a theme definition module, an inspiration gallery module, a relaxation space module, and a system setting module. The system is developed based on Swift language and SwiftUI on the Xcode platform, uses CoreGraphics for real-time painting rendering, and combines AVFoundation to process audio playback.

[0048] Creation canvas module. This module is the core functional area of the software, and users can freely draw on the canvas. The system analyzes the user's drawing behavior in real time. Specifically, the system calls a shape recognition algorithm every time the user finishes a stroke, such as when the user's finger or stylus leaves the touch interface, to determine the shape of the independent stroke. For complex graphics that may be composed of multiple strokes, such as when the user intends to draw a polygon but combines multiple independent lines, the current system design recognizes each independent line stroke and generates a note for it. Subsequently, by arranging the notes generated by each stroke from left to right according to the user's spatial arrangement or organizing them according to a certain temporal logic, an ordered note sequence is formed, which can then be played in a loop, thus constituting a continuous melody. This module design takes into account the interactivity of mood regulation and the freedom of creation. The system can recognize each stroke of the user's drawing, determine whether it is a line, a circle, or a polygon, and map it to a specific timbre. For example, a circle may correspond to a trumpet timbre, a line may correspond to a guitar timbre, and a polygon may correspond to a piano timbre. This mapping of stroke shape to timbre is pre-set by the system, and its rationality is mainly based on the emotional associations or auditory perceptions that different timbres can evoke in music psychology. For example, different musical instruments may produce timbres that are brighter or softer.

[0049] In this invention, stroke shape indirectly participates in shaping the emotional characteristics of music by influencing the selection of timbre. For example, simpler and smoother shapes (such as circles and lines) may be mapped to timbres that evoke a sense of tranquility or joy, while complex or irregular shapes (such as sharp polygons) may be mapped to timbres that are more tense or dynamic.

[0050] In the present invention, the vertical area of ​​the canvas is divided into several pitch areas, each area corresponds to a note in the selected scale, and the vertical position of the canvas determines the pitch of the note (such as do, re, mi, etc. in the scale). For example, in C major, it may be mapped to C, D, E, F, G, A, B, etc. from bottom to top. Under different emotional themes, the areas of the canvas will be mapped to scales in different ways. This embodiment presets four emotional themes: "Dawn Glow", "Morning Light", "Storm Coming", and "Dusk Silence" and their corresponding scales. For example, the "Dawn Glow" theme (high arousal, high valence) corresponds to C major, which is usually associated with positive and bright emotions; the "Dusk Silence" theme (low arousal, low valence) corresponds to D minor, which is often used to express relatively quiet or melancholic emotions. Table 1 shows the mapping relationship of the preset emotional themes.

[0051] Table 1

[0052]

[0053] The selection of these scales is based on research into the correlation between different modes (major, minor, specific scale patterns, etc.) and emotional expression in music theory, thus ensuring the objectivity of the design. The vertical center position of the user's brushstroke determines the pitch of the note generated by that brushstroke. As the user draws on the canvas, the system plays the notes in real time based on the classification and position of the brushstrokes. As the user continues to draw, when multiple brushstrokes are completed, the system arranges the notes generated by these brushstrokes according to their horizontal position on the canvas and plays them one by one in order from left to right. These strokes sound in sequence, just like the notes recorded from left to right on a musical score, forming a melody that brings the user a gradually accumulated emotional expression. In addition, users can switch between different themes by clicking the four theme buttons above the canvas to adjust the corresponding musical style and emotional adjustment mode. Each theme has a different mapping relationship between timbre, scale, and painting shape, helping users adjust the musical style and emotional expression according to different emotional needs.

[0054] Local Works Module. This module provides a work management space where each work completed by the user on the creative canvas can be saved and added to the local works list. This module is designed to facilitate users to batch manage and edit their emotion regulation works. After the user completes a work on the creative canvas, they can save the work to the local works module by clicking the Save button. Each saved work can be edited or reused at any time, allowing users to make adjustments and modifications. Users can manage saved works in batches, including deleting unnecessary works to keep the work library tidy. At the same time, users can also quickly enter the creative canvas by clicking on each work to make modifications or continue working.

[0055] Theme definition module. Through this module, the user can be deeply involved in the personalized customization of the system, increasing the sense of participation and control of the emotional regulation process. Since color is closely related to emotion, adjusting the warmth, brightness, etc. of the color can further enhance the effect of emotional regulation, therefore, the user can adjust the appearance and emotional regulation mode of the system according to personal preferences by customizing the color scheme of four themes, in order to enhance the emotional regulation benefit. Here, "color matching" specifically refers to the main color elements that constitute the visual environment of the theme, including the background color of the creation canvas and the visual feedback effect (line color, circle color, polygon color) that the painting brush may present under a certain theme. When the user paints on the canvas, it will directly be on the custom visual effect set by the selected theme. Different color matching schemes may be suitable for different emotions, for example, warm tones such as red or orange may be suitable for high-value emotions, while cool tones such as blue or green may help to relieve anxiety or depression. In addition to the color matching scheme, the user can also customize the name of each theme, thereby defining the way of emotional regulation according to personal feelings. Through the adjustment of the theme name, the user can create an environment that is closer to their emotional needs, enhancing the personalized experience of emotional healing. In the present invention, the user can customize the theme name, but cannot add or delete themes, and the main purpose of theme name customization is to allow the user to use labels that they understand and identify to mark a certain pre-set emotional theme. This behavior itself does not change the pre-set mapping relationship of the theme's timbre, pitch, and drawing shape.

[0056] Inspiration gallery module. This module mainly plays an auxiliary creation function, providing an option for users who lack drawing inspiration or may need external visual stimulation to start the creation process or guide emotions. Users can also choose not to use this module and directly enter the creation canvas module for free creation. In the inspiration gallery module, four emotional categories (happy time, quiet time, exciting time, and contemplative time) are provided, each containing six pictures. These pictures help users enter a specific emotional state and serve as a source of inspiration for drawing. Users can choose their favorite pictures to copy and form an emotional regulation background. The pictures in the gallery are classified according to the arousal and valence of emotions, and each picture corresponds to a certain emotional characteristic, helping users find the appropriate emotional regulation path in different emotional states. Among them, the "valence" of emotion refers to the positive or negative degree of emotion, that is, the continuous dimension from pleasant to unpleasant; the "arousal" of emotion refers to the intensity of emotion, that is, the level of physiological activation, from calm to excited. In this invention, pictures are classified according to their potential valence (e.g., high valence represents pleasure, low valence represents displeasure) and arousal (e.g., high arousal represents excitement, low arousal represents calmness). For example, the pictures under the "happy time" category aim to evoke positive emotions with high valence and medium-high arousal. In this module, users can select pictures from the gallery and use them as the background of the creation canvas for copying, further stimulating creative inspiration. With the help of the background, users can achieve a clearer emotional expression goal during the creation process, making the combination of music and vision more in line with personal emotional needs.

[0057] Relaxation space module. This module is designed to help users relax and relieve anxiety and stress through the combination of meditation and white noise. This module combines meditation background videos with ambient sounds and provides multiple themes to help users relax deeply. When the user chooses to start meditation, the system will provide a variety of background videos and corresponding white noise (such as rain, pine wind, bonfire, etc.) to help users enter a meditative state, reduce stress, and promote emotional stability. Users can also swipe left and right to switch different meditation themes according to their emotional needs. For example, "Rain Falls and Calms" is suitable for a quiet environment, while "Blue Sea Cleanses the Heart" provides a more gentle environment for anxious users and enhances the effect of emotional regulation. Although there are independent systems on the market that provide meditation and white noise functions, the innovation of the "Relaxation Space" module in the present invention is not reflected in its basic functions in isolation, but in its role as an organic part of the "active audio-visual fusion emotional healing system." Its value lies in: 1) Integration and complementarity: It provides users with a passive, guided way to relax in addition to the active and creative emotional expression through the creative canvas module. Users can use this module to pre-process their emotions (such as reducing anxiety) before engaging in high-intensity creative expression, or to settle and calm their emotions after creation, forming a more complete closed loop of emotional regulation experience. 2) System consistency: The design of this module (such as theme selection, visual style, etc.) can be consistent with the overall emotional healing concept of the system, providing users with a unified user experience. 3) Multimodal healing path: For some users or at certain specific moments, they may prefer or need passive relaxation methods rather than active creation. This module meets this diverse demand. Therefore, the relaxation space module and active modules such as the creative canvas together constitute the multi-dimensional, multi-path emotional healing solution of the present invention. Its relevance is reflected in providing users with a comprehensive set of emotional management tools, rather than just a stacking of functions. It enhances the applicability of the entire system in different situations and the user's freedom of choice, thereby improving the overall emotional healing effectiveness.

[0058] System Settings. This module allows users to personalize various aspects of the system to enhance the user experience. Users can adjust the system's operation according to their needs and preferences: they can customize brushstroke size and other factors to increase their creative freedom; they can set the playback speed and turn sound feedback on and off to adapt to different emotional states; and they can adjust the canvas's background grid, default theme, and auto-save to enhance the personalization of their creations.

[0059] Through the collaborative work of these modules, this invention constructs a closed-loop "behavior-perception-feedback" system. Every user's drawing action is instantly transformed into unique auditory and visual feedback. This active, creative interactive process not only effectively attracts the user's attention and diverts negative emotions, but also stimulates the user's desire to express their emotions. Through the "externalization" process of artistic creation, it achieves the effects of emotional counseling and psychological healing.

[0060] The basic interface layout of the system is as follows Figure 2 Specifically, it includes:

[0061] Left navigation bar: used to carry the entrance to the major functional modules of the system. Users can switch to different functional modules by clicking the buttons in this area, such as switching to the "Creation Canvas Module", "Local Works Module", "Theme Definition Module", "Inspiration Gallery Module", "Relaxation Space Module" or "System Settings Module".

[0062] Top information bar: used to display the system logo and slogan as well as the user's current login status information.

[0063] Theme Switcher: Located above the canvas area and below the top information bar, it directly corresponds to the theme selection function in the "Creation Canvas Module". Users can quickly switch between four preset or customized emotional themes (such as "Dawn Radiance" and "Morning Light Softness").

[0064] The Canvas Area is the core interactive area of ​​the Creative Canvas Module. Users create their drawings here, and the system converts audio and video in real time. If users select an image to copy from the Inspiration Gallery Module, that image will also be displayed as the background in this area.

[0065] Bottom control bar: may contain specific operation tools related to the "Creation Canvas Module", such as brush settings (corresponding to some functions in the "System Settings Module"), color selection, play / pause currently generated music, undo / redo, save work (corresponding to the entrance of the "Local Works Module"), etc.

[0066] These modules are implemented through Figure 2 The interface layout shown is organically organized and works together to provide users with a complete emotional healing experience from creation, feedback, regulation to personalized customization, helping users express emotions, alleviate negative emotions such as depression, and promote self-awareness.

[0067] The usage process of the emotional healing system based on active audio-visual fusion is as follows: Figure 3As shown in the figure, the user first enters the creative canvas module. The canvas appears as a blank area where the user can freely draw using the touch screen. Each stroke is mapped to a note in real time based on its shape and vertical position, and changes in timbre and pitch are immediately reflected on the screen. As the user draws, the system instantly recognizes the shape of the stroke and maps the pitch based on the position. The timbre and pitch of each stroke are continuously accumulated and played back during the user's creative process, forming a coherent melody.

[0068] The primary goal of this system's melody generation is not to pursue traditional musical beauty or professional composition standards. Its rationality and healing mechanism are mainly reflected in the following points:

[0069] 1) Structured Framework: The system presets specific scales (e.g., C major, D minor, etc.) and timbres under different emotional themes. This means that the notes produced by the user fall within a specific tonal framework, which itself provides a certain basis for musical harmony and avoids completely dissonant random notes.

[0070] 2) User-initiated creation and emotional externalization: The core of emotional healing lies in the process of "active" participation and emotional "externalization." Through the user's own painting behavior (brushstroke shape, position, speed, pressure, etc.), the user directly influences musical elements such as timbre, pitch, and rhythm. This creative process itself is a form of emotional expression and release. The immediate feedback of the music allows users to feel the connection between their inner state and the external creation.

[0071] 3) Personalization and Exploration: Even users without musical knowledge can explore the connection between sound and emotion through intuitive drawing. The generated music is personalized and directly reflects the user's current creative intent and state. This process of exploration and "play" helps divert attention and alleviate negative emotions.

[0072] 4) Multi-sensory synergistic stimulation: The synchronous integration of vision (painting) and hearing (music) enhances immersion and cognitive participation, which in itself has a positive effect on emotional regulation. The healing effect comes more from the experience of the creative process, the free expression of emotions, and the resulting sense of control and accomplishment.

[0073] This invention provides a tool and outlet for emotional expression. Experimental verification, combined with EEG data and subjective scale results, shows that under all four subject conditions, the system can regulate the subjects' depressive emotions to varying degrees and improve the subjects' depressive emotional state. The results of the subjective scale filling of the subjects' emotional regulation conditions for the four subjects are shown in Table 2, and the results of the EEG objective data analysis are shown in Table 3.

[0074] Table 2

[0075]

[0076] *p<0.05, **p<0.01

[0077] Table 3

[0078]

[0079]

[0080] *p<0.05, **p<0.01

[0081] Table 2 shows the changes in arousal and valence scores for the high and low emotion regulation difficulty groups under different emotion regulation themes. Paired t-tests were used to analyze the subjective scores for different emotion themes, revealing significant differences between the high and low emotion regulation difficulty groups for some emotion themes, as follows:

[0082] Regarding arousal scores, at baseline, the high (6.50±0.85) and low (6.30±0.70) groups with emotion regulation difficulties had higher arousal scores, indicating that all participants successfully entered a high-arousal, depressive state during the mood induction phase. Arousal scores significantly decreased in both groups during the Dawning theme, with those in the high (4.00±0.82) group with emotion regulation difficulties scoring higher compared to the low (3.00±0.45) group. This suggests that the theme had a more pronounced effect on reducing arousal in individuals with low emotion regulation difficulties (p<0.05*), effectively alleviating their high-arousal, depressive mood. Arousal scores in the Morning Light theme did not differ significantly between the two groups (high: 5.50±1.00, low: 5.80±1.10, p=0.08), indicating that the theme had a similar arousal-modulating effect in individuals with both high and low emotion regulation difficulties. For the Stormy theme, there was no significant difference between the high group (4.80±0.65) and the low group (4.60±0.60), indicating that the theme reduced arousal to some extent in both groups, but the regulatory effect was limited (p=0.09). For the Twilight Silence theme, arousal in both groups was significantly lower than baseline, with no significant difference between the high group (3.80±0.70) and the low group (3.60±0.50) (p=0.12), indicating that the theme effectively reduced arousal and enabled participants to enter a state of low arousal and relaxation.

[0083] Valence scores were low in both groups at baseline (high group: 3.80±0.55, low group: 4.10±0.65), indicating that participants were in a low-valence, depressive mood after the mood induction. Valence scores significantly increased with the Dawn Radiance theme, with the high group (5.00±0.82) significantly lower than the low group (5.80±0.45, p<0.05*), indicating that this theme had a more pronounced mood-enhancing effect on individuals with low mood regulation difficulties, making it easier for them to enter a positive mood state. Valence scores for the Morning Light theme showed a significant difference (p<0.01**) between the high group (6.00±0.50) and the low group (6.80±0.55), indicating that this theme had a more pronounced positive mood-regulating effect on individuals with low mood regulation difficulties, making it easier for them to experience pleasant emotions. For the "Storm Coming" theme, there was no significant difference in valence scores between the two groups (high group: 4.00±0.40, low group: 4.50±0.80, p=0.07), indicating that this theme had a weak valence-regulating effect on individuals with varying emotion regulation difficulties, failing to significantly enhance emotional well-being. For the "Twilight Silence" theme, valence scores were similar between the high group (5.20±0.60) and the low group (5.70±0.40), with no significant difference (p=0.10), indicating that this theme had a stable emotion-regulating effect across individuals and helped participants achieve a stable state of relaxation.

[0084] Table 3 shows the EEG data for subjects with high and low emotion regulation difficulties. It can be observed that within the high emotion regulation difficulty group, the EEG asymmetry index under different emotion themes exhibited significant differences at specific electrode pairs. For example, under the "Dawn Glow" theme, the asymmetry index at the T8-T7 electrodes was significantly higher in the high emotion regulation difficulty group (0.50±0.70) than in the low emotion regulation difficulty group (0.25±0.60), indicating that this theme effectively reduces arousal levels in individuals with high emotion regulation difficulties while simultaneously increasing valence, shifting them toward a moderately aroused, high-valence emotional state. Furthermore, at the P4-P3 electrode pair, the EEG asymmetry index for this theme was slightly lower in the high emotion regulation difficulty group (-0.30±0.55) than in the low emotion regulation difficulty group (-0.20±0.50), further demonstrating that "Dawn Glow" helps regulate negative emotions and enhance emotional pleasure.

[0085] Under the "Morning Light and Gentleness" theme, the asymmetry index of the F4-F3 (0.38±0.50) and T8-T7 (0.20±0.40) electrode pairs decreased in the low-emotional regulation difficulty group compared to the high-emotional regulation difficulty group (F4-F3: 0.40±0.55; T8-T7: 0.30±0.45). Furthermore, the P4-P3 index was lower in the low-emotional regulation difficulty group (-0.10±0.45) than in the high-emotional regulation difficulty group (-0.15±0.50). This suggests that this theme has a stronger emotional regulation effect in individuals with low emotional regulation difficulty, effectively reducing arousal levels and maintaining a high-valence emotional state, placing the subjects in a more calm and relaxed state.

[0086] For the "Storm is Coming" theme, the T8-T7 electrode index (0.60±0.80) of the high emotion regulation difficulty group was significantly higher than that of the low group (0.55±0.70). At the same time, the P4-P3 index in the high group (-0.10±0.45) was lower than that in the low group (-0.05±0.40), indicating that this theme may induce higher emotional fluctuations in individuals with high emotion regulation difficulty, and the regulatory effect is relatively limited, failing to significantly reduce their arousal level. However, for individuals with low emotion regulation difficulty, the regulatory effect of this theme is relatively good.

[0087] Under the "Twilight Silence" theme, the differences in indices across electrode pairs between the high and low emotion regulation difficulty groups were small, showing no significant changes. For example, the T8-T7 electrode asymmetry index was similar between the high (0.35±0.55) and low (0.32±0.50) groups, and the P4-P3 index was not significantly different between the high (-0.25±0.50) and low (-0.22±0.45) groups. This suggests that this theme can have a certain emotional stabilizing effect on individuals with different levels of emotion regulation difficulty, allowing them to maintain a low-arousal emotional state, but the overall regulatory effect is less prominent than that of other themes.

[0088] In summary, the four emotion themes exhibited varying regulatory effects across individuals with varying levels of emotion regulation difficulties. The "Dawn Glow" theme had a more pronounced regulatory effect on individuals with high emotion regulation difficulties, while the "Morning Light" theme was more effective for individuals with low emotion regulation difficulties. In contrast, the "Storm Approaches" theme may induce some degree of emotional volatility, while the "Dusk Silence" theme exhibited a moderate but somewhat stabilizing effect across all participants.

[0089] Users can tap four theme buttons above the canvas to switch between different musical styles and moods. Each theme adjusts the timbre, scale, and note mapping, helping users tailor their creative approach to their current mood. Each time a user completes a brushstroke, the system plays a corresponding sound effect for immediate feedback. When the user completes a painting or actively selects music for the entire canvas, the system links all the notes from the brushstrokes created on that canvas together. The notes play in a loop from left to right, creating a complete musical piece that can be appreciated, reviewed, and experienced, creating a mood-enhancing effect. Furthermore, works can be saved at any time during the creative process. Clicking "Save" moves the work to the "Local Works" section, where users can manage and continue editing their previous work. When users need inspiration, they can access the "Inspiration Library" section and select an image that matches their current mood to copy and use as a background for their work. This process, combining imagery with painting, enhances emotional regulation. Additionally, users can access the "Relaxation Space," where they can choose from a variety of meditation videos and white noise for emotional relaxation, perfect for those seeking to quickly manage anxiety or stress. After completing the creation, users can choose to save the work or share it directly through social platforms to enhance the spread and interactivity of the work.

[0090] The above-described embodiments merely illustrate the implementation methods of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An emotional healing system based on active audio-visual fusion, characterized by: include: The creative canvas module receives user drawing strokes and generates corresponding musical feedback in real time. Specifically, it receives user drawing strokes in real time, automatically classifies the strokes through shape recognition technology, and converts them into notes of specific timbre. The pitch is determined by the vertical position of the strokes on the canvas. As the user draws, a melody sequence is dynamically generated from left to right according to the horizontal position, supporting instant playback and looping playback. The local works module is used to store and manage the works generated by users in the creation canvas module; Theme definition module allows users to customize the color scheme and theme name of each emotional theme; The inspiration gallery module provides a gallery of images categorized by emotional valence and arousal, allowing users to select images as the background for their creative canvases. The Relaxation Space module provides a passive relaxation function combining meditation background videos and white noise; The system settings module is used to adjust brush stroke parameters, playback speed and canvas display properties.

2. The emotional healing system based on active audio-visual fusion according to claim 1 is characterized in that: The creative canvas module integrates multiple emotional theme systems. Each theme is independently configured with scale rules, timbre mapping relationships and visual color schemes. Users can switch themes instantly to change their creative style.

3. The emotional healing system based on active audio-visual fusion according to claim 2 is characterized in that: The creation canvas module includes: A stroke recognition unit that calls a shape recognition algorithm when the user completes a single stroke and classifies the stroke as a line, circle, or polygon; The timbre mapping unit maps the shape of the brush stroke to a specific timbre according to preset rules, where circles, lines, and polygons are associated with different categories of timbre respectively; The pitch mapping unit divides the vertical area of ​​the canvas into multiple pitch zones, each corresponding to a note in the musical scale. The vertical center position of the brush stroke determines the pitch of the generated note; The melody generation unit sorts the notes generated by multiple strokes from left to right according to their horizontal positions on the canvas to form a melody sequence that can be played in a loop; The theme switching unit provides at least four preset emotional themes, each of which is associated with an independent scale, timbre mapping rules and visual color scheme.

4. The emotional healing system based on active audio-visual fusion according to claim 2 is characterized in that: The preset rules of the tone mapping unit include: Circular brushstrokes map to a timbre in trumpet, flute, vocal choir, or vibraphone; Line strokes are mapped to a sound from fingerpicked bass, marimba, organ, or glockenspiel; The polygonal brushstrokes are mapped to a sound from a drum kit, percussion, or piano.

5. The emotional healing system based on active audio-visual fusion according to claim 2 is characterized in that: The theme switcher contains four preset themes: High-arousal, high-valence themes are mapped to the C major scale; The high-arousal, low-valence theme maps to the F major scale; The low-arousal, high-valence theme maps to the A minor scale; The low-arousal, low-valence theme maps to the D minor scale.

6. The emotional healing system based on active audio-visual fusion according to claim 2 is characterized in that: The working logic of the melody generation unit includes: Play the musical note corresponding to each brush stroke in real time while the user is drawing; When the user actively triggers playback or completes drawing, a continuous melody is generated from left to right according to the horizontal position of the brush strokes; Supports looping melody sequences to form complete music clips.

7. The emotional healing system based on active audio-visual fusion according to claim 1 is characterized in that: The color customization features of the theme definition module include: Adjust the canvas background color; Set the display colors of line, circle and polygon strokes respectively; The color scheme is associated with the emotional theme but does not change the timbre and scale mapping rules.

8. The emotional healing system based on active audio-visual fusion according to claim 1 is characterized in that: Images in the inspiration gallery module are categorized based on emotional arousal and valence. Valence refers to how positive or negative an emotion is, and arousal refers to the intensity of the emotion. The categories include high valence and high arousal category, high valence and low arousal category, low valence and high arousal category, and low valence and low arousal category. Each category contains multiple pictures with matching emotional features.

9. The emotional healing system based on active audio-visual fusion according to claim 1 is characterized in that: The relaxation space module contains multiple meditation theme combinations. Each theme is associated with a specific scene video and matching white noise. Quick selection can be achieved by swiping left and right to switch the theme interface.

10. The emotional healing system based on active audio-visual fusion according to claim 1 is characterized in that: The system settings module has the function of adjusting the thickness of the brush strokes, adjusting the melody playback speed, switching the canvas grid display, and switching the automatic save function.

Images