Emotion synchronous visual design system based on multi-modal biofeedback

Abstract

The invention provides an emotion synchronous visual design system based on multi-modal biological feedback, and aims to dynamically adjust visual design elements by analyzing physiological signals of a user in real time, so that the emotion experience of the user is enhanced. According to the system, various biological feedback sensors such as heart rate, skin conductance and brain wave sensors are utilized, an advanced data processing algorithm is integrated, and the emotional state of a user is accurately captured and analyzed. And then, the system changes visual elements such as colors, shapes and layouts in real time through a dynamic visual design adjustment module so as to match and respond to emotional changes of the user. In addition, the system further comprises a real-time feedback and interaction module and a personalized learning and optimization module so as to provide personalized visual experience and continuously optimize user interaction. The system not only improves the interactivity and attraction of the user interface, but also deepens the emotional relation between the brand and the consumer, and has important commercial application value and market potential.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to an emotion synchronization visual design system based on multimodal biofeedback. Background Art

[0002] With the continuous advancement of science and technology, human-computer interaction technology has evolved from simple functional operations to a stage that focuses on user experience and emotional connection. With the rapid development of digital media and information technology, users' demand for electronic products and software applications is no longer limited to basic functions, but is increasingly pursuing personalized, emotional and immersive interactive experiences. Traditional user interface design often ignores the importance of user emotional factors, resulting in a lack of emotional resonance in the interaction between users and devices, and an inability to meet users' deep needs.

[0003] To solve this problem, researchers began to explore how to incorporate the user's emotional state into interaction design to improve the user experience. Biofeedback technology, as an effective means, can reflect the user's emotional changes by monitoring the user's physiological signals. These physiological signals include heart rate, skin conductance, brain waves, etc., which are closely related to the individual's emotional state. However, existing biofeedback systems are mostly limited to the fields of medical and health monitoring, and often rely on a single physiological signal, which cannot fully capture the user's complex emotional state.

[0004] Existing visual design systems also fail to achieve real-time synchronization with the user's emotional state. User interfaces are often designed statically and lack the ability to dynamically adjust according to changes in user emotions. This limits the potential of user interfaces to provide personalized and emotional experiences. Therefore, it is necessary to develop a new type of emotional synchronization visual design system that can comprehensively utilize multimodal biofeedback technology to analyze the user's emotional state in real time and dynamically adjust visual design elements based on the analysis results to achieve a high degree of synchronization with the user's emotions and a personalized experience. This can not only enhance the user's emotional engagement, but also deepen the emotional connection between the user and the product, and improve user satisfaction and loyalty. Summary of the invention

[0005] The purpose of the present invention is to provide an emotion-synchronized visual design system based on multimodal biofeedback, which can capture and analyze the user's emotional state in real time and provide a personalized visual experience by dynamically adjusting visual design elements, thereby enhancing the user's emotional engagement and satisfaction. In order to achieve this goal, the present invention is as follows:

[0006] The multimodal biofeedback acquisition module is first started, and the user's physiological signals are collected in real time through devices such as heart rate sensors, skin conductance sensors, brain wave sensors, body temperature sensors, motion sensors, breathing sensors and eye tracking sensors. These raw data are then transmitted to the preprocessing unit, where they are filtered, amplified and digitized to eliminate noise and interference and ensure the clarity and accuracy of the signal. The preprocessed data is sent to the emotional state analysis module, which uses a deep learning algorithm to identify emotional states, such as pleasure, excitement, relaxation, tension, anxiety or pressure. The emotional state information obtained by the analysis then guides the dynamic visual design adjustment module, which selects a suitable design scheme from the visual design database according to the change of emotional state, or generates a new visual design, including the adjustment of elements such as color, shape, layout and animation. Then, the user interface rendering engine renders a new user interface in real time according to the instructions of the dynamic visual design adjustment module to ensure the smoothness of the interface and the high quality of the visual experience. At the same time, the real-time feedback and interaction module is responsible for displaying the adjusted user interface and collecting user feedback and interaction data, such as touch input, key operation, etc. The user behavior analysis unit analyzes these interactive data to identify the user's preferences and behavior patterns. The personalized learning and optimization module uses machine learning algorithms to continuously optimize the accuracy of emotional state recognition and the personalized adjustment strategy of visual design based on user feedback information, behavioral data and user behavior analysis results. The operating data and user information of all these modules are stored in the data storage module for subsequent analysis and optimization. Finally, the system control unit monitors the working status of all modules to ensure the stable operation of the system, and performs self-diagnosis and adjustment according to the system operation status. Through such modular design and close connection, the present invention realizes a visual design system that can respond to changes in the user's emotional state in real time, providing a highly personalized and emotional visual experience.

[0007] The multimodal biofeedback acquisition module integrates a variety of high-precision sensors, including a heart rate sensor (sampling frequency set to 250Hz), a skin conductance sensor (sampling frequency set to 128Hz), an EEG sensor (sampling frequency set to 512Hz), a body temperature sensor (measuring range 32°C to 42°C), a motion sensor (measuring range 0 to 10 meters per second), a breathing sensor (measuring frequency 0.1 to 2 Hz), an eye tracking sensor (tracking speed 120Hz), and at least one sound sensor, which is used to capture the user's sound signal to assist in analyzing the user's emotional state. The data of these sensors are transmitted to the preprocessing unit, where they are filtered, amplified and digitized to improve the signal-to-noise ratio of the signal and ensure the accuracy of the data. The preprocessed data is then sent to the emotional state analysis module, which uses a deep learning algorithm (such as a convolutional neural network CNN, with a learning rate set to 0.001) to analyze the physiological signals and identify the user's emotional state. The analysis results then guide the dynamic visual design adjustment module, which selects or generates a visual design scheme that matches the user's emotional state from the visual design database, such as adjusting the color saturation to a lower level and making the layout more spacious when the user feels relaxed, to create a peaceful atmosphere. The user interface rendering engine receives instructions from the dynamic visual design adjustment module and renders a new user interface in real time to ensure the smoothness of the interface and the high quality of the visual experience. The real-time feedback and interaction module is responsible for displaying the adjusted user interface and collecting user feedback and interaction data, such as touch input, key operation, etc. The response time of these data does not exceed 100 milliseconds to ensure instant feedback of user operations. The user behavior analysis unit analyzes these interaction data to identify user preferences and behavior patterns. The personalized learning and optimization module uses a machine learning algorithm (such as a collaborative filtering algorithm, with a regularization coefficient set to 0.01) to continuously optimize the accuracy of emotional state recognition and the personalized adjustment strategy of visual design based on user feedback information, behavior data and user behavior analysis results. The operating data and user information of all these modules are stored in the data storage module, which uses the AES-256 encryption algorithm to protect user data and ensure data security and privacy. Finally, the system control unit monitors the working status of all modules to ensure the stable operation of the system, and performs self-diagnosis and adjustment according to the system operation status. For example, when the system resource utilization rate exceeds 90%, the system control unit will automatically optimize resource allocation. Through such modular design and close connection, the present invention realizes a visual design system that can respond to changes in the user's emotional state in real time, providing a highly personalized and emotional visual experience.

[0008] The pre-processing unit collects the user's voice signal through an integrated sound sensor, which can capture the user's voice and breathing patterns and other sound features. The collected sound signal first passes through a bandpass filter, the cutoff frequency of which is set to 300Hz to 3400Hz to remove background noise and retain signals within the human voice frequency range. Next, the filtered sound signal is processed by a signal enhancement algorithm, which includes an adaptive gain control module, whose gain coefficient is dynamically adjusted according to the strength of the input signal to improve the signal-to-noise ratio (SNR) of the signal. For example, when a low signal strength is detected, the gain coefficient is automatically increased to enhance the voice signal; conversely, when the signal strength is high, the gain coefficient is reduced to avoid signal saturation and distortion. This process ensures the clarity and audibility of the sound signal, providing high-quality data input for subsequent emotional state analysis. The design and parameter adjustment of the signal enhancement algorithm are to optimize the signal-to-noise ratio and avoid or reduce signal distortion, which is crucial for accurate analysis of emotional state. In this way, the system can more accurately capture and analyze the user's emotional state, thereby providing a more personalized and emotional visual experience.

[0009] The emotional state analysis module significantly improves the accuracy of emotional state recognition by integrating deep learning algorithms. The module adopts advanced neural network architectures, such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), or Transformer models, which are capable of capturing complex emotional patterns and contextual relationships in text. For example, the LSTM model effectively solves the gradient vanishing problem in long sequence data through its unique gating mechanism, thereby performing well in processing long text data. In practical applications, the module first receives physiological signal data from the preprocessing unit, which is trained by a deep learning model to identify the user's emotional state, such as positive, negative, or neutral. The training of the deep learning model involves a large number of parameter adjustments, such as setting the learning rate to 0.001, the batch size to 32, and using the cross entropy loss function and the Adam optimizer. During the training process, the model optimizes its weights and biases through continuous iterative learning to minimize the difference between the predicted emotional state and the actual label. In addition, the model evaluation phase uses indicators such as accuracy, recall, and F1 score to measure the performance of the model to ensure its high accuracy in practical applications. On the output side of the emotional state analysis module, the deep learning model converts the processed data into prediction results of the emotional state, which are then transmitted to the dynamic visual design adjustment module. The receiving side, the dynamic visual design adjustment module, selects or generates a visual design solution that matches the user's emotional state from the visual design database based on the output of the emotional state analysis module, thereby achieving an emotionally synchronized visual experience. Through this deep learning-driven emotional state analysis, the system can more accurately capture and respond to the user's emotional changes, providing users with a more personalized and emotional visual experience. The application of this technology not only improves the interactivity and attractiveness of the user interface, but also deepens the emotional connection between users and products, improving user satisfaction and brand loyalty.

[0010] The dynamic visual design adjustment module is implemented through a visual design database, which is carefully constructed and contains a variety of preset visual design schemes. These schemes cover a wide range of styles and emotional expressions to adapt to the emotional states and personalized needs of different users. The visual design schemes in the database are stored parametrically, allowing the system to dynamically select or adjust the design schemes according to the results of different emotional state analysis. When the emotional state analysis module identifies that the user is in a relaxed state, the dynamic visual design adjustment module retrieves the design schemes that match the relaxed mood from the database, which may include warm color schemes, soft transition animations, and calm layouts. Each design scheme in the database is accompanied by detailed parameter descriptions, such as the RGB value of the color, the timing parameters of the animation, and the grid system of the layout, to ensure that the design scheme can accurately meet the needs of the emotional state. The system also has the ability to learn, and can continuously optimize the visual design schemes in the database through machine learning algorithms based on user feedback and interaction data, so that they are more in line with the user's emotional tendencies. This ability to dynamically adjust and continuously optimize enables the system to provide a highly personalized visual experience, thereby enhancing the user's emotional engagement and satisfaction. In actual application, the operation process of the dynamic visual design adjustment module includes: receiving the output of the emotional state analysis module, retrieving matching design solutions in the visual design database based on the output, adjusting the visual effects of the user interface, rendering the new user interface in real time, and collecting user feedback to optimize future design choices. This process ensures that the system can quickly respond to user emotional changes and provide instant visual feedback, thereby achieving an emotionally synchronized visual design experience.

[0011] The key component of the user interface rendering engine is a graphics processing unit (GPU), which is specially designed to accelerate the graphics rendering process. Through its parallel processing capabilities, the GPU can process thousands of pixels and vertex calculations at the same time, thereby significantly improving rendering efficiency. For example, a GPU may contain thousands of shader cores, each of which is capable of performing tasks such as floating-point operations and texture mapping to achieve complex visual effects. The application of the graphics processing unit (GPU) in the rendering process includes vertex processing, rasterization, and fragment shading. In the vertex processing stage, the GPU receives vertex data from the application and transforms it through the vertex shader to convert the vertex position from the model space to the screen space. In the rasterization stage, the GPU converts three-dimensional geometric graphics into two-dimensional pixel data in preparation for subsequent fragment shader processing. The fragment shader is responsible for calculating the final color and lighting effects of each pixel, realizing complex lighting models and material effects. In order to further improve rendering performance, the GPU uses a variety of optimization technologies, such as multi-threaded rendering, asynchronous computing, and unified memory architecture. These technologies enable the GPU to manage and schedule graphics rendering tasks more efficiently and reduce the communication overhead between the CPU and the GPU. For example, through asynchronous computing, the GPU can start executing other tasks in advance while waiting for certain rendering tasks to complete, thereby improving the overall rendering efficiency. In practical applications, the performance parameters of the GPU include core frequency (such as 1.8GHz), memory bandwidth (such as 384GB / s), and floating-point computing power (such as 10TFLOPS). These parameters directly affect the rendering speed and image quality of the rendering engine. By optimizing these parameters, it can be ensured that the user interface rendering engine can provide a smooth and high-quality visual experience when dealing with complex visual design solutions. The user interface rendering engine integrates a high-performance graphics processing unit (GPU) and uses its powerful parallel processing capabilities and optimization technology to achieve an efficient graphics rendering process, providing users with an emotionally synchronized visual design experience.

[0012] The key component of the real-time feedback and interaction module is a tactile feedback unit, which provides fine tactile feedback and enhances the user's interactive experience through integrated tactile feedback technology, such as vibration, electrotactile stimulation or piezoelectric effect. For example, when the user is playing a game or watching a video, the tactile feedback unit can synchronously generate corresponding vibration patterns according to the visual content, thereby simulating the texture of the object or the dynamics of the environment, providing the user with a more immersive experience. The design of the tactile feedback unit includes a series of parameter settings, such as the frequency of vibration (adjustable range of 20Hz to 300Hz), amplitude (adjustable range of 0.1G to 2.5G), and the intensity of electrotactile stimulation (adjustable range of 0 to 10mA), which can be adjusted according to the user's emotional state and preferences. For example, for users seeking a stimulating experience, the system may choose a higher vibration frequency and amplitude; while for users seeking a relaxing experience, a lower vibration frequency and amplitude may be selected. The tactile feedback unit can also intelligently adjust the intensity and mode of tactile feedback according to the user's operating behavior, such as the strength or speed of touching the screen, to provide a more personalized tactile experience. This intelligent adjustment is achieved by real-time monitoring of the user's interactive behavior, combining the user's historical interaction data, and using machine learning algorithms for prediction and optimization. In actual applications, the tactile feedback unit works closely with the user interface rendering engine to ensure the synchronization and coordination of tactile feedback with visual content. The user interface rendering engine is responsible for processing the rendering of visual content, while the tactile feedback unit adjusts the tactile feedback in real time according to the rendered content and the user's interactive behavior. The collaborative work of the two provides users with a seamless multi-sensory experience. The real-time feedback and interaction module is integrated with the tactile feedback unit, which not only enhances the user's interactive experience, but also achieves synchronization with the user's emotional state through intelligent adjustment and personalized settings, providing users with a highly personalized and emotional visual design experience.

[0013] The user behavior analysis unit uses a behavior pattern recognition algorithm to deeply understand the user's long-term behavior patterns. The algorithm first collects detailed user behavior data from the multimodal biofeedback acquisition module and the real-time feedback and interaction module, including physiological signals, interaction frequency and duration. Then, the algorithm identifies key behavioral features through a feature extraction process, and then uses deep learning models such as recurrent neural networks (RNN) to analyze these features to identify and predict user behavior patterns. The identified behavior patterns are then used to optimize the personalized learning and optimization module, which dynamically adjusts the user interface and interaction strategies to improve the user experience. In addition, the algorithm has the ability to continuously learn and can adapt to changes in user behavior to ensure the accuracy and timeliness of behavior recognition. Through this series of steps, the user behavior analysis unit not only enhances the system's insight into user behavior, but also improves the quality and efficiency of personalized services, thereby bringing users a richer and more emotional visual design experience.

[0014] The personalized learning and optimization module is able to predict the user's future emotional state and behavioral trends by integrating an advanced prediction model. The module uses deep learning techniques, especially recurrent neural networks (RNN) and long short-term memory networks (LSTM), which are able to process time series data and capture the long-term dependencies of user behavior. For example, the LSTM network effectively solves the gradient vanishing problem in long sequence training through its unique gating mechanism, thereby performing well in processing user behavior data. The training of the prediction model involves a large number of parameter adjustments, such as setting the learning rate to 0.001, the batch size to 32, and using the cross entropy loss function and the Adam optimizer. During the training process, the model optimizes its weights and biases through continuous iterative learning to minimize the difference between the predicted emotional state and the actual label. In addition, the model evaluation stage uses indicators such as accuracy, recall, and F1 score to measure the performance of the model to ensure its high accuracy in practical applications. In practical applications, the prediction model is able to predict the user's possible emotional state and behavior pattern in the future based on the user's historical behavior data and real-time feedback. For example, if the system predicts that the user may feel tired, it can automatically adjust the layout and color of the user interface to provide a more comfortable and relaxing visual experience. This predictive ability enables the system to proactively adapt to the needs of users and provide more personalized services. Through this deep learning-driven personalized learning and optimization, the system can not only provide a real-time emotionally synchronized visual experience, but also predict and adapt to the user's future needs, thereby achieving highly personalized and emotional user interaction. The application of this technology not only improves the interactivity and attractiveness of the user interface, but also deepens the emotional connection between users and products, improving user satisfaction and brand loyalty.

[0015] The operation method of the emotion synchronization visual design system based on multimodal biofeedback includes the following steps:

[0016] s1, start the multimodal biofeedback acquisition module to collect the user's heart rate (e.g., through a heart rate sensor with a sampling frequency set to 250 Hz), skin conductance (e.g., through a skin conductance sensor with a sampling frequency set to 128 Hz), brain waves (e.g., through a brain wave sensor with a sampling frequency set to 512 Hz), body temperature (measuring range 32°C to 42°C), body activity (e.g., through a motion sensor with a measuring range of 0 to 10 meters per second), breathing pattern (e.g., through a breathing sensor with a measuring frequency of 0.1 to 2 Hz), and eye movement data (e.g., through an eye tracking sensor with a tracking speed of 120 Hz);

[0017] s2, filtering (for example, using a bandpass filter with a cutoff frequency of 300 Hz to 3400 Hz), amplifying and digitizing (for example, using a 16-bit analog-to-digital converter) the collected physiological signals through a preprocessing unit to eliminate noise and interference and improve the signal-to-noise ratio of the signal;

[0018] s3, transmitting the preprocessed physiological signals to the emotional state analysis module, and analyzing the user's emotional state using a preset emotion recognition algorithm (e.g., a convolutional neural network (CNN) based on deep learning, with a learning rate set to 0.001);

[0019] s4, based on the result of the emotional state analysis module, the dynamic visual design adjustment module adjusts the visual design elements of the user interface (e.g., color saturation, layout symmetry, and animation smoothness) to match the user's emotional state;

[0020] s5, the user interface rendering engine renders the user interface in real time (e.g., resolution 1920x1080, frame rate 60fps) according to the instructions of the dynamic visual design adjustment module to ensure the smoothness and clarity of the visual experience;

[0021] s6, real-time feedback and interaction module displays the adjusted user interface and receives user interaction input (for example, the strength and speed of touching the screen). The response time of these data does not exceed 100 milliseconds to ensure instant feedback of user operations;

[0022] s7, the user behavior analysis unit analyzes the user's interaction patterns and preferences (for example, by clustering the user's click behavior and dwell time) to identify the user's long-term behavior trends;

[0023] s8, the personalized learning and optimization module optimizes the visual design adjustment strategy (for example, through a collaborative filtering algorithm with a regularization coefficient set to 0.01) based on user feedback information, behavior data, and user behavior analysis results to provide a personalized visual experience;

[0024] s9, the data storage module stores the user's emotional state data, visual design adjustment history, user interaction data, and user behavior analysis results (for example, using AES-256 encryption algorithm to protect user data) for further analysis and optimization;

[0025] s10, the system control unit monitors the working status of each module to ensure the stable operation of the system (for example, monitoring the usage of the CPU and GPU, memory usage, etc.), and performs self-diagnosis and adjustment according to the system operation status (for example, when the system resource usage exceeds 90%, it automatically optimizes resource allocation). BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of the emotional synchronization visual design system of the present invention

[0027] Figure 2 The operation method diagram of the emotional synchronization visual design system of the present invention DETAILED DESCRIPTION

[0028] The following will be combined with the attached embodiment of the present invention Figure 1 and Figure 2 , the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] The multimodal biofeedback acquisition module first collects the user's physiological signals in real time through a series of high-precision sensors, such as heart rate sensor (sampling frequency set to 250Hz), skin conductance sensor (sampling frequency set to 128Hz), brain wave sensor (sampling frequency set to 512Hz), body temperature sensor (measuring range 32℃ to 42℃), motion sensor (measuring range 0 to 10 meters per second), breathing sensor (measuring frequency 0.1 to 2 Hz) and eye tracking sensor (tracking speed 120Hz). The data of these sensors are filtered and amplified by the preprocessing unit to improve the signal-to-noise ratio of the signal and ensure the accuracy of the data. The preprocessed signal is sent to the emotional state analysis module, which uses a deep learning algorithm (such as a convolutional neural network CNN, with a learning rate set to 0.001) to analyze the physiological signal and identify the user's emotional state. The analysis results then guide the dynamic visual design adjustment module, which selects or generates a visual design scheme that matches the user's emotional state from the visual design database, such as adjusting the color saturation to a lower level and making the layout more spacious when the user feels relaxed, to create a peaceful atmosphere. The user interface rendering engine receives instructions from the dynamic visual design adjustment module and renders a new user interface in real time to ensure the smoothness of the interface and the high quality of the visual experience. The real-time feedback and interaction module is responsible for displaying the adjusted user interface and collecting user feedback and interaction data, such as touch input, key operation, etc. The response time of these data does not exceed 100 milliseconds to ensure instant feedback of user operations. The user behavior analysis unit analyzes these interaction data to identify user preferences and behavior patterns. The personalized learning and optimization module uses a machine learning algorithm (such as a collaborative filtering algorithm, with a regularization coefficient set to 0.01) to continuously optimize the accuracy of emotional state recognition and the personalized adjustment strategy of visual design based on user feedback information, behavior data and user behavior analysis results. The operating data and user information of all these modules are stored in the data storage module, which uses the AES-256 encryption algorithm to protect user data and ensure data security and privacy. Finally, the system control unit monitors the working status of all modules to ensure the stable operation of the system, and performs self-diagnosis and adjustment according to the system operation status. For example, when the system resource utilization rate exceeds 90%, the system control unit will automatically optimize resource allocation. Through such modular design and close connection, the present invention realizes a visual design system that can respond to changes in the user's emotional state in real time, providing a highly personalized and emotional visual experience.

[0030] The multimodal biofeedback acquisition module integrates a variety of high-precision sensors, including a heart rate sensor (with a sampling frequency set to 250Hz), a skin conductance sensor (with a sampling frequency set to 128Hz), an EEG sensor (with a sampling frequency set to 512Hz), a temperature sensor (measuring range 32°C to 42°C), a motion sensor (measuring range 0 to 10 meters per second), a breathing sensor (measuring frequency 0.1 to 2 Hz), an eye tracking sensor (tracking speed 120Hz), and at least one sound sensor, which is used to capture the user's sound signal to assist in analyzing the user's emotional state. The data from these sensors are transmitted to the preprocessing unit, where they are filtered, amplified, and digitized to improve the signal-to-noise ratio and ensure the accuracy of the data. The preprocessed data is then sent to the emotional state analysis module, which uses a deep learning algorithm (such as a convolutional neural network CNN, with a learning rate set to 0.001) to analyze the physiological signals and identify the user's emotional state. The analysis results then guide the dynamic visual design adjustment module, which selects or generates a visual design scheme that matches the user's emotional state from the visual design database, such as adjusting the color saturation to a lower level and making the layout more spacious when the user feels relaxed, to create a peaceful atmosphere. The user interface rendering engine receives instructions from the dynamic visual design adjustment module and renders a new user interface in real time to ensure the smoothness of the interface and the high quality of the visual experience. The real-time feedback and interaction module is responsible for displaying the adjusted user interface and collecting user feedback and interaction data, such as touch input, key operation, etc. The response time of these data does not exceed 100 milliseconds to ensure instant feedback of user operations. The user behavior analysis unit analyzes these interaction data to identify user preferences and behavior patterns. The personalized learning and optimization module uses a machine learning algorithm (such as a collaborative filtering algorithm, with a regularization coefficient set to 0.01) to continuously optimize the accuracy of emotional state recognition and the personalized adjustment strategy of visual design based on user feedback information, behavior data and user behavior analysis results. The operating data and user information of all these modules are stored in the data storage module, which uses the AES-256 encryption algorithm to protect user data and ensure data security and privacy. Finally, the system control unit monitors the working status of all modules to ensure the stable operation of the system, and performs self-diagnosis and adjustment according to the system operation status. For example, when the system resource utilization rate exceeds 90%, the system control unit will automatically optimize resource allocation. Through such modular design and close connection, the present invention realizes a visual design system that can respond to changes in the user's emotional state in real time, providing a highly personalized and emotional visual experience.

[0031] The pre-processing unit collects the user's voice signal through an integrated sound sensor that can capture the user's voice and breathing patterns and other sound features. The collected sound signal first passes through a bandpass filter with a cutoff frequency set to 300Hz to 3400Hz to remove background noise and retain signals within the human voice frequency range. Next, the filtered sound signal is processed by a signal enhancement algorithm, which includes an adaptive gain control module whose gain coefficient is dynamically adjusted according to the strength of the input signal to improve the signal-to-noise ratio (SNR) of the signal. For example, when a low signal strength is detected, the gain coefficient is automatically increased to enhance the voice signal; conversely, when the signal strength is high, the gain coefficient is reduced to avoid signal saturation and distortion. This process ensures the clarity and audibility of the sound signal, providing high-quality data input for subsequent emotional state analysis. The design and parameter adjustment of the signal enhancement algorithm are to optimize the signal-to-noise ratio and avoid or reduce signal distortion, which is crucial for accurate analysis of emotional state. In this way, the system can more accurately capture and analyze the user's emotional state, thereby providing a more personalized and emotional visual experience.

[0032] The emotional state analysis module analyzes the collected physiological signals through a deep learning algorithm to improve the accuracy of emotional state recognition. This module uses a combination of convolutional neural networks (CNN) and long short-term memory networks (LSTM), where CNN is used to extract the features of physiological signals, while LSTM is used to process time series data and capture the temporal dynamic changes of emotional states. During the training process, the cross entropy loss function and Adam optimizer are used, and the learning rate is set to 0.001, the batch size is 32, and the training cycle (epoch) is 10 to ensure the generalization ability of the model on different data sets. In addition, in order to improve the robustness of the model, a dropout mechanism is introduced with a dropout rate of 0.5 to reduce the risk of overfitting. Through the precise setting of these parameters and the training of the model, the emotional state analysis module can identify the user's emotional state with high accuracy and convert these states into inputs for visual design adjustments, thereby achieving an emotionally synchronized visual experience.

[0033] The key component of the dynamic visual design adjustment module is a visual design database that stores a variety of preset visual design schemes. These schemes are carefully designed to adapt to different emotional states, and each scheme contains a series of parameter settings to ensure that the user's emotional needs can be accurately met. For example, a scheme in the database may include the following parameters: the RGB value range of the color scheme (such as red R: 255, G: 0, B: 0), the grid system parameters of the layout (such as the grid size is set to 8x8 pixels), the timing parameters of the animation (such as the animation duration is set to 500 milliseconds), and the size and edge smoothness of the shape (such as the edge radius of the rounded rectangle is set to 10 pixels). These parameters are carefully adjusted according to the changes in the user's emotional state to ensure that the design scheme can produce the expected emotional response.

[0034] When the emotional state analysis module identifies the user's emotional state, it passes this information to the dynamic visual design adjustment module. The module then retrieves the design scheme that matches the user's current emotional state from the visual design database and adjusts the user interface according to the parameter settings in the scheme. For example, if the user is in a relaxed state, the system may choose a visual design scheme with a blue tone as the main color, a spacious layout, and smooth animation to enhance the user's relaxation experience. The system also has learning capabilities and can continuously optimize the visual design schemes in the database through machine learning algorithms based on user feedback and interaction data. This ability to dynamically adjust and continuously optimize enables the system to provide a highly personalized visual experience, thereby enhancing the user's emotional engagement and satisfaction. Through this meticulous parameter setting and dynamic adjustment, the present invention realizes a visual design system that can respond to changes in the user's emotional state in real time, providing users with a highly personalized and emotional visual experience.

[0035] A key component of the user interface rendering engine is a graphics processing unit (GPU), which is specially designed to accelerate the graphics rendering process. Through its parallel processing capabilities, the GPU is able to handle thousands of pixel and vertex calculations simultaneously, significantly improving rendering efficiency. For example, a GPU may contain thousands of shader cores, each of which is capable of performing tasks such as floating-point operations and texture mapping to achieve complex visual effects.

[0036] The application of graphics processing units (GPUs) in the rendering process includes vertex processing, rasterization, and fragment shading. In the vertex processing stage, the GPU receives vertex data from the application, transforms it through the vertex shader, and converts the vertex position from the model space to the screen space. In the rasterization stage, the GPU converts three-dimensional geometry into two-dimensional pixel data in preparation for subsequent fragment shader processing. The fragment shader is responsible for calculating the final color and lighting effects of each pixel, and realizing complex lighting models and material effects. In order to further improve rendering performance, the GPU uses a variety of optimization technologies, such as multi-threaded rendering, asynchronous computing, and unified memory architecture. These technologies enable the GPU to manage and schedule graphics rendering tasks more efficiently and reduce the communication overhead between the CPU and the GPU. For example, through asynchronous computing, the GPU can start executing other tasks in advance while waiting for certain rendering tasks to complete, thereby improving overall rendering efficiency.

[0037] In actual applications, the performance parameters of GPU include core frequency (such as 1.8GHz), memory bandwidth (such as 384GB / s) and floating-point computing power (such as 10TFLOPS). These parameters directly affect the rendering speed and image quality of the rendering engine. By optimizing these parameters, it is possible to ensure that the user interface rendering engine can provide a smooth and high-quality visual experience when dealing with complex visual design solutions. By integrating a high-performance graphics processing unit (GPU), the user interface rendering engine uses its powerful parallel processing capabilities and optimization technology to achieve an efficient graphics rendering process, providing users with an emotionally synchronized visual design experience.

[0038] The key component of the real-time feedback and interaction module is a tactile feedback unit, which provides fine tactile feedback and enhances the user's interactive experience through integrated tactile feedback technologies such as vibration, electrotactile stimulation or piezoelectric effect. For example, when the user is playing a game or watching a video, the tactile feedback unit can synchronize the corresponding vibration pattern according to the visual content, thereby simulating the texture of the object or the dynamics of the environment, providing the user with a more immersive experience. The design of the tactile feedback unit includes a series of parameter settings, such as the frequency of vibration (adjustable range of 20Hz to 300Hz), amplitude (adjustable range of 0.1G to 2.5G), and the intensity of electrotactile stimulation (adjustable range of 0 to 10mA), which can be adjusted according to the user's emotional state and preferences. For example, for users seeking a stimulating experience, the system may choose a higher vibration frequency and amplitude; while for users seeking a relaxing experience, a lower vibration frequency and amplitude may be selected. The tactile feedback unit can also intelligently adjust the intensity and pattern of tactile feedback according to the user's operating behavior, such as the strength or speed of touching the screen, to provide a more personalized tactile experience. This intelligent adjustment is achieved by real-time monitoring of the user's interactive behavior, combining the user's historical interaction data, and using machine learning algorithms for prediction and optimization. The tactile feedback unit works closely with the user interface rendering engine to ensure the synchronization and coordination of tactile feedback with visual content. The user interface rendering engine is responsible for processing the rendering of visual content, while the tactile feedback unit adjusts the tactile feedback in real time according to the rendered content and the user's interactive behavior. The collaborative work of the two provides users with a seamless multi-sensory experience. The real-time feedback and interaction module is integrated with the tactile feedback unit, which not only enhances the user's interactive experience, but also achieves synchronization with the user's emotional state through intelligent adjustment and personalized settings, providing users with a highly personalized and emotional visual design experience.

[0039] The user behavior analysis unit uses a behavior pattern recognition algorithm to deeply understand the user's long-term behavior patterns. The algorithm first collects detailed user behavior data from the multimodal biofeedback acquisition module and the real-time feedback and interaction module, including physiological signals, interaction frequency and duration. Then, the algorithm identifies key behavioral features through a feature extraction process, and then uses deep learning models such as recurrent neural networks (RNN) to analyze these features to identify and predict user behavior patterns. The identified behavior patterns are then used to optimize the personalized learning and optimization module, which dynamically adjusts the user interface and interaction strategies to improve the user experience. In addition, the algorithm has the ability to continuously learn and can adapt to changes in user behavior to ensure the accuracy and timeliness of behavior recognition. Through this series of steps, the user behavior analysis unit not only enhances the system's insight into user behavior, but also improves the quality and efficiency of personalized services, thereby bringing users a richer and more emotional visual design experience.

[0040] The personalized learning and optimization module integrates an advanced prediction model to predict users' future emotional states and behavioral trends. This module uses deep learning techniques, especially recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), which are capable of processing time series data and capturing the long-term dependencies of user behaviors. For example, the LSTM network effectively solves the gradient vanishing problem in long-sequence training through its unique gating mechanism, thus performing well in processing user behavior data.

[0041] The training of the prediction model involves a lot of parameter adjustments, such as setting the learning rate to 0.001, the batch size to 32, and using the cross entropy loss function and the Adam optimizer. During the training process, the model optimizes its weights and biases through continuous iterative learning to minimize the difference between the predicted emotional state and the actual label. In addition, the model evaluation phase uses indicators such as accuracy, recall, and F1 score to measure the performance of the model to ensure its high accuracy in practical applications.

[0042] The predictive model is able to predict the user's likely emotional state and behavior pattern in the future based on the user's historical behavior data and real-time feedback. For example, if the system predicts that the user may feel tired, it can automatically adjust the layout and color of the user interface to provide a more comfortable and relaxing visual experience. This predictive capability enables the system to proactively adapt to the needs of users and provide more personalized services. Through this deep learning-driven personalized learning and optimization, the system can not only provide a real-time emotionally synchronized visual experience, but also predict and adapt to the user's future needs, thereby achieving highly personalized and emotional user interaction. The application of this technology not only improves the interactivity and attractiveness of the user interface, but also deepens the emotional connection between users and products, improving user satisfaction and brand loyalty.

[0043] The operation method of the emotion synchronization visual design system based on multimodal biofeedback is characterized by comprising the following steps:

[0044] s1, start the multimodal biofeedback acquisition module to collect the user's heart rate (e.g., through a heart rate sensor with a sampling frequency set to 250 Hz), skin conductance (e.g., through a skin conductance sensor with a sampling frequency set to 128 Hz), brain waves (e.g., through a brain wave sensor with a sampling frequency set to 512 Hz), body temperature (measuring range 32°C to 42°C), body activity (e.g., through a motion sensor with a measuring range of 0 to 10 meters per second), breathing pattern (e.g., through a breathing sensor with a measuring frequency of 0.1 to 2 Hz), and eye movement data (e.g., through an eye tracking sensor with a tracking speed of 120 Hz);

[0045] s2, filtering (for example, using a bandpass filter with a cutoff frequency of 300 Hz to 3400 Hz), amplifying and digitizing (for example, using a 16-bit analog-to-digital converter) the collected physiological signals through a preprocessing unit to eliminate noise and interference and improve the signal-to-noise ratio of the signal;

[0046] s3, transmitting the preprocessed physiological signals to the emotional state analysis module, and analyzing the user's emotional state using a preset emotion recognition algorithm (e.g., a convolutional neural network (CNN) based on deep learning, with a learning rate set to 0.001);

[0047] s4, based on the result of the emotional state analysis module, the dynamic visual design adjustment module adjusts the visual design elements of the user interface (e.g., color saturation, layout symmetry, and animation smoothness) to match the user's emotional state;

[0048] s5, the user interface rendering engine renders the user interface in real time (e.g., resolution 1920x1080, frame rate 60fps) according to the instructions of the dynamic visual design adjustment module to ensure the smoothness and clarity of the visual experience;

[0049] s6, real-time feedback and interaction module displays the adjusted user interface and receives user interaction input (for example, the strength and speed of touching the screen). The response time of these data does not exceed 100 milliseconds to ensure instant feedback of user operations;

[0050] s7, the user behavior analysis unit analyzes the user's interaction patterns and preferences (for example, by clustering the user's click behavior and dwell time) to identify the user's long-term behavior trends;

[0051] s8, the personalized learning and optimization module optimizes the visual design adjustment strategy (for example, through a collaborative filtering algorithm with a regularization coefficient set to 0.01) based on user feedback information, behavior data, and user behavior analysis results to provide a personalized visual experience;

[0052] s9, the data storage module stores the user's emotional state data, visual design adjustment history, user interaction data, and user behavior analysis results (for example, using AES-256 encryption algorithm to protect user data) for further analysis and optimization;

[0053] s10, the system control unit monitors the working status of each module to ensure the stable operation of the system (for example, monitoring the usage of the CPU and GPU, memory usage, etc.), and performs self-diagnosis and adjustment according to the system operation status (for example, when the system resource usage exceeds 90%, it automatically optimizes resource allocation).

[0054] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. An emotional synchronization visual design system based on multimodal biofeedback, characterized in that: include: Multimodal biofeedback acquisition module, preprocessing unit, emotional state analysis module, dynamic visual design adjustment module, user interface rendering engine, real-time feedback and interaction module, user behavior analysis unit, personalized learning and optimization module, data storage module, system control unit, the multimodal biofeedback acquisition module first starts and collects multiple physiological signals of the user in real time, and then sends these signals to the preprocessing unit for filtering, amplification and digital processing to improve signal quality, then the emotional state analysis module uses a deep learning algorithm to analyze the processed data and identify the user's emotional state. According to these states, the dynamic visual design adjustment module selects or generates matching from the visual design database The system can adjust the visual design scheme and render a new user interface in real time through the user interface rendering engine. Meanwhile, the real-time feedback and interaction module displays the adjusted user interface and collects user feedback and interaction data. The user behavior analysis unit analyzes these data to identify user preferences. The personalized learning and optimization module uses these analysis results to continuously optimize the system performance and user experience. All data and analysis results are stored in the data storage module, and the system control unit monitors the operating status of all modules to ensure the stable operation and self-diagnosis of the entire system. The close collaboration of these modules constitutes a complete system that can respond to user emotional changes in real time and provide personalized visual experience.

2. The emotional synchronization visual design system according to claim 1, characterized in that: The multimodal biofeedback acquisition module also includes at least one sound sensor for collecting the user's sound signal to assist in analyzing the user's emotional state.

3. The emotional synchronization visual design system according to claim 1, characterized in that: The preprocessing unit further includes a signal enhancement algorithm for improving the signal-to-noise ratio of the physiological signal.

4. The emotional synchronization visual design system according to claim 1, characterized in that: The emotional state analysis module further includes a deep learning algorithm for improving the accuracy of emotional state recognition.

5. The emotional synchronization visual design system according to claim 1, characterized in that: The dynamic visual design adjustment module further includes a visual design database for storing a variety of preset visual design solutions.

6. The emotional synchronization visual design system according to claim 1, characterized in that: The user interface rendering engine further includes a graphics processing unit for accelerating the graphics rendering process.

7. The emotional synchronization visual design system according to claim 1, characterized in that: The real-time feedback and interaction module also includes a tactile feedback unit for providing tactile feedback to enhance the user's interactive experience.

8. The emotional synchronization visual design system according to claim 1, characterized in that: The user behavior analysis unit further includes a behavior pattern recognition algorithm for identifying the user's long-term behavior pattern.

9. The emotional synchronization visual design system according to claim 1, characterized in that: The personalized learning and optimization module further includes a prediction model for predicting the user's future emotional state and behavioral trends.

10. A method for operating the emotion synchronization visual design system based on multimodal biofeedback as claimed in claim 1, characterized in that: The following steps are involved: s1, start the multimodal biofeedback acquisition module to collect the user's heart rate, skin conductance, brain waves, body temperature, body activity, breathing pattern and eye movement data; s2, filtering, amplifying and digitizing the collected physiological signals through the preprocessing unit; s3, transmitting the preprocessed physiological signals to the emotional state analysis module, and analyzing the user's emotional state using a preset emotion recognition algorithm; s4, according to the result of the emotional state analysis module, the dynamic visual design adjustment module adjusts the visual design elements of the user interface; s5, the user interface rendering engine renders the user interface in real time according to the instructions of the dynamic visual design adjustment module; s6, the real-time feedback and interaction module displays the adjusted user interface and receives the user's interactive input; s7, the user behavior analysis unit analyzes the user's interaction patterns and preferences; s8, personalized learning and optimization module optimizes visual design adjustment strategies based on user feedback information, behavior data and user behavior analysis results; s9, the data storage module stores the user's emotional state data, visual design adjustment history, user interaction data, and user behavior analysis results; s10, the system control unit monitors the working status of each module to ensure the stable operation of the system, and performs self-diagnosis and adjustment according to the system operating status.

Images