Smart home control method for collecting emotion through brain-computer interface

Abstract

The invention discloses a smart home control method for collecting emotion through a brain-computer interface, and the method comprises the following steps: S1, carrying out the initialization processing of smart home equipment, enabling a user to start a smart home control system, enabling the smart home control system to carry out the self-inspection, and guaranteeing that all hardware and software components are in a normal working state; s2, collecting data such as physiological signals, behavior data, voice, texts and brain waves of the user by adopting an emotion intelligent sensing module through sensors such as wearable equipment, a camera and a microphone; the brain-computer interface technology and the emotion intelligent sensing technology are combined, the emotional state of the user can be sensed and recognized in real time, the home environment is automatically adjusted according to the emotional state of the user, the more comfortable and convenient living environment is provided for the user, and therefore more intelligent, convenient and personalized services are provided; and the life quality and the intelligent degree of the user are improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home control technology, and in particular to a smart home control method for collecting emotions through a brain-computer interface. Background Art

[0002] With the continuous development of artificial intelligence technology, smart home systems have gradually entered people's daily lives. However, the smart home systems currently on the market mainly rely on physical operations or voice commands for control, lack the perception and response to the user's emotional state, and have a low level of intelligence. For this reason, we propose a smart home control method that collects emotions through a brain-computer interface. Summary of the invention

[0003] Based on the technical problems existing in the background technology, the present invention proposes a smart home control method for collecting emotions through a brain-computer interface.

[0004] The present invention proposes a smart home control method for collecting emotions through a brain-computer interface, comprising the following steps:

[0005] S1: Initialize the smart home devices. The user starts the smart home control system, and the smart home control system performs a self-check to ensure that all hardware and software components are in normal working condition.

[0006] S2: Use wearable devices, cameras, microphones and other sensors to collect users' physiological signals, behavioral data, voice and text, brain waves and other data using emotional intelligence perception modules;

[0007] S3: Perform preprocessing operations such as standardization, cleaning, denoising, normalization or scaling on the data collected in S2 to improve data quality, and perform appropriate preprocessing operations on image and voice data. Image denoising is performed using Gaussian filtering. The mathematical expression of Gaussian filtering is: x and y represent the coordinates of the pixel;

[0008] S4: Extract emotion-related features from the preprocessed data in S3. For EEG data, use a specialized algorithm to extract neural signal features related to intention and emotion, where the clustering coefficient of each neural signal node is where k i represents the number of nodes adjacent to neural signal node i, and e(i) represents the actual number of edges;

[0009] S5: Use machine learning algorithms to perform sentiment analysis on the features extracted from S4 to identify the differences in the clustering coefficients of users’ emotional states in and is the clustering coefficient of the neural signal node i of different types of emotions, q1, q2…qn represent different emotional states, and identify the user's emotional state. For brain wave data, the user's intention instructions are identified through the classification algorithm;

[0010] S6: The emotional intelligence perception module outputs the emotional state identified in S5 to the smart home control module in a standardized manner for subsequent intelligent control;

[0011] S7: The brain wave signal collected in S2 is amplified by an isolation amplifier, and then filtered and digitized to improve the signal quality;

[0012] S8: The brain-computer interface module uses machine learning or deep learning models to analyze the brain wave signals processed in S7, recognize the user's intention commands, which include turning on the lights and adjusting the temperature, etc., convert the recognized intention commands into control signals, and send them to the smart home control module;

[0013] S9: The smart home control module receives the emotional state and control signals from the emotional intelligence perception module and the brain-computer interface module, and decides how to adjust the smart home devices according to preset rules and algorithms, combined with the user's emotional state and control signals. The smart home control module sends control instructions to the corresponding smart home devices to realize the switching and adjustment functions of the smart home devices, including lights, air conditioners, curtains, etc.;

[0014] S10: Store the user's emotional state, intention instructions, home device usage records and other data in the data center, use data analysis tools and technologies to mine and analyze user data, understand user behavior patterns, needs and preferences, and optimize system performance and provide personalized services based on the analysis results.

[0015] Preferably, in S2, when collecting brain wave data, a brain-computer interface module is used for collection. The brain-computer interface module adopts advanced brain-computer interface technology to read the user's brain wave signals in a non-invasive manner, identify the user's intention instructions, and convert them into control signals and send them to the smart home control module. The specific implementation method of the brain-computer interface module includes acquiring image frames from a camera source, performing image preprocessing, and then using a CNN model to extract important features and perform emotion classification. The image preprocessing operations include cropping, resizing, rotation, and color correction.

[0016] Preferably, in S9, when the status of smart home devices is automatically adjusted according to the user's intention instructions and emotional state, it includes automatically reducing the brightness of indoor lights and playing soothing music when the user feels nervous; when the user wants to watch a movie, the indoor lights can be automatically adjusted and the projector can be turned on.

[0017] Preferably, in S10, by analyzing the user data, the smart home control system can continuously optimize its own performance and provide personalized services that better meet the user's needs.

[0018] Preferably, in S2, the user's physiological data, such as heart rate, blood pressure, etc., are collected through wearable devices (such as smart watches, health bracelets);

[0019] Use cameras, sensors, etc. to monitor users' daily activities and behavior patterns to achieve the purpose of behavioral data collection;

[0020] Collect user's voice commands and text input through microphone and voice recognition technology to achieve the purpose of voice and text data collection;

[0021] The purpose of brain wave data collection is achieved by collecting brain wave signals through electrodes placed on the user's scalp.

[0022] Preferably, in S4, the emotion-related features include word frequency, part of speech, emotional vocabulary, pitch, speech speed, facial muscle movement, etc.

[0023] Preferably, in S5, common emotion state categories include positive emotion, negative emotion and neutral emotion.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. The emotional intelligence perception module can sense and identify the user's emotional state in real time, thereby automatically adjusting the home environment to meet the user's emotional needs. This emotion recognition and response function makes the smart home system more humane and intelligent, and improves the user's living experience;

[0026] 2. The use of advanced brain-computer interface technology and emotional intelligence perception technology makes the smart home system more intelligent. Users can control home appliances through simple thinking activities or emotional expressions, realizing true intelligent operation;

[0027] The present invention combines brain-computer interface technology with emotional intelligence perception technology, which can perceive and identify the user's emotional state in real time, automatically adjust the home environment according to the user's emotional state, and provide users with a more comfortable and convenient living environment, thereby providing more intelligent, convenient and personalized services, improving the user's quality of life and intelligence, while enhancing health and well-being, and has broad application prospects and market potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a smart home control method for collecting emotions through a brain-computer interface proposed by the present invention;

[0029] Figure 2 This is a block diagram of the connection between the emotional intelligent perception module, the brain-computer interface module, the smart home control module and the data center in the smart home control method for collecting emotions through a brain-computer interface proposed by the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further explained below in conjunction with specific embodiments.

[0031] Example

[0032] Reference Figure 1-2 , this embodiment proposes a smart home control method for collecting emotions through a brain-computer interface, comprising the following steps:

[0033] S1: Initialize the smart home devices. The user starts the smart home control system, and the smart home control system performs a self-check to ensure that all hardware and software components are in normal working condition.

[0034] S2: The emotional intelligence perception module is used to collect the user's physiological signals, behavioral data, voice and text, brain wave and other data through wearable devices, cameras, microphones and other sensors. When collecting brain wave data, the brain-computer interface module is used for collection. The brain-computer interface module uses advanced brain-computer interface technology to read the user's brain wave signals in a non-invasive manner, identify the user's intention instructions, and convert them into control signals and send them to the smart home control module. The specific implementation method of the brain-computer interface module includes obtaining image frames from the camera source, performing image preprocessing, and then using the CNN model to extract important features and perform emotion classification. The image preprocessing operations include cropping, resizing, rotation and color correction.

[0035] Among them, wearable devices (such as smart watches and health bracelets) are used to collect users' physiological data, such as heart rate and blood pressure;

[0036] Use cameras, sensors, etc. to monitor users' daily activities and behavior patterns to achieve the purpose of behavioral data collection;

[0037] Collect user's voice commands and text input through microphone and voice recognition technology to achieve the purpose of voice and text data collection;

[0038] The purpose of brain wave data collection is achieved by collecting brain wave signals through electrodes placed on the user's scalp;

[0039] S3: Perform preprocessing operations such as standardization, cleaning, denoising, normalization or scaling on the data collected in S2 to improve data quality, and perform appropriate preprocessing operations on image and voice data. Image denoising is performed using Gaussian filtering. The mathematical expression of Gaussian filtering is: x and y represent the coordinates of the pixel;

[0040] S4: Extract emotion-related features from the preprocessed data in S3. For EEG data, use a special algorithm to extract neural signal features related to intention and emotion. Emotion-related features include word frequency, part of speech, emotional vocabulary, pitch, speech rate, facial muscle movement, etc. The clustering coefficient of each neural signal node is where k i represents the number of nodes adjacent to neural signal node i, and e(i) represents the actual number of edges;

[0041] S5: Use machine learning algorithms to perform sentiment analysis on the features extracted from S4 to identify the differences in the clustering coefficients of users’ emotional states in and is the clustering coefficient of the neural signal node i of different types of emotions, q1, q2…qn represent different emotional states, and identify the user's emotional state. For brain wave data, the user's intention instructions are identified through classification algorithms. Common emotional state categories include positive emotions, negative emotions, and neutral emotions;

[0042] S6: The emotional intelligence perception module outputs the emotional state identified in S5 to the smart home control module in a standardized manner for subsequent intelligent control;

[0043] S7: The brain wave signal collected in S2 is amplified by an isolation amplifier, and then filtered and digitized to improve the signal quality;

[0044] S8: The brain-computer interface module uses machine learning or deep learning models to analyze the brain wave signals processed in S7, recognize the user's intention commands, which include turning on the lights and adjusting the temperature, etc., convert the recognized intention commands into control signals, and send them to the smart home control module;

[0045] S9: The smart home control module receives the emotional state and control signal from the emotional intelligence perception module and the brain-computer interface module, and decides how to adjust the smart home devices according to the preset rules and algorithms, combined with the user's emotional state and control signal. The smart home control module sends control instructions to the corresponding smart home devices to realize the functions of switching, adjusting, etc. of the smart home devices, wherein the smart home devices are automatically adjusted according to the user's intention instructions and emotional state, including when the user feels nervous, the indoor light brightness can be automatically reduced and soothing music can be played; when the user wants to watch a movie, the indoor light can be automatically adjusted and the projector can be turned on;

[0046] S10: storing the user's emotional state, intention instructions, home device usage records and other data in the data center, using data analysis tools and technologies to mine and analyze the user data, understand the user's behavior patterns, needs and preferences, and optimize the system's performance and provide personalized services based on the analysis results. By analyzing the user data, the smart home control system can continuously optimize its own performance and provide personalized services that better meet the user's needs;

[0047] This embodiment combines brain-computer interface technology and emotional intelligence perception technology, which can perceive and identify the user's emotional state in real time, automatically adjust the home environment according to the user's emotional state, and provide users with a more comfortable and convenient living environment, thereby providing more intelligent, convenient and personalized services, improving the user's quality of life and intelligence, while enhancing health and well-being, and has broad application prospects and market potential.

[0048] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A smart home control method for collecting emotions through a brain-computer interface, characterized in that: The following steps are involved: S1: Initialize the smart home devices. The user starts the smart home control system, and the smart home control system performs a self-check to ensure that all hardware and software components are in normal working condition. S2: Use wearable devices, cameras, and microphones to collect users' physiological signals, behavioral data, voice and text, and brain wave data using emotional intelligence perception modules; S3: Standardize, clean, denoise, normalize or scale the data collected in S2 to improve data quality, and perform appropriate preprocessing operations on image and voice data. Image denoising is performed using Gaussian filtering. The mathematical expression of Gaussian filtering is: x and y represent the coordinates of the pixel; S4: Extract emotion-related features from the preprocessed data in S3. For EEG data, use a specialized algorithm to extract neural signal features related to intention and emotion, where the clustering coefficient of each neural signal node is where k i represents the number of nodes adjacent to neural signal node i, and e(i) represents the actual number of edges; S5: Use machine learning algorithms to perform sentiment analysis on the features extracted from S4 to identify the differences in the clustering coefficients of users’ emotional states in and is the clustering coefficient of the neural signal node i of different types of emotions, q1, q2…qn represent different emotional states. For brain wave data, the user's intention instructions are identified through the classification algorithm; S6: The emotional intelligence perception module outputs the emotional state identified in S5 to the smart home control module in a standardized manner; S7: The brain wave signal collected in S2 is amplified by an isolation amplifier, and then filtered and digitized; S8: The brain-computer interface module uses machine learning or deep learning models to analyze the brain wave signals processed in S7, recognize the user's intention instructions, convert the recognized intention instructions into control signals, and send them to the smart home control module; S9: The smart home control module receives the emotional state and control signal from the emotional intelligence perception module and the brain-computer interface module, and decides how to adjust the smart home devices according to preset rules and algorithms in combination with the user's emotional state and control signal. The smart home control module sends control instructions to the corresponding smart home devices to realize the switch and adjustment functions of the smart home devices, and the smart home devices include lights, air conditioners and curtains; S10: Store the user's emotional state, intention instructions, and home device usage record data in the data center, use data analysis tools and technologies to mine and analyze user data, understand user behavior patterns, needs and preferences, and optimize system performance and provide personalized services based on the analysis results.

2. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In S2, when collecting brain wave data, a brain-computer interface module is used for collection. The brain-computer interface module adopts advanced brain-computer interface technology to read the user's brain wave signals in a non-invasive manner, identify the user's intention instructions, and convert them into control signals and send them to the smart home control module. The specific implementation method of the brain-computer interface module includes acquiring image frames from a camera source, performing image preprocessing, and then using a CNN model to extract important features and perform emotion classification. The image preprocessing operations include cropping, resizing, rotation and color correction.

3. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In the S9, the state of the smart home device is automatically adjusted according to the user's intention instructions and emotional state, which includes automatically reducing the brightness of the indoor light and playing soothing music when the user feels nervous; when the user wants to watch a movie, the indoor light can be automatically adjusted and the projector can be turned on.

4. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In S10, by analyzing the user data, the smart home control system can continuously optimize its own performance and provide personalized services that better meet the user's needs.

5. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In S2, the physiological data of the user is collected through a wearable device; Use cameras and sensors to monitor users' daily activities and behavior patterns to collect behavioral data; Collect user's voice commands and text input through microphone and voice recognition technology to achieve the purpose of voice and text data collection; The purpose of brain wave data collection is achieved by collecting brain wave signals through electrodes placed on the user's scalp.

6. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In S4, the emotion-related features include word frequency, part of speech, emotional vocabulary, pitch, speaking speed, and facial muscle movement.

7. The method for controlling a smart home by collecting emotions through a brain-computer interface according to claim 1, characterized in that: In S5, common emotional state categories include positive emotions, negative emotions, and neutral emotions.

Images