[0051] The invention discloses an intelligent voice recognition interactive lighting method based on emotion judgment, including the following steps:
[0052] Step 1. Collect the user's voice data
[0053] The voice data described in this solution refers to the voice information of the user when speaking, which is collected by the voice collection module, and the voice data can be saved in a wav format.
[0054] Step 2: Determine whether the voice data contains a voice command, and if it contains a voice command, adjust the lighting mode in the environment where the user is located according to the voice command.
[0055] After the voice collection module obtains the user’s voice data, the voice recognition module uses voice recognition technology to obtain the vocabulary in the user’s voice data, and then compares with the preset command vocabulary to determine whether the voice data contains Voice commands.
[0056] For example, the voice recognition module has pre-stored voice commands such as "turn off the lights", "turn on the lights", and "change colors", and set the adjustment control logic between these voice commands and the LED lights. If a certain voice command is recognized from the user's voice data, the lighting mode in the current environment is adjusted to correspond to the voice command. The lighting mode refers to the status of the brightness, color temperature, and color of the LED lights in the current environment. For example, when the voice command “turn off the lights” is recognized, the LED lights are turned off through the LED drive control module.
[0057] The above-mentioned light adjustment method is the first adjustment method in this solution, and it is also the most basic function. Once the voice command in the user's voice data is recognized, it is adjusted accordingly by the LED driving module, so that the user can freely adjust the lighting mode of the current environment.
[0058] As a further optimization of the above technical solution: the voice recognition module is also connected to the network through the WiFi wireless communication module, and is used to cooperate with the voice feedback module to achieve voice interaction with the user.
[0059] Step 3: When the user's voice data does not contain a voice command, extract voice features from the voice data, and perform pattern matching on the voice features with preset features stored in the emotion library, and obtain the matching result The current mood of the user, and then adjust the lighting mode in the environment where the user is located to correspond to the mood.
[0060] The voice feature refers to the prosody feature, voice quality feature, frequency spectrum feature, vocabulary feature and voiceprint feature extracted from voice data. Among them, the prosody feature is also called the super-sound quality feature or the super-segment feature, which refers to the changes in pitch, length and intensity in addition to the tone quality feature. The sound quality characteristics refer to the formants F1-F3, frequency band energy distribution, harmonic signal-to-noise ratio and short-term energy jitter in the audio. Spectrum feature, also called vibration spectrum feature, refers to a graph formed by decomposing complex oscillations into resonant oscillations of different amplitudes and frequencies, and arranged by the amplitudes of these resonant oscillations according to frequency. The spectrum characteristics are combined with the rhythm characteristics and the sound quality characteristics to improve the anti-noise effect of the characteristic parameters. Vocabulary features refer to the part-of-speech features of words in the voice data collected during the interaction between the system and the user. The part-of-speech feature is combined with other voice features in the voice data to help identify the emotional state of the user corresponding to the collected voice data. Voiceprint features refer to features related to users. The combination of voiceprint features and other voice features can effectively improve the accuracy of recognition in the process of emotion recognition. The specific extraction method is to save the voice data in the wav file format, obtain the PCM data by removing the file header of the wav file, and then extract the voice data through algorithms such as LPC (Linear Predictive Coding) and MFCC (Mel Frequency Cepstral Coefficent) Voice characteristics.
[0061] The extracted voice features are pattern-matched with the preset features stored in the emotion library. The speech feature extraction shown in this solution is completed in the emotion recognition module. The emotion recognition module is preset with an emotion library, and the emotion library stores the user's preset features. The preset feature here means that the user collects corresponding voice data samples under different emotions, and then extracts the voice feature from the sample, and uses the voice feature as the preset feature. The emotion described in this solution refers to normal , Happy, excited, sad, lost, lonely, anger, fear, and mocking these nine basic emotional models.
[0062] For example, in a happy mood, the user collects the user's voice data samples, and extracts the voice features (prosody feature, sound quality feature, frequency spectrum feature, vocabulary feature, and voiceprint feature) of the sample through the LPC algorithm and the MFCC algorithm. Use this voice feature as the user's preset feature in happy mood; the same method can be used to obtain the user's preset features in other moods.
[0063] Each mood corresponds to a lighting mode, and each lighting mode corresponds to a different brightness, color temperature, and color of the LED lights; for example, in a happy mood, the brightness, color temperature, and color temperature of the LED lights in the corresponding lighting mode The color saturation is high; and in the mood of loss and loneliness, the color temperature in the corresponding lighting mode is relatively low. In this solution, a mood corresponds to a preset lighting mode and is stored. For example, in the normal lighting mode, the brightness, color temperature, and color are A1, B1, and C1, respectively; in the lighting mode of Happy, the brightness, color temperature, and color are A2, B2, and C2, etc.; these corresponding relationships are stored in the emotion library in.
[0064] In this step, the extracted voice features are pattern-matched with the preset features in the emotion library. The emotion corresponding to the preset feature with the highest matching degree is determined as the current emotion of the user, and then the current LED The light is adjusted to the lighting mode corresponding to the mood.
[0065] As a further optimization of the above technical solution: the emotion recognition module uploads the result of pattern matching between the voice feature and the preset feature to the cloud storage through the WiFi wireless communication module, and then the voice feature corresponding to the same matching result multiple times Calculate the average value, and use the average value to update the preset feature. What this means here is that when in a networked state, the emotion database can be updated to achieve more accurate recognition results. The specific method is that each recognition result is uploaded to the cloud. For the same recognition result, for example, the emotion recognized as "happy" appears N times in total, then the average of the user voice characteristics corresponding to the N times of happy emotion is calculated The value is used as the preset feature corresponding to the new "happy" emotion to replace the previous preset feature. Through this online update method, the data in the emotion database can be more accurate.
[0066] The above process realizes the intelligent adjustment of the ambient light by judging the user's mood. In the method of the present invention, the corresponding adjustment of the ambient light and the voice interaction with the user can also be performed by formulating a schedule of work and rest.
[0067] Step 4. Determine whether the user has set a work schedule. If so, obtain the user's mood at the moment through voice interaction before the task arrives in the work schedule, and determine whether the lighting pattern corresponding to the mood at the moment corresponds to the task in the work schedule If the lighting patterns are the same, adjust the lighting pattern to correspond to the mood at the moment.
[0068] The user can establish a work schedule through the work and rest time input module. The content of the work schedule includes the task, the time corresponding to the task, and the lighting mode corresponding to the task. The tasks and times in the schedule are user-defined, including bedtime, wake-up time, work time, off-duty time, meal time, exercise time, etc.; for example, the task corresponding to 13 o'clock in the work schedule is rest, and 18 o'clock corresponds to The task is dining; when entering the task, the user can select or adjust the lighting mode corresponding to the task at the same time; for convenience, the corresponding lighting mode can be preset in each common task. If the user thinks it is not appropriate, you can Different parameters (brightness, color temperature and color) in the lighting mode are manually adjusted.
[0069] It is the third order to adjust the lighting mode of the LED lights through the work and rest time. The second is to change the lighting mode by judging the user's mood, namely step 3. The first is to change the lighting mode through voice commands, namely step 2. That is, when the user's voice data contains voice commands, they will be executed first.
[0070] In this step, preferably, if it is determined that the user has set a work schedule, then within 10-15 minutes before the arrival of the task closest to the current time in the work schedule, the user is reminded through voice interaction and the user’s current time is obtained. Emotions. For example, first broadcast the current time through the voice feedback module, and then ask the user's status, such as: "How is your mood today"; when the user feedbacks, collect the user's voice data, and use the method in step 3 to perform pattern matching to get the user's current mood , So as to determine whether the lighting mode corresponding to the user's current mood is consistent with the lighting mode corresponding to the task arriving next. There are two results after the judgment:
[0071] First, if the judgment result is that the two lighting modes are inconsistent, that is, they are inconsistent, the lighting mode corresponding to the recognized emotion is preferentially selected, and the lighting mode is adjusted to correspond to the emotion at the moment (the lighting mode corresponding to the emotion).
[0072] Second, if the judgment result is that there is no conflict between the two lighting modes, that is, the brightness, color temperature, and color are consistent, no operation is performed.
[0073] Regardless of the first or second case, when the time corresponding to the task is reached, the user is asked whether to perform the task on the schedule.
[0074] If the user replies to confirm the execution of the task on the schedule, the lighting mode is adjusted to the lighting mode corresponding to the task. The user reply refers to the voice command of “confirmation” contained in the collected user voice data.
[0075] If the user replies negatively to perform tasks on the schedule, then ask the user why and determine the user’s current mood; that is, when the user’s voice reply: “not executed”, the user will be asked the reason through the voice feedback module to determine the user’s current mood , To carry out related adjustments to emotions. If it is determined that the user has a positive emotion, the user is encouraged and reminded to follow the tasks on the timetable. The voice reply sentences and encouragement sentences and other sentences output through the voice feedback module are all stored in advance, or the voice feedback module is obtained through the WiFi wireless communication module through the Internet.
[0076] If it is determined that the user has negative emotions (sad, lost, lonely, anger, fear, sarcasm), change the lighting mode (such as lowering the brightness and warming the color temperature) to alleviate the user's negative emotions, and conduct a human-machine conversation with the user , Adjust the interaction tone according to the user’s current mood and interact with the user. Find out the corresponding conversation content from the voice library on the network or from the voice library established in this example (pre-stored conversation content). Conversation with the user. After the end, identify the user's emotion again (using the identification method in step 3). If the user's emotion is still negative, the tasks in the daily schedule will be terminated, that is, all tasks will no longer remind the user.
[0077] The present invention also provides an intelligent voice recognition interactive lighting system based on emotion judgment, including:
[0078] Voice acquisition module, voice recognition module, emotion recognition module, work time input module, voice feedback module, WiFi wireless communication module, LED drive control module and LED lights; among them:
[0079] The voice acquisition module, the voice recognition module, and the emotion recognition module are connected in sequence, the voice recognition module, the emotion recognition module, and the work time input module are jointly connected to the LED drive control module, and the LED drive control module is connected to the LED light; the voice feedback module , The WiFi wireless communication module is connected and jointly connected to the voice recognition module, emotion recognition module, and work time input module;
[0080] Wherein, the LED drive control module is used to change the lighting mode of the LED lamp;
[0081] The voice collection module is used to collect user voice data.
[0082] The voice collection module is used to collect user voice data;
[0083] The voice recognition module is used to determine whether the voice data contains a voice command, and if it contains a voice command, adjust the lighting mode in the environment where the user is located through the LED drive control module according to the voice command;
[0084] The emotion recognition module is used to extract voice features from the voice data, perform pattern matching between the voice features and preset features stored in the emotion library, obtain the user's current emotions through the matching results, and then The lighting mode in the user's environment is adjusted to correspond to the mood through the LED drive control module;
[0085] The work and rest time input module is used to determine whether the user has set a work and rest schedule. If there is, the user will be reminded by voice interaction before the task arrives in the work and rest schedule, and the user’s mood at the moment will be obtained, and the mood at the moment will be judged corresponding Whether the lighting mode is consistent with the lighting mode corresponding to the task in the schedule, if not, adjust the lighting mode through the LED drive control module to correspond to the mood at the moment;
[0086] The voice feedback module is used to realize voice interaction with the user;
[0087] The WiFi wireless communication module is used to interconnect the system and the network.
[0088] The power supply module is used to supply power to the system; the connection relationship of each module is as follows Figure 7 Shown.
[0089] Figure 4 In an embodiment of the present invention, the lighting mode conversion process when the user wakes up and sleeps:
[0090] Step 41: When the user enters the sleep task and time and wake-up task time in the work schedule, the system automatically calculates the sleep duration required by the user.
[0091] Step 42, 10 minutes before the sleep time on the work schedule, the system starts to gradually decrease the brightness of the light and the color temperature according to the sleep mode setting, and the entire lighting environment is at a warm color temperature.
[0092] In step 43, when the work and rest time is reached, the system reminds the user that after the user says a command word to indicate that he is ready to sleep, the light will continue during sleep, and the brightness of the light will continue to decrease until it reaches zero, and the entire environment is in a dark environment.
[0093] Step 44: After the user speaks a command word to indicate negative, the user's emotional state is judged after asking the reason. If the user's mood is good, the user's sleep time will be reminded every 15 minutes to allow the user to observe the time on the schedule .
[0094] If the user is in a bad mood, then according to the judged user mood, the lighting mode of the light is converted to adjust the mood of the person, and dialogue with the user, so that the user's mood improves and sleeps in a good mood Status, asking the user again every 30 minutes if he wants to go to sleep.
[0095] Step 45: Calculate the length of time between the user actually uttering the command word confirming sleep and the time to wake up.
[0096] Step 46: 15 minutes before the user sets the wake-up time of the schedule, the light is turned on and enters the lighting mode for wake-up. The brightness of the light gradually increases and the color temperature is lower. When the brightness changes, the color temperature is always low. .
[0097] Step 47: 5 minutes before the user wakes up, the brightness continues to increase, and the color temperature also increases, but does not exceed 3500K, which is similar to the sunlight at sunrise, and wakes the user with light.
[0098] Step 48: Compare the sleep duration required by the user with the actual sleep duration required by the user, and see the result of the comparison. If the actual sleep time of the user is just right, the system uses a normal tone to wake up the user; if the user actually sleeps If the sleep time is too long, the system will use a higher volume and a more active tone to wake up the user; if the user’s actual sleep time is less than the sleep time required by the user for more than 30 minutes, the system will wake up with a soft tone to wake up .
[0099] Step 49: Stop the wake-up mode after the user says "has gotten up", and make an introduction to the user about today's weather, traffic, air, etc. through the Internet, so that the user can understand the situation of the new day.
[0100] Figure 5 It is a schematic diagram of the user's emotional judgment and lighting mode conversion during and after get off work in another embodiment of the present invention.
[0101] Step 51, 5 minutes before the working time of the schedule set by the user, remind the user of the time in a urging tone, let the user go to work as soon as possible, and report the traffic situation online in real time.
[0102] Step 52: The system is connected to the network to remind the user whether to bring an umbrella according to the weather conditions of the day.
[0103] Step 53, the system turns on the lights according to the first 15 minutes of off-duty time, and adjusts the lighting mode of the lights according to the tired emotion recognition result.
[0104] Step 54: The user wakes up the system with a command word after arriving home, and can switch the lighting mode of the lamp.
[0105] Step 55: The system asks the user about the situation of the day, recognizes the user's mood, performs lighting mode conversion, and speaks to make the user's mood in a relaxed state.
[0106] Image 6 It is a schematic diagram of the user's emotional judgment and lighting mode conversion during a meal in another embodiment of the present invention.
[0107] Step 61: When it is time for the user to set the meal time on the schedule, the system reminds the user to eat on time. When the user agrees, the light is converted to the corresponding lighting mode.
[0108] Step 62: Recommend relevant recipes to the user according to the weather of the day and the current season, so that the user can choose, and when the user asks for the recipe, the voice broadcast is performed online to teach the user to cook.
[0109] In step 63, the system asks the user for the number of people dining, and divides them into different modes according to the number of people dining. There are visitor mode, reunion mode and personal mode. In visitor mode, the brightness of the light increases, and the color temperature rises to a medium level. Visitors and users can have better conversations. In the visitor mode, the functions of emotion recognition and voice recognition are temporarily disabled, and there will be no misrecognition caused by the content of the conversation. In the reunion mode, the color of the lights will be biased towards red and yellow, and the lights will bring out a festive atmosphere to make the family reunion atmosphere more intense.
[0110] Step 64: When there is only the user dining alone, the system will conduct voice communication with the user on the Internet according to the user's emotions, and the color temperature of the light is low, so that the user can maintain a happy mood while dining by himself.
[0111] The system asks the user for the name of the dishes, including Chinese, Western, noodles, hot pot, desserts, etc. The user can choose not to answer. After the user answers, the system can switch to different lighting environments according to different dish names to highlight the dishes on the table. The atmosphere shows different characteristics through lighting, which increases the user's appetite.
[0112] Such as Figure 8 Shown is the schematic diagram of the circuit structure of the speech recognition module and the emotion recognition module
[0113] Such as Figure 8 It is shown that when the user speaks a command word or answers a system question, the user's voice enters the noise reduction circuit through the microphone array of the voice acquisition module to remove noise, so that the recognition result is more accurate. Perform voice recognition in the voice recognition module. When the user speaks the command word directly, the LED light will be driven according to the original setting; after the user answers, the emotion recognition module will recognize the user’s emotions and adjust the LED lights according to different emotions Different lighting modes; and the voice feedback module will be connected to the Internet, and the system will output according to the content of the network, such as chat content, weather, traffic, etc., and then decode it through a decoder. After the audio amplifier circuit is amplified, the voice is performed in the speaker Play.
[0114] Picture 9 Shown is the functional block diagram of the LED drive control module
[0115] Such as Picture 9 As shown, after the MCU microcontroller in the LED drive control module receives the output result signals processed by the speech recognition module, the emotion recognition module, and the work time input module, it is processed by the communication protocol into information that can be recognized by the microcontroller, and this information is processed internally by the microcontroller It becomes the PWM output to control the brightness, color temperature and color of the lamp, and the light parameters of the lamp are set in the microcontroller, and finally it becomes the lighting mode.
[0116] Such as Picture 9 The LED drive circuit shown provides stable voltage and current, so that the LED lamp bead lighting tends to be stable. Because the functions of the LED of the present invention include brightness and darkness, color temperature, and color gradation, a drive circuit is required to ensure stable gradation of the light parameters of the LED , There will be no glare for a moment when the lights are adjusted in stages.
