Method, computer program product and system for providing personalized home service for user based on user voice interaction

Abstract

The invention provides a method, a computer program product and a system for providing personalized home service for a user based on user voice interaction, and the method comprises the steps: S1, carrying out the first-stage voice interaction with the user according to a preset first voice interaction mode; s2, determining the emotional state of the user according to the voice text content and voice features of the user in the first-stage voice interaction process; s3, selecting a proper voice interaction mode from a plurality of preset second voice interaction modes to perform second-stage voice interaction with the user according to the emotional state of the user; s4, according to the answers of the user to the questions in the second-stage voice interaction process, the personalized requirements of the user for at least one of the vision, the hearing, the smell and the touch are deduced; and S5, determining output parameters of the corresponding household equipment according to the personalized requirements. The smart home system can automatically provide personalized home service for the user by adopting the method.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a method, computer program product, and system for providing personalized home services based on user voice interaction. Background Technology

[0002] With the development of IoT technology, some manufacturers are using it to control and link home-related devices, creating smart home systems to provide users with a safe, convenient, and comfortable living environment. However, most existing smart home systems can only execute home service operations based on explicit user commands and cannot automatically provide personalized home services. Summary of the Invention

[0003] The technical problem to be solved by this invention is how a smart home system can automatically provide personalized home services to users.

[0004] To address the aforementioned technical problems, this invention provides a method for providing personalized home services based on user voice interaction, comprising the following steps: S1. Conduct the first stage of voice interaction with the user according to the preset first voice interaction mode; S2. Determine the user's emotional state based on the voice and text content and voice characteristics of the user during the first stage of voice interaction. S3. Based on the user's emotional state, select the appropriate mode from multiple preset second voice interaction modes to conduct the second stage of voice interaction with the user; S4. Based on the user's answers to various questions during the second stage of voice interaction, deduce the user's personalized needs for at least one of the visual, auditory, olfactory, and tactile senses. S5. Determine the output parameters of the corresponding home appliances based on the personalized requirements.

[0005] Furthermore, step S2 specifically involves inputting the user's voice and text content and voice features into a pre-trained emotion recognition AI model for identification, thereby obtaining the user's emotional state.

[0006] Furthermore, sound characteristics include at least one of tone, timbre, frequency, and amplitude.

[0007] Furthermore, S5 specifically: D1. The personalized needs include visual personalized needs, and the home appliances corresponding to the visual needs include projectors. The output image of the projector is determined. And / or D2, the personalized needs include auditory personalized needs, the home appliances corresponding to the auditory needs include audio players, and the type of audio played by the audio player is determined; And / or D3, the personalized needs include olfactory personalized needs, the home appliances corresponding to the sense of smell include aroma diffusers, and the type of essential oil released by the aroma diffuser is determined; And / or D4, the personalized needs include tactile personalized needs, the home appliances corresponding to tactile sensation include smart mattresses, and the firmness of the smart mattress is determined.

[0008] Furthermore, the D4 also determines the vibration frequency and / or heating temperature of the smart mattress.

[0009] Furthermore, D1 specifically determines the output image of the smart mattress's projector, D2 specifically determines the type of audio played by the smart mattress's audio player, and D3 specifically determines the type of essential oil released by the smart mattress's aromatherapy diffuser.

[0010] Furthermore, step S6 involves monitoring the user's sleep state and physiological indicators during sleep, adjusting the audio type played by the audio player according to the user's sleep state, and determining whether the user's physiological indicators are abnormal based on physiological indicator thresholds, and issuing an alarm if any abnormality is detected.

[0011] The present invention also provides a computer program product, including a computer program that, when executed, implements the method described above for providing personalized home services based on user voice interaction.

[0012] This invention also provides a smart home system that provides personalized home services based on user voice interaction, including a smart mattress and a smart doll that are interconnected; the smart mattress includes a mattress body and an audio player and an aromatherapy diffuser located at the corners of the mattress body, including a projector located at the head of the mattress, and the mattress body has built-in adjustable airbags, massage vibrators and heating modules; the smart mattress has a controller that controls and connects the adjustable airbags, massage vibrators, heating modules, audio player, aromatherapy diffuser and projector; the smart doll is specifically connected to the controller of the smart mattress.

[0013] Furthermore, the smart mattress includes a physiological monitoring sensor unit built into the mattress body and a pillow located at the head of the mattress body. The pillow has a built-in bone conduction unit that wirelessly connects to an audio player.

[0014] The present invention provides a method for providing personalized home services based on user voice interaction. S1 and S2 use user voice recognition in the first stage of voice interaction to obtain the user's emotional state. S3 performs the second stage of voice interaction based on the user's emotional state. S4 infers the user's personalized needs for at least one of the visual, auditory, olfactory, and tactile senses based on the second stage of voice interaction. S5 determines the output parameters of the corresponding home equipment accordingly to provide personalized home services to the user. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the method for providing personalized home services based on user voice interaction, as provided in this invention.

[0016] Figure 2 This is a structural block diagram of a smart home system that provides personalized home services based on user voice interaction, as presented in this invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to specific embodiments.

[0018] Smart home systems that provide personalized home services based on user voice interaction, such as Figure 2 As shown, the device includes a smart doll and a smart mattress that are interconnected. The smart mattress in this embodiment includes a mattress body, an audio player and an aromatherapy diffuser located at the corners of the mattress body, and a projector located at the head of the mattress. The mattress body has built-in adjustable airbags, a massage vibrator, and a heating module. A controller is installed within the smart mattress to control the adjustable airbags, massage vibrator, heating module, audio player, aromatherapy diffuser, and projector. The smart mattress can be referenced from prior art, such as patent documents CN120477541A, CN108125455A, and CN109043964A. The smart doll is specifically connected to the controller of the smart mattress.

[0019] The smart doll contains a computer program product, which includes a computer program. The smart mattress controller executes this computer program to achieve the following: Figure 1 The method shown provides personalized home services based on user voice interaction. The following example demonstrates how this smart home system is applied to a personal home, implementing this method to provide a personalized sleep mode service: S1. Conduct the first stage of voice interaction with the user according to the preset first voice interaction mode.

[0020] In this embodiment, a general first voice interaction mode is pre-set by technicians and configured in the smart doll. The first voice interaction mode is a casual conversation mode, and the interaction questions include, for example: "1. Good evening, how was your work today? 2. Are you tired after a long day? 3. Do you want me to tell you a joke? 4. Is there anything I need to remind you about tomorrow?" The smart doll detects whether someone is speaking in the room through its microphone. If it detects someone speaking in the room, it automatically initiates the first stage of voice interaction with the person in the room (hereinafter referred to as the user). Specifically, the smart doll conducts the first stage of voice interaction with the user according to the preset first voice interaction mode, that is, it asks the user questions according to the interaction questions of the first voice interaction mode and monitors the user's answers.

[0021] S2. Determine the user's emotional state based on the voice and text content and voice characteristics of the user during the first stage of voice interaction.

[0022] The intelligent doll performs text recognition on the monitored user's speech to obtain the speech-text content, and extracts sound features from the speech. Sound features include tone, timbre, frequency, and amplitude; alternatively, other embodiments may include at least one of tone, timbre, frequency, and amplitude. This embodiment pre-collects a large number of training samples (using text content and sound features as input, and emotional state as output, forming a set of training samples). These large training samples are used to train a large-scale artificial intelligence model (an existing general-purpose model in the industry), enabling it to recognize the user's emotional state based on text content and sound features. The trained large-scale artificial intelligence model is denoted as the emotion recognition artificial intelligence model. The intelligent doll inputs the user's speech-text content and sound features into the trained emotion recognition artificial intelligence model for recognition, obtaining the user's emotional state, such as recognizing the user as currently anxious.

[0023] S3. Based on the user's emotional state, select the appropriate mode from multiple preset second voice interaction modes to conduct the second stage of voice interaction with the user.

[0024] In this embodiment, technicians pre-set multiple second voice interaction modes corresponding to various emotional states and configured them in the smart doll. Emotional states include excitement, depression, calmness, irritability, and anxiety. The interaction issues for the second voice interaction modes corresponding to different emotional states differ, as detailed below: Interactive questions in the second voice interaction mode corresponding to an excited emotional state include: "1. My brain is excited and it's hard to fall asleep. Would you like to look at the vast starry sky to help calm my mind? Or is there anything else you'd like to see? 2. When my brain is excited at night, I can listen to some soothing music to help me fall asleep. Would you like me to play a song? 3. Do you prefer the smell of flowers or fruits? 4. Massage helps calm the brain, and you can also get heat therapy during a massage. Would you like that? 5. Do you prefer a firmer or softer mattress?"

[0025] Interactive questions in the second voice interaction mode corresponding to a low mood state include: "1. Beautiful flowers can make people feel happy and help them fall asleep quickly. Which flower do you like best? 2. Listening to light music can help you sleep. Would you like some? 3. Orange or grapefruit, which scent do you prefer? 4. If you need, I can provide you with a warm and soft bed. Would you like that?"

[0026] Interactive questions for the second voice interaction mode corresponding to a calm emotional state include: "1. Desert, ocean, beach, grassland, starry sky, which environment would you like to fall asleep in? 2. Would you like to listen to some music? 3. Do you prefer a light or strong fragrance? 4. Would you like a massage or heat therapy? 5. Do you prefer to sleep on a hard or soft bed?"

[0027] Interactive questions in the second voice interaction mode corresponding to an agitated emotional state include: "1. Which would you prefer to be in: a spring meadow, a summer beach, or an autumn wheat field? 2. Would you like to hear the sound of ocean waves? 3. Do you like the smell of lavender? It can relieve stress! 4. A massage can relax your mind and body. Would you like me to give you a massage? 5. Do you prefer to sleep on a hard bed or a soft bed?"

[0028] Interactive questions in the second voice interaction mode corresponding to an anxious emotional state include: "1. I've heard that looking at the stars can heal the soul. Shall we look at the starry sky? 2. Which do you prefer to hear: the sound of waves, rain, or the chirping of insects and birds? 3. The scent of chamomile can relieve anxiety. Shall we have some chamomile aromatherapy? 4. Gentle body vibrations help relax the mind and body. Shall I give you a low-frequency vibration massage while you sleep? 5. Do you want the feeling of being warmly enveloped and secure?"

[0029] After recognizing the user's emotional state, the smart doll selects a suitable second voice interaction mode from several preset modes and then uses that mode to engage in a second-stage voice interaction with the user. For example, if the doll recognizes that the user is currently anxious, it will use the second voice interaction mode corresponding to that emotional state to engage in a second-stage voice interaction, asking questions such as: "1. I've heard that looking at the stars can heal the soul. Shall we look at the starry sky?", "2. Which do you prefer to hear: the sound of waves, rain, or birdsong?", "3. The scent of chamomile can relieve anxiety. Shall we have some chamomile aromatherapy?", and monitoring the user's responses. Suppose the user's answer to question 1 is "okay"; their answer to question 2 is "the sounds of insects and birds"; their answer to question 3 is "okay"; their answer to question 4 is "good"; and their answer to question 5 is "let's try it." Each time the smart doll asks a question, it monitors the user's answer, then associates and saves the monitored user's voice response with the question.

[0030] S4. Based on the user's answers to various questions during the second stage of voice interaction, deduce the user's personalized needs for at least one of the visual, auditory, olfactory, and tactile senses.

[0031] The smart doll performs text recognition on the user's voice answers to various questions to obtain the corresponding voice-text content. Based on this analysis, it deduces the user's personalized needs for sight, hearing, smell, and touch. Taking the user's answers above as an example, the answer to question 1 is "okay"; the answer to question 2 is "insect chirping and bird calls"; the answer to question 3 is "okay"; the answer to question 4 is "okay"; and the answer to question 5 is "try it." Based on this, the user's personalized needs for sight, hearing, smell, and touch can be analyzed as follows: visual personalized need is to see the starry sky; auditory personalized need is insect chirping and bird calls; olfactory personalized need is the scent of chamomile; and tactile personalized need is low-frequency vibration massage and a warm, enveloping feeling.

[0032] S5. Determine the output parameters of the corresponding home appliances based on the personalized requirements.

[0033] The smart mattress in this embodiment integrates the mattress body, projector, audio player, and aroma diffuser. That is, it integrates the home appliances corresponding to vision (projector), hearing (audio player), smell (aroma diffuser), and touch (mattress body). The intelligent doll determines the output parameters of the corresponding home appliances based on the user's personalized needs for sight, hearing, smell, and touch. Specifically: if the personalized visual need is to see the starry sky, the output parameters of the corresponding projector will be: turn on the projector and project a starry sky image; if the personalized auditory need is the sound of insects and birds, the output parameters of the corresponding audio player will be: play the sound of insects and birds; if the personalized olfactory need is the scent of chamomile, the output parameters of the corresponding aromatherapy diffuser will be: release chamomile essential oil; and if the personalized tactile need is low-frequency vibration massage and a warm, enveloping feeling, the output parameters of the corresponding mattress will be: activate the massage vibrator for low-frequency vibration, activate the heating module for slight heating, and adjust the airbags to adjust the mattress firmness to the softest level. The smart doll packages the output parameters of all home appliances into a personalized sleep mode for the user, and provides the user with a corresponding sleep mode accordingly. Specifically, the smart doll sends the user's personalized sleep mode (containing the output parameters of all home appliances) to the controller of the smart mattress. The controller of the smart mattress adjusts the output parameters of the projector, audio player, aromatherapy diffuser, and mattress body corresponding to visual, auditory, olfactory, and tactile senses, respectively, to provide the user with a corresponding sleep mode. Specifically, the projector projects a starry sky onto the surrounding walls and ceiling, the audio player plays insect and bird sounds, the aromatherapy diffuser releases chamomile essential oil, the mattress body activates the massage vibrator to vibrate at a low frequency, the heating module activates to provide slight heating, and the airbags are adjusted to adjust the mattress firmness to the softest level. In this embodiment, the smart mattress includes a pillow located at the head of the mattress body. This pillow has a built-in bone conduction unit, which wirelessly connects to the audio player via, for example, Bluetooth. The audio player transmits the sounds of insects and birds to the pillow's bone conduction unit via Bluetooth. The bone conduction unit converts the sound into mechanical vibrations of different frequencies, generating sound waves that are then transmitted to the user through the skull.

[0034] S6. During the user's sleep, monitor the user's sleep state and physiological indicators, adjust the audio type played by the audio player according to the user's sleep state, and determine whether the user's body has abnormal physiological indicators based on physiological indicator thresholds. Once an abnormality is detected, an alarm will be issued.

[0035] The smart mattress includes a physiological monitoring sensor unit built into the mattress body, and a controller connected to the physiological monitoring sensor unit. The physiological monitoring sensor unit includes multiple flexible piezoelectric thin-film sensors and heart rate and blood pressure sensors (PPG (photoplethysmography) based sensors). Multiple flexible piezoelectric thin-film sensors are arranged on the mattress at preset row and column intervals, forming a flexible piezoelectric thin-film sensor array. This array can monitor pressure changes caused by the user's turning over, micro-limb movements, and breathing. The controller uses the signal intensity changes of the flexible piezoelectric thin-film sensors at different locations in the array to determine the user's positional changes, thereby determining the number of times the user turns over; furthermore, it uses the periodic pressure changes monitored by the flexible piezoelectric thin-film sensors at the same location to determine the user's respiratory rate.

[0036] During sleep, the smart mattress uses a flexible piezoelectric thin-film sensor array to monitor the user's sleep state—the number of times they toss and turn and their breathing rate. The smart mattress controller selects corresponding sleep-inducing audio based on the sleep state detected by the sensor array, then instructs the audio player to play the audio, which is transmitted to the user through the bone conduction unit to promote sleep. For example, if the sensor array detects that the user tosses and turns repeatedly for a long time or has large fluctuations in breathing rate, it means the user is in a light sleep stage and has difficulty entering deep sleep. Therefore, low-frequency sound waves (such as specific frequency sound waves synchronized with brain waves) are selected and played, transmitted to the sleeping user through the bone conduction unit built into the pillow, relaxing the brain and promoting deep sleep. The smart mattress controller also uses heart rate and blood pressure sensors (PPG (photoplethysmography) based sensors) to monitor the user's heart rate and blood pressure during sleep. Based on preset heart rate and blood pressure thresholds, it determines whether the user's heart rate and blood pressure are abnormal. If an abnormality is detected, an alarm is issued, such as automatically sending an alert to the user's emergency contact or automatically dialing the user's emergency contact via AI to inform them of the abnormal condition.

[0037] The present invention provides a method for providing personalized home services based on user voice interaction. S1 and S2 use user voice recognition in the first stage of voice interaction to obtain the user's emotional state. S3 performs a second stage of voice interaction based on the user's emotional state. S4 infers the user's personalized needs for at least one of the visual, auditory, olfactory, and tactile senses based on the second stage of voice interaction. S5 determines the output parameters of the corresponding home devices accordingly to provide personalized home services to the user, namely, providing personalized sleep mode services.

[0038] The above description is merely an embodiment of the present invention and does not limit the scope of patent protection. Any non-substantial changes or substitutions made by those skilled in the art based on the present invention will still fall within the scope of patent protection.

Claims

1. A method for providing personalized home services based on user voice interaction, characterized by: Includes the following steps: S1. Conduct the first stage of voice interaction with the user according to the preset first voice interaction mode; S2. Determine the user's emotional state based on the voice and text content and voice characteristics of the user during the first stage of voice interaction. S3. Based on the user's emotional state, select the appropriate mode from multiple preset second voice interaction modes to conduct the second stage of voice interaction with the user; S4. Based on the user's answers to various questions during the second stage of voice interaction, deduce the user's personalized needs for at least one of the visual, auditory, olfactory, and tactile senses. S5. Determine the output parameters of the corresponding home appliances based on the personalized requirements.

2. The method for providing personalized home services based on user voice interaction as described in claim 1, characterized in that, Step S2 specifically involves inputting the user's voice and text content and voice features into a pre-trained emotion recognition AI model for identification, thereby obtaining the user's emotional state.

3. The method for providing personalized home services based on user voice interaction as described in claim 2, characterized in that, Sound characteristics include at least one of tone, timbre, frequency and amplitude.

4. The method for providing personalized home services based on user voice interaction as described in claim 1, characterized in that, S5 specifically: D1. The personalized needs include visual personalized needs, and the home appliances corresponding to the visual needs include projectors. The output image of the projector is determined. And / or D2, the personalized needs include auditory personalized needs, the home appliances corresponding to the auditory needs include audio players, and the audio type of the audio player is determined; And / or D3, the personalized needs include olfactory personalized needs, the home appliances corresponding to the sense of smell include aroma diffusers, and the type of essential oil released by the aroma diffuser is determined; And / or D4, the personalized needs include tactile personalized needs, the home appliances corresponding to tactile sensation include smart mattresses, and the firmness of the smart mattress is determined.

5. The method for providing personalized home services based on user voice interaction as described in claim 4, characterized in that: The D4 also determines the vibration frequency and / or heating temperature of the smart mattress.

6. The method for providing personalized home services based on user voice interaction as described in claim 4, characterized in that: Specifically, D1 determines the output image of the projector of the smart mattress, D2 determines the type of audio played by the audio player of the smart mattress, and D3 determines the type of essential oil released by the aromatherapy diffuser of the smart mattress.

7. The method for providing personalized home services based on user voice interaction as described in claim 4, characterized in that, This includes step S6, which involves monitoring the user's sleep state and physiological indicators during sleep, adjusting the audio type played by the audio player according to the user's sleep state, and determining whether the user's physiological indicators are abnormal based on physiological indicator thresholds, and issuing an alarm if any abnormality is found.

8. A computer program product comprising a computer program, characterized in that, When the computer program is executed, it implements the method of providing personalized home services based on user voice interaction as described in any one of claims 1 to 6.

9. A smart home system that provides personalized home services based on user voice interaction, characterized by: The system includes a smart mattress and a smart doll that are interconnected. The smart mattress includes the mattress body and an audio player and aromatherapy diffuser located at the corners of the mattress body, as well as a projector located at the head of the mattress. The mattress body has built-in adjustable airbags, massage vibrators, and a heating module. The smart mattress has a controller that controls and connects the adjustable airbags, massage vibrators, heating module, audio player, aromatherapy diffuser, and projector. The smart doll is specifically connected to the controller of the smart mattress.

10. The smart home system as described in claim 9, which provides personalized home services based on user voice interaction, is characterized in that... The smart mattress includes a physiological monitoring sensor unit built into the mattress body and a pillow located at the head of the mattress body. The pillow has a built-in bone conduction unit that wirelessly connects to an audio player.

Images