Voice control method based on old-age care robot and old-age care robot thereof

By employing a human voice extraction model and multiple microphones, the elderly care robot effectively receives and responds to voice commands in noisy conditions, enhancing user interaction.

CN119068878AInactive Publication Date: 2024-12-03VIDEOSTRONG TECH CO LTD
View PDF 16 Cites 0 Cited by

Patent Information

Application Number
CN202411227659.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Elderly care robots generate significant noise during operation, interfering with their ability to receive voice commands effectively, leading to poor user interaction.

Method used

The elderly care robot is equipped with a microphone and a storage unit containing a human voice extraction model trained on annotated sound signals, utilizing multiple microphones to enhance voice signal extraction and recognition, enabling effective voice control even in noisy environments.

Benefits of technology

The solution allows users to interact with the elderly care robot through voice commands despite noise, improving user experience by ensuring accurate command reception and response.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068878A_ABST
    Figure CN119068878A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a voice control method based on an old-age care robot and the old-age care robot thereof, and the method comprises the steps: obtaining a first sound signal collected by a first microphone installed on the old-age care robot; a first human voice signal in the first sound signal is extracted based on human voice features stored in the storage unit, the human voice features are extracted based on a second sound signal collected by a second microphone, and the second microphone is installed on a base station of the old-age care robot or a terminal associated with the old-age care robot. The first human voice signal is obtained by extracting a first voice signal based on a human voice extraction model, and the human voice extraction model is obtained by training a sample voice signal marked with the human voice signal; and determining a voice control instruction based on the first human voice signal, and controlling the old-age care robot according to the voice control instruction. According to the invention, voice control based on the old-age care robot in a noise environment is realized, and the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a voice control method based on an elderly care robot and the elderly care robot. Background Art

[0002] With the rapid development of smart home technology, there are more and more automated and intelligent devices in the home environment. Among the many smart home devices, elderly care robots, such as sweeping robots, are widely used due to their automatic cleaning functions. At present, elderly care robots have also added voice control functions. Microphones and voice modules are installed on the body of the elderly care robot to receive user voice control commands and convert them into corresponding machine commands, and make corresponding actions according to voice commands, which provides a lot of convenience for users. However, since the various motors on the body of the elderly care robot will generate a lot of noise when working, users are easily disturbed by these noises when issuing voice commands to the elderly care robot, resulting in the elderly care robot not being able to receive commands well and respond to the commands accordingly. Summary of the invention

[0003] The present invention provides a voice control method based on an elderly care robot and the elderly care robot thereof, so as to solve the problem in the prior art that the motor on the elderly care robot generates relatively large noise when working, thus affecting the elderly care robot's ability to receive voice commands. Even in a noisy environment generated when the device is running, the user can effectively interact with the elderly care robot through voice commands, thereby enhancing the user experience.

[0004] In a first aspect, the present invention provides a voice control method based on an elderly care robot, which is applied to the elderly care robot. The elderly care robot is equipped with a microphone and a storage unit. The storage unit is embedded with a human voice extraction model. The human voice extraction model is trained by a sample sound signal of annotated human voice signals. The voice control method based on the elderly care robot includes:

[0005] Acquire a first sound signal collected by a first microphone installed on the elderly care robot;

[0006] Extracting a first human voice signal from the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model;

[0007] A voice control instruction is determined based on the first human voice signal, and the elderly care robot is controlled according to the voice control instruction.

[0008] According to the voice control method based on the elderly care robot provided by the present invention, the determining of the voice control instruction based on the first human voice signal includes:

[0009] Acquire a third sound signal collected simultaneously with the first sound signal, wherein the third sound signal is collected by the second microphone;

[0010] Extracting a third human voice signal from the third sound signal, and determining a human voice signal to be identified based on signal strengths of the first human voice signal and the third human voice signal;

[0011] Performing speech recognition on the human voice signal to be recognized to obtain a speech recognition result;

[0012] A voice control instruction is determined according to the voice recognition result.

[0013] According to the voice control method based on the elderly care robot provided by the present invention, the voice control method based on the elderly care robot further includes:

[0014] Performing sound source localization according to the first sound signal and the third sound signal to obtain a sound source position;

[0015] Control the elderly care robot to go to the location of the sound source.

[0016] According to the voice control method based on the elderly care robot provided by the present invention, the step of storing the human voice features in the storage unit includes:

[0017] Acquire a second sound signal collected by the second microphone;

[0018] Inputting the second sound signal into a pre-trained vocal feature extraction model to obtain vocal features in the second sound signal, wherein the vocal feature extraction model is trained based on an unsupervised learning algorithm;

[0019] Determining whether the human voice feature has been stored in the storage unit;

[0020] If not, the vocal feature is stored in the storage unit.

[0021] According to the voice control method based on the elderly care robot provided by the present invention, the voice control method based on the elderly care robot further includes:

[0022] In response to receiving a call request through the communication module built into the elderly care robot, a voice prompt is issued, wherein the voice prompt is used to remind the user of the call to be answered;

[0023] Obtaining a voice response from the user to the voice prompt;

[0024] determining a user location based on the voice response;

[0025] The elderly care robot is controlled to move to the user's position and wait for the user to answer the call.

[0026] According to the voice control method based on the elderly care robot provided by the present invention, the voice control method based on the elderly care robot further includes:

[0027] If no voice response is obtained from the user to the voice prompt within a preset time, the user's location is determined according to the indoor map stored in the elderly care robot and the user habit positioning model pre-built in the cloud, and the user habit positioning model is used to characterize the corresponding relationship between time and user location;

[0028] If no voice response from the user to the voice prompt is obtained within a predetermined time, a notification message is sent through the smart device associated with the elderly care robot, and the notification message includes unresponded call information.

[0029] According to the voice control method based on the elderly care robot provided by the present invention, the voice control method based on the elderly care robot further includes:

[0030] In response to receiving a voice instruction from a user indicating that a call needs to be made, determining a location of the user who needs to make a call based on the voice instruction;

[0031] Control the elderly care robot to move to the location of the user who needs to make a call;

[0032] Get a voice command containing information about a call to be made;

[0033] The communication module is controlled to make a call according to the voice command.

[0034] In a second aspect, the present invention provides a robot for the elderly, wherein a microphone and a storage unit are installed in the robot for the elderly, a human voice extraction model is embedded in the storage unit, and the human voice extraction model is obtained by training a sample sound signal of annotated human voice signals, and the robot for the elderly includes an acquisition unit, an extraction unit and a control unit;

[0035] The acquisition unit is used to acquire a first sound signal collected by a first microphone installed on the elderly care robot;

[0036] The extraction unit is used to extract a first human voice signal from the first sound signal based on the human voice feature stored in the storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model;

[0037] The control unit is used to determine a voice control instruction based on the first human voice signal, and control the elderly care robot according to the voice control instruction.

[0038] In a third aspect, the present invention further provides an elderly care robot, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the voice control method based on the elderly care robot as described in the first aspect is implemented.

[0039] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voice control method based on the elderly care robot as described in the first aspect.

[0040] In a fifth aspect, the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the voice control method based on the elderly care robot as described in the first aspect.

[0041] The voice control method based on the elderly care robot provided by the present invention obtains a first sound signal collected by a first microphone installed on the elderly care robot; extracts a first human voice signal in the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot. The first human voice signal is extracted from the first sound signal based on a human voice extraction model, and the human voice extraction model is trained by labeling sample sound signals of human voice signals; determines a voice control instruction based on the first human voice signal, and controls the elderly care robot according to the voice control instruction. Therefore, voice control based on the elderly care robot in a noisy environment is realized, and user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of the voice control method based on the elderly care robot provided by the present invention;

[0044] Figure 2 It is a structural schematic diagram of the elderly care robot provided by the present invention;

[0045] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0047] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0048] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0049] Figure 1 It is a flow chart of the voice control method based on the elderly robot provided by the present invention. It should be noted that the voice control method based on the elderly robot provided by the embodiment of the present invention is applicable to the voice control of the elderly and the elderly robot, such as a sweeping robot, a smart home control robot, an indoor cleaning robot, a life robot, etc. The elderly robot is equipped with a microphone and a storage unit, and a human voice extraction model is embedded in the storage unit. The human voice extraction model is obtained by training the sample sound signal of the labeled human voice signal. Therefore, the voice control method based on the elderly robot includes:

[0050] Step 10, obtaining a first sound signal collected by a first microphone installed on the elderly care robot;

[0051] In this embodiment, when someone makes a sound in the room, the first microphone starts to collect sound signals, which may include human voices, noises generated by the elderly care robot, and sounds of other objects.

[0052] Step 20, extracting the first human voice signal from the first sound signal based on the human voice features stored in the storage unit, wherein the human voice features are extracted based on the second sound signal collected by the second microphone, and the second microphone is installed in the base station of the elderly care robot or the terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on the human voice extraction model.

[0053] In this embodiment, the human voice feature is extracted based on the second sound signal collected by the second microphone, and the second microphone is installed in the base station of the elderly care robot or a terminal associated with the elderly care robot. The first microphone and the second microphone can be separate microphones or a microphone array.

[0054] Human voice features can include spectral features (such as Mel frequency cepstral coefficients), time domain features (such as short-time energy, short-time zero-crossing rate) and features extracted by deep learning models, which can reflect the uniqueness of human voice and its distinction from other sounds. Since the elderly care robot may have multiple users and the voice features of the same person may change at any time, a storage module is needed to store human voice features to facilitate the extraction and enhancement of human voice in noise.

[0055] As an example, when a user uses voice to interact with the elderly care robot for the first time, the base station will receive pure human voice input, analyze the sound features through the model, and store the sound features in the specified memory after obtaining them. After the human voice features are stored, when the user uses voice to interact with the elderly care robot again, the elderly care robot will match the human voice features stored by the base station, extract and strengthen the corresponding human voice signals, so that the elderly care robot can obtain clearer human voice control commands.

[0056] Specifically, the first sound signal can be matched with the stored vocal features. As an example, the first sound signal can be converted into the same representation as the vocal features (such as a spectrogram or a feature vector), and a similarity metric (such as Euclidean distance, cosine similarity, etc.) can be used to evaluate the similarity between them. During the matching process, some preprocessing steps such as filtering and denoising can be applied to improve the matching effect. In addition, the first vocal signal in the first sound signal can also be extracted based on the vocal features stored in the storage unit by constructing a model.

[0057] In one embodiment, the step of extracting the human voice feature includes: inputting the first sound signal into a pre-trained human voice extraction model to obtain the first human voice signal, wherein the human voice extraction model is trained by a sample sound signal with annotated human voice signals. The sample sound signal can be a human voice signal of the elderly care robot's own noise, and the annotated human voice signal is used as a data set for model training for training, so that the model can achieve the effect of extracting and enhancing the human voice signal in the noise according to the human voice signal characteristics. The construction of the human voice extraction model can be based on deep learning technology, such as convolutional neural network (CNN) or recurrent neural network (RNN), and supervised learning of human voice extraction can be performed using machine learning frameworks such as pytorch. PyTorch is an open source machine learning library that is widely used in applications such as computer vision and natural language processing. It provides powerful GPU-accelerated tensor computing capabilities and dynamic computational graphs for building deep learning models.

[0058] Step 30, determining a voice control instruction based on the first human voice signal, and controlling the elderly care robot according to the voice control instruction.

[0059] In this embodiment, voice recognition can be performed directly on the first human voice signal to determine the voice control instruction, or voice recognition can be performed in combination with the human voice signal collected by the second microphone to determine the voice control instruction. Specifically, it can be performed through various natural language processing algorithms.

[0060] Furthermore, the elderly care robot can perform corresponding actions according to the parsed and mapped commands, and then return to its cleaning task or wait for the next user instruction.

[0061] The embodiment of the present invention obtains a first sound signal collected by a first microphone installed on the elderly care robot; extracts a first human voice signal from the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; determines a voice control instruction based on the first human voice signal; controls the elderly care robot according to the voice control instruction, thereby realizing voice control based on the elderly care robot in a noisy environment and improving user experience.

[0062] In one embodiment, determining a voice control instruction based on a first human voice signal includes: obtaining a third sound signal collected simultaneously with the first sound signal, the third sound signal being collected by a second microphone; extracting a third human voice signal from the third sound signal; determining a human voice signal to be identified based on the signal strengths of the first human voice signal and the third human voice signal; performing voice recognition on the human voice signal to be identified to obtain a voice recognition result; and determining a voice control instruction based on the voice recognition result. The base station equipped with a microphone and a voice module is far away from the robot body, so the noise generated by the robot effectively avoids the interference of the robot's own noise on the user's voice command by the base station.

[0063] In one embodiment, based on the signal strengths of the first human voice signal and the third human voice signal, determining the human voice signal to be identified may be by comparing the signal strengths of the first human voice signal and the third human voice signal with a preset strength threshold, and determining the human voice signal greater than the strength threshold as the human voice signal to be identified.

[0064] If both the first human voice signal and the third human voice signal are less than the intensity threshold, the sound source can be located and the elderly care robot can be controlled to move to the sound source position to receive a human voice signal with a stronger signal strength. In addition, the signal strengths of the first human voice signal and the third human voice signal can be compared, and the human voice signal with a stronger signal strength can be determined as the human voice signal to be identified.

[0065] In one embodiment, the method further includes: locating the sound source according to the first sound signal and the third sound signal to obtain the sound source position; and controlling the elderly care robot to go to the sound source position. In addition, multi-dimensional positioning can be performed in combination with indoor maps, user habits and other data. Specifically, the sound signals obtained by the elderly care robot and the base station can be processed separately, and the processing process can be performed in the processor of the base station. When only a single sound signal is transmitted, the autocorrelation function is used to locate the sound source; when the elderly care robot and the base station both receive voice commands, a coordinate system is established with the elderly care robot and the base station as the x-axis. After the sound source is sent to both, the two different sound sources are processed, and the maximum controllable response power beamforming method is used to calculate which direction the sound source power is the largest, thereby determining the direction of the sound source. For the positioning of the sound source distance, it can be determined according to the time of sound propagation. The sound source is located together with the information sent back by the microphones located on the base station and the elderly care robot, and the direction of the sound source positioning will be more accurate.

[0066] In one embodiment, the step of storing the human voice feature in the storage unit includes: obtaining a second sound signal collected by a second microphone; inputting the second sound signal into a pre-trained human voice feature extraction model to obtain the human voice feature in the second sound signal, wherein the human voice feature extraction model is trained based on an unsupervised learning algorithm; determining whether the human voice feature has been stored in the storage unit;

[0067] If not, the human voice features are stored in the storage unit. The second microphone installed on the base station of the elderly care robot is used to collect the sound signal, which is relatively pure because the base station is far away from the noise source when the device is running. Using unsupervised learning algorithms for training can reduce the workload of labeling.

[0068] In one embodiment, unsupervised learning algorithms such as generative adversarial networks (GANs) and autoencoders can be used. Taking generative adversarial networks as an example, generative adversarial networks include generators and discriminators, where the goal of the generator is to generate features close to real human voices, and the goal of the discriminator is to distinguish the features generated by the generator from real human voice features. Through adversarial training, the generator and the discriminator compete with each other, the generator is constantly improved to generate more realistic human voice features, and the discriminator continuously improves its recognition ability.

[0069] In one embodiment, the voice control method based on the elderly care robot also includes: in response to receiving a call request through the built-in communication module of the elderly care robot, issuing a voice prompt, the voice prompt is used to remind the user of the pending call; obtaining the user's voice response to the voice prompt; determining the user's location based on the voice response; and controlling the elderly care robot to move to the user's location and wait for the user to answer the call.

[0070] In this embodiment, the communication module can be a module such as a 4G module and a SIM card that can be used for calls. In addition, taking the 4G module as an example, the 4G module has functions such as Beidou positioning. Using the positioning function, the local weather and climate information can be obtained according to the weather API. Based on this information, the elderly care robot can automatically customize the cleaning strategy of the day and interact with the user. Voice prompts are used to remind users of pending calls, and can also guide users to say their location. It can be a ringtone or a preset voice prompt, for example: "You have a new call request, please tell me your location."

[0071] In this embodiment, after the user responds to the voice prompt, the microphone or microphone array on the device can capture the user's voice signal. In addition, the captured voice signal can also be pre-processed by noise reduction, gain control, etc. to improve signal quality.

[0072] In this embodiment, voice recognition can be performed on the voice response to obtain the user's location; sound source positioning can also be performed based on the voice response to obtain the user's location; multi-dimensional positioning can also be performed in combination with indoor maps, user habits and other data.

[0073] In this embodiment, the elderly care robot has built or stored a home map inside, including information such as room layout and obstacle locations. The path planning algorithm can be used to calculate the best path to the user's location based on the user's location and the home map. After the elderly care robot moves to the user's location, it can also confirm that it has arrived at the user's location through sound or visual signals (such as flashing indicator lights), and wait at the user's location for the user to answer the call. After the call is over, the user can choose to hang up the call, and the device then returns to its cleaning task or waits for the next user instruction.

[0074] The embodiments of the present invention implement voice-controlled answering based on the elderly care robot, thereby improving the efficiency of answering calls and the user experience.

[0075] In one embodiment, determining the user location according to the voice response includes: performing voice recognition on the voice response to obtain a voice recognition result; determining whether the voice recognition result includes location information; and if so, determining the user location according to the location information. As an example, when a user hears a device announcement, he or she may respond, and the response may include the user location information, such as, I am in the kitchen, I am in the bedroom.

[0076] In one embodiment, the elderly care robot is provided with a first microphone, and the base station of the elderly care robot is provided with a second microphone. The user position is determined according to the voice response, including: locating the sound source according to the voice response collected by the first microphone and the second microphone to obtain the user position.

[0077] In one embodiment, a base station with a network interface and a microphone module can be used to locate the sound source together with the elderly robot. The microphone module on the base station and the microphone module on the elderly robot will respectively obtain a sound information. The elderly robot will send this sound information to the base station through the network, and the base station will uniformly package and process it. According to the algorithm, the location information of the sound source is obtained, and finally the location information is fed back to the elderly robot. The elderly robot locates according to the location information and moves to the location. The network interface of the base station is mainly used for information transmission between the elderly robot. Two microphones with different positions are used, which are located at the elderly robot body and the base station respectively. The specific location information of the user is finally determined by algorithm calculation, so as to reduce the cost of using the camera module for user positioning; at the same time, the microphone set on the base station also reduces the interference caused by the noise of the elderly robot itself. In addition, the network connection and sound source positioning function of the sweeping machine base station can be replaced by the terminal device, that is, the terminal device and the elderly robot are used to locate the sound source, or the terminal device itself is used to simulate multiple microphones to locate the sound source.

[0078] In one embodiment, the method also includes: if no voice response from the user to the voice prompt is obtained within a preset time, the user location is determined based on the indoor map stored in the elderly care robot and the user habit positioning model pre-built in the cloud, and the user habit positioning model is used to characterize the correspondence between time and user location.

[0079] In one embodiment, when the elderly care robot does not receive a voice response from the user within a certain period of time, the elderly care robot can determine the user's current location based on the user's habits, and move to the vicinity of the user to facilitate the user to hear the elderly care robot's voice broadcast; when the user hears the elderly care robot's broadcast and responds, the response may include the user's location information (for example: I'm in the kitchen, I'm in the bedroom), and the elderly care robot will move according to such instructions and map information and move to the room described by the user; when the user is in a large room, especially in the living room, the elderly care robot may not be within the effective range of the calling function based on the room positioning, so it is also necessary to accurately locate the user based on the sound, and move to the user's side based on the location information to perform the next step.

[0080] In one embodiment, it further includes: if the user's voice response to the voice prompt is not obtained within a predetermined time, a notification message is sent through the smart device associated with the elderly care robot, and the notification message includes unanswered call information. The elderly care robot can actively send a message to the user terminal APP to notify the user that there is a missed call that needs to be handled, and the user can handle the missed call according to his or her own situation.

[0081] In one embodiment, the method further includes a process of making a call, specifically: in response to receiving a voice instruction from a user indicating that a call needs to be made, determining the location of the user who needs to make a call according to the voice instruction; controlling the elderly care robot to move to the location of the user who needs to make a call; obtaining a voice command containing information about the call to be made; and controlling the communication module to make a call according to the voice command. Specifically, the method of determining the location of the user who needs to make a call according to the voice instruction can refer to the method of determining the user location according to the voice response in the above embodiment.

[0082] In one embodiment, the elderly care robot stores address book information, which is edited and saved by the user through the smart device associated with the elderly care robot. For the voice call control function, the elderly care robot can be equipped with a storage module for storing the address book required by the user, so that the user can use instructions to control the call to the person in need, for example, the elderly care robot, help me call someone. In addition, the following functions can be implemented in the application of the smart device: the user edits and saves the address book on the elderly care robot; the user manually edits the do not disturb mode of the elderly care robot's call answering function; the elderly care robot sends a missed call message to the user terminal when the call is not answered.

[0083] In one embodiment, the entire system includes an elderly care robot 201, a base station 203, and a cloud server 205. The elderly care robot 201 is equipped with a microphone 202, and the base station 203 is equipped with a microphone 204. The base station 203 and the elderly care robot 201 can use wired or wireless communication. For example, a network cable interface can be added to the base station 203, and the network cable can be used to connect to the home router to ensure that the base station 203 obtains the IP address of the same network segment as the elderly care robot 201. When the base station 203 and the elderly care robot 201 are in the same network, the two parties can realize the communication function; after the elderly care robot 201 obtains the sound signal, it can forward the sound signal to the base station 203, and the base station 203 sends these signals to the cloud server 205 for processing and receiving the results. After that, the base station 203 can forward the message with location information and voice instructions to the elderly care robot 201, and the elderly care robot 201 responds accordingly according to the received message. The base station 203 communicates with the elderly care robot 201 through the network, and also facilitates the base station's own information (water tank full, sewage tank full) to be sent to the elderly care robot 201, and the elderly care robot 201 can be used to exchange base station information with the user.

[0084] The elderly care robot provided by the present invention is described below. The elderly care robot described below and the voice control method based on the elderly care robot described above can be referred to each other. Figure 2 , Figure 2 This is a schematic diagram of an embodiment of the elderly care robot provided by an embodiment of the present invention. Figure 2 As shown, the elderly care robot provided by an embodiment of the present invention is equipped with a microphone and a storage unit, and a human voice extraction model is embedded in the storage unit. The human voice extraction model is obtained by training sample sound signals with labeled human voice signals. The elderly care robot includes: an acquisition unit 210, an extraction unit 220 and a control unit 230.

[0085] The acquisition unit 210 is used to acquire a first sound signal collected by a first microphone installed on the elderly care robot;

[0086] The extraction unit 220 is used to extract a first human voice signal from the first sound signal based on the human voice feature stored in the storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model;

[0087] The control unit 230 is used to determine a voice control instruction based on the first human voice signal, and control the elderly care robot according to the voice control instruction.

[0088] The elderly care robot provided by the present invention obtains a first sound signal collected by a first microphone installed on the elderly care robot; extracts a first human voice signal from the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model, and the human voice extraction model is trained by labeling sample sound signals of human voice signals; determines a voice control instruction based on the first human voice signal, and controls the elderly care robot according to the voice control instruction, thereby realizing voice control based on the elderly care robot in a noisy environment and improving user experience.

[0089] In one embodiment, the control unit 230 is further configured to:

[0090] Acquire a third sound signal collected simultaneously with the first sound signal, wherein the third sound signal is collected by the second microphone;

[0091] Extracting a third human voice signal from the third sound signal, and determining a human voice signal to be identified based on signal strengths of the first human voice signal and the third human voice signal;

[0092] Performing speech recognition on the human voice signal to be recognized to obtain a speech recognition result;

[0093] A voice control instruction is determined according to the voice recognition result.

[0094] In one embodiment, the elderly care robot is also used for:

[0095] Performing sound source localization according to the first sound signal and the third sound signal to obtain a sound source position;

[0096] Control the elderly care robot to go to the location of the sound source.

[0097] In one embodiment, the elderly care robot is also used for:

[0098] Acquire a second sound signal collected by the second microphone;

[0099] Inputting the second sound signal into a pre-trained vocal feature extraction model to obtain vocal features in the second sound signal, wherein the vocal feature extraction model is trained based on an unsupervised learning algorithm;

[0100] Determining whether the human voice feature has been stored in the storage unit;

[0101] If not, the vocal feature is stored in the storage unit.

[0102] In one embodiment, the elderly care robot is also used for:

[0103] In response to receiving a call request through the communication module built into the elderly care robot, a voice prompt is issued, wherein the voice prompt is used to remind the user of the call to be answered;

[0104] Obtaining a voice response from the user to the voice prompt;

[0105] determining a user location based on the voice response;

[0106] The elderly care robot is controlled to move to the user's position and wait for the user to answer the call.

[0107] In one embodiment, the elderly care robot is also used for:

[0108] If no voice response is obtained from the user to the voice prompt within a preset time, the user's location is determined according to the indoor map stored in the elderly care robot and the user habit positioning model pre-built in the cloud, and the user habit positioning model is used to characterize the corresponding relationship between time and user location;

[0109] If no voice response from the user to the voice prompt is obtained within a predetermined time, a notification message is sent through the smart device associated with the elderly care robot, and the notification message includes unresponded call information.

[0110] In one embodiment, the elderly care robot is also used for:

[0111] In response to receiving a voice instruction from a user indicating that a call needs to be made, determining a location of the user who needs to make a call based on the voice instruction;

[0112] Control the elderly care robot to move to the location of the user who needs to make a call;

[0113] Get a voice command containing information about a call to be made;

[0114] The communication module is controlled to make a call according to the voice command.

[0115] See also Figure 3 , Figure 3 is a schematic diagram of the structure of an electronic device provided by the present invention. An embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0116] Acquire a first sound signal collected by a first microphone installed on the elderly care robot;

[0117] Extracting a first human voice signal from the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model;

[0118] A voice control instruction is determined based on the first human voice signal, and the elderly care robot is controlled according to the voice control instruction.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A voice control method based on an elderly care robot, characterized in that: Applied to a robot for the elderly, the robot is equipped with a microphone and a storage unit, the storage unit is embedded with a voice extraction model, the voice extraction model is trained by a sample sound signal with annotated voice signals, and the voice control method based on the robot for the elderly includes: Acquire a first sound signal collected by a first microphone installed on the elderly care robot; Extracting a first human voice signal from the first sound signal based on a human voice feature stored in a storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model; A voice control instruction is determined based on the first human voice signal, and the elderly care robot is controlled according to the voice control instruction.

2. The voice control method based on the elderly care robot according to claim 1 is characterized in that: The determining the voice control instruction based on the first human voice signal includes: Acquire a third sound signal collected simultaneously with the first sound signal, wherein the third sound signal is collected by the second microphone; Extracting a third human voice signal from the third sound signal, and determining a human voice signal to be identified based on signal strengths of the first human voice signal and the third human voice signal; Performing speech recognition on the human voice signal to be recognized to obtain a speech recognition result; A voice control instruction is determined according to the voice recognition result.

3. The voice control method based on the elderly care robot according to claim 2 is characterized in that: The voice control method based on the elderly care robot also includes: Performing sound source localization according to the first sound signal and the third sound signal to obtain a sound source position; Control the elderly care robot to go to the location of the sound source.

4. The voice control method based on the elderly care robot according to claim 1 is characterized in that: The step of storing the vocal features in the storage unit comprises: Acquire a second sound signal collected by the second microphone; Inputting the second sound signal into a pre-trained vocal feature extraction model to obtain vocal features in the second sound signal, wherein the vocal feature extraction model is trained based on an unsupervised learning algorithm; Determining whether the human voice feature has been stored in the storage unit; If not, the vocal feature is stored in the storage unit.

5. The voice control method based on the elderly care robot according to claim 1 is characterized in that: The voice control method based on the elderly care robot also includes: In response to receiving a call request through the communication module built into the elderly care robot, a voice prompt is issued, wherein the voice prompt is used to remind the user of the call to be answered; Obtaining a voice response from the user to the voice prompt; determining a user location based on the voice response; The elderly care robot is controlled to move to the user's position and wait for the user to answer the call.

6. The voice control method based on the elderly care robot according to claim 1 is characterized in that: The voice control method based on the elderly care robot also includes: If no voice response is obtained from the user to the voice prompt within a preset time, the user's location is determined according to the indoor map stored in the elderly care robot and the user habit positioning model pre-built in the cloud, and the user habit positioning model is used to characterize the corresponding relationship between time and user location; If no voice response from the user to the voice prompt is obtained within a predetermined time, a notification message is sent through the smart device associated with the elderly care robot, and the notification message includes unresponded call information.

7. The voice control method based on the elderly care robot according to claim 1 is characterized in that: The voice control method based on the elderly care robot also includes: In response to receiving a voice instruction from a user indicating that a call needs to be made, determining a location of the user who needs to make a call based on the voice instruction; Control the elderly care robot to move to the location of the user who needs to make a call; Get a voice command containing information about a call to be made; The communication module is controlled to make a call according to the voice command.

8. A robot for elderly care, characterized in that: The elderly care robot is equipped with a microphone and a storage unit, the storage unit is embedded with a human voice extraction model, the human voice extraction model is obtained by training sample sound signals with labeled human voice signals, and the elderly care robot includes an acquisition unit, an extraction unit and a control unit; The acquisition unit is used to acquire a first sound signal collected by a first microphone installed on the elderly care robot; The extraction unit is used to extract a first human voice signal from the first sound signal based on the human voice feature stored in the storage unit, wherein the human voice feature is extracted based on a second sound signal collected by a second microphone, and the second microphone is installed in a base station of the elderly care robot or a terminal associated with the elderly care robot; the first human voice signal is extracted from the first sound signal based on a human voice extraction model; The control unit is used to determine a voice control instruction based on the first human voice signal, and control the elderly care robot according to the voice control instruction.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the voice control method based on the elderly care robot as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the voice control method based on the elderly care robot as described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Man-machine interaction method based on ROS of robot operation system

    CN106847279A

  • Improved sound source localization method based on progressive serial orthogonalization blind source separation algorithm, and implementation system for same

    CN107644650A

  • Air conditioner, control method and device therefor, and computer readable storage medium

    CN110578999A

  • Voice signal segmentation model training method, device and computer equipment

    CN111243619A

  • Method and system for entering dialects and making calls by robot for old people

    CN111899736A