Autonomous mobile device voice control method, apparatus, device, and readable storage medium

Abstract

The application discloses a voice control method, device and equipment of an autonomous mobile device and a readable storage medium. After the autonomous mobile device collects a second voice signal, at least two working areas are determined according to the second voice signal, and tasks indicated by the second voice signal are executed on the working areas in sequence. By using the scheme, a plurality of working areas can be indicated to the autonomous mobile device through one voice signal, and a user interacts with the autonomous mobile device through natural language to realize simple control on the autonomous mobile device, so that the accuracy is high and the process is simple.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a voice control method, apparatus, device, and readable storage medium for autonomous mobile devices. Background Technology

[0002] With the development of artificial intelligence technology, various autonomous mobile devices are increasingly entering people's lives, such as logistics robots, sweeping robots, lawnmower robots, and welcoming robots.

[0003] Traditional autonomous mobile devices are controlled via mobile applications (APPs), physical buttons on the device, or remote controls. Using an APP requires opening the app, a cumbersome process that can be difficult, especially for the elderly. Physical buttons only allow for simple controls such as starting and stopping the device. Remote controls are easily lost, leaving users unable to control the device if they cannot find them.

[0004] Therefore, how to conveniently and quickly control autonomous mobile devices is considered an urgent problem to be solved. Summary of the Invention

[0005] This application provides a voice control method, apparatus, device, and readable storage medium for autonomous mobile devices. It instructs multiple working areas to an autonomous mobile device through a single voice signal, enabling the autonomous mobile device to perform tasks on multiple working areas sequentially with high accuracy and a simple process.

[0006] In a first aspect, embodiments of this application provide a voice control method for an autonomous mobile device, applied to an autonomous mobile device, the method comprising:

[0007] Acquire the first speech signal;

[0008] When the first voice signal matches the wake-up command of the autonomous mobile device, the voice control function of the autonomous mobile device is activated.

[0009] The second voice signal is acquired when the voice control function is awake.

[0010] At least two working areas are determined based on the second voice signal;

[0011] The task indicated by the second voice signal is executed sequentially in the at least two working areas.

[0012] Secondly, embodiments of this application provide a voice control method for an autonomous mobile device, applied to an autonomous mobile device, the method comprising:

[0013] Acquire the second speech signal;

[0014] The working area is determined based on the second voice signal;

[0015] Determine the driving route based on the current location and the work area;

[0016] The working module is shut down and the vehicle proceeds to the working area according to the driving path.

[0017] If the user moves to the work area, the work module is activated to perform the task indicated by the second voice signal.

[0018] Thirdly, embodiments of this application provide an autonomous mobile device control device, comprising:

[0019] The acquisition module is used to acquire the first voice signal;

[0020] The processing module is used to wake up the voice control function of the autonomous mobile device when the first voice signal matches the wake-up command of the autonomous mobile device.

[0021] The acquisition module is also used to acquire a second voice signal when the voice control function is awake.

[0022] The processing module is further configured to determine at least two working areas based on the second voice signal;

[0023] An execution module is used to sequentially execute the tasks indicated by the second voice signal in the at least two working areas.

[0024] Fourthly, embodiments of this application provide an autonomous mobile device control device, comprising:

[0025] The acquisition module is used to acquire the second voice signal;

[0026] The processing module is used to determine the working area based on the second voice signal; and to determine the driving path based on the current location and the working area.

[0027] A driving module is used to shut down the working module and travel to the working area according to the driving path;

[0028] An execution module is configured to activate the working module to execute the task indicated by the second voice signal if the user travels to the working area.

[0029] Fifthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to implement the method described in the first aspect or various possible implementations of the first aspect.

[0030] In a sixth aspect, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the electronic device implements the method described in the second aspect above or various possible implementations of the second aspect.

[0031] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, are used to implement the method described in the first aspect or various possible implementations of the first aspect.

[0032] Eighthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed by a processor, are used to implement the method described in the second aspect above or various possible implementations of the second aspect.

[0033] Ninthly, embodiments of this application provide a computer program product comprising a computing program, wherein when the computer program is executed by a processor, it implements the method described in the first aspect or various possible implementations of the first aspect.

[0034] In a tenth aspect, embodiments of this application provide a computer program product comprising a computing program, wherein when the computer program is executed by a processor, it implements the method described in the second aspect above or various possible implementations of the second aspect.

[0035] The autonomous mobile device voice control method, apparatus, device, and readable storage medium provided in this application embodiment allow the autonomous mobile device to acquire a second voice signal, determine at least two working areas based on the second voice signal, and sequentially execute the tasks indicated by the second voice signal in each working area. Using this approach, multiple working areas can be indicated to the autonomous mobile device with a single voice signal, enabling the autonomous mobile device to sequentially execute tasks in multiple working areas. Users can interact with the autonomous mobile device through natural language, resulting in extremely simple control of the autonomous mobile device with high accuracy and a simple process. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1A This is a schematic diagram of the implementation environment of the autonomous mobile device voice control method provided in the embodiments of this application;

[0038] Figure 1B This is a schematic diagram of the structure of the sweeping robot provided in the embodiments of this application;

[0039] Figure 1C This is a schematic diagram of the sound signal acquisition device for autonomous mobile devices;

[0040] Figure 1D This is a schematic diagram of another embodiment of the autonomous mobile device provided in this application;

[0041] Figure 1E This is a voice control flowchart of an autonomous mobile device provided in an embodiment of this application;

[0042] Figure 1F This is another voice control flowchart of the autonomous mobile device provided in the embodiments of this application;

[0043] Figure 1G This is another voice control flowchart of the autonomous mobile device provided in the embodiments of this application;

[0044] Figure 2 This is a flowchart of the autonomous mobile device voice control method provided in the embodiments of this application;

[0045] Figure 3 This is a schematic diagram of the speech recognition process in an embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the carpeted area;

[0047] Figure 5 This is a diagram illustrating the furniture identification process;

[0048] Figure 6 This is a schematic diagram of the door recognition process;

[0049] Figure 7 This is a schematic diagram illustrating the synchronization process between the autonomous mobile device and the voice recognition server;

[0050] Figure 8A This is a schematic diagram for determining the position of the sound source relative to the center of the microphone array;

[0051] Figure 8B This is a schematic diagram of the microphone array and the body of the autonomous mobile device;

[0052] Figure 8C This is a schematic diagram illustrating the process of training a speech recognition model and recognizing speech.

[0053] Figure 9 This is a flowchart of the voice control logic for an autonomous mobile device provided in an embodiment of this application;

[0054] Figure 10 This is another flowchart of the autonomous mobile device voice control method provided in the embodiments of this application;

[0055] Figure 11 This is another flowchart of the autonomous mobile device voice control method provided in the embodiments of this application;

[0056] Figure 12 This is another flowchart of the autonomous mobile device voice control method provided in the embodiments of this application;

[0057] Figure 13 A schematic diagram of a voice control device for an autonomous mobile device provided in an embodiment of this application;

[0058] Figure 14 A schematic diagram of another autonomous mobile device voice control device provided in the embodiments of this application;

[0059] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0061] With the advancement of science and technology, robots have entered more and more people's lives, playing an important role. Currently, robots can be operated via physical buttons on the main unit, mobile applications (APPs), and remote controls. However, these methods all have drawbacks. Therefore, voice-controlled robots, due to their greater intelligence, are widely welcomed by users. Robots with voice recognition capabilities collect voice signals, recognize them, and execute related tasks.

[0062] However, current voice commands can only control the robot to turn on, perform tasks, stop, and charge. They cannot provide precise control. For example, if a user wants the robot to perform a task in a specific area, they need to move the robot to that area and then control it via voice. Similarly, if certain areas are prohibited, such as restrooms, the user must close the restroom door to prevent the robot from entering prohibited areas while it is working.

[0063] For example, sometimes users want a robot to perform tasks in multiple areas. This requires moving the robot to one area, completing the task there, and then moving it to another. While some robots can recognize the user's indicated work area, this only works if the user's voice command can only specify one area. If the user wants the robot to perform tasks in multiple areas, the user must manually indicate the next work area after each task, which is cumbersome.

[0064] Based on this, embodiments of this application provide a voice control method, apparatus, device, and readable storage medium for autonomous mobile devices. By using a single voice signal to instruct multiple working areas to the autonomous mobile device, the autonomous mobile device can sequentially perform tasks on multiple working areas with high accuracy and a simple process.

[0065] Figure 1A This is a schematic diagram illustrating the implementation environment of the autonomous mobile device voice control method provided in this application embodiment. Please refer to... Figure 1A The implementation environment includes autonomous mobile devices, such as robotic vacuum cleaners, self-propelled air purifiers, automatic lawnmowers, window cleaning robots, solar panel cleaning robots, butler robots, unmanned aerial vehicles, automated guided vehicles (AGVs), security robots, welcoming robots, and care robots.

[0066] Autonomous mobile devices are equipped with microphones and other sound signal acquisition devices to collect voice signals emitted by users. After collecting the voice signals, the autonomous mobile device recognizes them to obtain voice commands and executes the tasks instructed by the voice commands. In practice, the autonomous mobile device can recognize voice signals on its own. Alternatively, the autonomous mobile device establishes a network connection with a voice recognition server (not shown in the diagram). After collecting the voice signals, the autonomous mobile device sends them to the voice recognition server, which then recognizes the voice signals and sends the recognized voice commands back to the autonomous mobile device.

[0067] Below, we will take a robotic vacuum cleaner as an example to explain the structure of an autonomous mobile device in detail.

[0068] Figure 1B This is a schematic diagram of the structure of the sweeping robot provided in the embodiments of this application. The sweeping robot will be referred to as a robot below. Please refer to... Figure 1BThe symbol “→” represents the direction of propagation of the voice signal. The robot includes a robot shell 1, a drive element, a protruding structure 2, and a voice signal acquisition device 3; wherein, the drive element is disposed inside the robot shell 1 and is used to drive the robot shell 1 to move; the protruding structure 2 is disposed on the upper surface 10 of the robot shell 1, and the voice signal acquisition device 3 is disposed on the protruding structure 2.

[0069] Please refer to the following: Figure 1B The robot's outer shell 1 includes a top plate, an annular side plate, and a bottom plate. These three plates are assembled to form a receiving chamber, which houses a control unit and driving components. Additionally, the robot includes functional components such as drive wheels 6, side brushes 7, roller brushes, or fans mounted on the outer shell 1. The drive wheels 6 propel the robot under the action of the driving components. The side brushes 7 and roller brushes clean the working surface upon receiving signals from the control unit. The fan creates a negative pressure chamber within the dustbin to suck dust and debris from the working surface into the dustbin for cleaning. It should be noted that the structure and working principle of these functional components are essentially the same as those of existing robotic vacuum cleaners, and can be fully implemented by those skilled in the art using existing technology; therefore, they will not be described in detail here.

[0070] The upper surface 10 of the top plate of the robot shell 1 is provided with a raised structure 2. In some embodiments, the raised structure 2 and the top plate are integrally formed. In other embodiments, the raised structure 2 and the top plate are separately formed, and then the raised structure 2 is fixedly connected to the upper surface 10 of the top plate by means of bonding, threaded connection, etc. A sound signal acquisition device 3 is provided on the raised structure 2.

[0071] Typically, robot noise is generated by the drive components, side brushes 7, roller brushes, and / or fans, and these components are located inside the housing cavity or at its bottom. In this invention, the sound signal acquisition device is mounted on a raised structure 2 on the upper surface 10 of the robot's outer shell 1, so that the sound signal acquisition device 3 is away from the robot's noise source, thereby reducing the interference of the robot's own noise on the sound signal acquisition device 3, so that the robot can more accurately acquire the user's voice control commands. These user voice control commands include starting sweeping, playing music, stopping sweeping, and going to charging, etc. Those skilled in the art can set corresponding functions according to the actual needs of the robot.

[0072] Figure 1C This is a schematic diagram of the sound signal acquisition device for an autonomous mobile device. Please refer to it. Figure 1CThe sound signal acquisition device 3 includes a microphone (MIC). Specifically, in some embodiments, the sound signal acquisition device 3 includes a PCB board 30 (printed circuit board), a shock-absorbing housing 31, and a microphone chip 32. The shock-absorbing housing 31 is disposed on the PCB board 30 and forms an external enclosure of the sound signal acquisition device 3 with a receiving cavity. The microphone chip 32 is disposed within this receiving cavity. A pickup hole 310 communicating with the outside and the receiving cavity is provided in the central area of ​​the top of the shock-absorbing housing 31. The PCB board 30 is communicatively connected to the microphone chip 32 and the robot's control unit. After the microphone chip 32 acquires external sound signals from the pickup hole 310, it transmits the signals to the control unit through the PCB board 30. The control unit then controls the robot to execute user voice control commands contained in the sound signals.

[0073] It should be noted that the shock-absorbing housing 31 of the sound signal acquisition device 3 reduces the impact of vibrations generated during robot operation on the sound signal acquisition device 3. Furthermore, the shock-absorbing housing 31 absorbs noise from the robot itself. The microphone 310 is located in the central area of ​​the top of the shock-absorbing housing 31, and it only collects sound signals from above (usually voice control commands issued by the user). Especially for robotic vacuum cleaners, which typically operate on the ground while the user issues voice control commands from above, the microphone 310, located in the central area of ​​the top of the shock-absorbing housing 31, can easily collect the user's voice control signals. The noise emitted by the robot itself is blocked by the shock-absorbing housing 31 surrounding the microphone 310, reducing its interference with the sound signal acquisition signal collected by the sound signal acquisition device 3. In other embodiments, the shock-absorbing housing 31 includes shock-absorbing foam. This foam not only blocks noise from the robot itself from entering the microphone 310 but also absorbs some of the noise.

[0074] See also Figure 1C The sound signal acquisition device 3 also includes a waterproof and dustproof membrane 33, which is installed on the shock-absorbing housing 31 and covers the microphone hole 310 to prevent water or dust from falling onto the microphone chip 32 through the microphone hole 310 and affecting the sound signal acquisition effect of the microphone chip 32.

[0075] See also Figure 1C In this embodiment, the sound signal acquisition device 3 also includes an upper cover 34. The upper cover presses the shockproof cover 31 onto the PCB board and is fixedly connected to the protruding structure 2 or the distance sensor 3 by screws (not shown in the figure) or other connectors, thereby realizing a fixed connection between the sound signal acquisition device 3 and the robot shell 1 to prevent the sound signal acquisition device 3 from falling off the robot shell 1 during robot movement. In addition, a pickup hole is also opened on the top center area of ​​the upper cover 34 at the position corresponding to the pickup hole of the shockproof cover 31.

[0076] Furthermore, to enhance the sound signal acquisition capability of the sound signal acquisition device 3, it is necessary to ensure that the sound signal propagation path is as short and wide as possible. In some embodiments, this is achieved by limiting the aperture-to-depth ratio of the pickup hole 310. Specifically, the aperture (d1) to depth (d2) ratio of the pickup hole 310 is preferably greater than 1. In a more specific embodiment, the aperture (d1) to depth (d2) ratio of the pickup hole 310 is greater than 2:1.

[0077] To enable the robot to better acquire the user's voice control audio signals, in some embodiments, the robot includes at least three audio signal acquisition devices 3, which are evenly distributed in a ring. The evenly distributed ring of multiple audio signal acquisition devices 3 can collect audio signals transmitted from various angles in a balanced manner, ensuring the accuracy and consistency of the acquired user voice control signals.

[0078] Figure 1D This is a schematic diagram of another embodiment of the autonomous mobile device provided in this application. Please refer to... Figure 1D The robot includes three sound signal acquisition devices 3, which are evenly distributed in a ring. Specifically, the three sound signal acquisition devices 3 are located on a circle, with the distance from each device 3 to the center of the circle being the radius of that circle. The central angle between any two adjacent devices is 120°. Furthermore, to optimize the sound signal acquisition capability of the multiple sound signal acquisition devices 3, the diameter of the circle in which at least three sound signal acquisition devices 3 are evenly distributed is within the range of 60mm to 100mm.

[0079] In other embodiments, the robot includes three sound signal acquisition devices 3 arranged in a triangle, with one of the three devices located at the front of the upper surface 10 of the robot shell 1 relative to the other two. These three sound signal acquisition devices 3 can be evenly distributed in a ring, meaning they lie on the circumcircle of the triangle and the central angle between any two adjacent devices is 120°.

[0080] Of course, in other embodiments, the three sound signal acquisition devices 3 do not need to be evenly distributed in a ring; they only need to be arranged in a front-and-back configuration. The advantage of this arrangement is that when the robot vacuum moves forward, the voice control commands issued by the user are delayed in transmission through media such as air. As a result, the sound signal acquisition device 3 at the front of the upper surface 10 of the robot shell 1 will only collect a small amount of sound signal, while most of the sound signal needs to be collected by the sound signal acquisition device 3 located at the rear. Placing more sound signal acquisition devices 3 at the rear can better collect sound signals and ensure the accuracy of the collected sound signals.

[0081] Furthermore, in order to ensure the best sound signal acquisition effect of the sound signal acquisition device 3, some embodiments also provide selection criteria for the sound signal acquisition device 3, specifically: select an omnidirectional digital microphone with a signal-to-noise ratio (SNR) greater than 64dB(A), a sensitivity of -26 to +3dBFS, an acoustic overload point (AOP) of 120dB SPL, and a total harmonic distortion (THD) of preferably less than 0.5% at 94dB SPL@1kHz.

[0082] Furthermore, in some embodiments, the robot also includes a distance sensor 4, which is mounted on the robot's housing 1. This distance sensor measures the distance between the robot and an obstacle in its direction of movement. When the distance reaches a set threshold, the robot can stop moving or change its path to prevent collisions. In other embodiments, the distance sensor 4 is rotatably mounted on the robot's housing 1, allowing it to rotate 360 ​​degrees relative to the housing to detect the layout of furniture, walls, etc., within the workspace. This allows the robot to map the workspace and operate based on the map, improving work efficiency.

[0083] The distance sensor 4 includes DTOF and LDS. In some embodiments, the distance sensor 4 is disposed on the protruding structure 2, and the sound signal acquisition device 3 is disposed on the distance sensor 4. Therefore, the distance sensor 4 and the sound signal acquisition device 3 can utilize the protruding structure 2, eliminating the need for separate protrusions for each, thus simplifying the robot's structure and reducing its manufacturing cost.

[0084] In other embodiments, the protruding structure 2 includes a distance sensor 4. That is, the distance sensor 4 is directly mounted on the upper surface of the robot shell 1 to form a protruding structure 2, while the sound signal acquisition device 3 is mounted on the distance sensor 4, i.e., the sound signal acquisition device 3 is mounted on the protruding structure 2 formed by the distance sensor 4. Since the distance sensor 4 is directly mounted on the upper surface of the robot shell 1 to form the protruding structure 2, and the sound signal acquisition device 3 utilizes the characteristics of the distance sensor 4 itself to protrude onto the robot shell 1, there is no need for a separate protruding structure. The overall structure is simple and low-cost.

[0085] On the other hand, the distance sensor 4, located on the upper surface 10 of the robot's shell 1, can effectively avoid other structures on the robot itself, thus accurately sensing the position of obstacles. Meanwhile, the sound signal acquisition device 3 can be located as far away as possible from noisy components such as the robot's drive motor, roller brush, side brush 7, and fan, reducing the interference of noise generated by the robot itself on the sound signal.

[0086] In other embodiments, the robot further includes a sound signal playback device 5, which can be a speaker. The sound signal playback device 5 is mounted on the robot's outer shell 1 and is communicatively connected to the robot's control unit. The control unit is configured with the robot's audio playback mode, such as playing music. When a user controls the robot to enter this audio playback mode via a remote control or app, the music stored in the control unit is played through the sound signal playback device 5.

[0087] To prevent the sound signal played by the sound signal playback device 5 from interfering with the sound signal acquisition device 3's acquisition of the user's voice control sound signal, in some embodiments, the pickup hole 310 of the sound signal acquisition device 3 and the playback hole of the sound signal playback device 5 are oriented in different directions. More specifically, the pickup hole 310 of the sound signal acquisition device 3 is oriented perpendicular to the upper surface 10 of the robot shell 1, while the playback hole of the sound signal playback device 5 is oriented perpendicular to the outer surface 11 of the robot shell 1. That is, the pickup hole 310 of the sound signal acquisition device 3 and the playback hole of the sound signal playback device 5 are set at a 90° angle.

[0088] It should be noted that, under normal circumstances, the upper surface 10 and the outer facade 11 of the robot shell 1 are set perpendicular to each other. Of course, if the sound pickup hole 310 of the sound signal acquisition device 3 and the sound playback hole of the sound signal playback device 5 are oriented in different directions, the upper surface 10 and the outer facade 11 of the robot shell 1 are set at other included angles.

[0089] Furthermore, in some embodiments, the sound signal playback device 5 is located at the front of the robot shell 1, while the sound signal acquisition device 3 is located at the rear of the robot shell 1. In other embodiments, the sound signal playback device 5 is located at the rear of the robot shell 1, while the sound signal acquisition device 3 is located at the rear of the robot shell 1. The division between the front and rear of the robot shell 1 is based on the shape of the robot shell 1, dividing it into two parts along its front-to-back axis. The area located at the front of the robot shell 1 is the front part, and the area located at the rear of the robot shell 1 is the rear part. For example: Figure 1C In the embodiment shown, the circular robot shell 1 is divided into a front semicircular region and a rear semicircular region along the front-back direction. The front semicircular region defines the front part, and the rear semicircular region defines the rear part.

[0090] It is understandable that one of the sound signal acquisition device 3 and the sound signal playback device 5 is located at the front of the robot shell 1 and the other is located at the rear of the robot shell 1, so that the two are kept at a sufficient distance, thereby further reducing the interference of the sound signal played by the robot itself on the sound signal acquisition device 3. The robot can more accurately acquire the user's voice control command and execute the command accurately, thereby providing the user with a better user experience.

[0091] Furthermore, in order to reduce the interference of the sound signal played by the robot itself on the sound signal acquisition device 3, in some embodiments, the robot also includes a sound signal recovery device. The sound signal recovery device is communicatively connected to the robot's control unit and the sound signal playback device 5. It is used to recover the sound signal from the sound signal playback device 5. The control unit receives the sound signal recovered by the sound signal recovery device, filters the recovered sound signal from the sound signal acquired by the sound signal acquisition device 3, and then transmits the instructions contained in the filtered sound signal to the execution element to control the execution element to execute the instructions.

[0092] In some embodiments, the audio signal acquisition device includes a filtered acquisition circuit that is electrically connected to the control unit of the robot body via wires and to the audio signal playback device via wires.

[0093] In addition to the sound signal acquisition device, in some embodiments, the robot also includes a sound signal noise reduction device, which is communicatively connected to both the sound signal acquisition device 3 and the control unit. This device is used to perform noise reduction processing on the sound signal acquired by the sound signal acquisition device 3 to eliminate noise or invalid sound signal parts of the acquired sound signal.

[0094] In addition to the robot described above, this invention also provides a control method applicable to the robot described above, to eliminate invalid sound signals collected by the sound signal acquisition device 3, and in particular, to eliminate interference caused by the sound signals emitted by the robot itself on the signal acquisition of the sound acquisition signal. For example, please refer to... Figure 1E .

[0095] Figure 1E This is a voice control flowchart for an autonomous mobile device provided in an embodiment of this application. This embodiment includes:

[0096] S1. Use sound signal acquisition device 3 to acquire sound signals;

[0097] The sound signals collected by the sound signal acquisition device 3 mainly include the user's voice control commands to the robot. For example, the robot uses the sound signal acquisition device 3 to collect the sound signals contained in the user's voice control. However, in practice, the robot's drive motor, side brush 7, roller brush, and / or fan can also generate sound signals during operation, or the robot itself may have the ability to generate sound signals. For example, the robot can play music or read books during operation or when it is stopped. Since the main function of the sound signal acquisition device 3 is to collect user voice control, this paper refers to these sound signals generated by the robot itself as "invalid sound signals". Based on this, in order to eliminate the interference of these invalid sound signals on the signals collected by the sound signal acquisition device 3, the robot control method of the present invention further includes the following steps:

[0098] S2. The effective sound signal is obtained by filtering the sound signal collected by the sound signal acquisition device 3 from the sound signal played by the robot itself.

[0099] Figure 1F This is another voice control flowchart for the autonomous mobile device provided in this application embodiment. Please refer to... Figure 1F In some embodiments, the method for implementing step S2 in this control method includes the following steps:

[0100] S20. The sound signal played by the recovery robot itself is considered an invalid sound signal;

[0101] S21. After filtering the invalid sound signal from the sound signal acquired by the sound signal acquisition device 3, a valid sound signal is obtained.

[0102] Specifically, a sound signal playback device 5 is installed in the robot. The sound signal playback device 5 can be a speaker. The sound signal playback device 5 is installed on the robot shell 1 and is communicatively connected to the robot's control unit. The control unit is set with the robot's working modes, such as playing music. When the user controls the robot to enter the control mode through the remote control or APP, the music stored in the control unit is played out through the sound signal playback device 5.

[0103] The robot also includes a sound signal acquisition device, which is communicatively connected to the robot's control unit and sound signal playback device 5. It is used to acquire the sound signal from the sound signal playback device 5. The control unit receives the sound signal acquired by the sound signal acquisition device and filters the acquired sound signal from the sound signal acquisition device 3. Then, it transmits the instructions contained in the filtered sound signal to the execution element and controls the execution element to execute the instructions.

[0104] Figure 1G This is another voice control flowchart for an autonomous mobile device provided in this application embodiment. In this embodiment, the method for implementing step S2 includes the following steps:

[0105] S20' Determine if the robot is in broadcast mode;

[0106] S21' If so, then the sound signal played by the robot in the broadcasting mode is taken as an invalid sound signal;

[0107] S22': Filter the invalid sound signal from the sound signal acquired by the sound signal acquisition device 3 to obtain a valid sound signal.

[0108] In addition, in some other embodiments, the control method of the present invention uses a sound signal acquisition device 3 to acquire sound signals, first performs noise reduction processing on the sound signals, and then filters out the sound signals played by the robot to obtain effective sound signals, so as to further eliminate the influence of other sound signals besides the user's voice control commands.

[0109] After obtaining a valid sound signal in step S2, the control method executes the following steps:

[0110] S3. Execute the control commands contained in the valid sound signal to realize voice interaction between the robot and the user, thereby improving the user experience.

[0111] Examples of application scenarios are as follows:

[0112] 1. The robot vacuum is currently cleaning the floor. The user issues a voice control command to "play music," and the robot picks up the command and begins playing stored music. Alternatively, the user can request music from the robot's stored audio files; the voice control command simply needs to include the music title.

[0113] 2. The robot vacuum is currently in a stopped or standby state. When the user issues a voice control command to "sweep", the robot will receive the command and begin cleaning the floor according to the predetermined route.

[0114] 3. The robot vacuum is currently cleaning the floor and playing music at the same time. The user issues a voice control command to "stop playing music". The robot receives the command and stops playing music after filtering out invalid sound signals generated by the music playback.

[0115] Figure 2 This is a flowchart of a voice control method for an autonomous mobile device provided in an embodiment of this application. The executing entity of this embodiment is an autonomous mobile device, and this embodiment includes:

[0116] 201. Acquire the first speech signal.

[0117] 202. When the first voice signal matches the wake-up command of the autonomous mobile device, the voice control function of the autonomous mobile device is activated.

[0118] An autonomous mobile device is equipped with a sound signal acquisition device, such as a microphone or microphone array. When the voice control function of the autonomous mobile device is not activated, the sound signal acquisition device continuously acquires first voice signals from the surrounding environment, identifies the first voice signals, and if the first voice signal matches the wake-up command of the autonomous mobile device, the voice control function of the autonomous mobile device is activated; if the first voice signal does not match the wake-up command of the autonomous mobile device, the voice control function remains in a waiting-to-wake-up state.

[0119] When the voice control function is activated, the autonomous mobile device collects the second voice signal and sends it to the voice recognition server in the cloud so that the voice recognition server can determine whether the second voice signal matches the control command. When the voice control function is in a waiting-to-wake-up state, the autonomous mobile device recognizes and collects the first voice signal locally to see if it matches the wake-up command.

[0120] 203. Acquire the second voice signal in the voice control function wake-up state.

[0121] Once the voice control function of the autonomous mobile device is activated, the device can use a sound signal acquisition device to collect voice signals from the surrounding environment. For example, it can collect a second voice signal emitted by the user.

[0122] 204. Determine at least two working areas based on the second voice signal.

[0123] When the autonomous mobile device has its own voice recognition function, it recognizes the second voice signal to determine at least two working areas. Alternatively, the autonomous mobile device sends the second voice signal to a voice recognition server, which then determines at least two working areas and instructs the autonomous mobile device accordingly. For example, if the text content corresponding to the second voice signal is "clean Xiaoming's room and study," then the at least two working areas are Xiaoming's room and study; or, if the text content corresponding to the second voice signal is "clean all carpeted areas," then the working areas are multiple areas centered on the carpet and including the carpet.

[0124] Figure 3 This is a schematic diagram of the speech recognition process in an embodiment of this application. Please refer to... Figure 3 During the speech recognition process, the autonomous mobile device acquires a second speech signal, performs noise reduction processing on the second speech signal, and then uploads the noise-reduced second speech signal to the speech recognition server. The speech recognition server performs semantic recognition on the second speech signal to obtain the speech command and sends the speech command back to the autonomous mobile device. After receiving the speech command, the autonomous mobile device executes the task indicated by the speech command. The task can be sweeping, mopping, lawn mowing, air purification, etc.

[0125] It should be noted that, although Figure 3 The example described uses a self-operated mobile device to collect a second voice signal. However, this application embodiment is not limited to this. In other feasible implementations, the user can also open a client on the terminal device and then send a second voice signal, which is then collected by the terminal device. After noise reduction processing of the second voice signal, the noise-reduced second voice signal is uploaded to a voice recognition server.

[0126] 205. Perform the tasks indicated by the second voice signal sequentially on the at least two working areas.

[0127] The autonomous mobile device executes tasks in each work area in a specific order. For example, the autonomous mobile device randomly sorts the work areas to obtain a random queue, and then executes tasks in each work area in the order indicated by the random queue.

[0128] For example, if a user wants to perform cleaning tasks on multiple work areas, and prefers to prioritize cleaning a particular work area, a second voice signal can be issued according to priority. The autonomous mobile device determines the order in which the work areas appear in the at least two work areas within the second voice signal. Then, the task indicated by the second voice signal is executed sequentially on the at least two work areas according to this order.

[0129] Taking an autonomous mobile device as an air-purifying robot as an example, if a user wants to purify the air in the study and the nursery, and wants to prioritize purifying the nursery, the second voice signal is "Please purify the nursery and the study." Based on this second voice signal, the autonomous mobile device determines the working areas as the nursery and the study, prioritizing the nursery. Even if the study is closer to the autonomous mobile device than the nursery, the device will first proceed to the nursery to purify the air there, and then proceed to the study to purify the air there.

[0130] This approach allows autonomous mobile devices to execute tasks in multiple work areas sequentially based on priority, greatly satisfying user needs and making the process more user-friendly.

[0131] For example, if there are many work areas, and these work areas are spaced some distance apart, it would be energy-intensive and time-consuming for the autonomous mobile device to execute tasks in each work area in a random order. Therefore, after identifying multiple work areas, the autonomous mobile device determines the distance between itself and each of the at least two work areas. Then, it sorts the at least two work areas in order of increasing distance to form a queue; and executes the tasks indicated by the second voice signal in the at least two work areas sequentially according to the queue.

[0132] For example, if the second voice signal is "clean all carpeted areas," the autonomous mobile device will determine all areas containing carpet from the environmental map. For an example, please refer to... Figure 4 . Figure 4 This is a schematic diagram of the carpeted area.

[0133] Please refer to Figure 4 The diagram shows three carpeted areas: carpet area 41, carpet area 42, and carpet area 43. The autonomous mobile device 40 determines that carpet area 41 is the closest, followed by carpet area 42, and finally carpet area 43. Therefore, the cleaning order is carpet area 41, carpet area 42, and carpet area 43.

[0134] Using this approach, autonomous mobile devices perform tasks in each work area in order of distance from nearest to farthest, which can maximize energy savings and increase speed.

[0135] It should be noted that although the above describes operating an autonomous mobile device to perform tasks in multiple work areas via voice commands, this application is not limiting. Other feasible implementations may also allow operating an autonomous mobile device to perform tasks in a single work area via voice commands.

[0136] The autonomous mobile device voice control method provided in this application involves the autonomous mobile device acquiring a second voice signal, determining at least two working areas based on the second voice signal, and sequentially executing the tasks indicated by the second voice signal in each working area. This approach allows multiple working areas to be indicated to the autonomous mobile device with a single voice signal, enabling the autonomous mobile device to sequentially execute tasks in multiple working areas. Users interact with the autonomous mobile device through natural language, resulting in extremely simple control with high accuracy and a straightforward process.

[0137] Optionally, in the above embodiments, during the process of the autonomous mobile device sequentially executing the task indicated by the second voice signal in the at least two working areas, after completing the task in one of the at least two working areas, before moving to the next working area, the task execution is stopped.

[0138] After identifying at least two work areas, the autonomous mobile device performs tasks in each work area sequentially according to a set order. If the distance between two adjacent work areas is long, the autonomous mobile device can shut down its work modules while traveling from one work area to another along its travel path. In other words, the autonomous mobile device does not perform tasks such as cleaning or mowing while traveling along its travel path.

[0139] Please refer to the following: Figure 4 The second voice signal indicates that carpet areas 43, 42, and 41 are to be cleaned. The autonomous mobile device 40 moves from its current position to area 43. During this process, the working module of the autonomous mobile device 40 is turned off. After the autonomous mobile device 40 has finished cleaning carpet area 43, during its movement from carpet area 43 to carpet area 42, and after the autonomous mobile device 40 has finished cleaning carpet area 42, during its movement from carpet area 42 to carpet area 41, the working module is turned off along the travel path, as shown by the dashed arrow in the figure.

[0140] Using this approach, autonomous mobile devices can move from one work area to the next without activating their working modules, saving energy while increasing travel speed.

[0141] Optionally, in the above embodiments, before the autonomous mobile device moves from its current location to the first working area, or from the current working area to the next working area, it is further determined whether the length of the travel path is greater than a preset threshold. If the length of the travel path is greater than the preset length, the working module is turned off and the device moves towards the working area according to the travel path. If the length of the travel path is less than or equal to the preset length, the device moves towards the working area while the working module is on.

[0142] For example, sometimes the travel path between two work areas is relatively short; for instance, the travel path between a bedroom and a living room is almost negligible. To avoid damage caused by frequent switching of the work module, it is not necessary to turn off the work module when the travel path is short.

[0143] Optionally, in the above embodiments, when the autonomous mobile device determines at least two working areas based on the second voice signal, it first determines the area category based on the second voice signal. Then, it determines at least two working areas from the area set corresponding to the environmental map based on the area category.

[0144] In this embodiment, when the autonomous mobile device is in a completely unknown environment, it constructs an environmental map or receives an environmental map sent by other robots. Then, it uses a partitioning algorithm to segment the environmental map into multiple work areas, enabling the environmental map to represent each work area, such as a kitchen, bathroom, or bedroom. Furthermore, the environmental map can also represent the actual positions of different objects in the environment, allowing the autonomous mobile device to determine the placement of objects within each work area.

[0145] When a user wants to clean a specific type of work area, they include the category information in the first voice command. For example, "Clean all bedrooms." The autonomous mobile device will then recognize this voice signal and clean only the individual bedrooms.

[0146] For example, the voice signal "Clean the furniture in the living room" indicates the area category as "living room." Therefore, the autonomous mobile device identifies the living room from the environmental map. Furthermore, the voice signal also indicates the target object as "furniture." Therefore, the autonomous mobile device identifies the furniture in the living room, such as the sofa and coffee table. For each piece of furniture, an area containing that furniture is defined as the center, and that area is cleaned.

[0147] For example, the voice signal "Clean under all beds in the bedrooms" indicates the area category as "bedroom". Therefore, the autonomous mobile device identifies the bedrooms from the environmental map. Furthermore, the voice signal also indicates the target object as "bed". Therefore, the autonomous mobile device continues to determine the location of the beds within each bedroom. For each bed, an area containing the bed is defined as the center, and that area is cleaned.

[0148] For example, in the command "Start cleaning the sofa and dining table area," the autonomous mobile device behaves as follows: it walks to the sofa, draws a rectangle larger than the sofa with the center of the sofa as the origin, and uses this rectangle as the work area to clean. Then, it moves to the dining table area, draws a rectangle larger than the dining table with the center of the dining table as the origin, and uses this rectangle as the work area to clean.

[0149] For example, when the message "Start cleaning the sofa area" is displayed, the autonomous mobile device will move to the location of the sofa, draw a rectangle larger than the sofa with the center of the sofa as the origin, and use this rectangle as the working area to clean it.

[0150] This solution allows users to control an autonomous mobile device via voice to perform cleaning tasks on specific work areas, demonstrating a high level of intelligence. Furthermore, the voice signal can also indicate target objects, enabling the autonomous mobile device to perform cleaning tasks on localized areas, further enhancing its intelligence.

[0151] Optionally, in the above embodiments, the autonomous mobile device can divide the environmental map into multiple working areas to obtain the area set based on the environmental map, the location information of objects in the environmental map, or the location information of doors in the environmental map. Then, it updates the identifiers of each working area in the area set and sends update information to the speech recognition server, so that the speech recognition server updates the identifiers of each working area.

[0152] For example, a camera or other shooting device is installed on an autonomous mobile device. Figure 5 This is a diagram illustrating the furniture identification process. Please refer to it. Figure 5 The autonomous mobile device constructs an environmental map or continuously captures images during its movement to collect images. After acquisition, the images undergo preprocessing, including one or more of the following: contrast enhancement, lossless magnification, and feature extraction. Then, the autonomous mobile device uses a pre-deployed trained model to perform AI recognition on the preprocessed images. This allows the trained model to output recognition results such as the type and location coordinates of furniture in the images, and to map the recognition results from a three-dimensional (3D) environmental map to a two-dimensional (2D) environmental map for storage and display. The trained model, for example, is an AI model trained using various types of furniture as samples.

[0153] Figure 6 This is a schematic diagram of the gate recognition process. (Similar to the above...) Figure 5 The difference is that the training model is an AI model that is pre-trained with various gates as samples. After the image is input into the training model, the output is the recognition results such as the position coordinates of the gate, and the recognition results are mapped from the 3D environment map to the 2D environment map for storage and display.

[0154] After acquiring a 2D environment map, the autonomous mobile device integrates the recognition results of furniture and doors within the map and uses a partitioning algorithm to divide the 2D map into multiple regions. The autonomous mobile device then sends the partitioning results to an app server and a voice recognition server. The app server then forwards the partitioning results to a terminal device, which has an app installed to control the autonomous mobile device. Upon receiving the partitioning results, the terminal device displays them. For an example, please refer to [link to example]. Figure 7 .

[0155] Figure 7 This is a diagram illustrating the synchronization process between the autonomous mobile device and the speech recognition server. Please refer to it. Figure 7 After partitioning, the environmental map consists of three regions: Region 1, Region 2, and Region 3. Users can customize the identifiers for each region on the client side. For example, the identifiers for Region 1, Region 2, and Region 3 can be changed to Xiaoming's Room, Living Room, and Kitchen, respectively. The terminal device then sends the update information to the APP server, which in turn sends it to the autonomous mobile device. Upon receiving the update information, the autonomous mobile device updates the identifiers for each working area on its local environmental map.

[0156] In addition, after updating the identifiers of each work area, the autonomous mobile device also synchronously updates the identifiers of the work areas to the speech recognition server. Upon receiving the update information from the APP server, the autonomous mobile device detects the change in the work area identifiers. Then, while updating its local settings, the autonomous mobile device sends the update information to the speech recognition server, causing the speech recognition server to update and save the identifiers of each work area. Afterward, users can interact with the autonomous mobile device using custom naming conventions.

[0157] For example, if a user says, "Clean Xiaoming's room," then the autonomous mobile device will move to the room area that is custom-named "Xiaoming" and clean that area.

[0158] For example, if a user says, "Clean the living room and kitchen," the autonomous mobile device will first move to the area custom-named "living room" and clean that area, then move to the area custom-named "kitchen" and clean it.

[0159] This approach ensures that the identifiers of each work area stored on the speech recognition server are consistent with the identifiers of the corresponding work areas stored on the autonomous mobile device, thereby improving the accuracy of speech recognition on the speech recognition server.

[0160] In the above embodiments, when the second voice signal indicates a region category, the autonomous mobile device determines a working area that matches the region category from a pre-built set of regions. However, this application embodiment is not limited. In other feasible implementations, when the region indicated by the region category is a specific region, and the location of the specific region may be different each time a voice signal is sent, the autonomous mobile device can also determine the working area in real time based on the region category. For example, the specific region might be an area in the house with water stains. Obviously, the location of the water stains in the house is different at different times. In this case, the autonomous mobile device uses a camera or other means to collect images, recognizes the images, and thus determines at least two working areas that match the region category.

[0161] For example, the second voice signal is "Check for water in the house and dry it." After the autonomous mobile device collects this voice signal and performs semantic recognition, it continuously takes pictures and identifies images while moving. If there are water spots in the images, it wipes them dry. Then, it continues to move and take pictures, wiping away water spots each time they are identified.

[0162] For example, the second voice signal is "mop the oily areas in the kitchen." After the autonomous mobile device collects this voice signal and performs semantic recognition, it continuously takes pictures and recognizes the images as it moves through the kitchen, mopping the floor each time it detects an oily area.

[0163] This approach is used to achieve the goal of carrying out tasks in a specific area.

[0164] Optionally, in the above embodiments, when the autonomous mobile device sequentially executes the task indicated by the second voice signal in the at least two working areas, the operation mode can be determined according to the area category, and the task can be executed in the indicated two working areas at one time according to the operation mode.

[0165] For example, when a user issues a second voice signal, they may not specify a particular task; instead, the autonomous mobile device will determine and execute the task autonomously. For instance, the user might say, "Clean the grease in the kitchen." After continuously capturing images and identifying the grease-stained area, the autonomous mobile device will determine the task as: adding cleaning fluid and increasing the mopping frequency. Then, the autonomous mobile device will spray cleaning fluid onto the grease-stained area and mop vigorously.

[0166] For example, a user says, "Clean the water stains in the living room." After the autonomous mobile device continuously collects images to determine the water stain area, if there is a lot of water in the area, the task is determined to be: perform three mopping operations. Then, the autonomous mobile device mops the water stain area three times. If there is less water in the area, the task is determined to be: perform one mopping operation. Then, the autonomous mobile device mops the water stain area once.

[0167] This approach allows autonomous mobile devices to automatically determine the most suitable operating mode, thereby improving task execution efficiency.

[0168] Optionally, in the above embodiments, it is assumed that the autonomous mobile device is located in the initial region when it acquires the second voice signal, and this initial region is not any of the working regions in the second voice signal. Then, before moving from the initial region to the working region, the autonomous mobile device records the task execution status of the initial region. Afterwards, tasks are executed sequentially in each working region. After completing the task, it is determined based on the records whether the task in the initial region was not completed. If the autonomous mobile device has not completed the task in the initial region, it returns to the initial region and executes the task again.

[0169] Using this approach, after the autonomous mobile device completes its task in the area indicated by the second voice signal, it returns to the initial area to continue performing the task, thus avoiding the inability to complete the task in the initial area.

[0170] Optionally, in the above embodiments, the second voice signal may also include task parameters, etc. For example, the second voice signal may be: "Use the strong mopping mode to clean Xiaoming's room and study for 10 minutes each," or "Mop the oily areas twice," etc.

[0171] In the above embodiments, to prevent the autonomous mobile device from continuously recognizing voice signals when the user has no interaction needs, the voice control function of the autonomous mobile device is usually in a silent state. The voice control function can only be activated after the user utters a specific wake-up word. When the voice control function is in a silent state, the autonomous mobile device can be either stationary or actively operating.

[0172] During the operation of autonomous mobile devices, noise generated by motor rotation and other factors can potentially interfere with the accuracy of voice signal recognition. This application's embodiments avoid this drawback. The operating state of the autonomous mobile device after wake-up is referred to as the second operating state, and the operating state before wake-up is referred to as the first operating state. The volume of the sound generated by the autonomous mobile device in the second operating state is lower than the volume of the sound generated in the first operating state. When the autonomous mobile device acquires a second voice command in the first operating state, if the first voice signal matches the wake-up command, the autonomous mobile device automatically switches to the second operating state. That is, the autonomous mobile device switches to the second operating state by reducing output power consumption, and acquires the aforementioned second voice signal in the second operating state.

[0173] For example, when installing a microphone on an autonomous mobile device, the microphone is placed in a location with minimal and stable noise. Furthermore, the wake-up model is trained using a large number of samples, improving the wake-up rate of the autonomous mobile device across various operating states. Then, when the voice control function of the autonomous mobile device is activated while it is in operation, the device adjusts its operating state to reduce the volume of its own noise. Afterward, the user issues control voice commands at a normal volume, and the autonomous mobile device receives and executes the corresponding tasks.

[0174] For example, consider a robotic vacuum cleaner as an autonomous mobile device. In its first operating state, the device travels at a speed of 0.2 meters per second. When the user issues a first voice signal, if the signal matches a wake-up command, the device switches to a second operating state, traveling at 0.1 meters per second with lower noise levels. Subsequently, the user issues a second voice signal, which the device receives and executes the relevant tasks. The volume of the second voice signal can be lower than the volume of the first voice signal.

[0175] Furthermore, to ensure the autonomous mobile device can be woken up in high-noise environments, the wake-up rate can be improved in advance through multiple training sessions and algorithm updates. For example, when the autonomous mobile device is within 5 meters of the user and in its first working state, the wake-up rate using normal human voice can reach 85%. After being woken up, the autonomous mobile device switches to its first working state, where the voice recognition accuracy is almost equivalent to that of a smart speaker.

[0176] With this approach, once the autonomous mobile device is woken up, it automatically changes its operating state to reduce the noise it generates, thereby improving the accuracy of subsequent speech recognition.

[0177] Optionally, in the above embodiments, when the first voice signal matches the wake-up command of the autonomous mobile device, the autonomous mobile device determines the location of the sound source of the first voice signal. Then, the autonomous mobile device controls itself to switch from a first pose to a second pose based on the sound source location. When the autonomous mobile device is in the second pose, the distance between the microphone and the sound source location is less than the distance between the microphone and the sound source location when the autonomous mobile device is in the first pose. The microphone is a microphone mounted on the autonomous mobile device.

[0178] For example, intelligent voice technology has been widely applied in human-computer interaction, intelligent control, online services, and other fields. With the expansion of more application scenarios, intelligent voice technology has become the most convenient and effective means for people to obtain information and communicate. Intelligent voice technology includes speech recognition technology and speech synthesis technology. Sound source localization is a method of locating sound sources based on microphone arrays. Implementation methods can be divided into directional wave velocity formation and time delay estimation. Combining intelligent voice technology, microphone sound source localization technology, and autonomous mobile devices can design a wide range of application scenarios, such as issuing voice commands to autonomous mobile devices to perform tasks, interacting with autonomous mobile devices via voice to obtain corresponding guidance, and controlling the autonomous mobile device to turn based on sound sources.

[0179] In the field of autonomous mobile devices, microphone arrays are typically designed to receive sound source information for sound source localization. Based on this localization, the autonomous mobile device is controlled to turn in the direction of the sound source, increasing the interactivity and accuracy of subsequent speech recognition. The drawback of this type of application is the relatively large error in microphone array localization, typically around ±45°. The root cause of this error lies in the insufficient accuracy of estimating the time difference between the sound source and the microphone. Due to this error, the final effect of the autonomous mobile device turning towards the sound source may be inaccurate, resulting in a poor user experience.

[0180] Therefore, in this embodiment, when the voice control function of the autonomous mobile device is activated, the autonomous mobile device can adjust its posture to bring its microphone closer to the user, thereby improving the accuracy of voice acquisition. Sound source localization technology, speech recognition technology, and AI recognition technology are used to precisely control the autonomous mobile device to turn towards the speaker, i.e., towards the user. During this process, the autonomous mobile device captures the sound source through its microphone array, and after signal conversion, it is recognized as a predetermined wake-up word. Then, the position of the sound source relative to the microphone array is determined, and subsequently, the position of the sound source relative to the device itself is determined, thus determining the approximate rotation angle, i.e., locating the approximate position of the sound source. Finally, the autonomous mobile device rotates according to the rotation angle, and during the rotation process, AI recognition is used to accurately determine the specific position of the sound source, thereby controlling the autonomous mobile device to stop facing the user.

[0181] The following section provides a detailed explanation of how autonomous mobile devices can turn in the direction of sound source localization.

[0182] First, determine the first position, which is the position of the sound source relative to the center of the microphone array.

[0183] After the user speaks, the autonomous mobile device picks up the voice signal through the microphone array and processes the voice signal using the computing unit to obtain the voice recognition result. If the voice recognition result matches the wake word, the first position is determined; if the voice recognition result does not match the wake word, the device remains in a waiting-to-wake state.

[0184] Figure 8A This is a schematic diagram illustrating the position of the sound source relative to the center of the microphone array. Please refer to... Figure 8A The microphone array comprises six microphones, located at points S1-S6, evenly distributed on a circle of radius L1. The origin O of the spatial coordinate system is the center of the microphone array. After sound is emitted from the source, the time it takes for the sound to reach each microphone is different. Therefore, the autonomous mobile device can determine the initial position based on the speed of sound propagation, time delay, and the positions of each microphone. Here, time delay refers to the difference in the time it takes for different microphones to receive the sound.

[0185] It should be noted that although this example uses six microphones, the embodiments in this application are not limited to this. Other feasible implementations may use more or fewer microphones.

[0186] Secondly, the second position is determined based on the first position, and the rotation angle is determined based on the second position, where the second position is the position of the sound source relative to the center of the autonomous mobile device.

[0187] Typically, the microphone array is located at a fixed position on the body of the autonomous mobile device. Once the first position is determined, the autonomous mobile device can determine the second position based on the position of the microphone array and the first position.

[0188] Figure 8B This is a schematic diagram of the microphone array and the body of the autonomous mobile device. Please refer to... Figure 8B The center of the autonomous mobile device is the center of a great circle. While the center of the device and the center of the microphone array do not coincide, their relative positions are known. Therefore, once the autonomous mobile device determines its first position, it can determine its second position. Once the second position is determined, the rotation angle can be determined. The rotation angle refers to the angle of rotation of the autonomous mobile device from the first pose to the second pose.

[0189] The first pose is the pose of the autonomous mobile device before wake-up. The second pose is the pose of the autonomous mobile device with its microphone facing the user; the second pose can also be understood as the pose of the autonomous mobile device directly facing the user. The pose of the autonomous mobile device directly facing the user refers to the pose of the autonomous mobile device's camera facing the user.

[0190] Finally, rotate according to the rotation angle.

[0191] During rotation, to avoid affecting AI recognition performance due to excessively fast rotation and impacting user experience due to excessively slow rotation, the electronic device divides the rotation angle into a first angle and a second angle; it rotates at a first speed within the first angle and at a second speed within the second angle, where the first speed is greater than the second speed. The second angle is, for example, α degrees.

[0192] During the rotation, assuming the rotation angle is θ, the device first rotates rapidly by θ-α degrees, then rotates at a constant speed by α degrees, where α < θ. During this constant-speed rotation, the camera continuously captures images and performs AI recognition. If a user is detected, the rotation stops; otherwise, it rotates by α degrees and then stops. Here, α is related to the statistical error of the autonomous mobile device and can be 30 degrees, 60 degrees, etc. This application's embodiments are not limited to these parameters.

[0193] Taking a robotic vacuum cleaner as an example of an autonomous mobile device, please refer to... Figure 8B The functional components of a robotic vacuum cleaner include a camera, microphone array, laser rangefinder, infrared receiver sensor, side brush, and drive wheels. Additionally, it may include edge sensors (not shown in the diagram), anti-fall sensors, a suction fan, a motor, a roller brush, a computing and storage unit, a battery module, and a Wi-Fi module. In voice control scenarios, the robotic vacuum cleaner is in any operating state. The user issues a voice wake-up command, and the robot pauses its current operation, turns to the voice caller, and awaits the user's next interactive command.

[0194] For example, if a robot vacuum is cleaning in the living room and the user says "Xiao Q, Xiao Q," the robot vacuum will pause its cleaning, turn to the user, and simultaneously announce "I'm here" in voice, waiting for further instructions from the user, such as "Please leave the living room and clean another room." The robot vacuum will then announce "Okay" in voice and leave the living room to continue cleaning in the bedroom or other rooms.

[0195] In this voice interaction experience, the robot vacuum cleaner needs to accurately recognize the user's wake-up command and turn towards the user precisely and quickly, awaiting the next control command. If it cannot accurately locate the position of the voice command issuer, this interaction scenario will become very poor. The first scenario is inaccurate positioning, where the robot turns in another direction and does not accurately face the voice controller; the second scenario is that it locates the direction of the voice controller, but the rotation process is slow, the action takes a long time, and the interaction experience is poor.

[0196] Therefore, the robot vacuum pauses its cleaning process and turns towards the user. First, it determines the initial position of the sound source relative to the center of the microphone array. Then, based on the initial position and the position of the microphone array relative to the robot body, it determines the second position and rotation angle. Next, it rotates rapidly by θ-α degrees, then rotates at a constant speed of α degrees. During this constant speed rotation, it continuously captures images using its camera and performs AI recognition. If the user is detected, it stops rotating; otherwise, it rotates α degrees and then stops.

[0197] For example, Figure 8B If the user is located to the right of the robot vacuum, the rotation angle θ = 180 degrees. If α is 60 degrees, the autonomous mobile device first rotates rapidly clockwise 120 degrees, then rotates at a constant speed while acquiring images. If the user is detected based on the image when the device rotates to 170 degrees, the rotation stops. If the user is not detected, the device rotates at a constant speed for 60 degrees and then stops.

[0198] This approach, combining sound source localization, speech recognition, and AI recognition, makes the steering movements of autonomous mobile devices more accurate. Furthermore, the rotation process, which involves first rapid rotation followed by slow, uniform rotation based on the rotation angle, is smoother and the AI ​​recognition is more accurate.

[0199] In the above embodiments, speech recognition technology is a pattern recognition based on speech feature parameters. The speech recognition server can classify the input speech according to certain patterns, and then find the best matching result based on the judgment criteria. The principle framework diagram is as follows. Figure 8C As shown.

[0200] Figure 8C This is a schematic diagram illustrating the process of training a speech recognition model and recognizing speech. Please refer to it. Figure 8C During the training process, the input speech signal is preprocessed and then features are extracted. The extracted features are used to train the model, and the speech recognition model is then saved.

[0201] The trained speech recognition model is deployed on a speech recognition server. The user's speech signal is preprocessed and features are extracted. The speech recognition server then inputs the extracted features into the speech recognition model to improve the speech recognition results.

[0202] Figure 9 This is a flowchart illustrating the voice control logic of an autonomous mobile device provided in an embodiment of this application. This embodiment includes:

[0203] 901. The autonomous mobile device is in its first working state, and the voice control function is in a waiting-to-wake-up state.

[0204] 902. The autonomous mobile device collected the first voice signal.

[0205] For example, the user emits a first voice signal, and the autonomous mobile device collects the first voice signal using a sound signal acquisition device or the like.

[0206] 903. Does the first voice signal match the wake-up command of the autonomous mobile device? If the first voice signal matches the wake-up command, the autonomous mobile device is successfully woken up and step 904 is executed. If the first voice signal does not match the wake-up command, the autonomous mobile device fails to wake up and step 911 is executed.

[0207] In this step, the autonomous mobile device uses its own voice recognition function to determine whether the first voice signal and the wake-up command match, or the autonomous mobile device sends the first voice signal to the voice recognition server, which then determines whether the first voice signal and the wake-up command match.

[0208] 904. The autonomous mobile device switches from the first working state to the second working state.

[0209] For example, compared to the first operating state, the autonomous mobile device has lower power consumption and lower noise in the second operating state. For instance, in the first operating state, the drive wheel rotates normally, and the other sound-producing components operate normally; while in the second operating state, the drive wheel stops rotating, and the other sound-producing components reduce their operating power. Alternatively, in the second operating state, the drive wheel rotates at a reduced speed, and the other sound-producing components reduce their operating power.

[0210] 905. The autonomous mobile device determines whether it has collected the second voice signal within a preset time period. If the autonomous mobile device has collected the second voice signal within the preset time period, then proceed to step 906; if the autonomous mobile device has not collected the second voice signal within the preset time period, then proceed to step 912.

[0211] 906. The autonomous mobile device determines whether the second voice signal has been successfully parsed.

[0212] For example, the autonomous mobile device itself or the voice server parses the second voice signal. If the second voice signal is successfully parsed, step 907 is executed; if the second voice signal is not successfully parsed, step 913 is executed.

[0213] 907. The autonomous mobile device determines whether the parsing result matches the control command. If the parsing result matches the control command, proceed to step 908; if the parsing result does not match the control command, proceed to step 914.

[0214] For example, an autonomous mobile device or voice recognition server determines whether the parsing result corresponds to a task such as cleaning, sweeping, or mowing the lawn.

[0215] 908. The autonomous mobile device determines whether its own state meets the requirements for executing the task. If its own state meets the requirements for executing the task, then proceed to step 909; if its own state does not meet the requirements for executing the task, then proceed to step 915.

[0216] For example, the autonomous mobile device determines whether its own battery level, remaining dustbin space, and remaining water level in the water tank meet the task requirements.

[0217] 909. Execute the task and provide voice feedback to the user, then proceed to step 910.

[0218] For example, the autonomous mobile device starts moving and says to the user, "Okay, I'm about to clean Xiaoming's room and study."

[0219] 910. End this round of voice interaction. The voice control function enters the waiting wake-up state.

[0220] 911. The autonomous mobile device continues to operate in the first working state. After a preset time, proceed to step 910.

[0221] 912. The autonomous mobile device feedback command timed out and returned to the first working state. Then, proceed to step 910.

[0222] For example, the autonomous mobile device sends a voice feedback to the user: "Voice interaction timed out, please wake up again." Simultaneously, the autonomous mobile device re-enters its first working state. Then, step 910 is executed.

[0223] 913. The autonomous mobile device reports that it does not understand the user's intent and returns to the first working state. Then, proceed to step 910.

[0224] For example, the autonomous mobile device sends a voice feedback to the user: "No correct instruction received, please wake up again"; or "I didn't hear what you said clearly, please wake up again." Simultaneously, the autonomous mobile device re-enters its first working state. Then, step 910 is executed.

[0225] 914. The autonomous mobile device continues to operate in the second working state, engaging in voice interaction and dialogue with the user. After a preset time, proceed to step 910.

[0226] For example, an autonomous mobile device might send a message to the user such as, "This task is too difficult for me to perform. Please describe it in a different way," or "Do you want to clean under the bed in your bedroom?" to guide the user's interaction and help the system understand the user's intent.

[0227] 915. The autonomous mobile device's voice feedback indicates that it cannot perform the task and continues to operate in the second working state.

[0228] For example, the autonomous mobile device tells the user, "I need to charge before I can perform a task," "I need to return to the base station to refill water," "Please clean the dust box," etc., and continues to operate in the second working state. After a preset time, step 910 is executed.

[0229] Alternatively, the autonomous mobile device can tell the user, "I need to charge; I'll clean the bedroom after I'm fully charged," and then move to the base station to recharge and maintain its own condition. Afterward, the autonomous mobile device will then perform tasks such as cleaning.

[0230] Using this approach, autonomous mobile devices can determine their own status before executing a task, and decide whether to execute the task immediately or charge or rehydrate before executing the task, thus avoiding interruptions during task execution.

[0231] In the above embodiments, after the voice control function is activated, it re-enters the waiting-to-wake state after a preset time. For example, if no second voice signal is collected after the preset time following activation, the voice control function automatically enters the waiting-to-wake state. Similarly, after completing one wake-up and command issuance cycle, it automatically enters the waiting-to-wake state.

[0232] This approach can prevent accidental triggering of autonomous mobile devices by surrounding voices.

[0233] Optionally, in the above embodiments, the second voice signal can also indicate a mission restricted area. The autonomous mobile device determines the mission restricted area from the environmental map and identifies at least two work areas outside the mission restricted area.

[0234] If the no-go zones are fewer or easier to describe compared to the work area, the user can indicate the no-go zones in the second voice signal. The autonomous mobile device determines the no-go zones based on the second voice signal, and then designates the other areas as the work area and performs the task. For example, if the user says, "Don't clean under the bed," the autonomous mobile device will clean the area outside the bed.

[0235] This approach reduces the difficulty for users to issue voice commands.

[0236] In the above embodiments, voice signals are mainly used for controlling the work area. However, the embodiments of this application are not limited to this. Other feasible implementations can also use voice control to enable autonomous mobile devices to control other electronic devices in the home, to enable autonomous mobile devices to perform functions such as fixed-point patrol, monitoring and care, and to enable autonomous mobile devices to automatically locate items. These scenarios will be described below.

[0237] First, consider a scenario where a mobile device can control other electronic devices in the home via voice control. For an example, please refer to... Figure 10 . Figure 10 This is another flowchart of the voice control method for autonomous mobile devices provided in this application embodiment. This embodiment includes:

[0238] 1001. The autonomous mobile device is in its first working state, and the voice control function is in a waiting-to-wake-up state.

[0239] 1002. The user activates the voice control function through the first voice signal and issues the second voice signal.

[0240] For example, after the autonomous mobile device activates the voice control function, it automatically switches to the second working state, as described above, and will not be repeated here.

[0241] 1003. The autonomous mobile device receives a control command based on the second voice signal. This control command is used to instruct the autonomous mobile device to control a designated device.

[0242] For example, the autonomous mobile device parses the second voice signal itself, or a voice recognition server parses the voice signal to obtain control instructions. These control instructions instruct the autonomous mobile device to control designated devices within a specified area. The designated devices may be, for example, household appliances such as air conditioners, refrigerators, or curtains.

[0243] 1004. The autonomous mobile device moves to the designated area to complete the control of the designated device.

[0244] For example, when the autonomous mobile device is powered on, and the user is within its voice signal acquisition range (e.g., the user and the autonomous mobile device are both in the same location, or the autonomous mobile device is 5 meters away from the user), the user's second voice signal is "Turn on the master bedroom air conditioner, set it to 25°C." After the autonomous mobile device plans its route based on the environmental map and enters the master bedroom, it uses its own hardware remote control module to turn on the air conditioner, sets the mode to cooling mode, and sets the temperature to 25°C.

[0245] This approach combines voice control with other hardware and algorithms of autonomous mobile devices to enable value-added functions, making the autonomous mobile devices more intelligent.

[0246] Secondly, consider scenarios where autonomous mobile devices are used for fixed-point patrols via voice control. For an example, please refer to... Figure 11 . Figure 11 This is another flowchart of the autonomous mobile device voice control method provided in this application embodiment. This embodiment includes:

[0247] 1101. The autonomous mobile device is in its first working state, and the voice control function is in a waiting-to-wake-up state.

[0248] 1102. The user wakes up the voice control function with the first voice signal and issues the second voice signal.

[0249] For example, after the autonomous mobile device activates the voice control function, it automatically switches to the second working state, as described above, and will not be repeated here.

[0250] 1103. The autonomous mobile device receives a control command based on the second voice signal. This control command is used to instruct the autonomous mobile device to patrol at a fixed point.

[0251] For example, the autonomous mobile device can parse the second voice signal itself, or a voice recognition server can parse the voice signal to obtain control commands. These control commands are used to instruct the autonomous mobile device to perform tasks such as fixed-point patrols, monitoring, or caregiving.

[0252] 1104. Autonomous mobile devices perform fixed-point patrols, monitoring, or guarding.

[0253] For example, the user's second voice signal is "Go check on Dad's room." After the autonomous mobile device plans its route based on the environmental map and enters Dad's room, it travels to the previously set monitoring point, turns on the camera to record video, and sends the video back to the user's mobile client, realizing the function of monitoring the elderly across rooms.

[0254] This solution enables autonomous mobile devices to perform functions such as fixed-point patrols at home, monitoring designated areas, and guarding specific rooms according to the user's intentions through voice control.

[0255] Finally, consider the scenario of using voice control to automatically locate items using autonomous mobile devices. For an example, please refer to... Figure 12 . Figure 12 This is another flowchart of the autonomous mobile device voice control method provided in this application embodiment. This embodiment includes:

[0256] 1201. The autonomous mobile device is in its first working state, and the voice control function is in a waiting-to-wake-up state.

[0257] 1202. The user wakes up the voice control function with the first voice signal and issues the second voice signal.

[0258] 1203. The autonomous mobile device receives a control command based on the second voice signal. This control command is used to instruct the autonomous mobile device to find the target object.

[0259] 1204. The autonomous mobile device determines whether the location coordinates of the target object are marked in the environmental map. If the target object is marked in the environmental map, proceed to step 1205; if the target object is not marked in the environmental map, proceed to step 1208.

[0260] For example, during operation, the autonomous mobile device inputs the captured images into an AI training model to obtain the object's location coordinates, name, type, etc., and records this information in the environmental map for subsequent intelligent object finding. To prevent clutter on the client interface, the location coordinates of these objects may not be displayed on the environmental map. When the user issues a command to find the target object, the client displays the target object's location on the environmental map.

[0261] 1205. The autonomous mobile device asks the user via voice whether they need to search for the target object now. If the user responds that they need to search for the target object now, proceed to step 1206; if the user responds that they do not need to search for the target object now, proceed to step 1207.

[0262] 1206. Guide the user to the target object's location and display the target object's location on the environment map.

[0263] 1207. Display the location of the target object on the environment map and provide a voice prompt to the user: The location of the target object has been displayed on the client.

[0264] The location of the target object found during the search process on the environment map.

[0265] 1208. Voice prompt to user: No target object found. Please describe it in a different way.

[0266] For example, the first voice signal might be: "Please help me find my socks." After recognizing the first voice signal, the autonomous mobile device determines whether the socks' location coordinates have been marked locally. If the socks haven't been marked, the user is prompted that they can't find them. If the socks' location coordinates have been marked, a voice prompt is issued: "Do you want to go find the socks now?" If the user's response is an affirmative answer such as "Yes" or "Okay," the autonomous mobile device moves to guide the user to the socks' location and displays the socks' location coordinates on the client's environment map interface. If the user's response is a negative answer such as "No need," the autonomous mobile device simply instructs the client to display the socks' location coordinates.

[0267] This approach uses machine learning to train an AI training model, which then identifies specific objects. This allows users to use voice control to search for specific objects on a map or guide them to the object's location. This achieves the goal of intelligent object finding while expanding the uses of autonomous mobile devices and enhancing their intelligence.

[0268] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0269] Figure 13 This is a schematic diagram of a voice control device for an autonomous mobile device provided in an embodiment of this application. The voice control device 1300 for the autonomous mobile device includes: a data acquisition module 1301, a processing module 1302, and an execution module 1303.

[0270] Acquisition module 1301 is used to acquire the first voice signal;

[0271] The processing module 1302 is used to wake up the voice control function of the autonomous mobile device when the first voice signal matches the wake-up command of the autonomous mobile device.

[0272] The acquisition module 1301 is also used to acquire a second voice signal;

[0273] The processing module 1302 is used to determine at least two working areas based on the second voice signal;

[0274] The execution module 1303 is used to sequentially execute the task indicated by the second voice signal in the at least two working areas.

[0275] In one feasible design, the execution module 1303 is used to determine the order in which each working area appears in the at least two working areas of the second voice signal; and to execute the task indicated by the second voice signal in the at least two working areas in sequence according to the order.

[0276] In one feasible design, the execution module 1303 is used to determine the distance between the autonomous mobile device and each of the at least two working areas; sort the at least two working areas in order of distance from near to far to obtain a queue; and execute the task indicated by the second voice signal in the at least two working areas in sequence according to the queue.

[0277] In one feasible design, the execution module 1303 is used to stop executing the task after completing the task in one of the at least two work areas and before moving to the next work area.

[0278] In one feasible design, the processing module 1302 is used to determine the region category based on the second voice signal; and to determine the at least two working areas from the region set corresponding to the environmental map based on the region category.

[0279] In one feasible design, when the processing module 1302 determines the at least two working areas from the environment map according to the area category, it is used to determine the area containing the target object from the environment map with the target object as the center to obtain the at least two working areas when the area category indicates a target object.

[0280] Please refer to the following: Figure 13 In one feasible design, the aforementioned autonomous mobile device voice control device 1300 further includes a transceiver module 1304.

[0281] Before the processing module 1302 determines at least two working areas based on the second voice signal, it is further configured to divide the environment map into multiple working areas to obtain the area set based on the environment map, the location information of objects in the environment map, or the location information of doors in the environment map, and update the identifier of each working area in the area set.

[0282] The transceiver module 1304 is used to send update information to the speech recognition server so that the speech recognition server updates the identifiers of each working area.

[0283] In one feasible design, the processing module 1302 is further configured to control the autonomous mobile device to switch from a first working state to a second working state when the first voice signal matches the wake-up command of the autonomous mobile device, wherein the volume of the sound generated by the autonomous mobile device in the second working state is less than the volume of the sound generated in the first working state, and the wake-up command is used to wake up the voice control function of the autonomous mobile device; the acquisition module 1301 is configured to acquire the second voice signal in the second working state.

[0284] In one feasible design, the processing module 1302 is further configured to determine the sound source location of the first voice signal when the first voice signal matches the wake-up command of the autonomous mobile device; control the autonomous mobile device to switch from a first pose to a second pose according to the sound source location, wherein the distance between the microphone and the sound source location when the autonomous mobile device is in the second pose is less than the distance between the microphone and the sound source location when the autonomous mobile device is in the first pose, and the microphone is a microphone disposed on the autonomous mobile device.

[0285] In one feasible design, when the processing module 1302 controls the autonomous mobile device to switch from a first pose to a second pose based on the sound source location, it is used to determine a rotation angle based on the sound source location. The rotation angle is used to indicate the angle that the autonomous mobile device needs to rotate when switching from the first pose to the second pose. The rotation angle is divided into a first angle and a second angle. The device rotates at a first speed within the first angle and at a second speed within the second angle, wherein the first speed is greater than the second speed.

[0286] In one feasible design, when the first voice signal matches the wake-up command of the autonomous mobile device, after the processing module 1302 controls the autonomous mobile device to switch from the first working state to the second working state according to the location of the sound source, it is also used to control the voice control function to enter the waiting wake-up state after a preset time.

[0287] In one feasible design, the execution module 1303 is used to determine whether the self-state of the autonomous mobile device meets the requirements for executing the task; if the self-state does not meet the requirements for executing the task, then the autonomous mobile device is maintained.

[0288] In one feasible design, the processing module 1302 is used to determine the task restricted area from the environment map when the second voice signal indicates a task restricted area, and to determine the at least two work areas from the area outside the task restricted area.

[0289] In one feasible design, the processing module 1302 is used to determine the region category based on the second speech signal; acquire an image; and determine the region corresponding to the region category from the image to obtain the at least two working regions.

[0290] In one feasible design, the execution module 1303 is used to determine the operation mode according to the area category; and to execute tasks on the at least two work areas sequentially according to the operation mode.

[0291] In one feasible design, after the processing module 1302 sequentially executes the task indicated by the second voice signal in the at least two working areas, it is further configured to determine whether the task has been completed in the initial area, which is the area where the autonomous mobile device is located when it collects the second voice signal; if the task has not been completed in the initial area, it returns to the initial area to execute the task.

[0292] The autonomous mobile device voice control device provided in this application embodiment can execute the actions of the autonomous mobile device in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0293] Figure 14This is a schematic diagram of another autonomous mobile device voice control device provided in an embodiment of this application. The autonomous mobile device voice control device 1400 includes: a data acquisition module 1401, a processing module 1402, a driving module 1403, and an execution module 1404.

[0294] Acquisition module 1401 is used to acquire the second voice signal;

[0295] Processing module 1402 is used to determine the working area based on the second voice signal; and to determine the driving path based on the current location and the working area.

[0296] The driving module 1403 is used to shut down the working module and travel to the working area according to the driving path;

[0297] The execution module 1404 is configured to activate the working module to execute the task indicated by the second voice signal if the autonomous mobile device moves into the working area.

[0298] In one feasible design, the driving module 1403 is used to determine whether the length of the driving path is greater than a preset length. If the length of the driving path is greater than the preset length, the working module is shut down and the module travels to the working area according to the driving path.

[0299] In one feasible design, after the execution module 1404 activates the working module to execute the task indicated by the second voice signal, it is further configured to determine whether the task has been completed in the initial region, which is the region where the autonomous mobile device is located when it collects the second voice signal; if the task has not been completed in the initial region, the execution module returns to the initial region to perform the task.

[0300] In one feasible design, when the processing module 1402 determines the working area based on the second voice signal, it is used to determine the region category based on the second voice signal; acquire an image; and determine the region corresponding to the region category from the image to obtain the working area.

[0301] The autonomous mobile device voice control device provided in this application embodiment can execute the actions of the autonomous mobile device in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0302] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. As shown in FIG8, the electronic device 1500 is, for example, the aforementioned autonomous mobile device, which includes:

[0303] Processor 1501 and memory 1502;

[0304] The memory 1502 stores computer instructions;

[0305] The processor 1501 executes the computer instructions stored in the memory 1502, causing the processor 1501 to perform the autonomous mobile device voice control method implemented by the autonomous mobile device as described above.

[0306] The specific implementation process of processor 1501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0307] Optionally, the electronic device 1500 also includes a communication component 1503. The processor 1501, memory 1502, and communication component 1503 can be connected via a bus 1504.

[0308] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the autonomous mobile device voice control method implemented by the autonomous mobile device as described above.

[0309] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the autonomous mobile device voice control method described above.

[0310] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0311] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A voice control method of an autonomous mobile device, the method comprising: The method is applied to an autonomous mobile device, and comprises: collecting a first voice signal; when the first voice signal matches a wake-up instruction of the autonomous mobile device, waking up a voice control function of the autonomous mobile device; collecting a second voice signal in the wake-up state of the voice control function; determining at least two working areas according to the second voice signal, comprising: when the second voice signal indicates a task forbidden area, determining the task forbidden area from an environment map, and determining the at least two working areas from areas outside the task forbidden area; sequentially executing tasks indicated by the second voice signal on the at least two working areas; the sequentially executing tasks indicated by the second voice signal on the at least two working areas comprises: determining a working mode according to the area category; and sequentially executing tasks on the at least two working areas according to the working mode; obtaining a control instruction according to the second voice signal, the control instruction being used for instructing the autonomous mobile device to control a specified device; moving to a specified area to complete control on the specified device.

2. The method of claim 1, wherein, the sequentially executing tasks indicated by the second voice signal on the at least two working areas comprises: determining an order of appearance of each working area in the at least two working areas in the second voice signal; sequentially executing tasks indicated by the second voice signal on the at least two working areas according to the order.

3. The method of claim 1, wherein, the sequentially executing tasks indicated by the second voice signal on the at least two working areas comprises: determining distances between the autonomous mobile device and each working area in the at least two working areas; sorting the at least two working areas according to distances from near to far to obtain a queue; sequentially executing tasks indicated by the second voice signal on the at least two working areas according to the queue.

4. The method according to any one of claims 1 to 3, characterized in that, the sequentially executing tasks indicated by the second voice signal on the at least two working areas comprises: stopping executing tasks before moving to a next working area after executing tasks on one working area in the at least two working areas.

5. The method according to any one of claims 1 to 3, characterized in that, the determining at least two working areas according to the second voice signal comprises: determining an area category according to the second voice signal; determining the at least two working areas from a region set corresponding to an environment map according to the area category.

6. The method of claim 5, wherein, the determining the at least two working areas from the environment map according to the area category comprises: when the area category indicates a target object, determining a region containing the target object from the environment map to obtain the at least two working areas.

7. The method of claim 5, wherein, before the determining at least two working areas according to the second voice signal, the method further comprises: dividing the environment map into a plurality of working areas to obtain the region set according to the environment map, position information of objects in the environment map, or position information of doors in the environment map; updating an identifier of each working area in the region set; sending update information to a voice recognition server to enable the voice recognition server to update the identifier of each working area.

8. The method according to any one of claims 1 to 3, characterized in that, The second voice signal is collected in the voice control function wake-up state, and the method comprises the following steps of: When the first voice signal matches the wake-up instruction of the autonomous mobile device, the autonomous mobile device is controlled to switch from the first working state to the second working state, the volume of the sound produced by the autonomous mobile device in the second working state is smaller than the volume of the sound produced by the autonomous mobile device in the first working state, and the wake-up instruction is used to wake up the voice control function of the autonomous mobile device; The second voice signal is collected in the second working state.

9. The method of claim 8, wherein, Further comprising: When the first voice signal matches the wake-up instruction of the autonomous mobile device, the sound source position of the first voice signal is determined; According to the sound source position, the autonomous mobile device is controlled to switch from the first pose to the second pose, the distance between the microphone and the sound source position when the autonomous mobile device is in the second pose is smaller than the distance between the microphone and the sound source position when the autonomous mobile device is in the first pose, and the microphone is a microphone arranged on the autonomous mobile device.

10. The method of claim 9, wherein, The autonomous mobile device is controlled to switch from the first pose to the second pose according to the sound source position, and the method comprises the following steps of: According to the sound source position, a rotation angle is determined, the rotation angle is used to indicate the angle that the autonomous mobile device needs to rotate when switching from the first pose to the second pose; The rotation angle is divided into a first angle and a second angle; The first speed is greater than the second speed.

11. The method of claim 9, wherein, After the autonomous mobile device is controlled to switch from the first working state to the second working state according to the sound source position when the first voice signal matches the wake-up instruction of the autonomous mobile device, the method further comprises the following steps of: After a preset time period, the voice control function is controlled to enter a waiting wake-up state.

12. The method according to any one of claims 1 to 3, characterized in that, Before the tasks indicated by the second voice signal are sequentially performed on the at least two working areas, the method further comprises the following steps of: It is determined whether the self-state of the autonomous mobile device meets the requirements of performing tasks; If the self-state does not meet the requirements of performing tasks, the autonomous mobile device is maintained.

13. The method according to any one of claims 1 to 3, characterized in that, According to the second voice signal, at least two working areas are determined, and the method comprises the following steps of: According to the second voice signal, a region category is determined; An image is collected; From the image, a region corresponding to the region category is determined to obtain the at least two working areas.

14. The method according to any one of claims 1 to 3, characterized in that, After the tasks indicated by the second voice signal are sequentially performed on the at least two working areas, the method further comprises the following steps of: It is determined whether the initial region has been executed completely, and the initial region is a region where the autonomous mobile device collects the second voice signal; If the initial region has not been executed completely, the initial region is returned to perform tasks.

15. An autonomous mobile device voice control apparatus, comprising: Comprise: The collection module is used for collecting the first voice signal; The processing module is used for waking up the voice control function of the autonomous mobile device when the first voice signal matches the wake-up instruction of the autonomous mobile device; The collection module is also used for collecting the second voice signal in the voice control function wake-up state. The processing module is further configured to determine at least two work areas according to the second voice signal, including: when the second voice signal indicates a task forbidden area, determining the task forbidden area from an environmental map, and determining the at least two work areas from areas outside the task forbidden area. The execution module is configured to sequentially execute tasks indicated by the second voice signal on the at least two work areas. The execution module is specifically configured to determine a work mode according to the area category, and sequentially execute tasks on the at least two work areas according to the work mode. The execution module is specifically configured to obtain a control instruction according to the second voice signal, where the control instruction is used to instruct the autonomous mobile device to control a specified device and move to a specified area to complete control on the specified device.

16. An electronic device comprising a processor, a memory, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to enable the electronic device to implement the method in any one of claims 1 to 14.

17. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method in any one of claims 1 to 14.

Images