Method, apparatus and device for assisting visually impaired user and storage medium

Abstract

Embodiments of the present disclosure provide a visual impairment user assistance method and device, equipment and storage medium, which are applied to the field of robot technology. The method is applied to a robot and includes: receiving a voice assistance instruction issued by a visual impairment user; analyzing the voice assistance instruction to generate an assistance task; decomposing the assistance task into steps to obtain a plurality of assistance steps and an order of the plurality of assistance steps, and identifying an execution device for each assistance step; and calling the execution device for each assistance step in sequence according to the order of the plurality of assistance steps to execute the corresponding assistance step. In this way, the execution device corresponding to each assistance step in the assistance task can be called by the robot in sequence to execute the assistance step, linkage between devices is achieved, and therefore a complex assistance task can be completed conveniently, and the effect of the robot assisting the visual impairment user is improved.

Description

Technical Field

[0001] This disclosure relates to the field of robotics, and more particularly to a method, apparatus, device, and storage medium for assisting visually impaired users. Background Technology

[0002] Currently, there are tens of millions of visually impaired users in my country, whose lives are fraught with difficulties due to their visual impairment. While home mobile robots have seen continuous development, providing simple assistance such as navigation, they often fall short in handling complex tasks, particularly those involving multiple steps. Therefore, improving the effectiveness of robot assistance for visually impaired users has become a pressing technical challenge. Summary of the Invention

[0003] Embodiments of this disclosure provide a method, apparatus, device, and storage medium for assisting visually impaired users.

[0004] In a first aspect, embodiments of this disclosure provide a method for assisting visually impaired users, the method being applied to a robot, comprising:

[0005] Receive voice assistance commands from visually impaired users;

[0006] Analyze voice assistance commands to generate assistance tasks;

[0007] The assistance task is decomposed into steps to obtain multiple assistance steps and their order, and the execution devices of each assistance step are identified.

[0008] The execution devices for each assistance step are called sequentially according to the order of the multiple assistance steps to execute the corresponding assistance steps.

[0009] In some possible implementations of the first aspect, the method further includes, prior to receiving voice assistance commands from a visually impaired user:

[0010] Receive voice wake-up commands from users;

[0011] Voiceprint feature extraction for voice wake-up commands;

[0012] The extracted voiceprint features are matched with the pre-stored voiceprint features of visually impaired users. If the match is successful, the user is identified as a visually impaired user.

[0013] Among some possible implementations of the first aspect, the method also includes:

[0014] Based on the voice wake-up command, the sound source is located to determine its position.

[0015] If a match is successful, the user is identified as a visually impaired user, including:

[0016] If a match is successful, the device moves to a preset range of the sound source location and uses a camera to detect faces at the sound source location. If the detected face matches a pre-stored face of a visually impaired user, the user is identified as a visually impaired user.

[0017] In some possible implementations of the first aspect, the voice assistance instructions are analyzed to generate assistance tasks, including:

[0018] Speech recognition is performed on voice assistance commands to obtain text information;

[0019] User intent is obtained by analyzing text information using a user intent analysis model.

[0020] Generate assistance tasks based on user intent.

[0021] In some possible implementations of the first aspect, the execution devices for multiple assisting steps are identified, including:

[0022] For any assistance step, identify the functions required to execute the assistance step;

[0023] Locate any available devices with the necessary functions and use them as the devices to perform the assisting steps.

[0024] In some possible implementations of the first aspect, after sequentially invoking the execution device of each assistance step in the order of the multiple assistance steps to execute the corresponding assistance step, the method further includes:

[0025] Based on the results of each assistance step, generate an assistance task completion report;

[0026] The task completion report is delivered via voice broadcast.

[0027] In some possible implementations of the first aspect, after assisting with the task completion report via voice broadcast, the method further includes:

[0028] Receive voice improvement commands from visually impaired users regarding reports on the completion of assisted tasks;

[0029] The assistance steps that correspond to the voice improvement instructions among multiple assistance steps are identified as the assistance steps to be improved.

[0030] The device executing the assistance steps to be improved is improved in a manner that is based on the voice improvement instructions.

[0031] Secondly, embodiments of this disclosure provide an assistive device for visually impaired users, which is applied to a robot and includes:

[0032] The receiving module is used to receive voice assistance commands issued by visually impaired users;

[0033] The analysis module is used to analyze voice assistance commands and generate assistance tasks;

[0034] The decomposition module is used to decompose the collaborative task into steps, obtain multiple collaborative steps and their order, and identify the execution devices of each collaborative step.

[0035] The calling module is used to sequentially call the execution device of each assistance step to execute the corresponding assistance step according to the order of multiple assistance steps.

[0036] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods described above.

[0037] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described above.

[0038] In the embodiments of this disclosure, the robot receives voice assistance instructions from a visually impaired user; analyzes the voice assistance instructions to generate an assistance task; decomposes the assistance task into steps to obtain multiple assistance steps and their order, and identifies the execution devices for each assistance step; and sequentially calls the execution devices for each assistance step to execute the corresponding assistance step according to the order of the multiple assistance steps.

[0039] In this way, the robot can sequentially call the execution devices corresponding to each step in the assistance task to perform the steps, realizing the linkage between different devices, which makes it easier to complete more complex assistance tasks and improves the effectiveness of the robot in assisting visually impaired users.

[0040] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0041] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0042] Figure 1 A flowchart illustrating an embodiment of the present disclosure provides a method for assisting visually impaired users;

[0043] Figure 2 A structural diagram of an assistive device for visually impaired users provided in an embodiment of this disclosure is shown;

[0044] Figure 3 A structural diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0046] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0047] To address the problems in the background art, embodiments of this disclosure provide a method, apparatus, device, and storage medium for assisting visually impaired users. Specifically, a robot receives voice assistance commands from a visually impaired user; analyzes the voice assistance commands to generate an assistance task; decomposes the assistance task into multiple assistance steps and their order, and identifies the execution devices for each assistance step; and sequentially calls the execution devices of each assistance step to execute the corresponding assistance step according to the order of the multiple assistance steps.

[0048] In this way, the robot can sequentially call the execution devices corresponding to each step in the assistance task to perform the steps, realizing the linkage between different devices, which makes it easier to complete more complex assistance tasks and improves the effectiveness of the robot in assisting visually impaired users.

[0049] The following detailed description, with reference to the accompanying drawings, of the visually impaired user assistance method, apparatus, device, and storage medium provided by the embodiments of this disclosure through specific examples, will be provided in detail.

[0050] Figure 1 A flowchart illustrating an embodiment of an assistance method for visually impaired users provided in this disclosure is shown, such as... Figure 1 As shown, the visually impaired user assistance method 100 can be applied to a robot and includes the following steps:

[0051] S110 receives voice assistance commands from visually impaired users.

[0052] In some embodiments, before S110, a voice wake-up command issued by the user can be received, voiceprint features can be extracted from the voice wake-up command, and the extracted voiceprint features can be matched with pre-stored voiceprint features of visually impaired users. If the match is successful, the user can be accurately identified as a visually impaired user. At this time, the visually impaired user mode can be activated to provide assistance services to the visually impaired user.

[0053] It is worth noting that the voiceprint features involved here can be sound waveforms or sound spectrum diagrams, or image features in sound waveforms or sound spectrum diagrams, and there are no restrictions here.

[0054] Furthermore, to improve the accuracy of identifying visually impaired users, sound source localization can be performed based on voice wake-up commands (e.g., sound source localization based on time difference of arrival or time of arrival algorithms) to accurately determine the sound source location. Upon successful voiceprint feature matching, the system moves to a preset range (e.g., a range of 2m to 3m around the sound source location) and points the camera at the sound source location to perform face detection. If the detected face matches a pre-stored face of a visually impaired user, the user is identified as visually impaired.

[0055] In addition, the user's location can be determined through distance sensors or wearable smart devices (such as smartwatches, smart bracelets, and VR glasses). When the voiceprint feature is successfully matched, the camera moves to a preset range (e.g., a range of 2 to 3 meters around the user's location) and points at the user's location to perform face detection. If the detected face matches the pre-stored face of a visually impaired user, the user is identified as visually impaired.

[0056] S120 analyzes the voice assistance commands and generates assistance tasks.

[0057] In some embodiments, voice assistance commands can be recognized to obtain corresponding text information. Then, a user intent analysis model can be used to analyze the text information to obtain the user intent. Based on the user intent, assistance tasks, i.e. assistance plans, can be effectively generated.

[0058] The user intent analysis model can be trained through the following steps:

[0059] Obtain a user intent analysis training set. This training set includes multiple user intent analysis samples and their corresponding labels. Specifically, the user intent analysis samples include the text information corresponding to the user's voice assistance commands, and the labels include the user intent they intended to express when issuing the voice assistance commands.

[0060] The pre-defined neural network (e.g., deep neural network, convolutional neural network, generative adversarial neural network) is trained based on the user intent analysis training set, and the trained neural network is used as the user intent analysis model.

[0061] It is understood that assistance tasks include, but are not limited to, assisting in guiding and directing user behavior (such as directing users to move, use items and tools, etc.) and directly performing certain actions on behalf of users (such as delivering items and tools, opening curtains, feeding pets, etc.).

[0062] S130, decompose the assistance task into steps to obtain multiple assistance steps and their order, and identify the execution devices of each assistance step.

[0063] In some embodiments, the assistance task can be decomposed into steps using a task decomposition algorithm to obtain multiple assistance steps and their order. Then, for any assistance step, the function required for its execution is identified, an idle device with that function is found, and that device is used as the execution device for the assistance step, thereby quickly identifying the execution devices for multiple assistance steps.

[0064] For example, if an air purification function is required when a certain assistance step is performed, an available device with an air purification function can be found and used as the device to perform the assistance step.

[0065] It should be noted that robots can also be selected as execution devices for assisting steps, for example, when the functions required for the execution of assisting steps include the following:

[0066] (1) Guide visually impaired users to move to a designated location via voice, continuously identify the surrounding environment during the process, and assist users in avoiding obstacles.

[0067] (2) Identify the location of the items or tools that the visually impaired user wants to take, and guide the visually impaired user to take them through voice.

[0068] (3) Guide and assist visually impaired users to use tools in the correct position and direction.

[0069] (4) Tools for delivering items, etc.

[0070] S140, the execution devices of each assistance step are called sequentially according to the order of the multiple assistance steps to execute the corresponding assistance steps.

[0071] As an example, suppose there are 5 assistive steps, in the following order: Step 1, Step 2, Step 3, Step 4, and Step 5. In this case, the execution device for Step 1 can be invoked first to execute Step 1. After Step 1 is completed, the execution device for Step 2 can be invoked to execute Step 2, and so on, until all steps are executed.

[0072] As another example, suppose there are 5 assistive steps, in the order of steps 1-2, 3-4, and 5. Steps 1-2 are at the first level and are executed in parallel; steps 3-4 are at the second level and are executed in parallel; and step 5 is at the third level. In this case, the execution device for step 1-2 can be invoked first to execute step 1-2 in parallel. After step 1-2 is completed, the execution device for step 3-4 can be invoked to execute step 3-4 in parallel, and so on, until all steps are executed.

[0073] In the embodiments of this disclosure, the robot can sequentially call the execution devices corresponding to each step in the assistance task to execute the steps, thereby realizing the linkage between different devices, which facilitates the completion of highly complex assistance tasks and improves the effectiveness of the robot in assisting visually impaired users.

[0074] It is worth noting that, after sequentially calling the execution devices of each assistance step in the order of the multiple assistance steps to perform the corresponding assistance steps, the visually impaired user assistance method 100 may further include:

[0075] Based on the results of each assistance step, an assistance task completion report is generated and broadcast via voice so that users can promptly obtain information on the completion status of the assistance task.

[0076] Furthermore, in order to improve the execution effect of subsequent assistance steps, voice improvement instructions issued by visually impaired users in response to assistance task completion reports can be received. The assistance steps corresponding to the voice improvement instructions among multiple assistance steps are identified as assistance steps to be improved. The method used by the execution device to perform the assistance steps to be improved according to the voice improvement instructions is improved to make it more user-friendly.

[0077] The following is a detailed description of the assistance method for visually impaired users provided in this disclosure, using a specific embodiment as an example:

[0078] (1) Receive the user's voice wake-up command.

[0079] (2) Extract voiceprint features from the voice wake-up command, match the extracted voiceprint features with the pre-stored voiceprint features of visually impaired users, and if the match is successful, locate the sound source according to the voice wake-up command, determine the sound source location, move to the preset range of the sound source location, point the camera at the sound source location, and perform face detection at the sound source location. If the detected face is consistent with the pre-stored face of a visually impaired user, then the user is determined to be a visually impaired user.

[0080] (3) Receive voice assistance commands from visually impaired users.

[0081] (4) Perform speech recognition on the voice assistance instructions to obtain the corresponding text information, and then use the user intent analysis model to perform user intent analysis on the text information to obtain the user intent, and then generate the assistance task based on the user intent.

[0082] (5) Decompose the assistance task into steps to obtain multiple assistance steps and their order, and identify the execution devices of each assistance step.

[0083] (6) In the order of multiple assistance steps, call the execution device of each assistance step to execute the corresponding assistance step.

[0084] (7) Generate an assistance task completion report based on the execution results of each assistance step, and broadcast the assistance task completion report via voice so that users can obtain the assistance task completion status in a timely manner.

[0085] (8) Receive voice improvement instructions from visually impaired users for assistance task completion reports, identify the assistance steps corresponding to the voice improvement instructions among multiple assistance steps as assistance steps to be improved, and improve the method adopted by the execution device of the assistance steps to be improved according to the voice improvement instructions.

[0086] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0087] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0088] Figure 2 A structural diagram of an assistive device for visually impaired users provided in an embodiment of this disclosure is shown, such as... Figure 2As shown, the assistive device 200 for visually impaired users may include:

[0089] The receiving module 210 is used to receive voice assistance commands issued by visually impaired users.

[0090] Analysis module 220 is used to analyze voice assistance commands and generate assistance tasks.

[0091] The decomposition module 230 is used to decompose the assistance task into steps, obtain multiple assistance steps and their order, and identify the execution devices of each assistance step.

[0092] The calling module 240 is used to sequentially call the execution device of each assistance step to execute the corresponding assistance step according to the order of multiple assistance steps.

[0093] In some embodiments, the receiving module 210 is further configured to:

[0094] Before receiving voice assistance commands from visually impaired users, receive voice wake-up commands from users.

[0095] The assistive device 200 for visually impaired users also includes:

[0096] The extraction module is used to extract voiceprint features from voice wake-up commands.

[0097] The matching module is used to match the extracted voiceprint features with the pre-stored voiceprint features of visually impaired users. If the match is successful, the user is identified as a visually impaired user.

[0098] In some embodiments, the assistive device 200 for visually impaired users further includes:

[0099] The positioning module is used to locate the sound source based on the voice wake-up command and determine the location of the sound source.

[0100] The matching module is specifically used for:

[0101] If a match is successful, the device moves to a preset range of the sound source location and uses a camera to detect faces at the sound source location. If the detected face matches a pre-stored face of a visually impaired user, the user is identified as a visually impaired user.

[0102] In some embodiments, the analysis module 220 is specifically used for:

[0103] Speech recognition is performed on voice assistance commands to obtain text information;

[0104] User intent is obtained by analyzing text information using a user intent analysis model.

[0105] Generate assistance tasks based on user intent.

[0106] In some embodiments, the decomposition module 230 is specifically used for:

[0107] For any assistance step, identify the functions required to execute the assistance step;

[0108] Locate any available devices with the necessary functions and use them as the devices to perform the assisting steps.

[0109] In some embodiments, the assistive device 200 for visually impaired users further includes:

[0110] The generation module is used to generate an assistance task completion report based on the execution results of each assistance step after sequentially calling the execution device of each assistance step in the order of multiple assistance steps.

[0111] The broadcast module is used to assist in completing task reports through voice broadcast.

[0112] In some embodiments, the receiving module 210 is further configured to:

[0113] After the task completion report is read aloud via voice, the system receives voice improvement instructions from visually impaired users regarding the task completion report.

[0114] The assistive device 200 for visually impaired users also includes:

[0115] The determination module is used to identify the assistance steps that correspond to the voice improvement instructions among multiple assistance steps as the assistance steps to be improved.

[0116] An improvement module is used to improve the way the execution device performs the assistance steps to be improved according to the voice improvement instructions.

[0117] Understandable Figure 2 Each module / unit in the visually impaired user assistance device 200 shown has the ability to implement Figure 1 The functions of each step in the visually impaired user assistance method 100 shown, and the corresponding technical effects they achieve, will not be elaborated here for the sake of brevity.

[0118] Figure 3A structural diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Electronic device 300 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic device 300 may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] like Figure 3 As shown, the electronic device 300 may include a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0120] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer program product, including a computer program tangibly contained in a computer-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0122] The various embodiments described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this disclosure, a computer-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0125] It should be noted that this disclosure also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute method 100 and achieve the corresponding technical effects achieved by the embodiments of this disclosure in executing the method. For the sake of brevity, these will not be elaborated here.

[0126] In addition, this disclosure also provides a computer program product including a computer program that implements method 100 when executed by a processor.

[0127] To provide interaction with a user, the embodiments described above can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0128] The embodiments described above can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with the implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0129] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0130] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0131] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for assisting visually impaired users, characterized in that, The method is applied to robots, including: Receive voice assistance commands from visually impaired users; The voice assistance instructions are analyzed to generate assistance tasks; The assistance task is decomposed into steps to obtain multiple assistance steps and their order, and the execution devices of each of the multiple assistance steps are identified. The execution devices for each assistance step are invoked sequentially according to the order of the multiple assistance steps to execute the corresponding assistance steps.

2. The method according to claim 1, characterized in that, Prior to receiving voice assistance commands from a visually impaired user, the method further includes: Receive voice wake-up commands from users; Voiceprint features are extracted from the voice wake-up command; The extracted voiceprint features are matched with pre-stored voiceprint features of visually impaired users. If the match is successful, the user is identified as a visually impaired user.

3. The method according to claim 2, characterized in that, The method further includes: The sound source is located based on the voice wake-up command; If a match is successful, the user is determined to be a visually impaired user, including: If a match is successful, the user moves to a preset range within the sound source location and performs face detection at the sound source location using a camera. If the detected face matches a pre-stored face of a visually impaired user, the user is identified as a visually impaired user.

4. The method according to claim 1, characterized in that, The step of analyzing the voice assistance command and generating an assistance task includes: The voice assistance command is subjected to speech recognition to obtain text information; The user intent is obtained by performing user intent analysis on the text information using a user intent analysis model. Based on the user's intent, generate an assistance task.

5. The method according to claim 1, characterized in that, The execution devices that respectively identify the plurality of assisted steps include: For any assistance step, identify the functions required when the assistance step is executed; Locate an idle device with the aforementioned function and use it as the device to perform the assisting step.

6. The method according to claim 1, characterized in that, After the execution devices for each assistance step are sequentially invoked according to the order of the plurality of assistance steps to execute the corresponding assistance steps, the method further includes: Based on the results of each assistance step, generate an assistance task completion report; The report on the completion of the assisted task will be broadcast via voice.

7. The method according to claim 6, characterized in that, After broadcasting the assistance task completion report via voice, the method further includes: Receive voice improvement instructions from the visually impaired user in response to the completion report of the assistance task; The assistance step corresponding to the voice improvement instruction among the plurality of assistance steps is identified as the assistance step to be improved. The method by which the execution device of the assistance step to be improved performs the assistance step to be improved is improved according to the voice improvement instruction.

8. An assistive device for visually impaired users, characterized in that, The device is used in robots and includes: The receiving module is used to receive voice assistance commands issued by visually impaired users; The analysis module is used to analyze the voice assistance instructions and generate assistance tasks; The decomposition module is used to decompose the assistance task into steps, obtain multiple assistance steps and the order of the multiple assistance steps, and identify the execution devices of the multiple assistance steps respectively. The calling module is used to sequentially call the execution device of each assistance step to execute the corresponding assistance step according to the order of the multiple assistance steps.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images