Voice interaction method, electronic equipment and storage medium

Abstract

The invention discloses a voice interaction method, electronic equipment and a storage medium, and the voice interaction method comprises the steps: responding to a detected barrier-free event, determining a target window related to the barrier-free event, scanning the target window, and determining a control node included in the target window; and in response to the received user voice, scanning the control nodes, determining a target control node matched with the user voice in the control nodes, and performing corresponding control processing according to the target control node. Therefore, under the condition that the barrier-free event is triggered, the control nodes included in the target window related to the barrier-free event are scanned and stored in advance, and under the condition that the user voice is received, only the control node matched with the user voice needs to be searched from the part of control nodes to serve as the target control node. Therefore, corresponding control processing can be carried out, the scanning time of the control node in the voice interaction process is greatly shortened, the control information can be recognized in time, and the voice recognition and interaction efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice interaction method, electronic device and storage medium. Background Art

[0002] The barrier-free service of electronic devices is a voice interaction service designed for disabled users or users who are temporarily unable to fully interact with electronic devices, such as when they are driving. When the barrier-free service is turned on, the electronic device realizes the interaction between the user and the electronic device by scanning and identifying the control information included in the screen of the electronic device, as well as the content of the control on the screen of the electronic device spoken by the user, thereby realizing the voice see-say function. Based on this method, when encountering complex pictures (such as a large number of screen controls) and a large number of levels of pictures, the electronic device needs to scan and identify the control nodes (that is, the smallest unit that stores all the information of the control) one by one during the process of scanning the screen for control identification. The entire scanning process takes too long and the control information cannot be identified in time, which affects the efficiency and accuracy of voice recognition and interaction. Summary of the invention

[0003] The present application provides a voice interaction method, an electronic device and a storage medium, which are used to solve the problem in the prior art that, when an electronic device scans a screen to identify controls, it is necessary to scan and identify control nodes one by one, and the entire scanning process takes too long, and control information cannot be identified in time, affecting the efficiency and accuracy of voice recognition and interaction.

[0004] To solve the above technical problems, in the first aspect, the implementation method of the present application discloses a voice interaction method, which is applied to an electronic device, and the method includes: in response to detecting an accessibility event, determining a target window, where the target window is an associated window of the accessibility event; scanning the target window, and determining the control nodes included in the target window; in response to receiving a user voice, scanning the control nodes, and determining the target control node, where the target control node is a control node in the control nodes that matches the user voice; and performing corresponding control processing according to the target control node.

[0005] By adopting the above technical solution, in response to detecting an accessibility event, the associated target window of the accessibility event is determined, and then the control nodes included in the target window are determined by scanning. In response to receiving the user voice, the control nodes are scanned and the control nodes matching the user voice are determined as the target control nodes, and corresponding control processing is performed according to the target control nodes. Thus, in the case of an accessibility event being triggered, the control nodes included in the associated target window of the accessibility event are scanned and stored in advance. In the case of receiving the user voice, it is only necessary to find the control nodes matching the user voice from the part of the control nodes as the target control nodes, and then the corresponding control processing can be performed, that is, the WYSIWYG function is realized. When the user executes the WYSIWYG function, it is not necessary to scan all the screens (i.e., windows) and the control nodes included in each level to determine the control nodes matching the user voice. It is only necessary to scan the control nodes included in the target window related to the accessibility event, which greatly reduces the scanning time of the control nodes during the voice interaction process, can timely identify the control information, and improve the accuracy of voice recognition and interaction (i.e., WYSIWYG function).

[0006] According to another specific implementation of the present application, a voice interaction method disclosed in the implementation of the present application determines a target window in response to detecting an accessibility event, including: in response to detecting an accessibility event, determining the event type of the accessibility event; when the event type is a window change event, determining an application window corresponding to an application corresponding to the accessibility event as the target window; when the event type is a non-window change event, determining a current window corresponding to the accessibility event as the target window.

[0007] By adopting the above technical solution, in response to detecting an accessibility event, different target windows are determined according to the event type, so that the scanned control nodes are more accurate, thereby improving the accuracy of speech recognition and interaction (i.e., what you see is what you say function).

[0008] According to another specific implementation method of the present application, the implementation method of the present application discloses a voice interaction method, which includes: determining the application window corresponding to the application corresponding to the accessibility event in the following manner: obtaining the application information corresponding to the accessibility event, and traversing all window information displayed on the current desktop; determining the application window corresponding to the application corresponding to the accessibility event based on the application information and the window information.

[0009] By adopting the above technical solution, the application information corresponding to the accessibility event is obtained, and all window information displayed on the current desktop is traversed, and then the application window corresponding to the application corresponding to the accessibility event is determined according to the application information and window information. Thus, the accuracy of the determined target window is increased, and the accuracy of the control node obtained by scanning is further improved, thereby improving the accuracy of speech recognition and interaction (i.e., what you see is what you say function).

[0010] According to another specific implementation method of the present application, the implementation method of the present application discloses a voice interaction method, which includes: determining the current window corresponding to the accessibility event in the following manner: obtaining the source node information of the accessibility event; when it is determined according to the source node information that the source node does not have an accessibility service function turned on, obtaining and traversing all window information displayed on the current display desktop to determine the current window corresponding to the accessibility event.

[0011] By adopting the above technical solution, when it is determined according to the source node information that the source node of the accessibility event has not enabled the accessibility service function, all window information displayed on the current display desktop is obtained and traversed to determine the current window where the accessibility event occurs. Thus, the accuracy of the determined target window is increased, and the accuracy of the scanned control nodes is further improved, thereby improving the accuracy of speech recognition and interaction (i.e., what you see is what you say function).

[0012] According to another specific implementation of the present application, the implementation of the present application discloses a voice interaction method, which also includes: when the source node turns on the accessibility service function and the source node is visible, determining that the source node is a control node.

[0013] The above technical solution improves the accuracy of the control nodes obtained by scanning.

[0014] According to another specific implementation method of the present application, the implementation method of the present application discloses a voice interaction method, which scans the target window and determines the control nodes included in the target window, including: scanning the target window and determining the root node included in the target window; traversing the old control nodes determined when the previous accessibility event was triggered; obtaining the updated old control nodes based on the old control nodes; determining the control nodes included in the target window based on the root node and the updated old control nodes.

[0015] By adopting the above technical solution, old control nodes that do not exist in the corresponding window are deleted, thereby improving the accuracy of the obtained control nodes.

[0016] According to another specific implementation of the present application, a voice interaction method disclosed in the implementation of the present application obtains an updated old control node according to the old control node, including: deleting a first old control node in the old control node, the first old control node being an old control node that does not exist in the corresponding window; if it is determined that the second old control node is invisible, deleting the invisible second old control node to obtain an updated old control node, the second old control node being the old control node remaining after deleting the first old control node from the old control node; if it is determined that the second old control node is visible, determining whether the second old control node is allowed to be refreshed; if so, deleting the second old control node to obtain an updated old control node; if not, obtaining the updated old control node according to the second old control node.

[0017] By adopting the above technical solution, the accuracy of the updated old control node can be improved, thereby improving the accuracy of the obtained control node.

[0018] According to another specific implementation of the present application, a voice interaction method disclosed in the implementation of the present application determines a target window in response to detecting an accessibility event, including: in response to detecting an accessibility event, obtaining a currently displayed desktop and determining a target window.

[0019] In the second aspect, the implementation method of the present application also discloses a voice interaction device, including: a first processing module, used to determine a target window in response to detecting an accessibility event, the target window being the associated window of the accessibility event; a second processing module, used to scan the target window and determine the control nodes included in the target window; a third processing module, used to scan the control nodes in response to receiving a user voice, and determine the target control node, the target control node being the control node in the control nodes that matches the user voice; a fourth processing module, used to perform corresponding control processing according to the target control node.

[0020] In the third aspect, the implementation method of the present application also discloses an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores a computer program; the processor executes the computer program stored in the memory, so that the electronic device implements the voice interaction method provided by any implementation method of the first aspect mentioned above.

[0021] In a fourth aspect, the implementation method of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the voice interaction method provided in any one of the implementation methods of the first aspect above.

[0022] In a fifth aspect, an implementation method of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the voice interaction method provided in any one of the implementation methods of the first aspect above.

[0023] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can also be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of an accessibility service UI scanning level provided in an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of another barrier-free service UI scanning level provided in an embodiment of the present application;

[0026] Figure 3 It is a flowchart of a voice interaction method provided in an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a process for determining a target window provided in an embodiment of the present application;

[0028] Figure 5 It is a schematic diagram of a process of determining an application window corresponding to an application corresponding to an accessibility event provided by an embodiment of the present application;

[0029] Figure 6 It is a flowchart of determining the current window where an accessibility event occurs provided by an embodiment of the present application;

[0030] Figure 7 It is a flowchart of scanning a target window and determining control nodes included in the target window provided by an embodiment of the present application;

[0031] Figure 8 This is a flow chart of deleting a first old control node and obtaining an updated old control node provided by an embodiment of the present application;

[0032] Fig. 9 It is a flowchart of another voice interaction method provided in an embodiment of the present application;

[0033] Fig.10 is a structural schematic diagram of another voice interaction device provided in an embodiment of the present application;

[0034] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0035] Specific implementation method

[0036] As mentioned above, in the process of existing accessibility services (such as Android's accessibility services) between users and electronic devices to realize the voice see-say function, there is a problem that when the electronic device scans the screen to identify controls, it is necessary to scan and identify control nodes one by one. The entire scanning process takes too long and the control information cannot be identified in time, affecting the accuracy of voice recognition and interaction.

[0037] Specifically, the voice what you see is what you say function relies on the accessibility service to scan the information of the controls displayed on the screen, so as to meet the requirements of voice what you see is what you say command to recognize the control text and control. Figure 1 As shown, scanning nodes under all windows of a certain application (Application, App) in the display (such as Display1) where the accessibility event occurs (such as the root node (Root Node) under window window1 of App1, the root node (Root Node) under window window2, and the child nodes (Child Node) under the root node, and the sub-child nodes (SubChild Node) under the child nodes) takes about 200-500ms for a single App scan; scanning nodes of a certain window (such as the node Node under window window3 of App2) in the display (such as Display1) where the accessibility event occurs takes about 100-200ms for a single window scan; or as Figure 2 As shown, the scanning time for a single node is about 1-2ms when scanning a root node (such as the root node of window4 of App3) in the display (such as Display2) where the accessibility event occurs. When encountering complex screens (such as those with a lot of control information) and screens with many levels, if the traditional scanning method is used, the entire scanning process will be very long, and the longest time can reach 1.6s. Since the unscanned controls on the screen (also known as control text) cannot be recognized in time, the registration and recognition of text is delayed, which affects the recognition of the WYSIWYG command and affects the accuracy of the automatic speech recognition (ASR), that is, the WYSIWYG function.

[0038] Based on this, the present application provides a voice interaction method, an electronic device, and a storage medium. When an accessibility event is triggered, the control nodes included in the target window related to the accessibility event are scanned and stored in advance. When the user's voice is received, it is only necessary to find the control node that matches the user's voice from the part of the control nodes as the target control node, and then the corresponding control processing can be performed, that is, the WYSIWYG function is realized. When the user executes the WYSIWYG function, it is not necessary to scan all the screen images (i.e., windows) and the control nodes included in each level to determine the control nodes that match the user's voice. Only the control nodes included in the target window related to the accessibility event need to be scanned, which greatly reduces the scanning time of the control nodes during the voice interaction process, can timely identify the control information, and improve the accuracy of voice recognition and interaction (i.e., the WYSIWYG function).

[0039] Next, the voice interaction method provided by the implementation of the present application is described in detail in conjunction with the accompanying drawings.

[0040] The voice interaction method provided in the present application is applied to electronic devices, such as vehicle-mounted computers, computers, tablets, mobile phones and other electronic devices that can provide barrier-free services.

[0041] like Figure 3 As shown, in one implementation of the present application, the voice interaction method provided by the implementation of the present application specifically includes the following steps.

[0042] S100: In response to detecting an accessibility event, determining a target window.

[0043] The target window is the window associated with the accessibility event. For example, when an accessibility event occurs, if a new application is opened, the target window can be the corresponding window of the application; if no new application is opened, the corresponding window when the accessibility event occurs can be determined based on the source node information of the accessibility event.

[0044] S200, scanning a target window to determine control nodes included in the target window.

[0045] The determined control node is the control node that is most likely to participate in the voice interaction in the target window where the current accessibility event occurs. The identified control node is cached so that in the subsequent voice interaction process, the control node that matches the user's voice can be scanned more quickly and accurately, thereby improving the accuracy of voice recognition and interaction.

[0046] S300, in response to receiving a user voice, scanning control nodes and determining a target control node.

[0047] The target control node is a control node among the control nodes that matches the user's voice.

[0048] S400, performing corresponding control processing according to the target control node.

[0049] The control processing corresponding to the target control node determined by the user's voice is to achieve the control content of the electronic device when it is inconvenient for the user to fully interact with the electronic device at present.

[0050] The voice interaction method provided by the implementation of the present application, in response to detecting an accessibility event, determines the associated target window of the accessibility event, and then determines the control node included in the target window by scanning. In response to receiving the user voice, the control node is scanned and the control node matching the user voice is determined as the target control node, and the corresponding control processing is performed according to the target control node. Thus, in the case of an accessibility event being triggered, the control nodes included in the associated target window of the accessibility event are scanned and stored in advance. In the case of receiving the user voice, it is only necessary to find the control node matching the user voice from the part of the control nodes as the target control node, and then the corresponding control processing can be performed, that is, the WYSIWYG function is realized. It is not necessary to scan all the screens (i.e., windows) and the control nodes included in each level to determine the control node matching the user voice when the user executes the WYSIWYG function. It is only necessary to scan the control nodes included in the target window related to the accessibility event, which greatly reduces the scanning time of the control nodes during the voice interaction process, can timely identify the control information, and improve the accuracy of voice recognition and interaction (i.e., WYSIWYG function).

[0051] like Figure 4 As shown, in one implementation of the present application, in response to detecting an accessibility event, determining a target window includes the following steps.

[0052] S110: In response to detecting an accessibility event, determining an event type of the accessibility event.

[0053] S120: Determine whether the event type is a window change event, if so, proceed to step S130, if not, proceed to step S140.

[0054] S130: Determine the application window corresponding to the application corresponding to the accessibility event as the target window.

[0055] S140: Determine that the current window corresponding to the accessibility event is the target window.

[0056] If the event type is a window change event, it means that a new application has been opened and a new window has popped up or the active window of another application has been switched to. The current accessibility service will be performed on the new window, so the application window corresponding to the application corresponding to the barrier event is determined as the target window. If the event type is a non-window change event, it means that the current accessibility service is still performed on the original window, so the current window corresponding to the accessibility event is determined as the target window.

[0057] like Figure 5 As shown, in one implementation of the present application, the application window corresponding to the application corresponding to the accessibility event is determined through the following steps.

[0058] S131: Obtain application information corresponding to the accessibility event, and traverse all window information displayed on the current desktop.

[0059] S132: Determine an application window corresponding to the application corresponding to the accessibility event according to the application information and the window information.

[0060] Specifically, the App information where the accessibility event occurs can be first obtained, all windows on the current desktop displayed when the accessibility event occurs can be traversed and stored, and finally a window list corresponding to the App where the accessibility event occurs can be found to determine the application window.

[0061] like Figure 6 As shown, in one implementation of the present application, the current window corresponding to the accessibility event is determined through the following steps.

[0062] S141: Obtain source node information of accessibility events.

[0063] S142: When it is determined according to the source node information that the barrier-free service function is not enabled on the source node, all window information displayed on the current desktop is obtained and traversed to determine the current window corresponding to the barrier-free event.

[0064] In one implementation of the present application, when the barrier-free service function of the source node is turned on and the source node is visible, the source node is determined to be a control node.

[0065] In one implementation of the present application, when the barrier-free service function of the source node is turned on and the source node is not visible, the next node is scanned and it is determined whether the node is visible.

[0066] like Figure 7 As shown, in one implementation of the present application, scanning a target window and determining a control node included in the target window includes the following steps.

[0067] S210: Scan the target window to determine the root node included in the target window.

[0068] S220: Traverse the old control nodes determined when the previous accessibility event is triggered.

[0069] The previous accessibility event is the accessibility event that has been completed or conducted before the current accessibility event occurs, and the old control node is the control node included in the window related to the previous accessibility event stored when the previous accessibility event occurs.

[0070] S230: Obtain an updated old control node according to the old control node.

[0071] S240: Determine the control nodes included in the target window according to the root node and the updated old control nodes.

[0072] According to the root node included in the target window and the updated old control node, the control nodes included in the determined target window are more comprehensive and complete, which can effectively improve the accuracy of speech recognition and interaction.

[0073] like Figure 8 As shown, in one implementation of the present application, obtaining an updated old control node according to the old control node includes the following steps.

[0074] S231: Delete the first old control node in the old control nodes, where the first old control node is an old control node whose corresponding window does not exist. For example, the window corresponding to the first old control node has been closed, and the updated old control node is a control node whose corresponding window still exists and may participate in the currently occurring accessibility event.

[0075] S232: Determine whether the second old control node is invisible, if so, proceed to step S233, if not, proceed to step S234.

[0076] The second old control node is an old control node remaining after the first old control node is deleted from the old control nodes.

[0077] S233: Delete the invisible second old control node to obtain an updated old control node.

[0078] S234: Determine whether the second old control node is allowed to be refreshed, if so, proceed to step S235, if not, proceed to step S236.

[0079] S235: Delete the second old control node to obtain an updated old control node.

[0080] S236: Obtain an updated old control node according to the second old control node.

[0081] In one implementation of the present application, in response to detecting an accessibility event, determining a target window includes: in response to detecting an accessibility event, obtaining a current display desktop (i.e., a display desktop where the accessibility event occurs) and determining a target window.

[0082] In the case where the accessibility event is an Android accessibility service event, the voice interaction method provided by the present application can also be referred to as a method for accurately and efficiently scanning new nodes of a user interface (UI) control based on an Android accessibility service event. In one implementation of the present application, Fig. 9 As shown, the method includes the following steps.

[0083] S10: Determine whether an accessibility event is triggered (i.e. occurs).

[0084] S11: Obtain the display that triggers the accessibility event.

[0085] S12: Obtain the event type of the accessibility event.

[0086] S13: Determine whether it is a window change event according to the event type, if so, proceed to step S14, if not, proceed to step S20.

[0087] S14: Obtain the APP information where the accessibility event occurs.

[0088] S15: Traverse all windows in the display where the accessibility event occurs.

[0089] S16: Store all windows.

[0090] S17: Find the window list corresponding to the APP.

[0091] S18: Traverse the root node of the APP window.

[0092] S19: Determine whether the traversal of the node of the window is complete. If yes, proceed to step S31, if no, return to step S18.

[0093] S20: Obtain the source node information where the accessibility event occurs.

[0094] S21: Determine whether the node has accessibility support enabled. If so, proceed to step S22; if not, proceed to step S27.

[0095] S22: Execute root node=next node, that is, determine the next node as the root node.

[0096] S23: Scan the node.

[0097] S24: Check (i.e. determine) whether the node is visible, if yes, proceed to step S25, if no, return to step S22.

[0098] S25: store the node.

[0099] S26: Determine whether there are any nodes that have not been traversed. If so, return to step S22; if not, go to step S30.

[0100] S27: Get the window where the accessibility event occurs.

[0101] S28: Traverse all windows in the display where the accessibility event occurs.

[0102] S29: Find the corresponding window.

[0103] S30: Get the window root node.

[0104] S31: traverse all nodes in the cache (ie, old control nodes).

[0105] S32: Get the window corresponding to the node.

[0106] S33: Check (i.e. determine) whether the window of the cache node exists. If so, proceed to step S41; if not, proceed to step S34.

[0107] S34: Delete the node (ie, the first old control node).

[0108] S35: Determine whether the traversal is completed, if so, go to step S36, if not, return to step S31.

[0109] S36: traverse all nodes in the cache (ie, the second oldest control node).

[0110] S37: Check whether the cache node is visible, if so, go to step S38, if not, go to step S39.

[0111] S38: Determine whether the node can be refreshed, if so, go to step S39, if not, go to step S40.

[0112] S39: Delete the node.

[0113] S40: Determine whether the traversal is completed, if so, go to step S41, if not, return to step S36.

[0114] S41: Update and store a new node (ie, a control node).

[0115] The voice interaction method provided in this application, when scanning data, formulates an accurate scanning strategy based on the long-term use experience of actual accessibility services (such as Android accessibility services) and the accessibility event types and the applications, windows, screens, and root nodes that trigger accessibility events. At the same time, the old node removal strategy is used to greatly shorten the update time of control information.

[0116] The voice interaction method provided in this application may include:

[0117] Add node visibility judgment when scanning controls, and promptly exclude unnecessary scanning nodes to shorten the scanning time.

[0118] Use the three conditions of window invalidation + display + refresh to abandon the invalid old Node nodes in the screen.

[0119] Because the most important judgment when deleting a node is whether the node can be refreshed, but this operation is time-consuming. In order to minimize the execution of refresh operations, before judging whether to delete the old node, we will add whether the window where the node is located is still valid and whether it is displayed to reduce the refresh judgment. The two newly added strategies directly reduce 70% of the time consumption.

[0120] In another implementation of the present application, Fig.10 As shown, the present application also provides a voice interaction device, including: a first processing module, used to determine a target window in response to detecting an accessibility event, where the target window is an associated window of the accessibility event; a second processing module, used to scan the target window and determine the control nodes included in the target window; a third processing module, used to scan the control nodes in response to receiving a user voice, and determine the target control node, where the target control node is a control node in the control nodes that matches the user voice; and a fourth processing module, used to perform corresponding control processing according to the target control node.

[0121] See also Fig.11 , Fig.11 The figure shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.11 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0122] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes part of the technical solutions of the voice interaction method in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (NP), etc.; it can also be a digital data processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0123] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0124] By way of example and not limitation, the memory 123 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 123 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 123 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 123 is a non-volatile solid-state memory. In a particular embodiment, the memory 123 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0125] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0126] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0127] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the voice interaction method in the above embodiment.

[0128] An embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the technical solution of the voice interaction method of the above embodiment.

[0129] In some possible implementations, various aspects of the method provided in the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a processor of an electronic device, the program code is used to enable the processor of the electronic device to execute the steps of the method described above in this specification according to various exemplary implementations of the present application. For example, the electronic device can execute the voice interaction method recorded in the embodiments of the present application.

[0130] The program product may employ any combination of one or more readable media. The readable medium may be a readable data medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] The implementation method of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the voice interaction method in the above-mentioned embodiment.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable information processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable information processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable information processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable information processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary technical means in the art that are not disclosed in the present application.

[0136] It should be noted that, in addition to the implementation of the present application described in the above-mentioned specific embodiments, those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Although the description of the present application is introduced in conjunction with the preferred embodiment, this does not mean that the features of this invention are limited to the implementation. On the contrary, the purpose of introducing the invention in conjunction with the implementation is to cover other options or modifications that may extend based on the claims of the present application. In order to provide an in-depth understanding of the present application, the above description contains many specific details, and the present application may also be implemented without using these details. In addition, in order to avoid confusion or blurring the focus of the present application, some specific details will be omitted in the description. It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0137] It should be noted that in this specification, similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0138] Although the present application has been illustrated and described by referring to certain preferred implementations of the present application, it should be understood by those skilled in the art that the above contents are further detailed descriptions of the present application in combination with specific implementations, and it cannot be determined that the specific implementation of the present application is limited to these descriptions. Those skilled in the art may make various changes in form and details, including making several simple deductions or substitutions, without departing from the spirit and scope of the present application.

Claims

1. A voice interaction method, characterized in that: Applied to electronic equipment, the method comprises: In response to detecting an accessibility event, determining a target window, the target window being an associated window of the accessibility event; Scan the target window to determine the control nodes included in the target window; In response to receiving a user voice, scanning the control nodes and determining a target control node, the target control node being a control node among the control nodes that matches the user voice; Perform corresponding control processing according to the target control node.

2. The voice interaction method according to claim 1, characterized in that: In response to detecting an accessibility event, determining a target window includes: In response to detecting an accessibility event, determining an event type of the accessibility event; In a case where the event type is a window change event, determining an application window corresponding to the application corresponding to the accessibility event as the target window; When the event type is a non-window change event, the current window corresponding to the accessibility event is determined to be the target window.

3. The voice interaction method according to claim 2, characterized in that: The method includes determining an application window corresponding to the application corresponding to the accessibility event in the following manner: Obtain application information corresponding to the accessibility event, and traverse all window information currently displayed on the desktop; An application window corresponding to the application corresponding to the accessibility event is determined according to the application information and the window information.

4. The voice interaction method according to claim 2, characterized in that: The method includes determining the current window corresponding to the accessibility event in the following manner: Obtaining source node information of the accessibility event; When it is determined according to the source node information that the barrier-free service function is not enabled on the source node, all window information displayed on the current desktop is obtained and traversed to determine the current window corresponding to the barrier-free event.

5. The voice interaction method according to claim 4, characterized in that: The method further comprises: When the barrier-free service function is enabled on the source node and the source node is visible, the source node is determined to be the control node.

6. The voice interaction method according to any one of claims 1 to 5, characterized in that: Scanning the target window to determine the control nodes included in the target window includes: Scan the target window to determine the root node included in the target window; Traverse the old control nodes determined when the previous accessibility event was triggered; Obtain an updated old control node according to the old control node; The control nodes included in the target window are determined according to the root node and the updated old control nodes.

7. The voice interaction method according to claim 6, characterized in that: Obtaining an updated old control node according to the old control node includes: Deleting a first old control node in the old control nodes, where the first old control node is the old control node that does not exist in the corresponding window; If it is determined that the second old control node is invisible, the invisible second old control node is deleted to obtain an updated old control node, where the second old control node is an old control node remaining after the first old control node is deleted from the old control nodes; If it is determined that the second old control node is visible, determining whether the second old control node is allowed to be refreshed; If yes, then delete the second old control node to obtain an updated old control node; If not, an updated old control node is obtained according to the second old control node.

8. The voice interaction method according to any one of claims 1 to 7, characterized in that: In response to detecting an accessibility event, determining a target window includes: In response to detecting an accessibility event, the currently displayed desktop is acquired and the target window is determined.

9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores a computer program; The processor executes the computer program stored in the memory so that the electronic device implements the voice interaction method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, is used to implement the voice interaction method as described in any one of claims 1 to 8.

Images