Voice Control Method, Device, Electronic Device and Storage Medium

Abstract

The embodiments of the present application disclose a voice control method, device, electronic device and storage medium. The method includes: obtaining a first control identifier and a second control identifier from the acquired voice control instruction; if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the second control identifier, the target control is determined from the multiple controls corresponding to the first control identifier, and the target interface is the interface displayed when the voice control instruction is acquired, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control; and execute the control operation corresponding to the target control. Thus, the electronic device can more accurately determine the user's actual control intention through the above method.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a voice control method, device, electronic device, and storage medium. Background Art

[0002] Combining artificial intelligence technology and virtual personal assistants (VPAs) allows electronic devices to receive user voice commands through the auditory modality and complete corresponding interactive tasks. However, in many cases, users only clarify their interaction intentions after seeing the interactive interface and hope to directly operate the interactive interface or the objects in it. In addition, in some cases, there may be multiple controls in the interactive interface that match the voice commands triggered by the user, which will make it impossible for the electronic device to accurately determine the user's actual control intention. Summary of the Invention

[0003] In view of the above problems, the present application proposes a voice control method, device, electronic device and storage medium to improve the above problems.

[0004] In the first aspect, the present application provides a voice control method, the method comprising: obtaining a first control identifier and a second control identifier from an acquired voice control instruction; if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, the target interface being the interface displayed when the voice control instruction is acquired, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control; and executing the control operation corresponding to the target control.

[0005] In the second aspect, the present application provides a voice control device, which includes: an identifier acquisition unit, which is used to obtain a first control identifier and a second control identifier from the acquired voice control instruction; a control determination unit, which is used to determine the target control from the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target interface is the interface displayed when the voice control instruction is acquired, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control; a control unit, which is used to execute the control operation corresponding to the target control.

[0006] In a third aspect, the present application provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores program code, wherein the above method is executed when the program code is run.

[0008] The present application provides a voice control method, device, electronic device and storage medium, which first obtains a first control identifier and a second control identifier from an acquired voice control instruction, wherein the control corresponding to the first control identifier is a control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as a target control. If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control will be determined from the multiple controls corresponding to the first control identifier based on the second control identifier, and the control operation corresponding to the target control will be performed. Thus, in the above manner, when there are multiple controls to be determined corresponding to the voice control instruction (controls corresponding to the first control identifier) and the user's actual control intention cannot be clearly determined, the multiple controls to be determined can be determined with the help of the control corresponding to the second control identifier, thereby determining the control representing the user's actual control purpose from the multiple controls to be determined as the target control, thereby enabling the electronic device to accurately determine the user's actual control intention. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A schematic diagram showing an application scenario of a voice control method proposed in an embodiment of the present application is shown;

[0011] Figure 2 A schematic diagram showing an application scenario of another voice control method proposed in an embodiment of the present application is shown;

[0012] Figure 3 A flow chart of a voice control method proposed in an embodiment of the present application is shown;

[0013] Figure 4 A schematic diagram showing a user triggering a voice control command in an embodiment of the present application is shown;

[0014] Figure 5 A flow chart of a voice control method proposed in another embodiment of the present application is shown;

[0015] Figure 6 A schematic diagram of a target interface in an embodiment of the present application is shown;

[0016] Figure 7 A schematic diagram showing a control relationship structure diagram in an embodiment of the present application is shown;

[0017] Figure 8 A schematic diagram showing the relative positions of a control in an embodiment of the present application;

[0018] Figure 9 A flowchart of an implementation of S220 in an embodiment of the present application is shown;

[0019] Figure 10 A schematic diagram showing another target interface in an embodiment of the present application is shown;

[0020] Figure 11 A schematic diagram showing another control relationship structure diagram in an embodiment of the present application is shown;

[0021] Figure 12 A schematic diagram showing another control relationship structure diagram in an embodiment of the present application is shown;

[0022] Figure 13 A flow chart of a voice control method proposed in another embodiment of the present application is shown;

[0023] Figure 14A schematic diagram showing distance display in an embodiment of the present application is shown;

[0024] Figure 15 The following is a structural block diagram of a target object recognition device proposed in an embodiment of the present application;

[0025] Figure 16 Shows a structural block diagram of an electronic device proposed in this application;

[0026] Figure 17 It is a storage unit of an embodiment of the present application for storing or carrying program codes for implementing the voice control method according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0028] The widespread use of smart devices has brought numerous conveniences to our lives. Combining artificial intelligence (AI) with virtual personal assistants (VPAs) allows electronic devices to receive user voice commands through the auditory modality and complete corresponding interactive tasks. However, in many cases, users only clarify their interaction intentions after seeing the interactive interface and want to directly operate on the interface or objects within it.

[0029] However, the inventors discovered in their research that in some cases, there may be multiple controls in the interactive interface that match the voice command triggered by the user, which can cause the electronic device to be unable to accurately determine the user's actual control intent. Specifically, in the interface displayed by the electronic device, there may be multiple controls with the same name. Furthermore, the electronic device also recognizes that the voice control command sent by the user includes these multiple controls with the same name. Therefore, the electronic device may not be able to accurately determine which control the user actually intended to operate, making it impossible for the electronic device to accurately determine the user's actual control intent.

[0030] Therefore, the inventors proposed a voice control method, device, electronic device and storage medium in the present application. The method first obtains a first control identifier and a second control identifier from the acquired voice control instruction. When the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control, if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control will be determined from the multiple controls corresponding to the first control identifier based on the second control identifier, and the control operation corresponding to the target control will be executed.

[0031] Therefore, through the above method, when there are multiple controls to be determined (controls corresponding to the first control identifier) corresponding to the voice control instruction and the user's actual control intention cannot be clearly determined, the multiple controls to be determined can be determined with the help of the controls corresponding to the second control identifier, so as to determine the control representing the user's actual control purpose as the target control from the multiple controls to be determined, so that the electronic device can accurately determine the user's actual control intention.

[0032] The following first introduces the application scenarios involved in the embodiments of this application.

[0033] In the embodiment of the present application, the voice control method provided can be executed by an electronic device. In this manner, all steps in the voice control method provided by the embodiment of the present application can be executed by the electronic device. For example, Figure 1 As shown, the voice collection device of the electronic device 100 can collect voice control instructions, and transmit the collected voice collection instructions and the target interface to the processor, so that the processor can obtain the first control identifier and the second control identifier from the obtained voice control instructions, and then the processor uses the first control identifier and the second control identifier to determine the target control from the target interface to execute the control operation corresponding to the target control.

[0034] Furthermore, the voice control method provided in the embodiment of the present application can also be executed by a server. Correspondingly, in this way executed by the server, the electronic device can collect voice instructions, and the collected voice instructions and the target interface can be sent to the server synchronously, and then the server executes the voice control method provided in the embodiment of the present application to determine the target control, and then the server triggers the electronic device to perform the control operation corresponding to the target control. In addition, it can also be performed collaboratively by the electronic device and the server. In this way executed collaboratively by the electronic device and the server, some steps in the voice control method provided in the embodiment of the present application are performed by the electronic device, while other steps are performed by the server.

[0035] For example, Figure 2 As shown, the electronic device 100 can execute the voice control method including: obtaining a first control identifier and a second control identifier from the acquired voice control instruction, and then the server 200 executes if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, based on the control corresponding to the second control identifier, determining the target control from the multiple controls corresponding to the first control identifier, and generating a corresponding control instruction based on the target control, and then returning the generated control instruction to the electronic device 100, and triggering the electronic device 100 to execute the received control instruction.

[0036] It should be noted that in this method of collaborative execution by the electronic device and the server, the steps respectively executed by the electronic device and the server are not limited to the methods introduced in the above examples. In actual applications, the steps respectively executed by the electronic device and the server can be dynamically adjusted according to actual conditions.

[0037] The following describes the embodiments of the present application in conjunction with the accompanying drawings.

[0038] See also Figure 3 , the present application provides a voice control method, the method comprising:

[0039] S110: Acquire a first control identifier and a second control identifier from the acquired voice control instruction.

[0040] In an embodiment of the present application, the user can express his or her control target through voice. Correspondingly, the electronic device can use the voice issued by the user as a voice control instruction, and then determine the user's control target based on the received voice control instruction. Among them, the control target can be understood as the control that the user actually wants to operate in the interface displayed by the electronic device. It should be noted that when the user is using the electronic device, he or she may be talking and sending voice information. However, when the user sends the voice information, he or she may just be talking to others, and not necessarily wanting to control the electronic device. In order to avoid misidentification by the electronic device, the electronic device can start to obtain voice control instructions after obtaining the specified voice content. Among them, the specified voice content can be configured by the user according to his or her needs.

[0041] After obtaining the voice control instruction, it is possible to further obtain, from the voice control instruction, an identifier for obtaining the control target corresponding to the voice control instruction as the first control identifier, and an identifier for confirming the control corresponding to the control target of the voice control instruction as the second control identifier. That is to say, the second control identifier herein can be an identifier for assisting in confirming the control actually corresponding to the first control identifier.

[0042] Moreover, in the embodiments of this application, there can be multiple ways to obtain the first control identifier and the second control identifier.

[0043] As one way, the voice control instruction can be converted into corresponding text content, and then semantic understanding can be performed on the text content to obtain the first control identifier and the second control identifier. In this way, semantic extraction rules can be established in advance, and then identifiers can be obtained from the text content based on these semantic extraction rules. It should be noted that through research, the inventor found that the sentence patterns used by users when triggering voice control instructions are relatively fixed. For example, if a user hopes to download application A, the sentence pattern triggered may be "click the download button of application A", and this sentence pattern can be summarized as "action verb + {XXX} +'s + {XXX}". Or, the triggered sentence pattern can be "download application A". This sentence pattern can be summarized as "action verb + {XXX}".

[0044] In this way, after obtaining the text content, the words representing action verbs in the text content can be obtained based on the semantic extraction rules, and then the first control identifier and the second control identifier can be determined according to the sequential relationship before and after the action verb. For example, if the text content converted from the voice control instruction successfully matches the sentence pattern "action verb + {XXX} +'s + {XXX}", the content in the first "{XXX}" after the action verb can be used as the second control identifier, and the second "{XXX}" after the action verb can be used as the first control identifier. If the text content converted from the voice control instruction successfully matches the sentence pattern "action verb + {XXX}", the action verb can be recognized as the first control identifier, and the "{XXX}" after the action verb can be recognized as the second control identifier.

[0045] As another way, if the text content converted from the voice control instruction cannot be successfully matched with the pre-configured sentence pattern, the first control identifier and the second control identifier in the text content converted from the instruction can be extracted through a pre-trained neural network model.

[0046] S120: If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control is determined from the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, and the target interface is the interface displayed when the voice control instruction is obtained, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control.

[0047] Optionally, the target interface is the interface displayed by the electronic device when the voice control command is obtained. After receiving the voice control command, the electronic device can simultaneously start to identify the target interface to obtain the controls included in the target interface. In addition, in the embodiment of the present application, the controls included in the target interface can be identified in a variety of ways.

[0048] As a method, the target interface can be identified by code parsing. Optionally, the target interface can be identified by code parsing based on Google Accessibility. In this method, the identified control can correspond to the control ID, type, and description information, etc. Among them, the description information corresponding to the control is used to characterize the operations that can be implemented by the control. For example, if the control is a name used to characterize an application, the description information of the control will include the name of the represented application. Furthermore, if the control is used to trigger the download of the application, the description information of the control includes download.

[0049] As another method, the target interface can be identified by image and text recognition (for example, optical character recognition). In this method, a screenshot of the interface currently displayed by the electronic device can be taken. Then, image and text recognition is performed on the screenshot. In this method, the identified control can correspond to the control's location and the control's description information. In addition, in this method, the control's description information can include the text displayed in the control.

[0050] As another method, the target interface can be identified through icon recognition. In this method, a screenshot of the interface currently displayed on the electronic device can also be taken. The screenshot image can then be used for icon recognition. In this method, the identified control can be associated with the control's location and control description information. In addition, in this method, the control description information can include a description of the function of the identified control.

[0051] It should be noted that in the embodiment of the present application, when there are multiple ways to identify the target interface to obtain the controls in the target interface and the description information corresponding to the controls, one or more of the ways can be selected to identify the target interface according to the current actual needs. For example, if the target interface supports identification of the target interface based on code parsing, then the target interface can be directly identified by code parsing. If the target interface does not support identification of the interface by code parsing, then the target interface can be identified by both graphic and icon recognition.

[0052] In an embodiment of the present application, the electronic device may also determine whether the target interface supports identification of controls through code parsing in a variety of ways.

[0053] As one approach, an electronic device can store a data table containing a list of applications that support code recognition. Before the electronic device identifies a target interface, it can first query the data table to see whether the application to which the target interface to be identified belongs is stored. If the application to which the target interface to be identified belongs is included in the data table, it is determined that the target interface supports code parsing-based recognition, and the target interface can then be directly identified using code parsing.

[0054] If the data table does not contain the application to which the target interface to be identified belongs, it is determined that the target interface does not necessarily support identification based on code parsing. After determining that the target interface does not necessarily support identification based on code parsing, the target interface can be tentatively identified through code parsing. If the controls and their corresponding IDs, types, and descriptions can be identified, it is determined that the target interface supports identification based on code parsing. After obtaining the identification results, the application to which the target interface belongs can also be added to the data table.

[0055] If the control cannot be identified, it is determined that the target interface does not support recognition based on code parsing. Then, the target interface can be recognized by both image and text recognition and icon recognition.

[0056] After the target interface is identified, it can be determined whether the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier based on the controls identified in the target interface. If it is determined that the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control can be determined from the multiple controls corresponding to the first control identifier based on the second control identifier.

[0057] Among them, as mentioned above, after the target is identified, the controls included in the target interface and the description information of the controls can be obtained. Then, in the process of detecting whether the target interface includes controls corresponding to the first control identifier and the second control identifier, the first control identifier and the second control identifier can be matched with the description information of the controls identified from the target control respectively. If the description information of a control can be successfully matched with the first control identifier, it is determined that there is a control corresponding to the first control identifier in the target interface. If the description information of a control can be successfully matched with the second control identifier, it is determined that there is a control corresponding to the second control identifier in the target interface. In addition, the number of controls corresponding to the first control identifier and the number of controls corresponding to the second control identifier can also be determined by the number of successful matches.

[0058] Furthermore, in the process of matching the first control identifier and the second control identifier with the description information of the control identified from the target control, there may be multiple comparison methods.

[0059] As one approach, the first control identifier and the second control identifier can be directly text-matched with the description information. In this approach, if it is determined that the first control identifier and the description information have the same content, then the description information is determined to have successfully matched the first control identifier. Furthermore, if it is determined that the second control identifier and the description information have the same content, then the description information is determined to have successfully matched the second control identifier.

[0060] As another way, the first control identifier, the second control identifier and the description information can be converted into corresponding pinyin content respectively. Among them, the pinyin content corresponding to the first control identifier is the first pinyin content, the pinyin content corresponding to the second control identifier is the second pinyin content, and the pinyin content corresponding to the description information is the third pinyin content. In addition, the first pinyin content and the second pinyin content will also be replaced by phonemes based on the phoneme replacement table, and the pinyin content after the phoneme replacement of the first pinyin content will be used as the first replacement pinyin content, and the pinyin content after the phoneme replacement of the second pinyin content will be used as the second replacement pinyin content. Then, the first pinyin content, the second pinyin content, the first replacement pinyin content and the second replacement pinyin content are matched with the third pinyin content.

[0061] If there is a third pinyin content that successfully matches the first pinyin content, the control corresponding to the third pinyin content is used as the control corresponding to the first control identifier. If there is no third pinyin content that successfully matches the first pinyin content, the first replacement pinyin content is matched with the third pinyin content. If there is a first replacement pinyin content that successfully matches the first pinyin content, the control corresponding to the description information corresponding to the first replacement pinyin content that successfully matches the first pinyin content is used as the control corresponding to the first control identifier. Otherwise, it is determined that there is no control corresponding to the first control identifier in the target interface.

[0062] If there is a third pinyin content that successfully matches the second pinyin content, the control corresponding to the third pinyin content is used as the control corresponding to the second control identifier. If there is no third pinyin content that successfully matches the second pinyin content, the second replacement pinyin content is matched with the third pinyin content. If there is a second replacement pinyin content that successfully matches the second pinyin content, the control corresponding to the descriptive information corresponding to the second replacement pinyin content that successfully matches the second pinyin content is used as the control corresponding to the second control identifier. Otherwise, it is determined that there is no control corresponding to the second control identifier in the target interface.

[0063] like Figure 4 As shown, in Figure 4 In the scenario shown, if the voice control command triggered by the user is "Install Application A", then the first control identifier obtained according to the method in the embodiment of the present application can be installation, and the second control identifier is application A. And Figure 4 As shown in the interface diagram on the right, the electronic device currently displays eight controls in the description information for installation. Therefore, relying solely on the first control identifier alone may not clearly determine which application the user wants to install. However, combined with the second control identifier, which contains application A, it is clear that the user wants to trigger the installation of application A.

[0064] It should be noted that after the target interface is identified to obtain the recognition result, the recognition result can be stored so that the next time the description information of the control of the same target interface needs to be obtained, the recognition result obtained by the previous identification can be directly obtained without real-time identification, so as to improve the efficiency of responding to user operations.

[0065] S130: Execute a control operation corresponding to the target control.

[0066] After determining the target control, the electronic device can then execute the control operation corresponding to the target control. As a method, after determining the target control, a control instruction corresponding to the control operation corresponding to the target control can be generated, and then the electronic device triggers the execution of the control operation corresponding to the target control by executing the control instruction. The control instruction corresponding to the control operation corresponding to the target control can be generated by system injection (an operation method supported by Android) or by simulating screen clicks.

[0067] A voice control method provided by this embodiment first obtains a first control identifier and a second control identifier from the acquired voice control instruction. When the control corresponding to the first control identifier is the to-be-determined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the to-be-determined controls as the target control, if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control will be determined from the multiple controls corresponding to the first control identifier based on the second control identifier, and the control operation corresponding to the target control will be performed.

[0068] Therefore, through the above method, when there are multiple controls to be determined (controls corresponding to the first control identifier) corresponding to the voice control instruction and the user's actual control intention cannot be clearly determined, the multiple controls to be determined can be determined with the help of the controls corresponding to the second control identifier, so as to determine the control representing the user's actual control purpose as the target control from the multiple controls to be determined, so that the electronic device can accurately determine the user's actual control intention.

[0069] See also Figure 5 , the present application provides a voice control method, the method comprising:

[0070] S210: Obtain a first control identifier and a second control identifier from the acquired voice control instruction, wherein the control corresponding to the first control identifier is the to-be-determined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the actual control target of the user from the to-be-determined controls as the target control.

[0071] S220: If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, if there is one control corresponding to the second control identifier, then the target control is determined from the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, and the target interface is the interface displayed when the voice control instruction is obtained.

[0072] It should be noted that, in the embodiment of the present application, among the controls corresponding to the first control identifier, the control that the user wants to actually touch and the control corresponding to the second control identifier are usually associated with each other. Figure 6 As shown, if the voice control instruction triggered by the user is "install application B", the first control identifier obtained is installation, and the second control identifier is application B. Figure 5 In the interface shown, it can be identified that there are three controls corresponding to the first control identifier. However, the user actually wants to trigger the control that is adjacent to application B (the second control identifier) among the three controls corresponding to the first control identifier. Figure 6 Therefore, the association between the second control identifier and the control that the user actually wants to touch can be used to filter multiple controls corresponding to the first control identifier to select the control that the user actually wants to touch as the target control. The association between controls can include the distance between controls or the containment relationship between controls.

[0073] Furthermore, some controls in the target interface may be similar to each other. This similarity can be in the form of similar display styles, similar display positions, or similar inclusion relationships with other controls. In this case, controls similar to the control corresponding to the second control identifier can be combined to filter out the control that the user wants to touch from the multiple controls corresponding to the first control identifier as the target control.

[0074] As another embodiment, if there is one control corresponding to the second control identifier, determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: if there is one control corresponding to the second control identifier and the control corresponding to the second control identifier corresponds to a similar control, obtaining a control similar to the control corresponding to the second control identifier as a first similar control; and determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control. The first similar control may be one or multiple.

[0075] Then as a way, if the control corresponding to the second control identifier is one, then the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, including: if the control corresponding to the second control identifier is one, and the control corresponding to the second control identifier does not correspond to a similar control, then the target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier.

[0076] Among them, optionally, as a way to determine similar controls, the target interface can be first identified based on code parsing, so as to obtain the ID, type, position, size, inclusion relationship and description information of the controls included in the target interface, and then a control relationship structure diagram is constructed based on the ID, type, position, size, inclusion relationship and description information of the identified controls. In the control relationship structure diagram, there will be multiple nodes, each of which represents a control. Moreover, the control corresponding to the child node is included in the control corresponding to the parent node corresponding to the child node. It should be noted that in the control relationship structure diagram, the child node and the parent node exist relatively. If the control corresponding to a certain node is included in the control corresponding to a node adjacent to the node, then the node is a child node relative to the adjacent node, and correspondingly, the adjacent node is the parent node of the certain node. For example, for Figure 6 The interface shown can be analyzed to obtain Figure 7 As shown in the control relationship structure diagram. Figure 7 As shown, node 2 is adjacent to node 5, and from top to bottom, the level of the node corresponding to the lower the level of the control, the level of node 2 is higher than the level of node 5, and node 5 is a child node relative to node 2, and node 2 is a parent node relative to node 5. Figure 6 In the control relationship structure diagram shown, the node at the top is the root node. The root node represents the most basic control in the target interface. All controls in the interface, except for this most basic control, are contained within this most basic control. Furthermore, the closer the node is to the top, the closer its control hierarchy is to that of the most basic control.

[0077] In this way, obtaining a control similar to the control corresponding to the second control identifier as a first similar control can include: based on the properties of the control corresponding to the second control identifier, searching for a control similar to the control corresponding to the second control identifier in the control relationship structure diagram as the first similar control, the properties including at least one of the distance from the node corresponding to the control to the root node, the type of the control, the length and width of the control, and the relative position of the control in the corresponding parent control.

[0078] The distance from the node corresponding to the control to the root node represents the number of jumps required in the process of jumping from the node corresponding to the control to the root node. Figure 7Node 1 in the example needs 1 jump to reach the root node, so the distance between node 1 and the root node is 1. Node 8 needs 5 jumps to reach the root node, so the distance between node 8 and the root node is 5. For another example, node 5 needs 3 jumps to reach the root node, so the distance between node 5 and the root node is 3. Similarly, the distances between node 6 and node 8 and the root node are both 3.

[0079] The type of control can represent the purpose of the control in the interface. Optionally, based on the control type, the controls included in the interface can be divided into controls for outputting content, controls for displaying content, and controls for interacting with the user. Controls for outputting content can be text boxes. Controls for displaying content can be controls for displaying images or text content. Controls for interacting with the user can include buttons, etc.

[0080] The length and width of a control represent the size of the control itself. The relative position of a control in its corresponding parent control can be understood as the relative display position of the control in its parent control when it is displayed in the interface. Figure 8 As shown, control 1 includes control 11 and control 12, wherein control 11 is used to display the icon control, name control and installation trigger control corresponding to application A. Control 12 is used to display the icon control, name control and installation trigger control corresponding to application B. Figure 8 It can be seen that the relative position of the icon control corresponding to application A displayed in control 11 is the same as the relative position of the icon control corresponding to application B displayed in control 12.

[0081] In an embodiment of the present application, after obtaining the properties of the controls included in the target interface, controls similar to the control corresponding to the second control identifier can be filtered out as first similar controls based on the properties of the controls. Moreover, in the process of obtaining the first similar controls based on the properties, the first similar controls can be obtained based on one item in the control properties, or based on multiple items in the properties. For example, as a method, the first similar controls can be filtered based on the distance from the node corresponding to the control included in the properties to the root node. For example, please refer to Figure 7 If the second control is identified as application A, then the node corresponding to application A is node 5. The distance between node 5 and the root node is 3, and other nodes with a distance of 3 from the root node are at least node 6 and node 7. Therefore, it can be determined that the controls corresponding to node 6 and node 7 are the first similar controls.

[0082] For example Figure 8 As shown, if similar controls are determined based on their relative positions in the corresponding parent controls, then it can be determined Figure 8The icon control corresponding to application A and the icon control corresponding to application B are similar controls. If the icon control corresponding to application A is the control corresponding to the second control identifier, then it can be determined that the icon control corresponding to application B is a similar control.

[0083] Optional, such as Figure 9 As shown, the determining a target control from a plurality of controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control includes:

[0084] S221: Obtain the distances between the first node and multiple second nodes in the control relationship structure diagram to obtain multiple first distances, where the first node is used to represent the control corresponding to the second control identifier, and the second node is used to represent the control corresponding to the first control identifier.

[0085] Optionally, obtaining the distances between the first node and the plurality of second nodes in the control relationship structure diagram to obtain the plurality of first distances includes:

[0086] Obtain the nearest common parent node corresponding to the first node in the control relationship structure diagram and the second node currently performing the first distance calculation; obtain the distance from the first node to the nearest common parent node, and the distance from the second node currently performing the first distance calculation to the nearest common parent node, wherein the distances ensure the number of levels at which the two corresponding nodes jump to each other; use the sum of the distance from the first node to the nearest common parent node and the distance from the second node currently performing the first distance calculation to the nearest common parent node as the distance between the second node currently performing the first distance calculation and the first node to obtain multiple first distances.

[0087] For example, Figure 7 As shown, if the first control identifier is installation and the second control identifier is application A, then the nodes corresponding to the first control identifier are node 8, node 9, and node 10. The node corresponding to the second control identifier is node 5. Then the first node includes node 5, and the second node includes node 8, node 9, and node 10. Then, the first distance corresponding to node 5 and node 8, the first distance corresponding to node 5 and node 9, and the first distance corresponding to node 5 and node 10 are obtained respectively, thereby obtaining multiple first distances.

[0088] Among them, the common parent nodes corresponding to nodes 5 and 8 are node 2, node 1, and the root node, but node 2 is the parent node closest to nodes 5 and 8, so node 2 is the closest common parent node corresponding to nodes 5 and 8. Among them, the distance from node 5 to node 2 is 1, and the distance from node 8 to node 2 is 3, so the first distance corresponding to nodes 5 and 8 is 4. The common parent nodes corresponding to nodes 5 and 9 are node 1 and the root node, but node 1 is the parent node closest to nodes 5 and 9, so node 1 is the closest common parent node corresponding to nodes 5 and 9. Among them, the distance from node 5 to node 1 is 2, and the distance from node 9 to node 1 is 4, so the first distance corresponding to nodes 5 and 9 is 6. The common parent nodes corresponding to nodes 5 and 10 are node 1 and the root node, but node 1 is the parent node closest to nodes 5 and 10, so node 1 is the closest common parent node corresponding to nodes 5 and 10. Among them, the distance from node 5 to node 1 is 2, and the distance from node 10 to node 1 is 4, so the first distance corresponding to node 5 and node 10 is 6.

[0089] S222: Obtain the distances between the third node and multiple second nodes in the control relationship structure diagram to obtain multiple second distances, where the third node is the node corresponding to the first similar control.

[0090] Optionally, the method of obtaining the distances between the third node in the control relationship structure diagram and multiple second nodes to obtain multiple second distances includes: obtaining the nearest common parent node corresponding to the third node in the control relationship structure diagram and the second node currently performing the second distance calculation; obtaining the distance from the third node to the nearest common parent node, and the distance from the second node currently performing the second distance calculation to the nearest common parent node, wherein the distances ensure the number of levels at which the corresponding two nodes jump to each other; taking the sum of the distance from the third node to the nearest common parent node and the distance from the second node currently performing the second distance calculation to the nearest common parent node as the distance between the second node currently performing the second distance calculation and the third node to obtain multiple second distances.

[0091] For example, please refer to Figure 7As shown above, still taking the example that the first node includes node 5 and the second node includes node 8, node 9 and node 10, if the first similar control is determined by whether the distances to the root node are the same, then the nodes corresponding to the first similar control determined include node 6 and node 7. Then, according to the aforementioned access, the second distance between node 6 and node 8, the second distance between node 6 and node 9, and the second distance between node 6 and node 10 can be calculated. Furthermore, the second distance between node 7 and node 8, the second distance between node 7 and node 9, and the second distance between node 7 and node 10 will be calculated, thereby obtaining multiple second distances.

[0092] It should be noted that the method for calculating the second distance in this embodiment is the same as the method for calculating the first distance, and will not be described in detail here. Accordingly, the calculated second distance between nodes 6 and 8 is 6, the second distance between nodes 6 and 9 is 4, and the second distance between nodes 6 and 10 is 6. The calculated second distance between nodes 7 and 8 is 6, the second distance between nodes 7 and 9 is 6, and the second distance between nodes 7 and 10 is 4.

[0093] S223: Acquire multiple reference distances, where the multiple reference distances include the multiple first distances and the multiple second distances.

[0094] S224: Detect whether a minimum value among the multiple reference distances is consistent with a minimum value among the multiple first distances, and the number of the minimum value is one.

[0095] S225: If the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, and the number of the minimum values is one, then the control corresponding to the minimum value among the multiple controls corresponding to the first control identifier is used as the target control.

[0096] From the above example, we can find that the minimum value among the multiple reference distances is 4, and the minimum value among the multiple first distances is also 4. Then we can determine that the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances. Then, among the multiple controls corresponding to the first control identifier, the control corresponding to the minimum value among the multiple first distances can be used as the target control. For example, the controls corresponding to the second control identifier include Figure 7 The controls corresponding to nodes 8, 9, and 10 in the diagram, the control corresponding to the minimum value of the multiple first distances is the control corresponding to nodes 8 and 5, and the control corresponding to node 8 is the control corresponding to the minimum value of the multiple first distances and the second control identifier, so that the control corresponding to node 8 can be used as the target control.

[0097] S226: If the minimum value among the multiple reference distances is inconsistent with the minimum value among the multiple first distances, and the multiple first distances do not have a first distance that is the same as the minimum value among the multiple reference distances, obtain a second similar control, which is a control selected from the control relationship structure diagram based on the properties of the control corresponding to the first control identifier.

[0098] It should be noted that in some cases, due to a user's slip of the tongue, the control that the user wants to trigger may be different from the control target expressed by the user through the voice control instruction. Figure 10 As shown, in Figure 10 In the interface shown, the operation control corresponding to application A is Update, the operation control corresponding to application B is Update, and the operation control corresponding to application C is Install. Figure 10 The control relationship structure diagram obtained by identifying the controls in Figure 11 Based on Figure 11 As shown in the control relationship structure diagram, if the voice control instruction issued by the user is "Update Application C", the first control identifier obtained is Update, and the second control identifier is Application C. Based on the method of obtaining the first distance, the second distance, and the third distance described above, it can be found that the minimum value of the multiple reference distances is inconsistent with the minimum value of the multiple first distances, and it can be found that the minimum value of the multiple first distances is greater than the minimum value of the multiple reference distances. In this case, the property of the control corresponding to the first control identifier will be further obtained. Figure 11 The similar control selected in the control relationship structure diagram shown in the figure is used as the second similar control. For example, if the control is based on the root node ( Figure 11 The second similar control is selected by using the same distance to the node 1) in the root node, and then the control corresponding to the node 11 whose distance to the root node is also 4 can be used as the second similar control.

[0099] S227: Obtain a third distance, where the third distance includes a distance from a node corresponding to the second similar control to a node corresponding to the second control identifier.

[0100] Still Figure 11 As shown, the distance between the node 11 corresponding to the second similar control and the node 7 corresponding to the second control identifier can be obtained as 4, that is, the obtained third distance is 4.

[0101] S228: If there is a distance in the third distance that is uniquely consistent with the minimum value of the multiple reference distances, the control corresponding to the uniquely consistent distance is used as the target control.

[0102] Here, unique consistency can be understood as consistency and only one consistency. Correspondingly, the existence of a distance in the third distance that is uniquely consistent with the minimum value of the multiple reference distances can be understood as only one distance in the third distance is consistent with the minimum value of the multiple reference distances.

[0103] exist Figure 11 In the case shown, the minimum value among the multiple reference distances is also 4, so it can be determined that there is a distance in the third distance that is uniquely consistent with the minimum value among the multiple reference distances. Therefore, the control corresponding to the node (node 11) corresponding to the unique distance can be used as the target control. Thus, by obtaining the second similar control, even if the user makes a mistake and incorrectly issues a voice control instruction, the electronic device can intelligently correct the error in the user's voice control instruction, thereby increasing the probability of accurately executing the user's actual intention.

[0104] If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is only one control corresponding to the first control identifier, the control corresponding to the first control identifier is used as the target control.

[0105] Furthermore, in some cases, the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, but the minimum value may not be unique. Figure 12 As shown, after calculating in the above manner, it can be found that the control corresponding to the minimum value of the multiple first distances can be the control corresponding to node 8 or the control corresponding to node 9. Therefore, it is impossible to directly determine the target control. Then, if the method provided in the embodiment of the application cannot automatically determine the target control, the target control can be determined by querying the user.

[0106] Also, in some cases, there are two or more controls corresponding to the second control identifier. In this case, if there are also two or more controls corresponding to the first control identifier, it is impossible to determine the target control from the controls corresponding to multiple first control identifiers through the second control identifier. The target control can be determined by asking the user.

[0107] S230: Execute a control operation corresponding to the target control.

[0108] The present embodiment provides a voice control method, so that in the above-mentioned manner, when there are multiple controls to be determined (controls corresponding to the first control identifier) corresponding to the voice control instruction and the user's actual control intention cannot be clearly determined, the multiple controls to be determined can be determined with the help of the control corresponding to the second control identifier, so as to determine the control representing the user's actual control purpose as the target control from the multiple controls to be determined, thereby enabling the electronic device to accurately determine the user's actual control intention. Moreover, in the present embodiment, a control relationship structure diagram can be established based on the mutual inclusion relationship of the controls in the target interface, so that the control relationship structure diagram can be used to calculate the distance between the control corresponding to the second control identifier and the first similar control and the control corresponding to the first control identifier, and then based on the distance, the target control can be determined from the multiple controls corresponding to the first control identifier, so that the electronic device can more conveniently and accurately determine the target control.

[0109] See also Figure 13 , the present application provides a voice control method, the method comprising:

[0110] S310: Obtain a first control identifier and a second control identifier from the acquired voice control instruction, wherein the control corresponding to the first control identifier is the to-be-determined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the actual control target of the user from the to-be-determined controls as the target control.

[0111] S320: If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, obtain the display distance between each of the multiple controls corresponding to the first control identifier and the control corresponding to the second control identifier in the target interface. The target interface is the interface displayed when the voice control instruction is obtained.

[0112] In the embodiment of the present application, the display distance represents the pixel distance between the controls in the target interface. The display distance between two controls may include the distance between the center coordinates of the two controls.

[0113] like Figure 14 As shown, the display distance between control 20 and control 21 is d1, and the display distance between control 21 and control 22 is d2.

[0114] S330: Using the control with the smallest display distance among the controls corresponding to the first control identifier as the target control.

[0115] S340: Execute a control operation corresponding to the target control.

[0116] This embodiment provides a voice control method, which allows, through the above-mentioned method, when there are multiple controls to be determined (controls corresponding to the first control identifier) corresponding to the voice control instruction, making it impossible to clearly determine the user's actual control intention, to use the control corresponding to the second control identifier to determine the multiple controls to be determined, thereby determining the control representing the user's actual control purpose from the multiple controls to be determined as the target control, thereby enabling the electronic device to accurately determine the user's actual control intention. Moreover, in this embodiment, the target control can be determined from the multiple controls corresponding to the first control identifier directly based on the display distance between each of the multiple controls corresponding to the first control identifier and the control corresponding to the second control identifier, thereby improving the flexibility of obtaining the target control.

[0117] See also Figure 15 The present application provides a voice control device 400, which includes:

[0118] The identification acquisition unit 410 is used to obtain a first control identification and a second control identification from the acquired voice control instruction, wherein the control corresponding to the first control identification is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identification is used to determine the control representing the user's actual control target from the controls to be determined as the target control.

[0119] The control determination unit 420 is used to determine the target control from the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier. The target interface is the interface displayed when the voice control instruction is obtained.

[0120] The control unit 430 is configured to execute a control operation corresponding to the target control.

[0121] As one embodiment, the control determination unit 420 is specifically configured to, if there is one control corresponding to the second control identifier, determine a target control from among multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier. Optionally, the control determination unit 420 is specifically configured to, if there is one control corresponding to the second control identifier and there is no similar control corresponding to the control corresponding to the second control identifier, determine a target control from among multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier.

[0122] Optionally, the control determination unit 420 is specifically used to obtain a control similar to the control corresponding to the second control identifier as a first similar control if there is only one control corresponding to the second control identifier and there is a similar control corresponding to the control corresponding to the second control identifier; and determine a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control.

[0123] Optionally, based on the properties of the control corresponding to the second control identifier, a control similar to the control corresponding to the second control identifier is searched in the control relationship structure diagram as a first similar control, and the properties include at least one of the distance from the node corresponding to the control to the root node, the type of the control, the length and width of the control, and the relative position of the control in the corresponding parent control; wherein, the control relationship structure diagram is generated based on the inclusion relationship of the controls in the target interface, and the control corresponding to the child node in the control relationship structure diagram is included in the control corresponding to the parent node corresponding to the child node.

[0124] Optionally, the control determination unit 420 is specifically used to obtain the distances between the first node and multiple second nodes in the control relationship structure diagram, and obtain multiple first distances, where the first node is used to represent the control corresponding to the second control identifier, and the second node is used to represent the control corresponding to the first control identifier; obtain the distances between the third node and multiple second nodes in the control relationship structure diagram, and obtain multiple second distances, where the third node is the node corresponding to the first similar control; obtain multiple reference distances, where the multiple reference distances include the multiple first distances and the multiple second distances; if the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, and the number of the minimum values is one, then the control corresponding to the minimum value among the multiple controls corresponding to the first control identifier is used as the target control.

[0125] It is also specifically used to obtain a second similar control if the minimum value among the multiple reference distances is inconsistent with the minimum value among the multiple first distances, and the multiple first distances do not have a first distance that is the same as the minimum value among the multiple reference distances, the second similar control is a control selected from the control relationship structure diagram based on the attributes of the control corresponding to the first control identifier; obtain a third distance, the third distance includes the distance from the node corresponding to the second similar control to the node corresponding to the second control identifier; if there is a distance in the third distance that is uniquely consistent with the minimum value among the multiple reference distances, the control corresponding to the uniquely consistent distance is used as the target control.

[0126] As a method, the control determination unit 420 is specifically used to obtain the nearest common parent node corresponding to the first node in the control relationship structure diagram and the second node currently performing the first distance calculation; obtain the distance from the first node to the nearest common parent node, and the distance from the second node currently performing the first distance calculation to the nearest common parent node, and the distance ensures the number of levels that the corresponding two nodes jump to each other; take the sum of the distance from the first node to the nearest common parent node and the distance from the second node currently performing the first distance calculation to the nearest common parent node as the distance between the second node currently performing the first distance calculation and the first node, so as to obtain multiple first distances.

[0127] As a method, the control determination unit 420 is specifically used to obtain the nearest common parent node corresponding to the third node in the control relationship structure diagram and the second node currently performing the second distance calculation; obtain the distance from the third node to the nearest common parent node, and the distance from the second node currently performing the second distance calculation to the nearest common parent node, and the distance ensures the number of levels that the corresponding two nodes jump to each other; use the sum of the distance from the third node to the nearest common parent node and the distance from the second node currently performing the second distance calculation to the nearest common parent node as the distance between the second node currently performing the second distance calculation and the third node to obtain multiple second distances.

[0128] Among them, the control determination unit 420 is specifically used to use the control corresponding to the first control identifier as the target control if the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is only one control corresponding to the first control identifier.

[0129] As another method, the control determination unit 420 is specifically used to obtain the display distance between multiple controls corresponding to the first control identifier and the control corresponding to the second control identifier in the target interface; and the control with the smallest display distance among the controls corresponding to the first control identifier is used as the target control.

[0130] The present embodiment provides a voice control device, which first obtains a first control identifier and a second control identifier from an acquired voice control instruction, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control. If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control will be determined from the multiple controls corresponding to the first control identifier based on the second control identifier, and the control operation corresponding to the target control will be performed. Thus, in the above manner, when there are multiple controls to be determined corresponding to the voice control instruction (controls corresponding to the first control identifier) and it is impossible to clearly determine the user's actual control intention, the multiple controls to be determined can be determined with the help of the control corresponding to the second control identifier, thereby determining the control representing the user's actual control purpose from the multiple controls to be determined as the target control, thereby enabling the electronic device to accurately determine the user's actual control intention.

[0131] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present application, the coupling between modules can be electrical. In addition, the various functional modules in the various embodiments of the present application can be integrated into a processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0132] The following will be combined Figure 13 An electronic device provided by this application is described.

[0133] See also Figure 13 Based on the above-mentioned voice control method and device, the embodiments of the present application further provide an electronic device 1000 that can execute the above-mentioned voice control method. The electronic device 1000 includes one or more (only one is shown in the figure) processors 102, a memory 104, a camera 106, and an audio acquisition device 108 that are coupled to each other. The memory 104 stores a program that can execute the content of the above-mentioned embodiments, and the processor 102 can execute the program stored in the memory 104.

[0134] The processor 102 may include one or more processing cores. The processor 102 utilizes various interfaces and circuits to connect various components within the electronic device 1000. It executes instructions, programs, code sets, or instruction sets stored in the memory 104, and accesses data stored in the memory 104 to perform various functions and process data within the electronic device 1000. Optionally, the processor 102 may be implemented in at least one hardware form: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 102 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 102 but may be implemented separately via a communication chip. Alternatively, the processor 102 may be a neural network chip. For example, it can be an embedded neural network chip (NPU).

[0135] The memory 104 may include a random access memory (RAM) or a read-only memory (ROM). The memory 104 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 104 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.

[0136] Furthermore, in addition to the aforementioned components, the electronic device 1000 may further include a network module 110 and a sensor module 112 .

[0137] The network module 110 is used to implement information exchange between the electronic device 1000 and other devices, such as transmitting device control instructions, operation request instructions, and status information acquisition instructions, etc. When the electronic device 200 is specifically a different device, the corresponding network module 110 may be different.

[0138] The sensor module 112 may include at least one sensor. Specifically, the sensor module 112 may include, but is not limited to: a level, a light sensor, a motion sensor, a pressure sensor, an infrared heat sensor, a distance sensor, an acceleration sensor, and other sensors.

[0139] Among them, the pressure sensor can be a sensor that detects the pressure generated by pressing on the electronic device 1000. That is, the pressure sensor detects the pressure generated by contact or pressing between the user and the electronic device, such as the pressure generated by contact or pressing between the user's ear and the mobile terminal. Therefore, the pressure sensor can be used to determine whether contact or pressing occurs between the user and the electronic device 1000, as well as the magnitude of the pressure.

[0140] The accelerometer can detect the magnitude of acceleration in all directions (generally three axes) and the magnitude and direction of gravity when stationary. This can be used for applications that recognize the posture of the electronic device 1000 (such as switching between landscape and portrait modes, related games, and magnetometer posture calibration), vibration recognition-related functions (such as pedometers and tapping), etc. In addition, the electronic device 1000 may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, and thermometer, which will not be detailed here.

[0141] The audio collection device 108 is used to collect audio signals. Optionally, the audio collection device 108 includes multiple audio collection devices, which may be microphones.

[0142] As a way, the network module of the electronic device 1000 is a radio frequency module, which is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The radio frequency module may include various existing circuit components for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption / decryption chip, a user identity module (SIM) card, a memory, and the like. For example, the radio frequency module can interact with an external device by sending or receiving electromagnetic waves. For example, the radio frequency module can send instructions to a target device.

[0143] Please refer to Figure 14 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 800 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0144] The computer-readable storage medium 800 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk, or a ROM. Alternatively, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 for executing any of the method steps described above. These program codes can be read from or written to one or more computer program products. The program code 810 can be compressed, for example, in a suitable form.

[0145] In summary, the present application provides a voice control method, device, electronic device and storage medium, which first obtains a first control identifier and a second control identifier from the acquired voice control instruction, wherein the control corresponding to the first control identifier is the control to be determined corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the user's actual control target from the controls to be determined as the target control. If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control will be determined from the multiple controls corresponding to the first control identifier based on the second control identifier, and the control operation corresponding to the target control will be executed. Thus, in the above manner, when there are multiple controls to be determined corresponding to the voice control instruction (the controls corresponding to the first control identifier) and the user's actual control intention cannot be clearly determined, the multiple controls to be determined can be determined with the help of the controls corresponding to the second control identifier, thereby determining the control representing the user's actual control purpose from the multiple controls to be determined as the target control, thereby enabling the electronic device to accurately determine the user's actual control intention.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A voice control method, characterized in that: The method comprises: Acquire a first control identifier and a second control identifier from the acquired voice control instruction; If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, the target control is determined from the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, and the target interface is the interface displayed when the voice control instruction is obtained, wherein the control corresponding to the first control identifier is the to-be-determined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine the control representing the actual control target of the user from the to-be-determined controls through the association between controls as the target control, wherein the association between controls includes the distance between controls or the inclusion relationship between controls; Execute a control operation corresponding to the target control.

2. The method according to claim 1, characterized in that The determining a target control from a plurality of controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: If there is only one control corresponding to the second control identifier, a target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier.

3. The method according to claim 2, characterized in that If there is only one control corresponding to the second control identifier, determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: If there is only one control corresponding to the second control identifier and no similar control corresponds to the control corresponding to the second control identifier, a target control is determined from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier.

4. The method according to claim 2, characterized in that If there is only one control corresponding to the second control identifier, determining a target control from multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: If there is one control corresponding to the second control identifier, and the control corresponding to the second control identifier corresponds to a similar control, obtaining a control similar to the control corresponding to the second control identifier as a first similar control; Based on the control corresponding to the second control identifier and the first similar control, a target control is determined from multiple controls corresponding to the first control identifier.

5. The method according to claim 4, characterized in that The acquiring a control similar to the control corresponding to the second control identifier as a first similar control includes: Based on the properties of the control corresponding to the second control identifier, searching the control relationship structure diagram for a control similar to the control corresponding to the second control identifier as a first similar control, where the properties include at least one of a distance from a node corresponding to the control to a root node, a type of the control, a length and width of the control, and a relative position of the control in a corresponding parent control; The control relationship structure diagram is generated based on the inclusion relationship of controls in the target interface, and the control corresponding to a child node in the control relationship structure diagram is included in the control corresponding to the parent node corresponding to the child node.

6. The method according to claim 5, characterized in that The determining a target control from a plurality of controls corresponding to the first control identifier based on the control corresponding to the second control identifier and the first similar control includes: Obtaining distances between a first node and a plurality of second nodes in a control relationship structure diagram to obtain a plurality of first distances, wherein the first node is used to represent a control corresponding to the second control identifier, and the second node is used to represent a control corresponding to the first control identifier; Obtaining distances between a third node and a plurality of second nodes in the control relationship structure diagram to obtain a plurality of second distances, wherein the third node is a node corresponding to the first similar control; Acquire a plurality of reference distances, the plurality of reference distances including the plurality of first distances and the plurality of second distances; If the minimum value among the multiple reference distances is consistent with the minimum value among the multiple first distances, and the number of the minimum values is one, then the control corresponding to the minimum value among the multiple first distances among the multiple controls corresponding to the first control identifier is used as the target control.

7. The method according to claim 6, characterized in that The method further comprises: If the minimum value among the multiple reference distances is inconsistent with the minimum value among the multiple first distances, and the multiple first distances do not have a first distance that is the same as the minimum value among the multiple reference distances, obtaining a second similar control, where the second similar control is a control selected from the control relationship structure diagram based on the attribute of the control corresponding to the first control identifier; Acquire a third distance, where the third distance includes a distance from a node corresponding to the second similar control to a node corresponding to the second control identifier; If there is a distance in the third distance that is uniquely consistent with the minimum value in the multiple reference distances, the control corresponding to the uniquely consistent distance is used as the target control.

8. The method according to claim 6, characterized in that The step of obtaining the distances between the first node and the plurality of second nodes in the control relationship structure diagram to obtain the plurality of first distances includes: Obtain the nearest common parent node corresponding to the first node in the control relationship structure diagram and the second node currently performing the first distance calculation; Obtaining the distance between the first node and the nearest common parent node, and the distance between the second node currently performing the first distance calculation and the nearest common parent node; The sum of the distance from the first node to the nearest common parent node and the distance from the second node currently performing the first distance calculation to the nearest common parent node is used as the distance between the second node currently performing the first distance calculation and the first node to obtain multiple first distances.

9. The method according to claim 6, characterized in that The step of obtaining the distances between the third node and the plurality of second nodes in the control relationship structure diagram to obtain the plurality of second distances includes: Get the nearest common parent node corresponding to the third node in the control relationship structure diagram and the second node currently performing the second distance calculation; Obtaining a distance between the third node and the nearest common parent node, and a distance between the second node currently performing the second distance calculation and the nearest common parent node, wherein the distance represents the number of levels that the two nodes jump between each other; The sum of the distance from the third node to the nearest common parent node and the distance from the second node currently performing the second distance calculation to the nearest common parent node is used as the distance between the second node currently performing the second distance calculation and the third node to obtain multiple second distances.

10. The method according to claim 1, characterized in that The determining a target control from a plurality of controls corresponding to the first control identifier based on the control corresponding to the second control identifier includes: Obtaining, in the target interface, a display distance between each of a plurality of controls corresponding to the first control identifier and a control corresponding to the second control identifier; The control with the smallest display distance among the controls corresponding to the first control identifier is used as the target control.

11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: If the target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there is only one control corresponding to the first control identifier, the control corresponding to the first control identifier is used as the target control.

12. A voice control device, characterized in that: The device comprises: an identification acquisition unit, configured to acquire a first control identification and a second control identification from the acquired voice control instruction; a control determination unit for determining a target control from among the multiple controls corresponding to the first control identifier based on the control corresponding to the second control identifier, if a target interface includes a control corresponding to the first control identifier and a control corresponding to the second control identifier, and there are multiple controls corresponding to the first control identifier, wherein the target interface is the interface displayed when the voice control instruction is obtained, wherein the control corresponding to the first control identifier is the to-be-determined control corresponding to the voice control instruction, and the control corresponding to the second control identifier is used to determine, from among the to-be-determined controls, a control representing an actual control target of the user as the target control through an association between controls, wherein the association between controls includes a distance between controls or an inclusion relationship between controls; A control unit is used to execute a control operation corresponding to the target control.

13. An electronic device, characterized in that: including one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, wherein when the program code is run, the method according to any one of claims 1 to 11 is executed.

Images