Instruction execution program determination method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of software, and discloses an instruction execution program determination method and device, electronic equipment and a storage medium, the determination method is applied to a first application program, and the determination method comprises the following steps: matching an identified voice instruction with preset words in each second application program, determining a target second application program of which the matching degree is located in a preset matching degree interval; wherein the second application program is a program displayed in the virtual page; wherein all preset matching degrees in the preset matching degree interval are smaller than 100%; and if the matching degree of the display text of the target control in the target second application and the voice instruction exceeds a preset matching degree, sending the display text of the target control to the target second application so as to enable the target second application to serve as an instruction execution program, and determining a to-be-executed target instruction according to the display text of the target control. According to the method, the instruction execution program can be flexibly determined to meet the expectation of the user, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of software technology, and in particular to a method, device, electronic device and storage medium for determining an instruction execution program. Background Art

[0002] When different applications are running in parallel, due to confusion in instruction permissions, it is impossible to correctly determine the application that responds to the relevant instructions, that is, it is impossible to accurately determine the execution program of the relevant instructions, resulting in an inability to accurately respond to the relevant instructions that the user expects to execute.

[0003] The related technology regards the program running in the foreground of the virtual page as the main program, which responds to all instructions issued by the user. However, for some of the instructions, the user does not expect the program running in the foreground to execute, which causes the user to execute the related instructions unexpectedly and fail to obtain the instruction execution results that meet the user's expectations. Summary of the invention

[0004] In view of the above problems, the present application provides a method, device, electronic device and storage medium for determining an instruction execution program, which are used to accurately determine the instruction execution program and the target instruction to be executed.

[0005] According to one aspect of the present application, a method for determining an instruction execution program is provided, and the determination method is applied to a first application, and the determination method includes: matching the recognized voice instruction with preset words in each second application, and determining a target second application whose matching degree is within a preset matching degree interval; wherein the second application is a program displayed in a virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%; if the display text of a target control in the target second application and the voice instruction match more than a preset matching degree, then sending the display text of the target control to the target second application, so that the target second application acts as an instruction execution program, and determines the target instruction to be executed according to the display text of the target control.

[0006] In an optional manner, each second application includes at least one control; the determination method also includes: matching the voice instruction with the display text of each control in the target second application, and determining the control whose matching degree exceeds the preset matching degree as the control to be responded; if the position information of the target control to be responded meets the preset position information, and the correlation between the target control to be responded and the voice instruction meets the preset correlation, then the target control to be responded is used as the target control; wherein, the target control to be responded is any control among the controls to be responded.

[0007] In an optional manner, the determination method also includes: if the position information of all controls to be responded to does not satisfy the preset position information, or the correlation between all controls to be responded to and the voice command does not satisfy the preset correlation, then the first application is used as the command execution program, and the voice command is used as the target command to be executed by the first application.

[0008] In an optional manner, the position information of the control to be responded to includes the coordinates of each boundary point of the control to be responded to, and the layer of the control to be responded to in the virtual page; the determination method also includes: matching the coordinates of each boundary point of the target control to be responded to with the preset point coordinates in a preset area to obtain the number of successfully matched boundary point coordinates; if the number of successfully matched boundary point coordinates is greater than the preset number, and the layer of the target control to be responded to in the virtual page is within the preset layer range, then it is determined that the position information of the target control to be responded to meets the preset position information.

[0009] In an optional manner, the determination method also includes: if the matching degrees corresponding to all second applications are less than the minimum end value of the preset matching degree interval, or the matching degrees between the display texts of all controls in the virtual page and the voice instructions do not exceed the preset matching degree, then the first application is used as the instruction execution program, and the voice instruction is used as the target instruction to be executed by the first application.

[0010] In an optional manner, the recognized voice command is matched with preset words in each second application, including: if the matching degree corresponding to the designated second application is 100%, the voice command is sent to the designated second application, so that the designated second application acts as an instruction execution program and determines the target instruction to be executed according to the voice command.

[0011] In an optional manner, the matching of the recognized voice instructions with preset words in each second application includes: matching the words in the voice instructions recognized by the first application with the preset words in the traversed second application to obtain the words successfully matched by the traversed second application; dividing the number of words successfully matched by the traversed second application by the number of preset words in the traversed second application to obtain the matching degree corresponding to the traversed second application, so as to determine the matching degree corresponding to each second application.

[0012] According to another aspect of the present application, a device for determining an instruction execution program is provided, and the determination device is applied to a first application, and the determination device includes: a matching module, which matches the recognized voice instruction with preset words in each second application, and determines a target second application whose matching degree is within a preset matching degree interval; wherein the second application is a program displayed in a virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%; a first determination module, which sends the display text of the target control to the target second application if the matching degree between the display text of the target control in the target second application and the voice instruction exceeds a preset matching degree, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

[0013] According to one aspect of the present application, an electronic device is provided, including: a controller; and a memory for storing one or more programs, wherein when the one or more programs are executed by the controller, the above-mentioned determination method is executed.

[0014] According to one aspect of the present application, a computer-readable storage medium is also provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the above-mentioned determination method.

[0015] According to one aspect of the present application, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned determination method.

[0016] The first application of this application matches and analyzes the voice instructions recognized by itself and the relevant information of the second application displayed in the virtual page; when the matching degree between the display text of the target control and the voice instruction in the target second application exceeds the preset matching degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control; wherein the voice instruction recognition program and the instruction execution program are not the same, achieving the purpose of cross-program instruction execution and breaking the mental confinement of instruction recognition and instruction execution in the same program. This application can flexibly determine the instruction execution program and the target instruction to be executed, so that the determined instruction execution program meets the user's expectations to the greatest extent, thereby obtaining an instruction execution result that meets the user's expectations.

[0017] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flowchart of a method for determining an instruction execution program shown in an exemplary embodiment of the present application.

[0020] Figure 2 is based on Figure 1 The exemplary embodiment shown is a flowchart of another method for determining an instruction execution program.

[0021] Figure 3 is based on Figure 2 The exemplary embodiment shown is a flowchart of another method for determining an instruction execution program.

[0022] Figure 4 is based on Figure 1 The exemplary embodiment shown is a flowchart of another method for determining an instruction execution program.

[0023] Figure 5 It is a schematic diagram of the application scenario of the method for determining the instruction execution program of the present application.

[0024] Figure 6 It is a structural diagram of a device for determining an instruction execution program shown in an exemplary embodiment of the present application.

[0025] Figure 7 It is a schematic diagram of the structure of a computer system of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0029] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0030] The program running in the foreground of the virtual page is generally the main program with the highest permission level, which responds to all instructions issued by the user. However, for some of these instructions, the user does not expect the program running in the foreground to execute, which causes the user's unexpected program to execute related instructions and fails to obtain the instruction execution results that meet the user's expectations.

[0031] To this end, one aspect of the present application provides a method for determining an instruction execution program. Figure 1 , Figure 1 1 is a flow chart of a method for determining an instruction execution program shown in an exemplary embodiment of the present application. The determination method is applied to a first application program, and at least includes S110 to S120, which are described in detail as follows:

[0032] S110: Match the recognized voice command with the preset words in each second application to determine the target second application whose matching degree is within a preset matching degree interval; wherein the second application is the program displayed in the virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%.

[0033] In this embodiment, the first application is used as the execution subject to recognize the voice or voice text input by the user to identify the voice command therein. In the related art, the recognition of voice commands and the execution of commands are limited to the same application, but this application breaks through this thinking and determines the command execution program that is closest to the user's expectations according to the specific situation to improve the user's experience.

[0034] The second application and the first application are programs running in different processes. The second application is a program displayed in the foreground page (virtual page), and the number of the second application is at least one; the display pages of each second application can be distributed in various areas of the foreground page, and the display pages of each second application can be displayed overlappingly, that is, displayed in different page layers. This application does not specifically limit the size and page layer of the display page of each application. The target second application is a second application whose matching degree of the preset word is within the preset matching degree interval. In this embodiment, the target second application determined can be two or more.

[0035] The preset words are the words displayed in the respective display pages of all controls in each second application. They can be all displayed words, hot words with high triggering frequency, button labels, menu options, link texts or other interactive elements in the display page, etc. Users can activate or operate the corresponding preset words in the target application so that the target application executes the preset execution instructions for the preset words.

[0036] The preset matching degree interval is an interval value used to determine the target second application program, and the maximum end value of the interval must be less than 100%, that is, the preset matching interval is a percentage interval consisting of values ​​within 100%.

[0037] In a certain embodiment, if the matching degree corresponding to the designated second application is 100%, it indicates that the voice instruction completely matches the preset word in the second application of the instruction, indicating that the designated second application is the instruction execution program expected by the user, and indicating that the user expects to execute the instruction associated with the preset word that completely matches. In this case, it indicates that the display text of the relevant control in the designated second application is accurately hit, that is, the hit display text is exactly the same as the voice instruction, and there is no need for complex analysis and processing. The first application can directly send the voice instruction to the designated second application, so that the designated second application acts as the instruction execution program and directly executes the preset execution instruction corresponding to the hit display text, or the second application determines the target instruction to be executed based on the voice instruction (that is, the hit display text).

[0038] The matching operation in this step can be a matching operation at the text level. For example, the words in the voice command recognized by the first application are matched with the preset words in the traversed second application to obtain the words that are successfully matched by the traversed second application; the number of words that are successfully matched by the traversed second application is divided by the number of preset words in the traversed second application to obtain the matching degree corresponding to the traversed second application, so as to determine the matching degree corresponding to each second application.

[0039] The traversal method can be used to match the preset words in all second applications without omission, so as to calculate the matching degree corresponding to each second application, and then accurately and without omission determine the target second application whose matching degree is within the preset matching degree range. Among them, the matching degree calculation method for each second application is to divide the number of related words to simply and quickly obtain the matching degree of each second application.

[0040] S120: If the matching degree between the display text of the target control in the target second application and the voice instruction exceeds a preset matching degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

[0041] The preset matching degree value is less than 100%, which can be any percentage in the preset matching interval or a percentage outside the preset matching interval. For example, the preset matching interval is (60, 90), and the preset matching degree is 80% or 95%, or 50%.

[0042] The second application displayed in the virtual page includes corresponding controls, and the corresponding display text is displayed in the controls. Here, the display text of the corresponding control is matched with the voice command in a targeted manner, that is, the matching operation is performed at the control text level to determine the control that fits the voice command (the control that meets the user's execution expectations).

[0043] If the match degree between the display text of the target control in the target second application and the voice instruction exceeds the preset match degree, it indicates that the voice instruction hits the hot word in the target second application (i.e., the display text of the target control), and the first application sends the hit hot word to the target second application to hand over the execution action after the hit hot word to the target second application, that is, transfers the subsequent processing authority to the target second application, and the target second application directly executes the preset operation instruction corresponding to the hit hot word, or the target second application determines the target instruction to be executed based on the hit hot word, thereby improving the customizability of the target second application for the target instruction and being able to flexibly adjust the execution instruction.

[0044] In this embodiment, the first application performs matching and analysis based on the voice instructions recognized by itself and the relevant information of the second application displayed in the virtual page; when the matching degree between the display text of the target control in the target second application and the voice instruction exceeds the preset matching degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control; wherein, the voice instruction recognition program and the instruction execution program are not the same, achieving the purpose of cross-program instruction execution and breaking the mental confinement of instruction recognition and instruction execution in the same program. This embodiment can flexibly determine the instruction execution program and the target instruction to be executed, so that the determined instruction execution program meets the user's expectations to the greatest extent, thereby obtaining an instruction execution result that meets the user's expectations.

[0045] In another exemplary embodiment of the present application, how to determine the target control is described in detail. Figure 2 , Figure 2 is based on Figure 1 The exemplary embodiment shown is a flow chart of another method for determining an instruction execution program. The determination method is as follows Figure 1 On the basis of S110 to S120 shown, at least S210 to S220 are also included; wherein each second application includes at least one control, which is described in detail as follows:

[0046] S210: Match the voice instruction with the display text of each control in the target second application, and determine the control whose matching degree exceeds a preset matching degree as the control to be responded.

[0047] There may be multiple controls to be responded to, and each control to be responded to needs to be analyzed specifically to determine the target control.

[0048] S220: If the position information of the target control to be responded to satisfies the preset position information, and the correlation between the target control to be responded to and the voice command satisfies the preset correlation, the target control to be responded to is taken as the target control; wherein the target control to be responded to is any control among the controls to be responded to.

[0049] The position information is information such as the display position, display layer, and display pixel of the control in the virtual page, and is used to represent the visual display parameters related to each control to be responded.

[0050] The relevance is the degree to which the control fits the voice command. The higher the relevance, the more the control fits the user's command intention. In some embodiments, the magnitude of the relevance depends on the degree of match between the display text of the control and the voice command, for example: relevance = preset coefficient × degree of match between the display text and the voice command. In other embodiments, in order to better reflect the impact of relevance on determining the target control from different dimensions, the relevance also depends on the function of the control, the size of the control, etc., for example: relevance = first weight coefficient × degree of match between the display text and the voice command + second weight coefficient × kinetic energy value of the control + third weight coefficient × size of the control; wherein the first weight coefficient is greater than the second weight coefficient, and greater than the third weight coefficient, and the sum of the three weight coefficients is 1.

[0051] If the position information of all the controls to be responded to does not meet the preset position information, or the correlation between all the controls to be responded to and the voice command does not meet the preset correlation, the first application is used as the command execution program, and the voice command is used as the target command to be executed by the first application.

[0052] Among them, the first application is an application that executes global semantics, which is a program hidden under the foreground page (virtual page). Because if the user expects the second application to execute instructions, the voice command issued by the user will inevitably hit the display text of the control of the relevant second application in the foreground page, and this situation indicates that the hit display text is empty, and all the controls to be responded to do not conform to the user's instruction intention, that is, all the controls to be responded to are not the controls that the user expects to trigger, and then determine that all the second applications displayed in the virtual page are not the instruction execution programs expected by the user. It can be inferred that the application that the user expects to recognize voice instructions is the instruction execution program, then the first application will use itself as the instruction execution program, and directly use the recognized voice instructions as the target instructions to be instructed.

[0053] The relevant technology does not involve the hitting process at the control level, nor does it involve how to determine the target control, let alone the process of verifying the location information of the control to be responded to.

[0054] To this end, in another exemplary embodiment of the present application, how to determine whether the position information of each to-be-responded control satisfies the preset position information is described in detail. Figure 3 , Figure 3 is based on Figure 2 The exemplary embodiment shown is a flow chart of another method for determining an instruction execution program. The determination method is as follows Figure 2 On the basis of S210 to S220 shown, at least S310 to S320 are also included; wherein the position information of the control to be responded to includes the coordinates of each boundary point of the control to be responded to, and the layer of the control to be responded to in the virtual page, which is described in detail as follows:

[0055] S310: Match the coordinates of each boundary point of the target control to be responded to with the coordinates of the preset points in the preset area to obtain the number of successfully matched boundary point coordinates.

[0056] The shape, size, area, etc. of the control can be determined based on the coordinates of the control's boundary points.

[0057] The preset area is a designated display area in a virtual page. In some embodiments, the user views the display text of the target control in the virtual page, and if the user wishes to trigger a preset operation instruction corresponding to the display text, a voice instruction similar to or equal to the display text will be issued.

[0058] Usually, the user's sight will focus on the visual focus area of ​​the virtual page. The target control that the user expects to trigger is mostly located in the visual focus area. By setting the preset area as the visual focus area, the relevant control to be responded to is judged, thereby providing a condition for determining whether the relevant control to be responded to is the target control, and combining the user's visual habits to infer the user's expectations. Among them, the preset area can be the central area of ​​the virtual page, and the central area of ​​the virtual page is the visual focus area of ​​most users.

[0059] S320: If the number of successfully matched boundary point coordinates is greater than a preset number, and the layer of the target control to be responded to in the virtual page is within the preset layer range, it is determined that the position information of the target control to be responded to satisfies the preset position information.

[0060] If the number of successfully matched boundary point coordinates is greater than the preset number, the relevant boundary points representing the target control to be responded to are included in the preset area, and its display area overlaps with the preset area. If the number of successfully matched boundary point coordinates is greater, the overlap area representing the target control to be responded to and the preset area is greater, and the display area of ​​the target control to be responded to is easier for the user to focus on.

[0061] The preset layer range is the layer range with obvious display effect in the virtual page, for example, the first layer and the second layer from top to bottom, the user can clearly view the display page of the relevant layer. If the layer of the target control to be responded to in the virtual page is within the preset layer range, it indicates that the layer to which the target control to be responded to belongs is a layer that is easily noticed by the user, that is, a layer with obvious display effect in the virtual page.

[0062] This embodiment combines the display area of ​​the target control to be responded to with the layer in which it is located, so as to make a pre-determination of the target control at the position information level, and further determine whether the target control to be responded to is the target control.

[0063] In another exemplary embodiment of the present application, how to determine whether the position information of each control to be responded to satisfies the preset position information is described in detail. Figure 4 , Figure 4 is based on Figure 1 The exemplary embodiment shown is a flow chart of another method for determining an instruction execution program. The determination method is as follows Figure 1 Based on the S110 to S120 shown, at least S410 is also included, which is described in detail as follows:

[0064] S410: If the matching degrees corresponding to all second applications are less than the minimum end value of the preset matching degree interval, or the matching degrees of the displayed texts and voice commands of all controls in the virtual page do not exceed the preset matching degree, the first application is used as the instruction execution program, and the voice command is used as the target instruction to be executed by the first application.

[0065] Exemplarily, the preset matching degree interval is (60%, 90%), and the preset matching degree is 50%. If the matching degrees corresponding to all second applications are less than 60%, or the matching degrees of the display text and voice commands of all controls in the virtual page are less than 50%, then the display text representing the hit is empty, and all second applications displayed in the virtual page are not the instruction execution programs expected by the user. It can be inferred that the application that the user expects to recognize voice commands is the instruction execution program. The first application will use itself as the instruction execution program and directly use the recognized voice command as the target instruction to be instructed.

[0066] In another exemplary embodiment of the present application, the application scenarios of the above-mentioned multiple determination methods are exemplarily described. Figure 5 , Figure 5 It is a schematic diagram of an application scenario of the method for determining the instruction execution program of the present application. It includes a first application 100 and a foreground application 200. The first application 100 uses ASR (Automatic Speech Recognition) for voice instruction recognition, and uses NLP (Natural Language Processing) for voice processing. The foreground application 200 includes at least one second application displayed in a virtual page. After the foreground application is started, the Activity is registered with SDK (Software Development Kit).

[0067] The first application 100, as an execution subject, executes the determination method shown in any of the above exemplary embodiments, which is exemplarily described as follows:

[0068] After the first application 100 is awakened, one thread pulls preset words, such as hot words, from the foreground application 200 and caches them locally; another thread recognizes the voice information or voice text input by the user to identify the voice instructions therein.

[0069] The first application 100 matches the recognized voice command with the preset words in each second application; if the matching degree corresponding to a second application is 100%, the first application 100 sends the voice command to the second application so that the second application acts as a command execution program and determines the target command to be executed according to the voice command.

[0070] If the matching degree is not 100%, and the matching degree is within the preset matching degree interval, the second application is the target second application, and the first application 100 inputs the screenshot of the current foreground page (i.e., the virtual page) and the voice command into the preset model so that the preset model determines the command execution program. Among them, all preset matching degrees in the preset matching degree interval are less than 100%.

[0071] The preset model matches the voice command with the display text of each control in the target second application, and determines the target control from the controls to be responded whose matching degree exceeds the preset matching degree, and determines whether the target control exists according to the position information of the control and the correlation between the control and the voice command. Among them, the control whose position information among the controls to be responded meets the preset position information and whose correlation with the voice command meets the preset correlation is determined as the target control.

[0072] If the preset model outputs the target control, the first application 100 sends the display text of the target control to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

[0073] If the location information of all controls to be responded to does not meet the preset location information, or the correlation between all controls to be responded to and the voice command does not meet the preset correlation, the preset model outputs the result that the target control is empty, and the first application 100 uses itself as the command execution program and uses the voice command as the target command to be executed.

[0074] The first application 100 of the present application performs matching and analysis based on the voice instructions recognized by itself and the relevant information of the second application displayed in the virtual page; when the matching degree between the display text of the target control in the target second application and the voice instruction exceeds the preset matching degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control; wherein the voice instruction recognition program and the instruction execution program are not the same, achieving the purpose of cross-program instruction execution and breaking the mental confinement of instruction recognition and instruction execution in the same program. The present application can flexibly determine the instruction execution program and the target instruction to be executed, so that the determined instruction execution program meets the user's expectations to the greatest extent, thereby obtaining an instruction execution result that meets the user's expectations.

[0075] Another aspect of the present application also provides a device for determining an instruction execution program, such as Figure 6 As shown, Figure 6 600 is a schematic diagram of a structure of a determination device for an instruction execution program shown in an exemplary embodiment of the present application. The determination device 600 is applied to a first application program, and the determination device 600 includes:

[0076] The matching module 610 matches the recognized voice command with the preset words in each second application program to determine a target second application program whose matching degree is within a preset matching degree interval; wherein the second application program is the program displayed in the virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%;

[0077] The first determination module 630 sends the display text of the target control to the target second application if the match between the display text of the target control in the target second application and the voice instruction exceeds a preset match, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

[0078] In another exemplary embodiment, each second application includes at least one control; the determining device 600 further includes:

[0079] A text matching module matches the voice command with the display text of each control in the target second application, and determines the control with a matching degree exceeding a preset matching degree as a control to be responded to;

[0080] The target control determination module uses the target control to be responded to as the target control if the position information of the target control to be responded to satisfies the preset position information and the correlation between the target control to be responded to and the voice command satisfies the preset correlation; wherein the target control to be responded to is any control among the controls to be responded to.

[0081] In another exemplary embodiment, the determining device 600 further includes:

[0082] The second determination module uses the first application as the instruction execution program and the voice instruction as the target instruction to be executed by the first application if the position information of all the controls to be responded to does not meet the preset position information, or the correlation between all the controls to be responded to and the voice instruction does not meet the preset correlation.

[0083] In another exemplary embodiment, the position information of the control to be responded to includes the coordinates of each boundary point of the control to be responded to and the layer of the control to be responded to in the virtual page; the determining device 600 further includes:

[0084] The boundary point matching module matches the coordinates of each boundary point of the target control to be responded to with the coordinates of the preset points in the preset area to obtain the number of successfully matched boundary point coordinates;

[0085] The matching success module determines that the position information of the target control to be responded to satisfies the preset position information if the number of the successfully matched boundary point coordinates is greater than the preset number and the layer of the target control to be responded to in the virtual page is within the preset layer range.

[0086] In another exemplary embodiment, the determining device 600 further includes:

[0087] The third determination module uses the first application as the instruction execution program and the voice instruction as the target instruction to be executed by the first application if the matching degree corresponding to all second applications is less than the minimum end value of the preset matching degree interval, or the matching degree between the display text and the voice instruction of all controls in the virtual page does not exceed the preset matching degree.

[0088] In another exemplary embodiment, the matching module 610 includes:

[0089] The matching result unit sends the voice instruction to the designated second application if the matching degree corresponding to the designated second application is 100%, so that the designated second application is used as the instruction execution program to determine the target instruction to be executed according to the voice instruction.

[0090] In another exemplary embodiment, the matching module 610 includes:

[0091] A traversal matching unit matches the words in the voice instruction recognized by the first application with the preset words in the traversed second application to obtain the words that are successfully matched by the traversed second application;

[0092] The matching degree determination unit divides the number of words successfully matched by the traversed second application by the number of preset words in the traversed second application to obtain the matching degree corresponding to the traversed second application, so as to determine the matching degree corresponding to each second application.

[0093] The first application of this application matches and analyzes the voice instructions recognized by itself and the relevant information of the second application displayed in the virtual page; when the matching degree between the display text of the target control and the voice instruction in the target second application exceeds the preset matching degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control; wherein the voice instruction recognition program and the instruction execution program are not the same, achieving the purpose of cross-program instruction execution and breaking the mental confinement of instruction recognition and instruction execution in the same program. This application can flexibly determine the instruction execution program and the target instruction to be executed, so that the determined instruction execution program meets the user's expectations to the greatest extent, thereby obtaining an instruction execution result that meets the user's expectations.

[0094] It should be noted that the determination device provided in the above embodiment and the determination method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here.

[0095] Another aspect of the present application provides an electronic device, including: a controller; and a memory for storing one or more programs, which, when executed by the controller, executes the above-mentioned determination method.

[0096] See also Figure 7 , Figure 7 1 is a schematic diagram of the structure of a computer system of an electronic device shown in an exemplary embodiment of the present application, which shows a schematic diagram of the structure of a computer system of an electronic device suitable for implementing an embodiment of the present application.

[0097] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0098] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 to the random access memory (RAM) 703, such as executing the method in the above embodiment. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, ROM 702 and RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0099] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0100] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 709, and / or installed from a removable medium 711. When the computer program is executed by a central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0101] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0102] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0104] Another aspect of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the above determination method when executed by a processor. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0105] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the determination method provided in each of the above embodiments.

[0106] According to one aspect of an embodiment of the present application, a computer system is also provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM), such as executing the method in the above embodiment. In RAM, various programs and data required for system operation are also stored. CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0107] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0108] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A method for determining an instruction execution program, characterized in that: The determination method is applied to a first application program, and the determination method includes: Matching the recognized voice command with the preset words in each second application program to determine a target second application program whose matching degree is within a preset matching degree interval; wherein the second application program is a program displayed in the virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%; If the display text of the target control in the target second application and the voice instruction match more than a preset degree, the display text of the target control is sent to the target second application, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

2. The determination method according to claim 1, characterized in that: Each second application includes at least one control; the determination method further includes: Matching the voice command with the displayed text of each control in the target second application, and determining the control whose matching degree exceeds the preset matching degree as the control to be responded; If the position information of the target control to be responded to meets the preset position information, and the correlation between the target control to be responded to and the voice instruction meets the preset correlation, the target control to be responded to is used as the target control; wherein the target control to be responded to is any control among the controls to be responded to.

3. The determination method according to claim 2, characterized in that: The determination method further comprises: If the position information of all controls to be responded to does not meet the preset position information, or the correlation between all controls to be responded to and the voice command does not meet the preset correlation, the first application is used as the command execution program, and the voice command is used as the target command to be executed by the first application.

4. The determination method according to claim 2, characterized in that: The position information of the control to be responded to includes the coordinates of each boundary point of the control to be responded to and the layer of the control to be responded to in the virtual page; the determination method also includes: Matching the coordinates of each boundary point of the target control to be responded to with the coordinates of the preset points in the preset area to obtain the number of successfully matched boundary point coordinates; If the number of the successfully matched boundary point coordinates is greater than a preset number, and the layer of the target control to be responded to in the virtual page is within the preset layer range, it is determined that the position information of the target control to be responded to satisfies the preset position information.

5. The determination method according to claim 1, characterized in that: The determination method further comprises: If the matching degrees corresponding to all second applications are less than the minimum end value of the preset matching degree interval, or the matching degrees between the display texts of all controls in the virtual page and the voice command do not exceed the preset matching degree, the first application is used as the command execution program, and the voice command is used as the target command to be executed by the first application.

6. The determination method according to any one of claims 1 to 5, characterized in that: The matching of the recognized voice command with the preset words in each second application program includes: If the matching degree corresponding to the designated second application is 100%, the voice instruction is sent to the designated second application, so that the designated second application is used as an instruction execution program to determine the target instruction to be executed according to the voice instruction.

7. The determination method according to any one of claims 1 to 5, characterized in that: The matching of the recognized voice command with the preset words in each second application program includes: Matching the words in the voice command recognized by the first application with the preset words in the traversed second application to obtain the words successfully matched by the traversed second application; The number of words that are successfully matched by the traversed second application is divided by the number of preset words in the traversed second application to obtain the matching degree corresponding to the traversed second application, so as to determine the matching degree corresponding to each second application.

8. A device for determining an instruction execution program, characterized in that: The determining device is applied to the first application program, and the determining device includes: a matching module, matching the recognized voice command with preset words in each second application program, and determining a target second application program whose matching degree is within a preset matching degree interval; wherein the second application program is a program displayed in the virtual page; wherein all preset matching degrees in the preset matching degree interval are less than 100%; The first determination module sends the display text of the target control in the target second application to the target second application if the matching degree between the display text of the target control and the voice instruction exceeds a preset matching degree, so that the target second application acts as an instruction execution program and determines the target instruction to be executed according to the display text of the target control.

9. An electronic device, characterized in that: include: Controller; A memory for storing one or more programs, which, when executed by a controller, enables the controller to implement the determination method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the determination method according to any one of claims 1 to 7.

Images