Instruction execution method, device, terminal and storage medium based on state machine

Abstract

The present application discloses a method, device, terminal and storage medium for executing instructions based on a state machine, and relates to the field of human-computer interaction. The method comprises: receiving a voice instruction, the voice instruction is used to instruct to realize a target event through a third-party application; obtaining a script corresponding to the target event indicated by the voice instruction, the script comprising n processing steps for realizing the target event, the target processing step in the n processing steps is configured with a state machine, the state machine is used to trace back the target processing step according to a state transition condition, and n is a positive integer; and running the script. The method can improve the success rate of the voice assistant realizing the target event through a third-party application.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of human-computer interaction technology, and in particular to a state machine-based instruction execution method, device, terminal, and storage medium. Background Art

[0002] Voice assistant refers to software that provides corresponding services to users based on their voice input, and it is widely used in terminals.

[0003] In the related art, a user can operate a third-party application through a voice assistant. Specifically, the user sends a voice command to the voice assistant, the voice assistant obtains the corresponding script, and then operates the third-party application by executing the script. Summary of the invention

[0004] The embodiment of the present application provides a method, device, terminal and storage medium for executing instructions based on a state machine. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides an instruction execution method based on a state machine, the method comprising:

[0006] receiving a voice instruction, wherein the voice instruction is used to instruct to achieve a target event through a third-party application;

[0007] Obtaining a script corresponding to the target event indicated by the voice command;

[0008] The script is run, wherein the script includes n processing steps for realizing the target event, wherein the target processing step among the n processing steps is configured with a state machine, and the state machine is used to trace back the target processing step according to a state transfer condition, and n is a positive integer.

[0009] On the other hand, an embodiment of the present application provides an instruction execution device based on a state machine, and the method device includes:

[0010] An interaction module, configured to receive a voice instruction, wherein the voice instruction is used to instruct to achieve a target event through a third-party application;

[0011] An acquisition module, used to acquire a script corresponding to the target event indicated by the voice instruction;

[0012] A running module is used to run the script, wherein the script includes n processing steps for realizing the target event, wherein the target processing step among the n processing steps is configured with a state machine, and the state machine is used to backtrack the target processing step according to a state transfer condition, and n is a positive integer.

[0013] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the state machine-based instruction execution method as described in the first aspect.

[0014] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the state machine-based instruction execution method as described in the first aspect.

[0015] In another aspect, an embodiment of the present application provides a computer program product, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned instruction execution method based on a state machine.

[0016] The technical solution provided in the embodiments of the present application can bring at least the following beneficial effects:

[0017] By setting a state machine for one or more processing steps in the relevant script when operating a third-party application based on voice commands, before the state machine completes the state transfer, the terminal repeatedly executes the processing step set with the state machine until the processing step is successfully executed, at which time the terminal jumps to the next processing step. When there are fuzzy searches, retries, and advertisements, the terminal can complete fuzzy searches based on the state machine, execute the next step after the retry is successful, and execute the next step after the advertisement is played, thereby reducing the probability of script execution failure and improving the success rate of voice assistants operating third-party applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a structural block diagram of a computer device provided by an embodiment of the present application;

[0019] Figure 2 is a method flow chart of a state machine-based instruction execution method provided by another embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an instruction execution method based on a state machine provided in another embodiment of the present application;

[0021] Figure 4 is a method flow chart of a state machine-based instruction execution method provided by another embodiment of the present application;

[0022] Figure 5 is a schematic diagram of an instruction execution method based on a state machine provided in another embodiment of the present application;

[0023] Figure 6 is a schematic diagram of an instruction execution method based on a state machine provided in another embodiment of the present application;

[0024] Figure 7 is a method flow chart of a state machine-based instruction execution method provided by another embodiment of the present application;

[0025] Figure 8 is a block diagram of an instruction execution device based on a state machine provided by another embodiment of the present application;

[0026] Fig. 9 It is a structural diagram of a terminal provided in another embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0028] The following is an introduction to the relevant terms involved in the embodiments of the present application.

[0029] State machine: It is composed of state registers and combinational logic circuits. It can transfer states according to pre-set states based on control signals. It is the control center that coordinates related signal actions and completes specific operations.

[0030] Voice assistant: software that provides users with corresponding services based on their voice input.

[0031] Script: An executable file written in a specific descriptive language and in a certain format.

[0032] Figure 1 A schematic diagram of a computer device provided by an exemplary embodiment of the present application is shown, and the computer device includes a terminal 101.

[0033] The terminal 101 may include at least one of a smart phone, a laptop computer, a desktop computer, a tablet computer, a smart speaker, a smart robot, and a smart wearable device.

[0034] The terminal 101 runs a voice assistant 103 , a background service 102 , a third-party application 104 , and an accessibility management service 105 (AccessibleManagerService, also known as “auxiliary service”).

[0035] Voice assistant refers to software that provides corresponding services to users based on the user's voice input. The voice assistant receives the voice command issued by the user, performs voice recognition on the voice command to obtain the recognition result, determines the target event that the user needs to complete based on the recognition result, and then runs the script corresponding to the target event to realize the target event indicated by the user on the terminal. For example, the voice assistant receives the voice command "Call A" issued by the user, runs the call script, and enables the terminal to automatically call the contact "A".

[0036] The third-party application refers to an application running on the terminal other than the voice assistant 103 and the barrier-free management service 105. The third-party application includes but is not limited to: chat applications, shopping applications, music applications, video applications, game applications, etc.

[0037] The accessibility management service is used to monitor the operating status of the terminal, or to monitor the operating status of each application on the terminal. Exemplarily, when the voice assistant subscribes to the operating status of a third-party application to the accessibility management service through the accessibility service interface, the accessibility service management will send the operating data of the third-party application to the voice assistant. Exemplarily, the operating data includes the user interface of the third-party application. Exemplarily, the accessibility management service can be set in the operating system of the terminal or in the voice assistant.

[0038] The background service 102 provides background services for the voice assistant 103. Exemplarily, the background service 102 is used to receive the voice command sent by the voice assistant, perform automatic voice recognition and / or natural language understanding on the voice command to obtain a script corresponding to the voice command, and send the script to the voice assistant.

[0039] The terminal includes a first memory and a first processor. The first memory stores an instruction execution algorithm based on a state machine; the instruction execution algorithm based on the state machine is called and executed by the first processor to implement the instruction execution method based on the state machine provided in the present application. The first memory may include but is not limited to the following: random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read-only memory (Programmable Read-Only Memory, PROM), erasable read-only memory (Erasable Programmable Read-Only Memory, EPROM), and electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM).

[0040] The first processor may be composed of one or more integrated circuit chips. Optionally, the first processor may be a general-purpose processor, such as a central processing unit (CPU) or a network processor (NP). Optionally, the first processor may implement the state machine-based instruction execution method provided in the present application by running a program or code.

[0041] refer to Figure 2 , which shows a flow chart of a method provided by an embodiment of the present application, the method is used Figure 1 The voice assistant on the terminal in the embodiment.

[0042] Step 201: receiving a voice instruction, where the voice instruction is used to instruct to implement a target event through a third-party application.

[0043] The voice command is an audio command. Exemplarily, the voice command may be an audio command issued by a user and received by a microphone of the terminal, or the voice command may be an audio command sent by other terminals and received by the terminal, or the voice command may be an audio command stored locally in the terminal.

[0044] Exemplarily, the voice command contains text information, and voice recognition of the voice command can obtain the text command corresponding to the voice command. Exemplarily, the voice command is used to instruct the terminal (voice assistant) to complete the target event through a third-party application, and the target event refers to an event that needs to be implemented in the third-party application. Exemplarily, the terminal automatically performs certain operations through the third-party application according to the voice command to achieve the target event indicated in the voice command.

[0045] The voice instruction corresponds to at least one third-party application. Exemplarily, the text instruction corresponding to the voice instruction may include the program name of the third-party application. Of course, the voice instruction may not include the program name of the third-party application.

[0046] For example, when the user says to the voice assistant "Send a text message to Zhang San", the voice assistant receives the voice command "Send a text message to Zhang San", and the third-party application corresponding to the voice command is the "SMS" application.

[0047] For example, the user can also issue a voice command to the voice assistant that requires the cooperation of multiple third-party applications. For example, the voice assistant receives a command from the user "Send today's weather to Zhang San", and the third-party applications corresponding to the voice command include "SMS" and "Weather" applications. The terminal should first query the weather information in the "Weather" application, and then edit the weather information into a text message in the "SMS" application and send it to Zhang San.

[0048] Step 202, obtain a script corresponding to the target event indicated by the voice command, the script includes n processing steps for implementing the target event, the target processing step among the n processing steps is configured with a state machine, the state machine is used to backtrack the target processing step according to the state transfer condition, and n is a positive integer.

[0049] The voice assistant obtains the script corresponding to the voice instruction according to the voice instruction. Exemplarily, the voice assistant performs voice recognition on the voice instruction to obtain the text instruction corresponding to the voice instruction, determines the third-party application and the target event indicated by the voice instruction according to the text instruction, and determines the script corresponding to the voice instruction according to the third-party application and the target event.

[0050] A script is a program code used to implement a target event.

[0051] The script is divided into at least one processing step, which can refer to multiple actual steps to complete the target event. For example, when the target event is "send a text message to Zhang San", at least one processing step can include four processing steps: opening the text message application, searching for Zhang San's number, editing the text message content, and sending the text message to Zhang San. The processing step can also be to split the script program of the target event into multiple program codes, each program code is a processing step.

[0052] Exemplarily, at least one of the n processing steps is configured with a state machine.

[0053] The state machine is used to send the processing result to the specified processing step, or to the code step specified in the processing step, according to the state transfer condition satisfied by the processing result of the processing step, so as to complete the state transfer of each code step in the processing step by using the state machine.

[0054] Exemplarily, input data is input into a state machine, and the state machine runs the program code in the state machine according to the input data to obtain a processing result. Different processing results satisfy different state transition conditions. The state transition condition includes the processing result and the corresponding code step or processing step to be transferred. When the processing result satisfies a certain state transition condition, the processing result is sent to the code step or processing step corresponding to the state transition condition, so as to realize the backtracking operation of the processing step. Exemplarily, a processing step is composed of multiple code steps, and each code step corresponds to a section of logic code in the program code of the processing step.

[0055] For example, Figure 3As shown, this processing step 301 includes a first code step 302 and a state machine 303. When running this processing step 301, the first code step 302 is run first, and then the state machine 303 is run. When the processing result output by the state machine is the first result, the first result is returned to the first code step 302 according to the state transfer condition satisfied by the first result. The first code step 302 determines whether to continue running the state machine 303. If the state machine 303 continues to run, the repeated running of this processing step is completed.

[0056] Step 203, run the script.

[0057] The voice assistant runs the script program of the voice command, and the script program interacts with the third-party application to implement the target event indicated by the voice command on the third-party application.

[0058] To sum up, the method provided in this embodiment sets a state machine for one or more processing steps in the relevant script when operating a third-party application based on voice commands. Before the state machine completes the state transfer, the terminal repeatedly executes the processing step set with the state machine until the processing step is successfully executed. At this time, the terminal jumps to the next processing step. When there are fuzzy searches, retries, and advertisements, the terminal can complete fuzzy searches based on the state machine, execute the next step after the retry is successful, and execute the next step after the advertisement is played. This reduces the probability of script execution failure and improves the success rate of the voice assistant operating the third-party application.

[0059] The method in the related technology is that when there are fuzzy searches, retries, advertisements and other reasons during the operation of a third-party application, the script execution fails and the third-party application cannot be operated through the voice assistant.

[0060] Based on this, an embodiment of the present application provides an instruction execution method based on a state machine. When operating a third-party application based on voice instructions, a state machine is set for one or more processing steps in a related script. The state machine is used to enable the terminal to repeatedly run a processing step in the script until the processing step is successfully executed. At this time, the terminal jumps to the next processing step. When there are fuzzy searches, retries, and advertisements, the terminal can complete fuzzy searches based on the state machine, execute the next step after the retry is successful, and execute the next step after the advertisement is played. This reduces the probability of script execution failure and improves the success rate of the voice assistant operating the third-party application.

[0061] Figure 4 A state machine-based instruction execution method provided by an embodiment of the present application is shown. Figure 1 The voice assistant on the terminal in the embodiment. The method includes:

[0062] Step 201: receiving a voice instruction, where the voice instruction is used to instruct to implement a target event through a third-party application.

[0063] The voice command is issued by the user and is used to instruct the terminal to perform a certain operation based on the voice command to provide the user with corresponding services.

[0064] Optionally, the voice assistant calls the microphone component to receive the voice command. Optionally, the voice assistant calls the microphone component to receive the voice command when it is in the awake state. After the terminal detects the wake-up word of the voice assistant, or receives the operation signal corresponding to the designated button, the terminal switches the voice assistant to the awake state.

[0065] For example, the voice command is to send the message "telephone conference at 2 o'clock this afternoon" to Wang Wu.

[0066] Step 202, obtain a script corresponding to the target event indicated by the voice command, the script includes n processing steps for implementing the target event, the target processing step among the n processing steps is configured with a state machine, the state machine is used to backtrack the target processing step according to the state transfer condition, and n is a positive integer.

[0067] A script is an executable file written in a certain format using a specific descriptive language. The script includes at least one processing step for executing a voice command.

[0068] Taking the voice command of sending the message "phone conference at 2 o'clock this afternoon" to Wang Wu as an example, the processing steps included in the corresponding script include: opening the SMS application, searching for the contact Wang Wu, displaying the SMS editing interface of the contact Wang Wu, entering the content "phone conference at 2 o'clock this afternoon", and sending the edited SMS.

[0069] In one possible implementation, the voice assistant obtains the script from the background service, which stores the scripts corresponding to different target events of different third-party applications. The background service parses the third-party application to be operated and the target event indicated by the voice command, and then obtains the script corresponding to the third-party application and the target event. In another possible implementation, the voice assistant stores the script locally.

[0070] Exemplarily, the voice assistant sends a voice command to the background service, and the background service is used to recognize the voice command and obtain the script corresponding to the voice command according to the recognition result. The background service performs voice recognition on the voice command, determines the third-party application and target event indicated by the voice command according to the recognition result, and then determines the script and sends the script to the voice assistant. The voice assistant receives the script corresponding to the target event indicated by the voice command sent by the background service.

[0071] Step 2031, running the i-th processing step, where i is a positive integer not greater than n.

[0072] The voice assistant starts from the first processing step among the n processing steps and runs the n processing steps of the script sequentially.

[0073] Exemplarily, the i-th processing step includes m code steps, at least one of the m code steps is implemented by a state machine, and m is a positive integer. At least one state machine is provided in each processing step. Exemplarily, the state machine is used to trace the result back to the code step before the state machine.

[0074] For example, Figure 5 As shown, in an i-th processing step, 7 code steps are included: a first determination step 401, a search state machine 402, a second determination step 403, a first pre-state machine 404, a third determination step 405, a second pre-state machine 406 and an action program 407.

[0075] Step 2032, in response to the state machine being configured for the i-th processing step, backtrack to the i-th processing step according to the state transition condition satisfied by the processing result of the i-th processing step, where i is a positive integer not greater than n; or, in response to the state machine being configured for the i-th processing step, continue to run the i+1-th processing step according to the state transition condition satisfied by the processing result of the i-th processing step.

[0076] Exemplarily, the processing result of the above-mentioned i-th processing step refers to the intermediate result output by the state machine, and the state machine performs state transition according to the intermediate result. In response to the i-th processing step being configured with a state machine, according to the state transition condition satisfied by the intermediate result output by the state machine in the i-th processing step, the intermediate result is traced back to the code step located before the state machine.

[0077] This embodiment provides two functional state machines: a search state machine and a pre-state machine. The search state machine is used to implement multiple searches and fuzzy searches, and the pre-state machine is used to perform pre-processing on input data to obtain output data.

[0078] Wherein, when the i-th processing step is configured with a search state machine, the voice assistant, in response to the i-th processing step being configured with a search state machine and the i-th processing step not being successfully executed, backtracks and executes the i-th processing step; or, in response to the i-th processing step being configured with a search state machine and the number of executions of the i-th processing step not reaching the number threshold, backtracks and executes the i-th processing step; or, in response to the i-th processing step being configured with a search state machine and the i-th processing step not being successfully executed and the number of executions of the i-th processing step not reaching the number threshold, backtracks and executes the i-th processing step. In response to the i-th processing step being configured with a search state machine and the i-th processing step being successfully executed, continue to execute the i+1-th processing step; or, in response to the i-th processing step being configured with a search state machine and the number of executions of the i-th processing step reaching the number threshold, continue to execute the i+1-th processing step.

[0079] The search state machine includes a search program. Running the search state machine means running the search program, sending search instructions to third-party applications, and third-party applications perform search tasks according to the search instructions. The accessibility management service monitors the running status of third-party application data and forwards the running data of third-party applications to the voice assistant. The voice assistant determines the running result of the search program based on the received running data.

[0080] Exemplarily, when the i-th processing step is configured with a search state machine, the voice assistant backtracks to run the search state machine in response to the i-th processing step being configured with a search state machine and the search program in the search state machine has not been successfully executed; or, in response to the i-th processing step being configured with a search state machine and the number of times the search program in the search state machine has been executed does not reach a number threshold, the voice assistant backtracks to run the search state machine; or, in response to the i-th processing step being configured with a search state machine and the search program in the search state machine has not been successfully executed and the number of times the search program in the search state machine has not reached a number threshold, the voice assistant backtracks to run the search state machine. In response to the i-th processing step being configured with a search state machine and the search program in the search state machine running successfully, continue to run the i+1-th processing step, or continue to run the code step located after the search state machine in the i-th processing step; or, in response to the i-th processing step being configured with a search state machine and the number of times the search program in the i-th search state machine runs reaches a number threshold, continue to run the i+1-th processing step, or continue to run the code step located after the search state machine in the i-th processing step.

[0081] The state transition condition of the search state machine includes: in response to not obtaining the search result (running result), returning to the code step before the search state machine that there is no search result. In response to obtaining the search result (running result), returning to the code step before the search state machine that there is a search result.

[0082] The code step before the search state machine can be a determination step, which is used to determine whether the search times of the current search state machine reaches a threshold. When the threshold is reached, the next code step or the next processing step is run; when the threshold is not reached, the search state machine continues to run for searching.

[0083] When the i-th processing step is configured with a pre-state machine, the voice assistant performs pre-processing on the processing result of the i-th processing step in response to the i-th processing step being configured with a pre-state machine to obtain the pre-processing result; and returns the pre-processing result.

[0084] Exemplarily, when the i-th processing step is configured with a pre-state machine, the voice assistant performs pre-processing on the processing result of the input pre-state machine in response to the i-th processing step being configured with a pre-state machine to obtain the pre-processing result; returns the pre-processing result to the code step before the pre-state machine, or returns the pre-processing result to the code step after the pre-state machine.

[0085] Preprocessing refers to any form of data processing on input data. For example, preprocessing can be used to identify the correctness of search results and determine whether the search results are correct by using fuzzy matching. Alternatively, preprocessing can be used to perform some type of data processing on the intermediate results output by a code step, and the processing results can be input into the next code step, or returned to the previous code step.

[0086] Exemplarily, at least one state machine may be configured in a processing step. Figure 6 As shown, the script includes four processing steps: step 1, step 2, step 3 and step 4. Step 2 is configured with a third pre-state machine 408, and step 3 is configured with a search state machine 402 and a fourth pre-state machine 409.

[0087] For example, Figure 5As shown, the i-th processing step is configured with a search state machine and two pre-processing state machines. First, the first judgment step 401 is executed to determine whether a search is required. If a search is required, the search state machine 402 is executed to search, and the search results are sent to the second judgment step 403 to determine whether pre-processing is required. If pre-processing is required, the first pre-processing state machine 404 is executed to pre-process the search results (for example, determine the correctness of the search results) to obtain the pre-processing results, and the pre-processing results are returned to the second judgment step 403. The second judgment step 403 determines whether to continue pre-processing. If pre-processing is not required, the pre-processing results are returned to the search state machine 402, and the search state machine returns the pre-processing results to the first judgment step 401. The first judgment step 401 determines whether to continue searching. If searching is continued, the search state machine 402 is continuously executed; until the first judgment step 401 determines that the number of searches reaches the threshold, or the correct search results are obtained, then there is no need to search, and the third judgment step 405 is executed. The third judgment step 405 determines whether pre-processing is required. If pre-processing is required, the pre-processing state machine 406 is run to perform pre-processing to obtain the pre-processing result, and the pre-processing result is returned to the action program 407, and the action program 407 is run; if pre-processing is not required, the action program 407 is directly run.

[0088] Step 2033: In response to the state machine not being configured for the i-th processing step, continue to run the i+1-th step.

[0089] For example, some of the n processing steps need to send instructions to the third-party application to control the third-party application to achieve the target event. Or, some of the n processing steps need to receive the operation data of the third-party application sent by the accessibility management service.

[0090] The voice assistant runs the i-th processing step and sends the i-th instruction corresponding to the i-th processing step to the third-party application; receives the running data of the third-party application sent by the accessibility service, and the accessibility service is used to monitor the running status of the third-party application; determines the processing result of the i-th processing step based on the running data.

[0091] Step 2034, repeat the above steps to run n processing steps of the script.

[0092] Exemplarily, the voice assistant sequentially runs n processing steps in the script according to the above method.

[0093] To summarize, the technical solution provided by the embodiment of the present application sets a state machine for one or more processing steps in the relevant script when operating a third-party application based on voice commands. Before the state machine completes the state transfer, the terminal repeatedly executes the processing step set with the state machine until the processing step is successfully executed. At this time, the terminal jumps to the next processing step. When there are fuzzy searches, retries, and advertisements, the terminal can complete fuzzy searches based on the state machine, execute the next step after the retry is successful, and execute the next step after the advertisement is played. This reduces the probability of script execution failure and improves the success rate of the voice assistant operating the third-party application.

[0094] Figure 7 A state machine-based instruction execution method provided by an embodiment of the present application is shown. Figure 1 The voice assistant on the terminal in the embodiment. The method includes:

[0095] Step 501: The voice assistant initializes the voice engine proxy.

[0096] Step 502: The voice assistant receives a voice command issued by a user.

[0097] Step 503: The voice assistant requests the background service to update the simulated click package.

[0098] Step 504: the background service determines the package name that supports simulated clicks.

[0099] Step 505: The background service sends a simulated click package name to the voice assistant.

[0100] Step 506: The voice assistant determines the version number based on the simulated click package name and sends the version number of the simulated click package to the background service.

[0101] Step 507: The background service obtains the simulated click package name and version number.

[0102] Step 508: The background service returns a simulated click packet to the voice assistant.

[0103] Step 509: The voice assistant sends a subscription request to the accessibility management service, requesting the accessibility management service to start synchronizing the running data of the third-party application to the voice assistant.

[0104] Step 510: The voice assistant sends a voice command to the background service by calling the local service.

[0105] Step 511, the background service performs data analysis on the voice command, and obtains the script corresponding to the voice command according to the data analysis result.

[0106] Step 512: The background service sends the script to the voice assistant.

[0107] Step 513: The voice assistant performs data analysis on the script.

[0108] Step 514: The voice assistant starts executing the voice command and sends the command to the third-party application.

[0109] Step 515, the voice assistant executes the processing steps in the script.

[0110] Exemplarily, there is at least one processing step configured with a state machine, and the voice assistant backtracks and runs the processing step according to the program running results and the state transition conditions of the state machine, so as to repeat the processing step when the processing step is not successfully run, thereby improving the success rate of script running.

[0111] Step 516: The voice assistant feeds back the script running status to the background service.

[0112] Step 517: The background service performs data analysis on the running status and updates the script according to the data analysis result.

[0113] Step 518: The voice assistant receives the update script.

[0114] Step 519: The third-party application receives the instruction and executes the action indicated by the instruction.

[0115] Step 520: The third-party application synchronizes operation data with the accessibility management service.

[0116] Step 521: The accessibility management service synchronizes the running data of the third-party application to the voice assistant.

[0117] Step 522: The voice assistant checks whether the voice command has been executed based on the operating data.

[0118] Step 523, if the execution is completed, the voice assistant sends an end service instruction to the accessibility management service so that the accessibility management service no longer synchronizes the running data of the third-party application to the voice assistant.

[0119] Step 524, the voice assistant closes the interface display.

[0120] To summarize, the technical solution provided by the embodiment of the present application sets a state machine for one or more processing steps in the relevant script when operating a third-party application based on voice commands. Before the state machine completes the state transfer, the terminal repeatedly executes the processing step set with the state machine until the processing step is successfully executed. At this time, the terminal jumps to the next processing step. When there are fuzzy searches, retries, and advertisements, the terminal can complete fuzzy searches based on the state machine, execute the next step after the retry is successful, and execute the next step after the advertisement is played. This reduces the probability of script execution failure and improves the success rate of the voice assistant operating the third-party application.

[0121] The following is an embodiment of the device of the present application. For parts not described in detail in the embodiment of the device, reference may be made to the technical details disclosed in the above method embodiment.

[0122] Please refer to Figure 8 , which shows a block diagram of an instruction execution device based on a state machine provided by an exemplary embodiment of the present application. The instruction execution device based on the state machine can be implemented as all or part of the terminal through software, hardware or a combination of both. The instruction execution device based on the state machine includes:

[0123] An interaction module 601 is used to receive a voice instruction, where the voice instruction is used to instruct to achieve a target event through a third-party application program;

[0124] An acquisition module 602 is used to acquire a script corresponding to the target event indicated by the voice instruction, wherein the script includes n processing steps for implementing the target event, wherein a target processing step among the n processing steps is configured with a state machine, and the state machine is used to backtrack the target processing step according to a state transition condition, and n is a positive integer;

[0125] The running module 603 is used to run the script.

[0126] In an optional embodiment, the running module 603 is used to run the i-th processing step, where i is a positive integer not greater than n;

[0127] The running module 603 is configured to, in response to the state machine being configured for the i-th processing step, trace back the i-th processing step according to the state transition condition satisfied by the processing result of the i-th processing step, where i is a positive integer not greater than n; or, in response to the state machine being configured for the i-th processing step, continue to run the i+1-th processing step according to the state transition condition satisfied by the processing result of the i-th processing step;

[0128] The running module 603 is used for continuing to run the (i+1)th step in response to the state machine not being configured in the i-th processing step;

[0129] The running module 603 is used to repeat the above steps to run the n processing steps of the script.

[0130] In an optional embodiment, the state machine includes a search state machine;

[0131] The running module 603 is used for backtracking and running the i-th processing step in response to the i-th processing step being configured with the search state machine and the i-th processing step being unsuccessful;

[0132] or,

[0133] The running module 603 is used for backtracking and running the i-th processing step in response to the i-th processing step being configured with the search state machine and the number of executions of the i-th processing step not reaching a number threshold;

[0134] or,

[0135] The running module 603 is used to backtrack and run the i-th processing step in response to the i-th processing step being configured with the search state machine, the i-th processing step not being successfully run, and the number of runs of the i-th processing step not reaching a number threshold.

[0136] In an optional embodiment, the state machine includes a search state machine;

[0137] The running module 603 is used for continuing to run the (i+1)th processing step in response to the i-th processing step being configured with the search state machine and the i-th processing step being successfully run;

[0138] or,

[0139] The running module 603 is used to continue running the (i+1)th processing step in response to the i-th processing step being configured with the search state machine and the i-th processing step having been run a number of times reaching a threshold.

[0140] In an optional embodiment, the state machine includes a front state machine;

[0141] The operation module 603 is used for performing pre-processing on the processing result of the i-th processing step in response to the i-th processing step being configured with the pre-state machine to obtain a pre-processing result;

[0142] The operation module 603 is used to return the pre-processing result.

[0143] In an optional embodiment, the device further comprises:

[0144] A sending module 605, configured to execute the i-th processing step to send the i-th instruction corresponding to the i-th processing step to the third-party application;

[0145] A receiving module 604 is used to receive the running data of the third-party application sent by the accessibility service, and the accessibility service is used to monitor the running status of the third-party application;

[0146] The determination module 606 is used to determine the processing result of the i-th processing step based on the operation data.

[0147] In an optional embodiment, the device further comprises:

[0148] The sending module 605 is used to send the voice command to the background service, and the background service is used to recognize the voice command and obtain the script corresponding to the voice command according to the recognition result;

[0149] The receiving module 604 is used to receive the script corresponding to the target event indicated by the voice instruction sent by the background service.

[0150] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0151] The present application also provides a terminal, which includes a processor and a memory, wherein at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the state machine-based instruction execution method provided by the above-mentioned various method embodiments. It should be noted that the terminal can be as follows Fig. 9 The terminal provided.

[0152] Fig. 9The structure block diagram of a terminal 900 provided by an exemplary embodiment of the present application is shown. The terminal 900 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 900 may also be called a user account device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0153] Typically, the terminal 900 includes a processor 901 and a memory 902 .

[0154] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0155] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 to implement the method for generating a health index or the method for generating a health index provided in the method embodiment of the present application.

[0156] In some embodiments, the terminal 900 may further optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, a positioning assembly 908 and a power supply 909.

[0157] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0158] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Exemplarily, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user account identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0159] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, and the front panel of the terminal 900 is set; in other embodiments, the display screen 905 can be at least two, which are respectively set on different surfaces of the terminal 900 or are folded; in some other embodiments, the display screen 905 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 900. Even, the display screen 905 can also be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0160] The camera assembly 906 is used to capture images or videos. Exemplarily, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0161] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user account and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 900. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0162] The positioning component 908 is used to locate the current geographical location of the terminal 900 to implement navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system or Russia's Galileo system.

[0163] The power supply 909 is used to power various components in the terminal 900. The power supply 909 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0164] In some embodiments, the terminal 900 further includes one or more sensors 910 , including but not limited to: an acceleration sensor 911 , a gyroscope sensor 912 , a pressure sensor 913 , a fingerprint sensor 914 , an optical sensor 915 , and a proximity sensor 916 .

[0165] The acceleration sensor 911 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 900. For example, the acceleration sensor 911 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the display screen 905 to display the user account interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used to collect motion data of games or user accounts.

[0166] The gyro sensor 912 can detect the body direction and rotation angle of the terminal 900, and the gyro sensor 912 can cooperate with the acceleration sensor 911 to collect the 3D action of the user account on the terminal 900. The processor 901 can implement the following functions based on the data collected by the gyro sensor 912: motion sensing (such as changing the UI according to the tilt operation of the user account), image stabilization during shooting, game control, and inertial navigation.

[0167] The pressure sensor 913 can be set on the side frame of the terminal 900 and / or the lower layer of the display screen 905. When the pressure sensor 913 is set on the side frame of the terminal 900, it can detect the holding signal of the user account on the terminal 900, and the processor 901 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is set on the lower layer of the display screen 905, the processor 901 controls the operability controls on the UI interface according to the pressure operation of the user account on the display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0168] The fingerprint sensor 914 is used to collect the fingerprint of the user account, and the processor 901 identifies the identity of the user account based on the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the identity of the user account based on the collected fingerprint. When the identity of the user account is identified as a trusted identity, the processor 901 authorizes the user account to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 914 can be set on the front, back or side of the terminal 900. When a physical button or a manufacturer logo is set on the terminal 900, the fingerprint sensor 914 can be integrated with the physical button or the manufacturer logo.

[0169] The optical sensor 915 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 according to the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera component 906 according to the ambient light intensity collected by the optical sensor 915.

[0170] The proximity sensor 916, also called a distance sensor, is usually arranged on the front panel of the terminal 900. The proximity sensor 916 is used to collect the distance between the user account and the front of the terminal 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user account and the front of the terminal 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 916 detects that the distance between the user account and the front of the terminal 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from the screen-off state to the screen-on state.

[0171] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal 900, and the terminal 900 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0172] The memory also includes one or more programs, which are stored in the memory, and the one or more programs include instructions for executing the state machine-based instruction execution method provided in the embodiment of the present application.

[0173] The present application also provides a computer device, which includes: a processor and a memory, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the state machine-based instruction execution method provided in the above-mentioned method embodiments.

[0174] The present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the state machine-based instruction execution method provided by the above-mentioned method embodiments.

[0175] The present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the instruction execution method based on the state machine provided in the above optional implementation.

[0176] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0177] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0178] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for executing instructions based on a state machine, characterized in that: The method comprises: receiving a voice instruction, wherein the voice instruction is used to instruct to achieve a target event through a third-party application; Obtaining a script corresponding to the target event indicated by the voice command, the script comprising n processing steps for implementing the target event, a target processing step among the n processing steps being configured with a state machine, the state machine being used to backtrack the target processing step according to a state transition condition, and n being a positive integer; Run the i-th processing step, where i is a positive integer not greater than n; In response to the state machine being configured for the i-th processing step, the i-th processing step is backtracked according to a first state transition condition satisfied by a processing result of the i-th processing step; or, in response to the state machine being configured for the i-th processing step, the i+1-th processing step is continued to be executed according to a second state transition condition satisfied by a processing result of the i-th processing step; In response to the state machine not being configured in the i-th processing step, continuing to execute the i+1-th step; Repeat the above steps to run the n processing steps of the script; Wherein, the state machine includes a search state machine, and the search state machine is used to implement multiple searches and / or fuzzy searches; The first state transition condition includes: the i-th processing step configured with the search state machine is not successfully executed and / or the number of executions does not reach a number threshold; The second state transition condition includes: the i-th processing step configured with the search state machine runs successfully or the number of runs reaches the number threshold.

2. The method according to claim 1, characterized in that The state machine further includes a pre-state machine, wherein the state machine is configured in response to the i-th processing step, and backtracking the i-th processing step according to a first state transition condition satisfied by a processing result of the i-th processing step includes: In response to the i-th processing step being configured with the pre-state machine, pre-processing the processing result of the i-th processing step to obtain a pre-processing result; Returns the pre-processing result.

3. The method according to claim 1 or 2, characterized in that: The running of the i-th processing step comprises: Running the i-th processing step to send an i-th instruction corresponding to the i-th processing step to the third-party application; receiving the running data of the third-party application sent by the accessibility service, wherein the accessibility service is used to monitor the running status of the third-party application; The processing result of the i-th processing step is determined based on the operating data.

4. The method according to claim 1 or 2, characterized in that: The step of obtaining a script corresponding to the target event indicated by the voice instruction includes: Sending the voice command to a background service, the background service is used to recognize the voice command and obtain a script corresponding to the voice command according to the recognition result; Receive a script corresponding to the target event indicated by the voice instruction sent by the background service.

5. An instruction execution device based on a state machine, characterized in that: The device comprises: An interaction module, configured to receive a voice instruction, wherein the voice instruction is used to instruct to achieve a target event through a third-party application; an acquisition module, used for acquiring a script corresponding to the target event indicated by the voice instruction, wherein the script includes n processing steps for realizing the target event, wherein a target processing step among the n processing steps is configured with a state machine, and the state machine is used for backtracking the target processing step according to a state transition condition, and n is a positive integer; A running module, used for running the i-th processing step, where i is a positive integer not greater than n; The running module is configured to, in response to the state machine being configured for the i-th processing step, trace back the i-th processing step according to a first state transition condition satisfied by a processing result of the i-th processing step; or, in response to the state machine being configured for the i-th processing step, continue to run the i+1-th processing step according to a second state transition condition satisfied by a processing result of the i-th processing step; The running module is used for continuing to run the (i+1)th step in response to the state machine not being configured in the i-th processing step; The running module is used to repeat the above steps to run the n processing steps of the script; Wherein, the state machine includes a search state machine, and the search state machine is used to implement multiple searches and / or fuzzy searches; The first state transition condition includes: the i-th processing step configured with the search state machine is not successfully executed and / or the number of executions does not reach a number threshold; The second state transition condition includes: the i-th processing step configured with the search state machine runs successfully or the number of runs reaches the number threshold.

6. A terminal, characterized in that: The terminal includes a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded by the processor and executes the state machine-based instruction execution method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the state machine-based instruction execution method according to any one of claims 1 to 4.

Images