Data processing method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of artificial intelligence, and provides a data processing method and device, electronic equipment, a storage medium and a program product, the data processing method is applied to the electronic equipment comprising a voice assistant program, and the method comprises the following steps: adding a function of processing a voice assistant program request for a preset application program through a configuration file, defining an intention type which can be processed by the preset application program; accessing a preset application program to the voice assistant program to obtain an application intention relation table; obtaining a voice instruction through the voice assistant program, and recognizing a target intention type according to the voice instruction; according to the target intention type and the application intention relation table, a target application program is determined from the preset application programs through the voice assistant program, and an operation task corresponding to the target intention type in the target application program is executed. According to the invention, through the voice assistant program, the voice instruction is transmitted to the corresponding application program and the corresponding operation is executed in the application program, so that the convenience of user operation is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a data processing method, apparatus, electronic device, storage medium, and program product. Background Art

[0002] On mobile devices, voice assistants have become an important means for users to interact with the devices. Voice assistants usually use "wake-up words" or specific instructions to launch specific applications or perform certain operations. To achieve such voice interaction, the voice assistant system in the device processes the user's natural language input, analyzes the user's instruction intention through natural language processing and machine learning algorithms, and then matches it to the corresponding system service or application function. However, the solutions of the prior art can only open the specified application, and the voice assistant cannot interfere with the operations of the application, so it cannot further open the specified page of the application, etc. Summary of the Invention

[0003] The present invention provides a data processing method, apparatus, electronic device, storage medium, and program product to solve the problem in the prior art that the voice assistant can only open the specified application and cannot further operate on the application.

[0004] The present invention provides a data processing method applied to an electronic device. The electronic device includes a voice assistant program. The data processing method includes: adding a function of processing requests of the voice assistant program to a preset application through a configuration file, and defining the type of intention that the preset application can process; connecting the preset application to the voice assistant program, obtaining the type of intention that the preset application can process reported by the preset application, and obtaining an application-intention relationship table; where the application-intention relationship table records the corresponding relationship between the type of intention and the preset application; obtaining a voice command through the voice assistant program, and identifying a target intention type according to the voice command; according to the target intention type and the application-intention relationship table, determining a target application from the preset applications through the voice assistant program, and executing an operation task corresponding to the target intention type in the target application.

[0005] According to a data processing method provided by the present invention, identifying a target intention type according to a voice command further includes: binding the type of intention and a preset voice command to obtain an intention-command relationship table; where the intention-command relationship table records the corresponding relationship between the type of intention and the preset voice command; obtaining the target intention type according to the voice command and the intention-command relationship table.

[0006] A data processing method provided by the present invention further includes: displaying an intent instruction relationship table on a voice instruction management page; obtaining a management instruction of a user on the voice instruction management page; where the management instruction includes an addition instruction and an editing instruction, the addition instruction is used to add a correspondence between a new intent type and a preset voice instruction; the editing instruction is used to adjust the correspondence between an existing intent type and a preset voice instruction; and updating the intent instruction relationship table according to the management instruction.

[0007] A data processing method provided by the present invention adds a function of processing voice assistant program requests to a preset application program through a configuration file, and defines intent types that the preset application program can process, including: adding a function of processing voice assistant program requests to the preset application program by configuring a first file; describing the reason for using the voice assistant program when requesting user authorization by configuring a second file; and defining intent types by creating a third file.

[0008] A data processing method provided by the present invention further includes: editing the second file through a main application module to add intent types that the preset application program can process; the main application module is provided by the preset application program; editing the second file through an intent processing module to describe intent types that the intent processing module can process; the intent processing module is provided by the voice assistant program.

[0009] After performing an operation task corresponding to a target intent type in a target application program, a data processing method provided by the present invention further includes: displaying a page after performing the operation task corresponding to the target intent type in the target application program.

[0010] The present invention also provides a data processing device deployed on an electronic device, the electronic device includes a voice assistant program, and the data processing device includes: a configuration module, configured to add a function of processing voice assistant program requests to a preset application program through a configuration file, and define intent types that the preset application program can process; an integration module, configured to connect the preset application program to the voice assistant program, obtain the intent types that the preset application program can process reported, and obtain an application intent relationship table; where the application intent relationship table records the correspondence between intent types and the preset application program; a voice recognition module, configured to obtain a voice instruction through the voice assistant program, and recognize a target intent type according to the voice instruction; an intent execution module, configured to determine a target application program from the preset application programs through the voice assistant program according to the target intent type and the application intent relationship table, and perform an operation task corresponding to the target intent type in the target application program.

[0011] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the data processing method as described in any one of the above is implemented.

[0012] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the data processing method as described in any one of the above is implemented.

[0013] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the data processing method as described in any one of the above is implemented.

[0014] The present invention provides a data processing method, apparatus, electronic device, storage medium, and program product. The data processing method is applied to an electronic device, and the electronic device includes a voice assistant program. The present invention can expose the capabilities and features of an application program to the voice assistant program. Without the user stating the name of the application program, by only telling the voice assistant program a voice instruction including the execution purpose, the corresponding application program can be started and the corresponding operation can be performed within the application, meeting the specific needs of the user; the present invention enables the user to automatically complete specific tasks through the voice assistant program; by applying the wake-up and instruction parsing functions of the voice assistant program, the voice instruction is transmitted to the corresponding application program and the corresponding operation is performed within the application program; it is applicable to the integration of the voice assistant program, application program, and its background service to ensure convenient operation in all scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0016] Figure 1 is a schematic flowchart of the data processing method provided by an embodiment of the present invention.

[0017] Figure 2 is an interactive schematic diagram of the preliminary configuration in the data processing method provided by an embodiment of the present invention.

[0018] Figure 3 is an interactive schematic diagram of the installation stage in the data processing method provided by an embodiment of the present invention.

[0019] Figure 4 is an interactive schematic diagram of the running stage in the data processing method provided by an embodiment of the present invention.

[0020] Figure 5 It is a schematic structural diagram of a data processing device provided by an embodiment of the present invention.

[0021] Figure 6 It is a schematic entity structure diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0022] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0023] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the embodiments of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0024] In the related art, the user needs to wake up the voice assistant, say the name of the application to be opened, and the voice assistant queries the name in the application database according to the result of voice recognition, and decides which application to open according to the query result, or prompts the user to re-enter the voice, or selects any application from multiple query results to open.

[0025] However, this method requires the prior establishment of an application database. If an application is installed or uninstalled, it also involves the detection of the installation and uninstallation events of the application and the addition and deletion operations of the database; and it can only open the specified application and cannot further open the specified page of the application, etc.

[0026] Based on this, the present invention provides a data processing method, which is applied to an electronic device, and the electronic device includes a voice assistant program. The purpose of the present invention is to open the functions and features of the application to the voice assistant, so that the user can start the application and meet the specific needs of the user only by describing his intention to the voice assistant without explicitly mentioning the name of the application.

[0027] Please refer to Figure 1 ,Figure 1 It is a schematic flowchart of the data processing method provided by an embodiment of the present invention. In this embodiment, the data processing method may include steps S110 to S140, and the specific steps are as follows: S110: Add the function of processing voice assistant program requests to a preset application through a configuration file, and define the intent types that the preset application can process. S120: Connect the preset application to the voice assistant program, obtain the intent types that the preset application can process reported by the preset application, and obtain an application-intent relationship table; where the application-intent relationship table records the corresponding relationship between the intent type and the preset application. S130: Obtain a voice command through the voice assistant program, and identify the target intent type according to the voice command.

[0028] S140: According to the target intent type and the application-intent relationship table, determine the target application from the preset applications through the voice assistant program, and execute the operation task corresponding to the target intent type in the target application.

[0029] The configuration file is the key to realizing the interaction between the preset application and the voice assistant in this embodiment. It defines which requests of the voice assistant the application can understand and process, and how to respond to these requests. Through the configuration file, developers can flexibly expand the functions of the application without modifying the core code of the application.

[0030] It can be understood that the preset application is an application selected by the developer that can implement the instructions of the voice assistant program, which enables the preset application to quickly adapt to new voice assistant functions or changes in user needs.

[0031] In this embodiment, it is necessary to establish a communication mechanism between the voice assistant program and the preset application so that requests can be sent and responses can be received. Optionally, during the process of installing the preset application in the electronic device, the preset application can be connected to the voice assistant program.

[0032] Optionally, during the process of starting the preset application in the electronic device, it can also be detected whether the preset application is connected to the voice assistant program. If not, the user can be asked to obtain user authorization.

[0033] After the preset application is connected to the voice assistant program, it will report the intent types it supports to the voice assistant program. The voice assistant program constructs an application-intent relationship table according to the intent information reported by the preset application. The application-intent relationship table records the corresponding relationship between the intent type and the preset application, that is, which preset applications support each intent type.

[0034] The voice assistant program can capture the user's voice commands through the built-in microphone or the connected external microphone of the electronic device, and then cooperate with voice recognition to obtain the target intention type of the user. The voice assistant looks up the application program that can handle this intention, that is, the target application program, in the application-intention relationship table. The application-intention relationship table records the corresponding relationship between the intention type and the preset application program.

[0035] For example, the application-intention relationship table records that the "query weather" intention is supported by the "weather query application", and the "play music" intention is supported by the "music playback application". Then the "weather query application" is the target application program corresponding to the target intention type of "query weather", and the "music playback application" is the target application program corresponding to the target intention type of "play music".

[0036] This embodiment provides a full-scenario wake-up mechanism for the voice assistant based on the mobile system, enabling users to automatically complete specific tasks through the voice assistant, such as querying phone bills and paying bills in the communication operator scenario. This embodiment is applicable to the integration of the voice assistant, application programs, and their background services to ensure convenient operations in all scenarios. By applying the wake-up and instruction parsing functions of the voice assistant, the voice commands are transmitted to the corresponding operator application and the corresponding operations are executed within the application.

[0037] In this embodiment, the application-intention relationship table is a data structure used to record the intention types that the preset application programs can handle and the corresponding relationship between these intention types and the application programs. The role of the application-intention relationship table is to help the voice assistant program quickly identify the intention of the user's instructions and correctly route the instructions to the application program that can handle this intention. Through the application-intention relationship table, the voice assistant program can efficiently achieve seamless docking between the user's instructions and the functions of the application programs.

[0038] It should be noted that each intention type has a unique identifier for differentiation in the application-intention relationship table; each preset application program also has a unique identifier for identifying the specific application program in the application-intention relationship table.

[0039] Optionally, the intention type and the preset application program can be in a one-to-one correspondence relationship. For example, one application program can handle one intention type; the preset application program and the intention type can also be in a one-to-many relationship. For example, one preset application program can handle multiple intention types; or one intention type may correspond to multiple preset application programs. In this case, it is also necessary to set the default preset application program corresponding to this intention type.

[0040] The embodiments of the present invention solve the problem of natural interaction between users and different application programs, enabling users to quickly jump to a specified function page (such as querying phone bills) through a voice assistant program. Compared with the in-app menu navigation in the related art, this interaction method of direct access greatly reduces the operation steps, meets the user's pursuit of convenience and immediacy, and is particularly friendly to users who are not familiar with the application operation process (such as elderly users).

[0041] Exemplarily, multiple application programs are installed on a mobile phone, including application program A of a communication operator and application program B for music playback. Application program A has functions such as querying phone bills, and application program B has functions such as playing music.

[0042] When the user says the voice command: "Query my phone bill balance" to the voice assistant program on the mobile phone, the voice assistant program can jump to the "Query Phone Bill" function page of application program A, and this page will automatically load and display relevant information such as the user's current phone bill balance. In addition, the user can further perform other relevant operations on this page, such as viewing the phone bill consumption details, recharging the phone bill, etc., without manually opening the communication operator's application program and searching for the "Query Phone Bill" function step by step.

[0043] When the user says the voice command: "Play the favorite song C of mine" to the voice assistant program, the voice assistant program can jump to the "My Favorites" or "My Music" page of application program B, automatically search for and play song C.

[0044] In some embodiments, the step of identifying the target intent type according to the voice command may specifically further include: Binding the intent type and the preset voice command to obtain an intent command relationship table; where the intent command relationship table records the corresponding relationship between the intent type and the preset voice command; according to the voice command and the intent command relationship table, obtaining the target intent type.

[0045] The voice command is a command or request input by the user through voice, and the preset voice command refers to a predefined and recognizable voice command template.

[0046] The intent command relationship table records the association relationship between the intent type and the preset voice command. When a voice command sent by the user is received, the voice command can be compared with the preset voice commands in the intent command relationship table to determine the preset voice command that can match the voice command in the intent command relationship table. Then, in the intent command relationship table, the intent type corresponding to the matching preset voice command is used as the target intent type to determine the specific intent that the user wants to achieve.

[0047] Optionally, the intent type and the preset voice command can be in a one-to-one binding relationship. For example, one preset voice command is bound to one intent type; the intent type and the preset voice command can also be in a one-to-many relationship. For example, multiple preset voice commands can be bound to the same intent type, which means that the user can use multiple different expressions to trigger the same intent type. This embodiment can also recognize these different expressions and map them to the same intent.

[0048] Understandably, with the help of the preset voice commands in the intent instruction relationship table that can be matched with the voice command, the intent type of the user can be determined through the voice command.

[0049] Specifically, the voice assistant program converts the voice command into text through speech recognition technology, and then determines the intent type of the user through the intent recognition algorithm. Intent recognition is a process of matching the text content of the voice command with the preset intent type. The intent recognition process can include speech recognition, feature extraction, intent matching, and context analysis, etc.

[0050] Speech recognition is to convert the user's voice command into text. Feature extraction is to extract features such as the spectrum, sound intensity, and fundamental frequency of the speech. Intent matching is to match the text content and speech features with the preset intent rule library. Context analysis is to further improve the accuracy of intent recognition by combining context information (such as historical conversation content).

[0051] Therefore, the voice assistant program converts the user's voice command into text through speech recognition technology, then analyzes the text content using the intent recognition algorithm, and combines context information (such as historical conversations, user preferences, etc.) to finally determine the target intent type of the specific operation requirements expressed by the user's voice command. This process enables the voice assistant to accurately understand the user's needs and route the command to the corresponding application program or function module, thereby achieving an efficient and natural interaction experience.

[0052] In this embodiment, the intent types are bound to the preset voice commands, and an intent command relationship table is established, which records the corresponding relationships between the intent types and the preset voice commands. Through this design, the target intent type can be accurately identified based on the voice commands and the intent command relationship table. With the clear intent command relationship table, the system can better understand the user's voice commands, reduce the possibility of misrecognition, and improve the accuracy of voice recognition. Users do not need to remember complex command formats or application names, but only need to naturally describe their intents to execute the target functions, making the interaction process more natural and convenient, and enhancing the user experience. It should also be noted that the intent command relationship table can include multiple intent types and their corresponding preset voice commands, enabling the system to support more complex multi-intent recognition and meet the diverse needs of users; and it can be easily updated and extended by adding new intent types and preset voice commands, enabling the system to continuously adapt to new application scenarios and user needs.

[0053] In some embodiments, the data processing method may further include the following steps: Display the intent command relationship table on the voice command management page; obtain the management commands of the user on the voice command management page; where the management commands include add commands and edit commands, and the add command is used to add the corresponding relationship between a new intent type and a preset voice command; the edit command is used to adjust the corresponding relationship between the existing intent type and the preset voice command; update the intent command relationship table according to the management commands.

[0054] This embodiment can further improve the flexibility and scalability of the system. Specifically, the add command allows users to add new intent types and preset voice commands at any time, enabling the system to quickly adapt to new application scenarios and user needs. The edit command allows users to adjust the corresponding relationship between the existing intent types and preset voice commands to ensure that the system always meets the latest usage requirements. Moreover, through the voice command management page, the display of the intent command relationship table enables users to intuitively see the current corresponding relationships between intents and commands, and users can centrally manage all intent types and preset voice commands, improving the management efficiency.

[0055] In some embodiments, a delete command may also be included, and the delete command allows users to delete the corresponding relationship between the existing intent type and the preset voice command. Therefore, when users find that certain intents or commands are no longer applicable or need to be adjusted, they can delete the unnecessary intent types, reducing the complexity of the voice assistant. This helps to clean up the unused intents and avoid confusion and misrecognition.

[0056] Therefore, through the add command, edit command, and delete command, users can customize the intents and commands of the voice assistant according to their own needs and preferences, thereby improving the response efficiency and accuracy of the voice assistant program.

[0057] In some embodiments, the steps after performing the operation task corresponding to the target intent type in the target application may further include: In the target application, display the page after performing the operation task corresponding to the target intent type.

[0058] In this embodiment, by displaying the page after performing the operation task corresponding to the target intent type in the target application, user feedback can be significantly enhanced. Users can immediately see the operation result and confirm that the system has correctly executed their instructions; user efficiency can be improved. Users do not need to manually check the operation result, and the system automatically displays the page after the operation, reducing the user's operation steps and maintaining the coherence of the operation process, thereby improving the intelligence of the voice assistant program.

[0059] In some embodiments, the steps of adding the function of processing the voice assistant program request to the preset application through the configuration file and defining the intent types that the preset application can process may specifically include: Add the function of processing the voice assistant program request to the preset application by configuring the first file; describe the reason for using the voice assistant program when requesting user authorization by configuring the second file; define the intent type by creating a new third file.

[0060] In this embodiment, the first file, the second file, and the third file are configuration files / definition files for defining and configuring various functions, permissions, and behaviors of the application.

[0061] Specifically, the first file can be regarded as a permission configuration file, mainly used to define the permissions of the application; the second file can be regarded as an application configuration file, used to define various metadata and configuration information of the application; the third file can be regarded as an intent definition file, used to define the intent types supported by the application and their parameters.

[0062] It can be understood that in different systems, the above configuration files / definition files may have different naming methods, but the functions that can be executed are common.

[0063] Taking the application of the SiriKit technology framework in the iOS system as an example, the first file can be the entitlements file, whose full name is "code signing entitlements". It is a configuration file used to define the permissions required by iOS or macOS applications during development and is presented as a plist file in XML format. It grants specific functions or security permissions to the application, is an important link in application signing and distribution, and is also an important guarantee for application security.

[0064] The second file can be the info.plist file. The info.plist file is a key configuration file in iOS app development, storing various configuration information of the application. It tells the iOS operating system how to handle the app, including the app's metadata and various settings. When the app needs to access the user's private data or functions, the info.plist file must provide corresponding permission request descriptions, which will be shown to the user when the app first requests these permissions.

[0065] The third file can be the intentdefinition file. In iOS development, especially when using SiriKit, the intentdefinition file is used to define custom intents. It is mainly used to support SiriKit or other voice assistant functions, enabling the application to handle the user's voice commands.

[0066] In some embodiments, the data processing method may further include the following steps: Editing the second file through the main application module to add intent types that the preset application can handle; the main application module is provided by the preset application; editing the second file through the intent processing module to describe the intent types that the intent processing module can handle; the intent processing module is provided by the voice assistant program.

[0067] Among them, the main application module refers to the main application program part containing the core functions and configuration information. The preset application can be regarded as a part of the main application module or an application component closely integrated with the main application module. The voice assistant program is provided with a dedicated intent processing module, which is responsible for processing the user's intent requests.

[0068] Taking the implementation of the SiriKit technology framework in the iOS system to enable the voice assistant to complete specific tasks in the target application scenario as an example, the following is a detailed explanation.

[0069] Please refer to Figures 2 - 4 , Figure 2 which is the interaction schematic diagram of the preliminary configuration in the data processing method provided by the embodiments of the present invention, Figure 3 which is the interaction schematic diagram of the installation stage in the data processing method provided by the embodiments of the present invention, Figure 4 which is the interaction schematic diagram of the running stage in the data processing method provided by the embodiments of the present invention.

[0070] In Figure 2 the preliminary configuration, the developer interacts with the intent processing module and the main application module respectively, adds the function of processing the voice assistant program requests through configuring and editing files, and defines the intent types that the preset application can handle. The content of the specific configuration file includes: Configure the entitlements file: Add the ability for the app to handle voice assistant requests.

[0071] Configure the info.plist file: Describe the reasons for using the voice assistant when requesting user authorization.

[0072] Create an intentdefinition file: Inform the system of the defined intents, such as checking the phone bill.

[0073] Edit the info.plist file: Add the intent types that this app can handle, such as being able to handle the intent of checking the phone bill.

[0074] Edit the info.plist file: Describe the intent types that this processing module can handle.

[0075] In Figure 3 During the installation phase, the system module interacts with the intent processing module and the main app module respectively. The main app module can inform the system module to register custom intents: for example, inform the system module that there is an intent to check the phone bill, inform the system module that this app can handle voice assistant requests, and inform the system module what intents this app can handle. The intent processing module can also inform the system module what intents this module can handle.

[0076] In Figure 4 During the running phase, the user, the system module, the intent processing module, the main app module, the voice command management module, and the quick voice command management service all interact.

[0077] The user enters the voice command management page. The voice command management module sends getAllVoiceShortcuts() to the quick voice command management service. The quick voice command management service returns all the added voice shortcuts. The voice command management module displays the currently editable voice shortcuts and the voice shortcuts that can be newly added. The user adds or edits voice shortcuts to the voice command management module. The voice command management module displays the new interface or the edit interface, which is provided by SiriKit and will automatically save the results.

[0078] Among them, getAllVoiceShortcuts() is a function related to the shortcuts of the voice assistant program, used to obtain all the voice shortcuts added by the user to the voice assistant program.

[0079] The user wakes up the voice assistant and speaks a voice command, such as "Check phone bill". The voice assistant program in the system module responds to the voice command according to the voice recognition intent. The system module processes the intent through handle() to the intent processing module, and the intent processing module returns an NSUserActivity object, which has a webpageURL property whose value is an http link. At this time, the processing result type is continueInApp, which means that the system needs to open the application and display the corresponding content according to the link specified by webpageURL.

[0080] Among them, handle() is a general function or method name, usually used to handle certain events or operations; NSUserActivity is a class in iOS and macOS, used to represent the user's activities in the application. webpageURL is a property in the NSUserActivity class, used to specify the web page address related to the user activity. http is the abbreviation of Hypertext Transfer Protocol, which is a protocol used to transfer data between the client and the server. continueInApp is a boolean property, usually used in NSUserActivity or other similar contexts, used to indicate whether it is necessary to continue processing the current activity within the application.

[0081] The system module starts the application (App) through the main application module, calls the continueUserActivity() method, and passes in the NSUserActivity object. The main application module obtains the http connection in the object, calls the urlJumpProcess() method, and finally the main application module displays the page, such as the phone bill details page.

[0082] Among them, continueUserActivity() is a commonly used method in iOS application development and belongs to the UIApplicationDelegate protocol. It is used to handle the user's transition from one activity (the scenario of continuing the operation), and urlJumpProcess() is used to handle the logic of URL jumping.

[0083] In the example, the operation path to wake up the target application and enter the specified page using the voice assistant is as follows: First, the user needs to authorize the voice assistant to use the target application; then the user manually adds a voice shortcut command and customizes the trigger phrase; finally, the user issues a voice command (such as "Check phone bill") through the voice assistant, triggers the corresponding intent, and automatically opens the target application and enters the specified scenario (such as the "phone bill details page"). The detailed introduction of each module is as follows: (1) System module System modules are various services provided by the official of the electronic device, including a voice assistant program.

[0084] When adding a voice command, it is necessary to bind the voice command to an intent. When the voice command is spoken, the system will automatically recognize the intent. For example, bind a voice command of "Check phone bill" to the intent of querying the phone bill. When saying "Check phone bill" to the voice assistant program, the system will recognize it as the intent of querying the phone bill.

[0085] Through the configuration file, when the application is installed, the system can be informed which intent requests the main application module and the intent processing module can handle. Therefore, when the system recognizes an intent, it will hand over the intent request to the corresponding module for processing. Generally, it is given to the intent processing module first, and the next step is determined according to the processing result, which can start the application program, or further require the user to give clear parameters, or display a custom UI interface, or inform the user that the recognition fails and cannot be recognized. For example, the intent processing module registers with the system that it can handle the intent of querying the phone bill. When the system recognizes the intent of querying the phone bill, it will initiate an intent processing request to the intent processing module.

[0086] In addition, SiriKit provides a quick voice command management service (INVoiceShortcutCenter), which can query all newly added voice quick commands in this application. This service is used in the voice management module of this embodiment.

[0087] (2) Intent processing module The intent processing module is provided by SiriKit, can be independent of the main application module, and can run in the background.

[0088] First, it is necessary to specify in the configuration file which intents this module can handle. When the system recognizes the intent issued by the user, it will check which modules are registered as capable of handling this intent. If this module is registered as capable of handling this intent, the system can hand over this intent to this module for processing.

[0089] It should be noted that the system official may pre-define some intent types, such as taking a taxi, ordering food, paying, sending messages, etc. When the intent processing modules of multiple applications are registered as capable of handling the same intent type, the user needs to specify which application will handle this intent. In this example, the intent processing module is set to be capable of handling intents such as querying the phone bill, querying traffic, and recharging the phone bill.

[0090] After receiving the intent request from the system, the intent processing module will return an intent handler, and this handler will process the intent request. The steps for processing an intent request include: parsing parameters - confirming details - executing tasks.

[0091] The parsing parameter is to confirm whether the parameters provided by the user meet the requirements. The confirmation details are to verify the parameters and check whether the service is available, and this step can be ignored in some embodiments. Executing the task is to process the intent request and return the processing result to the system.

[0092] Among them, the processing result can include the following situations: ContinueInApp: Indicates that the operation needs to continue in the app.

[0093] InProgress: Indicates that the processing is in progress.

[0094] Success: Indicates that the processing is successful.

[0095] Failure: Indicates that the processing fails.

[0096] FailureRequiringAppLaunch: Indicates that the processing fails, but the user needs to launch the app to complete a specific operation.

[0097] When the return result is ContinueInApp, a custom user activity (NSUSerActivity) can be provided, and its function is to provide the context required when the app is awakened. In this example, the webpageURL property of the user activity object can be set to the target link, such as the link to the phone bill details page.

[0098] (3) Main app module The main app module can be understood as the target application itself.

[0099] The startup method of the application is not limited to the user's manual operation, but can also be automatically started by the system according to specific conditions. When the system wakes up the application through the user activity, the application can extract relevant context information from the user activity. Specifically in this example, when the system wakes up the application, the application extracts the target link from the user activity and performs a page jump according to the link. This process shows that by passing context information through user activities, the application can automatically enter a specific function or page after startup, thereby improving the user experience and operation efficiency.

[0100] A file of type Intentdefinition is added to the main app module. This file is provided by SiriKit, and developers only need to fill in the relevant information of the intent in this file, such as the name, description, parameters and their requirements, etc. After completing the filling of this information, compiling the project can automatically generate the classes corresponding to these intents. In this example, the IntentDefinition file defines the following three intents: Query phone bill (CMCCQueryFeeIntent): Used to handle the operation of users querying phone bills.

[0101] Query data usage (CMCCQueryCurrencyIntent): Used to handle the operation of users querying data usage.

[0102] Recharge phone bill (CMCCReChargeIntent): Used to handle the operation of users recharging phone bills.

[0103] In this way, developers can easily define and manage the voice assistant program intents supported by the application, ensuring that the voice assistant program can accurately understand and process users' voice commands, and enhancing the user experience.

[0104] It should also be noted that the reasons for setting which intents the main application module can handle are as follows: 1. Endows the ability to handle voice assistant program requests: The system needs to clarify which intent types can be processed by this application to ensure that requests can be correctly responded to.

[0105] 2. Backup mechanism of the intent processing module: In most cases, the system will give priority to handing intent requests to the intent processing module for processing. However, if the intent processing module crashes or fails to respond, the system will continue to hand the intent request to the main application module for processing. In this case, the system will prompt the user that the target program system can handle this request and ask the user if they want to open this application for processing.

[0106] (4) Voice command management module The voice command management module is a voice command management page. In SiriKit, custom intents need to be bound to voice commands so that the system can recognize intents from voices. Therefore, in this example, a new voice management page is added for users to edit and add voice commands.

[0107] In this example, intents for querying phone bills, querying data usage, and recharging phone bills are defined, and the intent processing module can handle these three intents. Using the quick voice command management service provided by SiriKit, the added voice quick commands in the target voice program are obtained, that is, the editable voice quick commands are obtained. By comparing with the intents that can be processed, the voice quick commands that can be added are obtained.

[0108] When a user needs to add or edit a voice shortcut command, two UI interfaces provided by SiriKit can be used: INUIAddVoiceShortcutViewController and INUIEditVoiceShortcutViewController. After the user finishes the operation on the UI interface, if there are changes such as addition or deletion of the voice shortcut command, the service is used again to obtain the currently added voice shortcut commands and update the voice command management page of the voice assistant program.

[0109] Based on the functions and roles of the above-described modules, the steps to implement the data processing method may include the following: 1. Increase the ability to handle Siri requests.

[0110] 2. Check the user authorization status when the App starts. If the user has not authorized, request the user to authorize.

[0111] 3. Create a new intentdefinition file, and customize the intents in this file. This file defines three intents, namely querying the phone bill, querying the traffic, and recharging the phone bill. The Title and Descrption attributes are used to describe the intent. After compiling this file, the classes and processing protocols corresponding to the intents will be automatically generated.

[0112] 4. Configure the info.plist files of the main application module and the intent processing module respectively, and define the intent types that they can handle.

[0113] 5. Handle the intent requests in the intent processing module, show how to return the handler, and the processing logic of the query phone bill intent. Among them, the method for obtaining the handler (handlerForIntent) and the method for handling the intent request (handleQueryFee:completion) are both automatically called by the system.

[0114] 6. Handle the user activity in the main application, extract the destination link for redirection. The method application:continueUserActivity:restorationHandler is automatically called when the application is awakened in the way of user activity.

[0115] Among them, application:continueUserActivity:restorationHandler is a method in iOS development, which is used to handle the user activity when the application is awakened through a universal link or other means.

[0116] 7. Add a voice command management page. When the user enters the page, first use the voice command management service provided by SiriKit to obtain all the voice shortcuts added to this application, and then compare them with the intents that the intent processing module can handle to obtain the voice shortcuts that can be added.

[0117] 8. Display the new and edit UI interfaces. SiriKit does not allow developers to customize the UI for adding and editing voice shortcuts and can only use the UI classes it provides. When creating the UI, it is also necessary to pass in the corresponding shortcut (INShortcut) or voice shortcut (INVoiceShortcut) for the intent. For example, when adding a voice shortcut for querying phone bills, a shortcut corresponding to the intent of querying phone bills needs to be passed in.

[0118] In the data processing method provided by the embodiment of the present invention above, direct page jumps in the application scenario can be realized. Through the customized intent, the intent processing module, and the function of adding shortcut voice commands, the user can directly wake up and jump to the specific function page of the target application (such as the phone bill query page) through the voice assistant program, rather than just opening the main interface of the target application. It does not adopt the simple application opening method in the related technology, but significantly simplifies the user operation process by directly accessing the internal functions of the application, improving the fluency and convenience of the user experience.

[0119] In addition, in this embodiment, through the customized voice suggestion phrases, for example, the phrase suggested when adding a shortcut voice command for querying phone bills is "query phone bills", it helps users complete complex operations with more natural voice commands. The related technology does not provide voice phrase prompts or suggestions. The voice phrase suggestions can enable users to adapt to the functions within the application more quickly, while enhancing the intelligent interaction experience of the voice assistant. Compared with the traditional voice assistant wake-up method, it is more in line with the natural language usage habits of users.

[0120] Moreover, the present invention also differs from the related technology in that when the voice recognition result is unclear or there are multiple matching results, it is necessary to prompt the user to re-enter or make a selection; while in the embodiment of the present invention, specific voice phrases (such as "query phone bills") and user context data are used to ensure that the voice assistant can uniquely identify and execute the target operation, reducing unnecessary confirmation steps. Without multiple-step confirmation and reducing user interaction, this improvement further optimizes the user experience by reducing interaction steps, enhancing the response efficiency of the intelligent assistant, and increasing user satisfaction.

[0121] The present invention also provides a data processing device. The data processing device provided by the present invention will be described below. The data processing device described below can be mutually corresponding and referred to with the data processing method described above.

[0122] Please refer to Figure 5, Figure 5 It is a schematic structural diagram of a data processing device provided by an embodiment of the present invention. In this embodiment, the data processing device is deployed in an electronic device, and the electronic device includes a voice assistant program. Specifically, the data processing device may include a configuration module 510, an integration module 520, a speech recognition module 530, and an intent execution module 540.

[0123] The configuration module 510 is used to add the function of processing voice assistant program requests to a preset application program through a configuration file and define the intent types that the preset application program can process.

[0124] The integration module 520 is used to connect the preset application program to the voice assistant program, obtain the intent types that the preset application program can process reported by the preset application program, and obtain an application intent relationship table; where the application intent relationship table records the corresponding relationship between the intent types and the preset application programs.

[0125] The speech recognition module 530 is used to obtain a voice command through the voice assistant program and identify the target intent type according to the voice command.

[0126] The intent execution module 540 is used to determine a target application program from the preset application programs through the voice assistant program according to the target intent type and the application intent relationship table, and execute the operation task corresponding to the target intent type in the target application program.

[0127] In some embodiments, the speech recognition module 530 is used to: bind the intent type and a preset voice command to obtain an intent command relationship table; where the intent command relationship table records the corresponding relationship between the intent type and the preset voice command; and obtain the target intent type according to the voice command and the intent command relationship table.

[0128] In some embodiments, the data processing device further includes an adjustment module, and the adjustment module is used to: display the intent command relationship table on a voice command management page; obtain a management command of a user on the voice command management page; where the management command includes a new command and an edit command, the new command is used to add a corresponding relationship between a new intent type and a preset voice command; the edit command is used to adjust the corresponding relationship between an existing intent type and a preset voice command; and update the intent command relationship table according to the management command.

[0129] In some embodiments, the configuration module 510 may specifically be used to: add the function of processing voice assistant program requests to a preset application program by configuring a first file; describe the reason for using the voice assistant program when requesting user authorization by configuring a second file; and define the intent type by creating a third file.

[0130] In some embodiments, the configuration module 510 may specifically be further configured to: edit a second file through the main application module to add intent types that the preset application can handle; the main application module is provided by the preset application; edit the second file through the intent processing module to describe the intent types that the intent processing module can handle; the intent processing module is provided by the voice assistant program.

[0131] In some embodiments, the data processing device further includes a display module, and the display module is configured to: display, in a target application, a page after performing an operation task corresponding to a target intent type.

[0132] As described above, the embodiments of the present invention provide a data processing device. Through a voice command, it is possible to directly jump to a specific page within an application, such as a call fee query interface, rather than just opening the application. Compared with the related art, the steps of user interaction are reduced, and the operation efficiency is improved. In addition, in the related art, when the recognition result is ambiguous or there are multiple matching items, it is usually necessary for the user to confirm or select again, which increases the operation burden of the user. However, the embodiments of the present invention improve the user experience by reducing these intermediate confirmation steps, enabling the user to complete the query operation in a single voice interaction. In addition, the embodiments of the present invention also provide customized voice phrase suggestions. These preset common phrases conform to the user's natural language habits, enabling the user to complete the operation more naturally and quickly. This function further simplifies the interaction between the user and the application and enhances the usability of the application.

[0133] On the other hand, the embodiments of the present invention also provide an electronic device. Please refer to Figure 6 , Figure 6 is a schematic physical structure diagram of the electronic device provided by the embodiments of the present invention. As shown in Figure 6 , the electronic device may include a memory 620, a processor 610, and a computer program stored in the memory 620 and executable on the processor 610. When the processor 610 executes the program, it implements the data processing method provided by each of the above methods.

[0134] Optionally, the electronic device may further include a communication bus 630 and a communication interface 640. Among them, the processor 610, the communication interface 640, and the memory 620 complete mutual communication through the communication bus 630. The processor 610 may call the computer program in the memory 620 to execute the data processing method, and the method may include: Add the function of processing voice assistant program requests to a preset application through a configuration file, and define the types of intents that the preset application can process; connect the preset application to the voice assistant program, obtain the types of intents that the preset application can process reported by the preset application, and obtain an application-intent relationship table; the application-intent relationship table records the corresponding relationship between the intent type and the preset application; obtain a voice command through the voice assistant program, and identify the target intent type according to the voice command; according to the target intent type and the application-intent relationship table, determine the target application from the preset applications through the voice assistant program, and execute the operation task corresponding to the target intent type in the target application.

[0135] In addition, when the logical instructions in the above-mentioned memory 620 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0136] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data processing methods provided by the above-mentioned various methods. The steps and principles have been introduced in detail in the above methods and will not be repeated here.

[0137] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the data processing methods provided by the above-mentioned various methods. The steps and principles have been introduced in detail in the above methods and will not be repeated here.

[0138] A non-transitory computer-readable storage medium can be any available medium or data storage device accessible by a processor, including but not limited to magnetic memories (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memories (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor memories (such as ROM, EPROM, EEPROM, non-volatile memories (NAND FLASH), solid-state drives (SSD)), etc.

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disks, optical discs, etc., including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A data processing method, characterized in that: Applied to an electronic device, the electronic device includes a voice assistant program, and the data processing method includes: Adding a function of processing voice assistant program requests to a preset application through a configuration file, and defining the intent types that the preset application can handle; Connecting the preset application to the voice assistant program, obtaining the intent type that can be processed reported by the preset application, and obtaining an application intent relationship table; wherein the application intent relationship table records the correspondence between the intent type and the preset application; Acquire voice instructions through the voice assistant program, and identify the target intention type according to the voice instructions; According to the target intent type and the application intent relationship table, the target application is determined from the preset applications through the voice assistant program, and an operation task corresponding to the target intent type is executed in the target application.

2. The data processing method according to claim 1, characterized in that: The identifying the target intention type according to the voice instruction also includes: Binding the intention type and the preset voice command to obtain an intention-command relationship table; wherein the intention-command relationship table records the correspondence between the intention type and the preset voice command; The target intention type is obtained according to the voice instruction and the intention instruction relationship table.

3. The data processing method according to claim 2, characterized in that: Also includes: Displaying the intention-command relationship table in the voice command management page; Obtaining management instructions in the voice instruction management page; wherein the management instructions include adding instructions and editing instructions, the adding instructions are used to add a correspondence between a new intent type and a preset voice instruction; the editing instructions are used to adjust the correspondence between an existing intent type and a preset voice instruction; The intention instruction relationship table is updated according to the management instruction.

4. The data processing method according to claim 1, characterized in that: The configuration file is used to add the function of processing voice assistant program requests to the preset application, and defines the intent types that the preset application can handle, including: Adding a function of processing the voice assistant program request to the preset application by configuring the first file; The reason for using the voice assistant program when requesting user authorization is described by configuring a second file; Define the intent type by creating a new third file.

5. The data processing method according to claim 4, characterized in that: Also includes: Editing the second file through the main application module to increase the types of intents that can be processed by the preset application; The main application module is provided by the preset application program; The second file is edited by the intent processing module to describe the intent types that the intent processing module can handle; the intent processing module is provided by the voice assistant program.

6. The data processing method according to any one of claims 1 to 5, characterized in that: After executing the operation task corresponding to the target intention type in the target application, the method further includes: In the target application, a page after executing the operation task corresponding to the target intention type is displayed.

7. A data processing device, characterized in that: Deployed in an electronic device, the electronic device includes a voice assistant program, and the data processing device includes: A configuration module, used to add a function of processing voice assistant program requests to a preset application through a configuration file, and define the intent types that the preset application can process; An integration module, used to connect the preset application to the voice assistant program, obtain the intent type that can be processed reported by the application, and obtain an application intent relationship table; wherein the application intent relationship table records the correspondence between the intent type and the preset application; A voice recognition module, used to obtain voice instructions through the voice assistant program and identify the target intention type according to the voice instructions; An intention execution module is used to select a target application from the preset applications through the voice assistant program according to the target intention type and the application intention relationship table, and execute an operation task corresponding to the target intention type in the target application.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the data processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 6 is implemented.

Images