Data processing method and electronic equipment

Abstract

The invention discloses a data processing method and electronic equipment, and the method comprises the steps: obtaining a target file which belongs to a target function module; analyzing the target file to obtain at least one target capability of the target function module and a calling service for calling the target capability; wherein each target capability comprises a capability name and a capability description, and the capability description accords with a natural language; each calling service of the target capability comprises a first service providing input parameters processed by the target function module and a second service obtaining an output result from the target function module.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method and an electronic device. Background Art

[0002] When executing a task in an electronic device, the corresponding functional module in the electronic device is usually called. Therefore, it is necessary to collect the capabilities of each functional module in the electronic device in advance.

[0003] However, when collecting the capabilities of the functional modules on the electronic device, the user needs to manually set the capabilities of each functional module on the electronic device and write a calling method and parameters for each functional module. Therefore, there is a problem of high operational complexity. Summary of the invention

[0004] In view of this, the present application provides a data processing method and an electronic device as follows:

[0005] A data processing method, comprising:

[0006] Obtaining a target file, wherein the target file belongs to a target function module;

[0007] Parsing the target file to obtain at least one target capability of the target functional module and a calling service for calling the target capability;

[0008] in,

[0009] Each of the target capabilities includes a capability name and a capability description, and the capability description complies with natural language;

[0010] The calling service of each target capability includes a first service that provides input parameters for processing by the target function module and a second service that obtains output results from the target function module.

[0011] The above method, preferably, further comprises:

[0012] A tag of the target function module is established, wherein the tag represents a corresponding relationship between the target function module and each of the target capabilities possessed by the target function module and a calling service for calling the target capabilities.

[0013] The above method, preferably, further comprises:

[0014] Obtaining a call request for calling a first capability, wherein the call request includes input parameters;

[0015] Based on the corresponding relationship, the input parameters in the call request are sent to the first functional module through the first service of the first call service, and the output result corresponding to the first capability is obtained from the first functional module through the second service of the first call service.

[0016] In the above method, preferably, the calling request is: a request generated by processing the user demand obtained by the target application and each capability description;

[0017] The output result corresponding to the first capability is used to provide to the target application as feedback information of the user demand.

[0018] The above method, preferably, further comprises:

[0019] Based on the intelligent engine, processing the user requirements obtained through the target application and each of the capability descriptions to generate the call request; the intelligent engine has each of the capability descriptions;

[0020] Based on the intelligent engine, an output result corresponding to the first capability is provided to the target application as feedback information of the user demand.

[0021] An electronic device, comprising:

[0022] At least one target function module, each of the at least one target function module has at least one target capability and a calling service for calling the target capability;

[0023] A capability management module is used to obtain a target file, the target file belongs to the target function module; parse the target file to obtain at least one target capability of the target function module and a calling service for calling the target capability;

[0024] in,

[0025] Each of the target capabilities includes a capability name and a capability description, and the capability description complies with natural language;

[0026] The calling service of each target capability includes a first service that provides input parameters for processing by the target function module and a second service that obtains output results from the target function module.

[0027] The above electronic device is preferably:

[0028] The capability management module is further configured to: obtain a call request for calling a first capability, the call request including input parameters; based on the corresponding relationship in the capability management module, send the input parameters in the call request to a first functional module through a first service of a first call service, and obtain an output result corresponding to the first capability from the first functional module through a second service of the first call service;

[0029] The capability management module establishes a tag of the target function module, the tag represents the corresponding relationship, and the corresponding relationship is: the corresponding relationship between the target function module and the target capability of the target function module and the calling service of the target capability.

[0030] The above electronic device preferably further includes: an intelligent engine;

[0031] The intelligent engine is used to: process the user requirements obtained through the target application and each capability description to generate the call request; the intelligent engine has each capability description;

[0032] The intelligent engine is used to provide the output result corresponding to the first capability to the target application as feedback information of the user demand.

[0033] In the above electronic device, preferably, the target functional module includes: a second functional module deployed on the electronic device, the second functional module being capable of establishing a connection channel between the electronic device and the target device;

[0034] The intelligent engine is further used to: obtain, from the capability management module, a capability description of at least one target capability possessed by the third functional module deployed on the target device;

[0035] The capability description of at least one target capability possessed by the third functional module is sent from the target device to the capability management module through the connection channel.

[0036] In the above electronic device, preferably, the capability management module is further used for:

[0037] In the case where the call request corresponds to the target device, the call request is sent to the target device through the connection channel, the target device calls the first service of the first call service through the deployed capability management module, sends the input parameters in the call request to the third function module, and obtains the output result corresponding to the first capability from the third function module through the second service of the first call service;

[0038] An output result corresponding to the first capability sent by the target device is received through the connection channel. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0041] Figure 2 This is an example diagram of deploying target functional modules of an electronic device in an embodiment of the present application;

[0042] Figure 3 A partial flow chart of a data processing method provided in an embodiment of the present application;

[0043] Figure 4 This is an example diagram of implementing capability invocation through AS in an embodiment of the present application;

[0044] Figure 5 Another example diagram of implementing capability invocation through AS in an embodiment of the present application;

[0045] Figure 6 This is an example diagram of the interconnection between the electronic device and the target device in the embodiment of the present application;

[0046] Figure 7 This is an example diagram of obtaining a capability description of a target device on an electronic device in an embodiment of the present application;

[0047] Figure 8 This is an example diagram of implementing capability calling on an electronic device in an embodiment of the present application;

[0048] Fig. 9 This is an example diagram of calling capabilities on a target device on an electronic device in an embodiment of the present application;

[0049] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0050] Fig.11 Another structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0051] Fig.12 Another structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0052] Fig.13Another structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0053] Fig.14 This is a schematic diagram of the structure of the interconnection between the electronic device and the target device in the embodiment of the present application;

[0054] Fig.15 This is an example diagram of obtaining a capability description of a target device on an electronic device in an embodiment of the present application;

[0055] Fig.16 This is an example diagram of implementing capability calling on an electronic device in an embodiment of the present application;

[0056] Fig.17 This is an example diagram of calling capabilities on a target device on an electronic device in an embodiment of the present application;

[0057] Fig.18 This is an example diagram of the application for implementing capability collection and capability calling on a mobile phone;

[0058] Fig.19 This application is applicable to the interactive timing diagram of the mobile phone realizing capability collection and capability calling in the scenario where the mobile phone and the notebook are interconnected. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0060] refer to Figure 1 As shown in FIG. 1 , a flowchart of a data processing method provided in an embodiment of the present application is provided. The method can be applied to electronic devices that are deployed with target functional modules and can perform data processing, such as mobile phones, tablet devices, notebooks or servers. Figure 2 As shown in , one or more target function modules may be deployed in the electronic device, each of which can be used to implement one or more corresponding functions, which may be referred to as target capabilities. The technical solution in this embodiment is mainly used to reduce the complexity of the operation of collecting target capabilities on the electronic device.

[0061] Specifically, the method in this embodiment may include the following steps:

[0062] Step 101: Obtain the target file.

[0063] The target file belongs to the target function module, each target function module has a target file, and the target file can be a file in XML format. The target file records at least one target capability of the target function module and a calling service for calling the target capability.

[0064] In a specific implementation, the target file may be provided by the developer of the target function module. When the target function module is developed or upgraded, the developer adds the target file to the target function module, thereby enabling the target file of each target function module to be obtained on the electronic device.

[0065] For example, when a social application deployed on an electronic device is upgraded, an upgrade package is downloaded from the application provider's server. The upgrade package includes a target file of the interactive application. The target file records at least one target capability of the social application and a calling service for calling the target capability.

[0066] For another example, when installing a map application on an electronic device, an installation package may be downloaded from the application provider's server. The installation package includes a target file of the map application. The target file records at least one target capability of the map application and a calling service for calling the target capability.

[0067] In one implementation, in this embodiment, a package manager such as PackageManager in the operating system may be used to scan target files belonging to each target functional module to obtain the target files.

[0068] Step 102: Parse the target file to obtain at least one target capability of the target functional module and a calling service for calling the target capability.

[0069] Each target capability includes a capability name and a capability description, and the capability description conforms to natural language. Moreover, the calling service of each target capability includes a first service that provides input parameters for processing by the target function module and a second service that obtains output results from the target function module. The capability name is used to uniquely represent the corresponding target capability, and the capability description describes the function implemented by the corresponding target capability in natural language.

[0070] For example, taking a target function module as an example, the target function module's own target capability regarding artificial intelligence (AI) is independently declared in a target file such as XML. Take the following code snippet as an example:

[0071] <? xml version="1.0" encoding="utf-8"? >

[0072] <capabilitys>

[0073] <capability> / / Declare a target capability

[0074] <method

[0075] name = "catchMeUp" / / target ability name

[0076] description = "summarize for notifications" / / target capability description

[0077] sourceName="phone" / >

[0078] <returnparameters>

[0079] <parameter

[0080] name="response"

[0081] description="summary result" / >

[0082] < / returnparameters>

[0083] < / capability>

[0084] <capability> / / Declare a target capability

[0085] <method

[0086] name = "imageGenerate" / / target capability name

[0087] description = "generate image for text" / / target capability description

[0088] sourceName="phone" / >

[0089] <parameters>

[0090] <parameter

[0091] name="imageDesc"

[0092] description="To generate the picture,describe the details of thepicture in detail"

[0093] required="true" / >

[0094] < / parameters>

[0095] <returnparameters>

[0096] <parameter

[0097] name="image"

[0098] description="byte array for image" / >

[0099] < / returnparameters>

[0100] < / capability>

[0101] < / capabilitys>

[0102] The above example declares two AI capabilities in the "description" field, which is the capability description:

[0103] (1)summarize for notifications;

[0104] (2)generate image for text.

[0105] The calling service for calling the target capability is also declared in the XML file, as shown in the following code:

[0106] class ExportCapabilityService:MotoAiCapabilityService(){

[0107] override fun invokeAiCapability(bundle:Bundle,callback:IMotoAiCapabilityCallback){

[0108] val method=bundle.getString("methodName")

[0109] logD{"ExportAbilityService-->invokeAiCapability(),method=$method"}

[0110] when(method){

[0111] "catchMeUp"->{

[0112] CatchMeUpHandler(this,bundle,callback).execute()

[0113] }

[0114] "imageGenerate"->{

[0115] ImageGeneratorHandler(this,bundle,callback).execute()

[0116] }

[0117] }

[0118] }

[0119] }

[0120] Among them, "CatchMeUpHandler(this,bundle,callback).execute()" is the calling service for calling the target capability "catchMeUp". The "bundle" is the first service, which is used to provide input parameters for processing by the target functional module, that is, the first service provides input parameters to the target functional module so that the target functional module can process them. And "callback" is the second service, which is used to obtain output results from the target functional module, that is, the target functional module processes the input parameters to realize the target capability of "catchMeUp" and obtain output results.

[0121] Thus, an XML file of the target function module is obtained in the electronic device, and the XML file is parsed to obtain the capability name, capability description and the first service and the second service in the calling service of each target capability of the target function module.

[0122] Specifically, in the electronic device, the parsed target capabilities may be stored in a capability list.

[0123] For example, as shown in the following code:

[0124] private void scanAllAiCapabilities(){ / / Scan XML file

[0125] List <resolveinfo>services; / / Define a list of capabilities to save target capabilities

[0126] Intent intent=new Intent(AI_ACTION);

[0127] services = mPkgManager.queryIntentServices(intent, PackageManager.GET_META_DATA / / Scan the xml file through the package manager

[0128] |PackageManager.MATCH_DISABLED_UNTIL_USED_COMPONENTS);

[0129] for(ResolveInfo service:services){ / / Traverse the installation package of the target function module

[0130] ServiceInfo serviceInfo=service.serviceInfo;

[0131] Log.d(TAG,"pkg:"+serviceInfo.packageName

[0132] +"resid:"+serviceInfo.metaData.getInt("android.moto_ai_capability"));

[0133] try{ / / Parsing of xml

[0134] Resources resources=mPkgManager.getResourcesForApplication(serviceInfo.packageName);

[0135] XmlResourceParser xmlResourceParser = resources.getXml(serviceInfo.metaData.getInt("android.moto_ai_capability")); / / The parsed result of xml is placed in "resources"

[0136] parseXml(xmlResourceParser,serviceInfo.packageName);

[0137] }catch(PackageManager.NameNotFoundException e){ / / Handle parsing error Log.e(TAG,"errMsg:",e);

[0138] }

[0139] }

[0140] }

[0141] It can be seen that on the electronic device, the xml files can be scanned by a package manager such as PackageManager, and then the xml files can be parsed to obtain the target capabilities of each target functional module on the electronic device and the calling service for calling the target capabilities.

[0142] It should be noted that the method in this embodiment can be implemented by a capability management module deployed in an electronic device, such as an agent service module AS (Agent Service).

[0143] In a specific implementation, the target function module is deployed with function submodules corresponding to each target capability it has, and each function submodule is used to realize the corresponding target capability. Based on this, the first service provides input parameters to one of the function submodules in the target function module, so that the function submodule processes the input parameters to realize the target capability corresponding to the function submodule, thereby obtaining the output result corresponding to the target capability; then, the second service obtains the output result of realizing the corresponding target capability from the function submodule in the target function module.

[0144] It can be seen from the above technical scheme that in a data processing method provided by an embodiment of the present application, by parsing the target files belonging to the target function modules, the target capabilities of these target function modules and the calling services for calling these target capabilities can be obtained. In this way, when these target capabilities need to be called, these target capabilities can be used through the calling services of the target capabilities. In this process, the user does not need to manually set the capabilities of each function module on the electronic device, nor does the user need to specifically write calling methods and parameters for each function module. This can reduce the operational complexity of collecting capabilities in electronic devices and improve the user experience of using electronic devices.

[0145] In one implementation, in this embodiment, a tag of the target function module may also be established, and the tag of the target function module represents the corresponding relationship between the target function module and each target capability possessed by the target function module and the calling service that calls the target capability.

[0146] Therefore, by recording the correspondence between the target functional module, the target capability and the calling service in the electronic device, it can be identified that the calling service parsed from the target file corresponds to the corresponding target functional module.

[0147] For example, after parsing the XML file of the social application, a tag of the social application is established. The tag of the social application represents: the correspondence between the social application and the target capabilities and calling services of the parsed social application. Based on the correspondence, the target functional module to which the target capability belongs can be identified (the capability description of the target capability can also represent the target functional module to which the target capability belongs), and the calling service corresponding to the target capability can be located.

[0148] Based on the above implementation, the present embodiment may also include the following processing, such as Figure 3 As shown in:

[0149] Step 301: Obtain a call request for calling a first capability.

[0150] The calling request includes input parameters.

[0151] It should be noted that the first ability is the target ability required to perform the task, such as timing ability or text and image ability.

[0152] Step 302: Based on the corresponding relationship, the input parameters in the call request are sent to the first functional module through the first service of the first call service.

[0153] Step 303: Obtain an output result corresponding to the first capability from the first functional module through the second service of the first calling service.

[0154] Among them, the first function module is the target function module to which the first capability belongs. In this embodiment, the tags of each target function module can be used to find the target function module to which the first capability belongs and the calling service that calls the first capability, that is, the first function module, the first service, and the second service. Based on this, in this embodiment, the input parameters of the call request can be sent to the first function module through the first service, so that the first function module processes the input parameters to implement the first capability, thereby obtaining the output result corresponding to the first capability. Afterwards, the output result obtained by the first function module implementing the first capability can be obtained through the second service.

[0155] Based on the above implementation scheme, the call request can be: a request generated by processing the user demand obtained by the target application and each capability description, and the output result corresponding to the first capability is provided to the target application as feedback information of the user demand.

[0156] For example, Figure 4 As shown in , the target application can be an artificial intelligence assistant AI assistant, such as a voice assistant in a mobile phone, etc. The voice assistant can receive input data such as text or voice that represents user needs. Based on this, the AS implementing the method in this embodiment processes the input data representing the user needs and the capability description of each target capability to generate a call request, which represents the need to call the first capability, and the call request contains input parameters that need to be transmitted to the functional submodule in the first functional module corresponding to the first capability. Based on this, the AS implementing the method in this embodiment determines the first service and the second service in the call service that calls the first capability based on the corresponding relationship represented by the tag, and then sends the input parameters in the call request to the functional submodule corresponding to the first capability in the first functional module through the first service, and the functional submodule corresponding to the first capability processes the input parameters to obtain the output result, that is, to realize the first capability, and then obtains the output result obtained from the functional submodule corresponding to the first capability in the first functional module through the second service. Finally, the AS implementing the method in this embodiment provides the obtained output result as feedback information for the user's needs to the voice assistant, and the voice assistant provides it to the user through voice playback or text output.

[0157] In one implementation, the embodiment can process the user requirements and each capability description obtained through the target application based on the intelligent engine to generate a call request. The intelligent engine here has each capability description, and the capability description in the intelligent engine can be provided by the embodiment.

[0158] For example, Figure 5 As shown in , the AS implementing the method in this embodiment sends the collected capability description of each target capability to the intelligent engine, whereby the intelligent engine can process the user needs and these capability descriptions obtained by the voice assistant to determine the target capability required to meet the user needs, i.e., the first capability.

[0159] Furthermore, in this embodiment, based on the intelligent engine, the output result corresponding to the first capability can be provided to the target application as feedback information of the user demand.

[0160] For example, Figure 5 As shown in , after the call request is generated based on the intelligent engine, the call request represents the need to call the first capability, and the call request contains input parameters that need to be transmitted to the functional submodule corresponding to the first capability. The AS implementing the method in this embodiment can determine the first service and the second service in the call service that calls the first capability based on the corresponding relationship represented by the tag, and then send the input parameters in the call request to the functional submodule corresponding to the first capability in the first functional module through the first service, and the functional submodule corresponding to the first capability processes the input parameters to obtain the output result, that is, to realize the first capability. After that, the output result obtained is obtained from the functional submodule corresponding to the first capability in the first functional module through the second service. Finally, the AS implementing the method in this embodiment sends the obtained output result to the intelligent engine, thereby using the intelligent engine to provide the output result as feedback information for user needs to the voice assistant, and the voice assistant provides it to the user through voice playback or text output.

[0161] In a specific implementation, the intelligent engine may include: an artificial intelligence core AI Core deployed in an electronic device. AICore has the ability to understand the intent of natural language. Alternatively, the intelligent engine may include: an AI Core deployed in an electronic device, and a large language model LLM (Large Language Models), which can be deployed in electronic devices, other devices, or the cloud. Based on this, in this embodiment, call requests can be generated based on AI Core first. If AI Core cannot process user requirements and capability descriptions, that is, it cannot generate call requests, then call requests can be generated based on LLM.

[0162] In one implementation, the electronic device can be interconnected with the target device. Based on this, the electronic device can also collect the target capabilities of each third functional module on the target device, the third functional module is a functional module deployed on the target device, the third functional module has at least one target capability and a calling service for calling the target capability, and the target capability has a capability name and a capability description; similarly, the target device can also collect the target capabilities of each target functional module on the electronic device.

[0163] Specifically, the target functional module in the electronic device includes: a second functional module deployed on the electronic device, the second functional module can establish a connection channel between the electronic device and the target device. For example, the second functional module can be a super interconnected application that can realize device interconnection, and is used to realize the connection channel between the target device and the electronic device.

[0164] For example, Figure 6 As shown in , both the electronic device and the target device are deployed with a super interconnection application, which is an upper-layer application of the operating system and can establish a connection channel between the electronic device and the target device.

[0165] Based on this, the intelligent engine on the electronic device can also obtain the capability description of at least one target capability of the third functional module deployed on the target device from the capability management module AS implementing the method in this embodiment. The capability description of at least one target capability of the third functional module is sent from the target device to the capability management module AS implementing the method in this embodiment through the connection channel between the target device and the electronic device.

[0166] In one implementation, in addition to recording: at least one target capability possessed by the target function module itself and a calling service for calling the target capability, the target file of the second function module may also record: the target capability possessed by the third function module deployed on the target device and a calling service for calling the target capability. The target file recording the target capability possessed by the third function module deployed on the target device and the calling service is transmitted to the second function module by the target device through the connection channel established by the second function module. Based on this, the capability management module AS on the electronic device can obtain the target capability possessed by the third function module deployed on the target device and the corresponding calling service by parsing the target file of the second function module, thereby obtaining the capability description of the target capability possessed by the third function module deployed on the target device in the capability management module AS.

[0167] Furthermore, the capability management module AS implementing the method in this embodiment in the electronic device can also send the call request to the target device through the connection channel when the call request corresponds to the target device. The target device can send the input parameters in the call request to the third functional module corresponding to the first capability through the deployed capability management module by calling the first service of the first call service and obtain the output result corresponding to the first capability from the third functional module through the second service of the first call service. Finally, the output result corresponding to the first capability sent by the target device is received through the connection channel. Afterwards, the capability management module AS implementing the method in this embodiment in the electronic device can provide the output result corresponding to the first capability to the target application as feedback information of the user demand based on the intelligent engine.

[0168] For example, Figure 7 As shown in , the capability management module AS1 is deployed on the electronic device, the capability management module AS2 is deployed on the target device, and super-interconnected applications such as C1 and C2 are deployed on both the electronic device and the target device. A connection channel D is established between the super-interconnected applications. AS2 can collect the target capability of each third functional module deployed on the target device and the calling service for calling the target capability through the method described above. AS2 sends the capability description of the collected target capability to the super-interconnected application C2 on the target device. The super-interconnected application C2 on the target device sends the capability description collected by AS2 to the super-interconnected application C1 on the electronic device through the connection channel D. The super-interconnected application C1 on the electronic device sends the capability description collected by AS2 to AS1. AS1 sends both the capability description collected by itself and the capability description collected by AS2 to the intelligent engine. Thus, the intelligent engine can process the user needs and these capability descriptions obtained by the voice assistant to determine the target capability required to meet the user needs, that is, the first capability, thereby generating a call request and providing it to AS1. Afterwards, it can be determined whether the first functional module is the target functional module deployed on the electronic device or the third functional module deployed on the target device through the capability description of the first capability.

[0169] If the first functional module is a target functional module deployed on the electronic device, AS1 can determine the first service and the second service in the calling service for calling the first capability based on the corresponding relationship represented by the tag established on the electronic device, and then send the input parameters in the calling request to the functional submodule corresponding to the first capability in the first functional module in the electronic device through the first service, and the functional submodule corresponding to the first capability processes the input parameters to obtain the output result, that is, to realize the first capability, and then obtain the output result obtained from the functional submodule corresponding to the first capability in the first functional module through the second service, such as Figure 8 As shown in , finally, AS1 of this embodiment sends the obtained output result to the intelligent engine, thereby using the intelligent engine to provide the output result as feedback information for user needs to the voice assistant, and the voice assistant provides it to the user through voice playback or text output.

[0170] If the first functional module is a third functional module deployed on the target device, AS1 may send a call request to the hyperconnected application C1 on the electronic device, such as Fig. 9 As shown in , the super interconnected application C1 on the electronic device transmits the call request to the super interconnected application C2 on the target device through the connection channel D, and the super interconnected application C2 on the target device sends the call request to AS2. AS2 can determine the first service and the second service in the call service that calls the first capability based on the corresponding relationship represented by the tag established on the target device, and then send the input parameters in the call request to the functional submodule corresponding to the first capability in the first functional module in the target device through the first service. The functional submodule corresponding to the first capability processes the input parameters to obtain the output result, that is, to realize the first capability. After that, the functional submodule corresponding to the first capability in the first functional module is obtained from the output result obtained by the second service, and AS2 sends the obtained output result to the super interconnected application C2 on the target device. The super interconnected application C2 on the target device transmits the output result to the super interconnected application C1 on the electronic device through the connection channel D. The super interconnected application C1 on the electronic device sends the output result to AS1, and AS1 sends the received output result to the intelligent engine, thereby using the intelligent engine to provide the output result as feedback information for user needs to the voice assistant, and the voice assistant provides it to the user through voice playback or text output.

[0171] refer to Fig.10 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and the electronic device may include the following structure:

[0172] At least one target function module 1001, each of the at least one target function module 1001 has at least one target capability and a calling service for calling the target capability;

[0173] The capability management module 1002, such as AS, is used to obtain a target file, which belongs to the target function module 1001; parse the target file to obtain at least one target capability of the target function module 1001 and a calling service for calling the target capability;

[0174] in,

[0175] Each of the target capabilities includes a capability name and a capability description, and the capability description complies with natural language;

[0176] Each of the target capability invocation services includes a first service that provides input parameters to the target function module 1001 for processing and a second service that obtains output results from the target function module 1001 .

[0177] It can be seen from the above technical scheme that in an electronic device provided by an embodiment of the present application, the capability management module can be used to parse the target files belonging to the target function modules to obtain the target capabilities of these target function modules and the calling services for calling these target capabilities. In this way, when these target capabilities need to be called, these target capabilities can be used through the target capability calling services. In this process, the user does not need to manually set the capabilities of each function module on the electronic device, nor does the user need to specifically write calling methods and parameters for each function module. This can reduce the operational complexity of collecting capabilities in the electronic device and improve the user experience of the electronic device.

[0178] In one implementation, Fig.11 As shown in , the capability management module 1002 is also used to: obtain a call request for calling a first capability, the call request including input parameters; based on the corresponding relationship in the capability management module 1002, send the input parameters in the call request to the first function module 1011 through the first service of the first call service, and obtain the output result corresponding to the first capability from the first function module through the second service of the first call service;

[0179] The capability management module 1002 establishes a tag for the target function module 1001, and the tag represents the corresponding relationship, which is the corresponding relationship between the target function module 1001 and the target capability of the target function module 1001 and the calling service of the target capability.

[0180] In one implementation, the electronic device may further include: an intelligent engine 1003, such as an AI Core and / or LLM, etc.; Fig.12 as shown in .

[0181] The intelligent engine 1003 is used to: process the user requirements obtained through the target application 1031 and each capability description to generate the call request; the intelligent engine has each capability description;

[0182] The intelligent engine 1003 is used to provide the output result corresponding to the first capability to the target application 1031 as feedback information of the user demand.

[0183] Based on the above implementation, the target functional module 1001 includes: a second functional module 1012 deployed on the electronic device, the second functional module 1012 can establish a connection channel 1004 between the electronic device and the target device, such as Fig.13 As shown in;

[0184] The intelligent engine 1003 is further used to: obtain, from the capability management module 1002, a capability description of at least one target capability possessed by the third functional module 1005 deployed on the target device;

[0185] The capability description of at least one target capability possessed by the third functional module 1005 is sent from the target device to the capability management module 1002 through the connection channel 1004 .

[0186] Further, such as Fig.14 As shown in , the capability management module 1002 is also used for:

[0187] In the case where the call request corresponds to the target device, the call request is sent to the target device through the connection channel 1004, the target device calls the first service of the first call service through the deployed capability management module 1006, sends the input parameters in the call request to the third function module 1005, and obtains the output result corresponding to the first capability from the third function module 1005 through the second service of the first call service;

[0188] An output result corresponding to the first capability sent by the target device is received through the connection channel 1004.

[0189] For example, Fig.15 As shown in , a capability management module 1002 is deployed on the electronic device, a capability management module 1006 is deployed on the target device, super-interconnected applications such as 1012 and 1013 are deployed on both the electronic device and the target device, and a connection channel 1004 is established between the super-interconnected applications. The capability management module 1006 can collect the target capability of each third functional module 1005 deployed on the target device and the calling service of calling the target capability through the method described above. The capability management module 1006 sends the capability description of the collected target capability to the super-interconnected application 1013 on the target device, and the super-interconnected application 1013 on the target device collects the capability description of the target capability from the capability management module 1006. The received capability description is sent to the super interconnected application 1012 on the electronic device through the connection channel 1004, and the super interconnected application 1012 on the electronic device sends the capability description collected by the capability management module 1006 to the capability management module 1002. The capability management module 1002 sends both the capability description collected by itself and the received capability description collected by the capability management module 1006 to the intelligent engine 1003, whereby the intelligent engine 1003 can process the user needs and these capability descriptions obtained by the voice assistant 1031 to determine the target capability, i.e., the first capability, required to meet the user needs, thereby generating a call request and providing it to the capability management module 1002. Afterwards, it can be determined through the capability description of the first capability whether the first function module 1011 is the target function module 1001 deployed on the electronic device or the third function module 1005 deployed on the target device.

[0190] If the first functional module 1011 is the target functional module 1001 deployed on the electronic device, the capability management module 1002 can determine the first service and the second service in the calling service of calling the first capability based on the corresponding relationship represented by the tag established on the electronic device, and then send the input parameters in the calling request to the functional sub-module corresponding to the first capability in the first functional module 1011 in the electronic device through the first service, and the functional sub-module corresponding to the first capability processes the input parameters to obtain an output result, that is, to realize the first capability, such as Fig.16 As shown in , after that, the output result is obtained from the functional sub-module corresponding to the first capability in the first functional module 1011 through the second service. Finally, the capability management module 1002 of this embodiment sends the obtained output result to the intelligent engine 1003, thereby using the intelligent engine 1003 to provide the output result as feedback information for user needs to the voice assistant 1031, and the voice assistant 1031 provides it to the user through voice playback or text output.

[0191] If the first functional module is the third functional module 1005 deployed on the target device, the capability management module 1002 can send a call request to the super interconnected application 1012 on the electronic device, the super interconnected application 1012 on the electronic device transmits the call request to the super interconnected application 1013 on the target device through the connection channel 1004, the super interconnected application 1013 on the target device sends the call request to the capability management module 1006, the capability management module 1006 can determine the first service and the second service in the call service of calling the first capability based on the corresponding relationship represented by the tag established on the target device, and then send the input parameters in the call request to the functional sub-module corresponding to the first capability in the first functional module 1011 in the target device through the first service, and the functional sub-module corresponding to the first capability processes the input parameters to obtain an output result, that is, to realize the first capability, such as Fig.17 As shown in , after that, the output result obtained is obtained from the functional submodule corresponding to the first capability in the first functional module 1011 through the second service, the capability management module 1006 sends the obtained output result to the super-interconnected application 1013 on the target device, the super-interconnected application 1013 on the target device transmits the output result to the super-interconnected application 1012 on the electronic device through the connection channel 1004, the super-interconnected application 1012 on the electronic device sends the output result to the capability management module 1002, the capability management module 1002 sends the received output result to the intelligent engine 1003, thereby using the intelligent engine 1003 to provide the output result as feedback information for user needs to the voice assistant 1031, and the voice assistant provides it to the user through voice playback or text output.

[0192] Taking a mobile phone as an example, the technical solution of the present application is described below:

[0193] This application provides a unified method and mechanism that allows system applications / third-party applications in the mobile phone system to declare their own AI capabilities and required parameters (i.e., input parameters) and return values ​​(i.e., output results) according to the mechanism. After the system is turned on, there will be an independent service module, namely the capability management module AS mentioned above, which scans all AI capabilities in the system and provides them to the caller (i.e., the voice assistant mentioned above). When the caller needs to interact with the large language model LLM, the acquired AI capabilities can be used to let the large language model (i.e., the intelligent engine mentioned above) decide what capabilities to use to complete the user's needs (i.e., generate call requests in the above text). Through this method, the caller does not need to care about what capabilities are provided in the system, let alone collect these capabilities by himself to complete the interaction with the large language model, which greatly enriches the scenarios and interactive experience of the large language model to complete user instructions.

[0194] Among them, system applications / third-party applications can declare their own AI capabilities through the xml file mentioned above, such as the xml file "<?xml version="1.0" encoding="utf-8"?>" and call the service "ExportCapabilityService" in the previous article. The system module AS in the mobile phone can scan the system code to declare all AI capabilities, and provide the capability descriptions of all parsed AI capabilities to the intelligent engine, such as the large language model or AI Core, thereby expanding the capabilities of the intelligent engine.

[0195] like Fig.18 As shown in the figure, it is a framework diagram for realizing capability collection and capability calling in a mobile phone. Among them, AS contains multiple implementation modules:

[0196] (1) The user interacts with the AI ​​assistants, and the AI ​​assistants input information such as voice or text that represents the user's needs as prompts; or the AI ​​assistants provide output results as feedback information to the user;

[0197] (2) AI assistants send user requirements to AI Core; or, AI Core sends output results to AI assistants;

[0198] (3) AI Core can process the user requirements and the capability description provided by AS in combination with LLM, generate a call request, and send the call request to the implementation module 1 in AS; or, the implementation module 1 in AS sends the output result to AI Core; wherein the call request corresponds to the first capability and includes input parameters; the LLM here can be deployed on a local mobile phone or in the cloud;

[0199] (4) The implementation module 1 in the AS sends the input parameters in the call request to the functional submodule corresponding to the first capability through the first service in the call service corresponding to the first capability. The functional submodule may be a functional submodule in the target functional module deployed on the local mobile phone, or a functional submodule in other functional modules deployed on other devices connected to the mobile phone, such as a tablet device; or, the implementation module 1 in the AS obtains the output result sent by the functional submodule corresponding to the first capability through the second service in the call service corresponding to the first capability;

[0200] (5) The implementation module 2 in the AS scans the declaration files of all applications in the mobile phone through the package manager in the operating system to collect the capabilities of all applications deployed on the local mobile phone; and the implementation module 2 in the AS receives the capabilities collected by other interconnected devices through the super interconnected application;

[0201] (6) The implementation module 2 in the AS synchronizes the capability descriptions of all capabilities to the AI ​​assistants;

[0202] (7) AS can also implement other functions by implementing module 3, such as providing screen information.

[0203] Among them, AS can scan and call the AI ​​capabilities in the system. ASSDK provides a unified software development kit SDK (Software Development Kit) for each application to integrate and implement AI capability declaration and call.

[0204] like Fig.19 As shown in the figure, it is an interactive sequence diagram for realizing capability collection and capability call through the super interconnection application SC (SmartConnect) in the device interconnection scenario. In this diagram, an AI application with AI capabilities such as AI image generation is deployed in the notebook, and an application that needs to use the AI ​​image generation capability is deployed in the mobile phone. Both the notebook and the mobile phone are deployed with SC. The specific process is as follows:

[0205] First, each application deployed on the notebook can declare its own capabilities through the target file, such as the AI ​​image generation capability declared by the AI ​​application, the super interconnection capability declared by the SC, and the screen projection capability; the AS on the notebook can collect the AI ​​capabilities of all applications by parsing the target file and establish tags between AI applications and AI capabilities and calling services. After that, the AS on the notebook sends the capability description of the AI ​​capabilities received from the mobile phone to the SC on the notebook.

[0206] In addition, the AS on the notebook can also provide the capability descriptions of all capabilities to the AI ​​Core of the notebook. The AI ​​Core can process user requirements and the capability descriptions provided by the AS in combination with the LLM to generate a call request.

[0207] Similarly, each application deployed on the mobile phone declares its own capabilities, such as the super-connectivity capability declared by the SC; the AS on the mobile phone can collect the capabilities of all applications and establish tags between the applications and the corresponding capabilities and the calling services. The AS on the mobile phone then sends the capability descriptions of the collected capabilities to the SC on the mobile phone.

[0208] In addition, the AS on the mobile phone can also provide the capability descriptions of all capabilities to the AI ​​Core of the mobile phone. The AI ​​Core can process user requirements and the capability descriptions provided by the AS in combination with the LLM to generate a call request.

[0209] Based on this, the SC on the mobile phone and the SC on the laptop communicate the capability descriptions of the capabilities they have collected through the established connection channel. Specifically, the SC on the mobile phone provides the capability description transmitted by the SC on the laptop to the AS on the mobile phone, and the AS on the mobile phone provides the capability description to the AI ​​Core on the mobile phone; the SC on the laptop provides the capability description transmitted by the SC on the mobile phone to the AS on the laptop, and the AS on the laptop provides the capability description to the AI ​​Core on the laptop.

[0210] Therefore, the AS in any device participating in the interconnection stores the capability description of all devices; the AI ​​Core in any device participating in the interconnection stores the capability description of all devices.

[0211] Specifically, the SC of the notebook can manage the capability description obtained from the AS through the interconnection capability manager MGR (SmartConnect CapabilityManager) and save it to the capability list; the SC of the mobile phone can manage the capability description obtained from the AS through the MGR and save it to the capability list. Moreover, the notebook and the mobile phone can establish a private connection channel through their respective SCs, and communicate the capability descriptions in the capability lists of their respective MGRs to each other through the connection channel.

[0212] Alternatively, the target file of the SC on the notebook also includes a target file sent by the SC of the mobile phone that declares the capabilities of the mobile phone. The AS on the notebook can directly obtain the capability description of the capabilities of each application on the mobile phone by parsing the target file, and the AS on the notebook provides these capability descriptions to the AI ​​Core on the notebook. Similarly, the target file of the SC on the mobile phone also includes a target file sent by the SC of the notebook that declares the capabilities of the notebook (such as AI capabilities). The AS on the mobile phone can directly obtain the capability description of the capabilities of each application (such as AI application) on the notebook by parsing the target file, and the AS on the mobile phone provides the capability description to the AI ​​Core on the mobile phone.

[0213] Therefore, after the capability descriptions of the respective SCs on the laptop and the mobile phone are communicated with each other, the capability list of the MGR of the SC on the laptop includes not only the capability description of the capabilities of the local device, but also the capability description of the capabilities of each application on the mobile phone. Correspondingly, the AS of the notebook includes not only the capability description of the capabilities of the local device, but also the capability description of the capabilities of each application on the mobile phone. In addition to the capability description of the capabilities of the local device, the capability list of the MGR of the SC on the mobile phone also includes the capability description of the capabilities of each application on the laptop, such as the AI ​​image generation capability. Correspondingly, the AS of the mobile phone includes not only the capability description of the capabilities of the local device, but also the capability description of the capabilities of each application on the laptop, such as the AI ​​image generation capability.

[0214] In addition, the SCs of the laptop and the phone can update the capabilities of the other device's applications to other local applications in real time. For example, if the AI ​​image generation capability of the laptop changes, the SC on the phone will obtain the new AI image generation capability of the laptop through the connection channel and transmit it to the AS on the phone.

[0215] It should be noted that after capability intercommunication, other applications on the mobile phone can query the super-connectivity capabilities declared by the SC and the AI ​​capabilities of the notebook received by the SC through the AS, such as the AI ​​image generation capability.

[0216] Based on the above scheme, the mobile phone can receive input data such as text or voice representing user needs through the voice assistant, and the input data represents that the user needs to use the AI ​​image generation capability. Based on this, the AI ​​Core of the mobile phone processes the input data representing the user needs and the capability description of all capabilities to generate a call request, and the call request corresponds to the AI ​​image generation capability. After that, the AI ​​Core of the mobile phone sends the call request corresponding to the AI ​​image generation capability to the SC on the mobile phone, and the SC on the mobile phone sends the call request for the AI ​​image generation capability to the SC on the notebook through the connection channel. The SC on the notebook calls the AI ​​image generation capability in the AI ​​application on the notebook by calling the service, for example, the input parameters in the call request are given to the AI ​​application through the first service. After the AI ​​application generates an image, the image generated by the AI ​​application is obtained through the second service and the generated image, that is, the output result, is returned to the SC on the notebook. The SC on the notebook sends the output result to the SC on the mobile phone through the connection channel, and the SC on the mobile phone returns the output result to the voice assistant on the mobile phone, thereby realizing the AI ​​image generation capability on the notebook on the voice assistant on the mobile phone.

[0217] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0218] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0219] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0220] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.< / resolveinfo>

Claims

1. A data processing method, comprising: Obtaining a target file, wherein the target file belongs to a target function module; Parsing the target file to obtain at least one target capability of the target functional module and a calling service for calling the target capability; in, Each of the target capabilities includes a capability name and a capability description, and the capability description complies with natural language; The calling service of each target capability includes a first service that provides input parameters for processing by the target function module and a second service that obtains output results from the target function module.

2. The method according to claim 1, further comprising: A tag of the target function module is established, wherein the tag represents a corresponding relationship between the target function module and each of the target capabilities possessed by the target function module and a calling service for calling the target capabilities.

3. The method according to claim 2, further comprising: Obtaining a call request for calling a first capability, wherein the call request includes input parameters; Based on the corresponding relationship, the input parameters in the call request are sent to the first functional module through the first service of the first call service, and the output result corresponding to the first capability is obtained from the first functional module through the second service of the first call service.

4. The method according to claim 3, wherein the calling request is: a request generated by processing the user requirements obtained by the target application and each of the capability descriptions; in, The output result corresponding to the first capability is used to provide to the target application as feedback information of the user demand.

5. The method according to claim 4, further comprising: Based on the intelligent engine, processing the user requirements obtained through the target application and each capability description to generate the call request; The intelligence engine has each of the capability descriptions; Based on the intelligent engine, an output result corresponding to the first capability is provided to the target application as feedback information of the user demand.

6. An electronic device comprising: At least one target function module, each of the at least one target function module has at least one target capability and a calling service for calling the target capability; A capability management module is used to obtain a target file, the target file belongs to the target function module; parse the target file to obtain at least one target capability of the target function module and a calling service for calling the target capability; in, Each of the target capabilities includes a capability name and a capability description, and the capability description complies with natural language; The calling service of each target capability includes a first service that provides input parameters for processing by the target function module and a second service that obtains output results from the target function module.

7. The electronic device according to claim 6, wherein: The capability management module is further used to: obtain a call request for calling the first capability, wherein the call request includes input parameters; Based on the corresponding relationship in the capability management module, sending the input parameter in the call request to the first function module through the first service of the first call service and obtaining the output result corresponding to the first capability from the first function module through the second service of the first call service; The capability management module establishes a tag of the target function module, the tag represents the corresponding relationship, and the corresponding relationship is: the corresponding relationship between the target function module and the target capability of the target function module and the calling service of the target capability.

8. The electronic device according to claim 7, further comprising: Intelligent engine; The intelligent engine is used to: process the user requirements obtained through the target application and each capability description to generate the call request; The intelligence engine has each of the capability descriptions; The intelligent engine is used to provide the output result corresponding to the first capability to the target application as feedback information of the user demand.

9. The electronic device according to claim 8, wherein the target function module comprises: A second functional module deployed on the electronic device, the second functional module being capable of establishing a connection channel between the electronic device and a target device; The intelligent engine is further used to: obtain, from the capability management module, a capability description of at least one target capability possessed by the third functional module deployed on the target device; The capability description of at least one target capability possessed by the third functional module is sent from the target device to the capability management module through the connection channel.

10. The electronic device according to claim 9, wherein the capability management module is further configured to: In the case where the call request corresponds to the target device, the call request is sent to the target device through the connection channel, the target device calls the first service of the first call service through the deployed capability management module, sends the input parameters in the call request to the third function module, and obtains the output result corresponding to the first capability from the third function module through the second service of the first call service; An output result corresponding to the first capability sent by the target device is received through the connection channel.

Images