Plug-in processing method and apparatus, device, storage medium, and program product
By responding to user interaction information in the digital assistant, determining plugin calls and establishing communication connections, the problem of traditional plugins being unable to operate client devices is solved, thus expanding functionality and service capabilities.
Patent Information
- Application Number
- PCT/CN2025/103296
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-06-30
- Filing Date
- 2025-06-25
- Publication Date
- 2026-01-08
AI Technical Summary
Traditional digital assistant plugins cannot directly operate client devices, limiting their functional expansion and service capabilities.
By responding to user interaction information, the plugin call is determined, the target communication connection is established, and the call parameters are used to request the client device to execute the function.
It expands the functionality and service capabilities of the digital assistant and enhances the diversity of plugin usage.
Smart Images

Figure CN2025103296_08012026_PF_FP_ABST
Abstract
Description
Method, device, equipment, storage medium and program product for plug-in processing
[0001] The present application claims priority to the Chinese patent application No. 202410870005.3, filed on June 30, 2024, entitled “Method, device, equipment, storage medium and program product for plug-in processing”, the whole content of which is incorporated herein by reference. TECHNICAL FIELD
[0002] The example embodiments of the present disclosure generally relate to the field of computer, and in particular, to a method, device, electronic equipment, computer readable storage medium and computer program product for plug-in processing. BACKGROUND
[0003] With the development of information technology, various client devices can provide people with various services in work and life, etc. For example, an application providing services can be deployed in the client device. The client device or the application can provide a user with a digital assistant type function to assist the user in using the client device or the application. How to improve the functional diversity of the digital assistant is a technical problem to be explored at present. SUMMARY
[0004] In a first aspect of the present disclosure, a method for plug-in processing is provided. The method comprises: in response to receiving interaction information of a user from a client device, determining, based on the interaction information, a call to a predetermined plug-in, the predetermined plug-in being configured to request execution of at least one function at the client device; determining, based on the interaction information, a call parameter of a target function in the at least one function and a target communication connection mode with the client device; establishing a communication connection with the client device based on the target communication connection mode; and requesting the client device to execute the target function by using the call parameter via the communication connection.
[0005] In a second aspect of the present disclosure, a method for plug-in processing is provided. The method comprises: in response to receiving interaction information of a user, sending the interaction information to a server device; establishing a corresponding communication connection with the server device based on a target communication connection mode; receiving, from the server device via the communication connection, a call parameter of a target function at a client device; and executing the target function based on the call parameter.
[0006] In a third aspect of the present disclosure, an apparatus for plug-in processing is provided. The apparatus includes an interaction information receiving module configured to, in response to receiving interaction information of a user from a client device, determine, based on the interaction information, a call to a predetermined plug-in configured to request execution of at least one function at the client device; a communication mode determining module configured to determine, based on the interaction information, a call parameter for a target function in the at least one function and a target communication connection mode with the client device; a first connection establishing module configured to establish, based on the target communication connection mode, a communication connection with the client device; and a target function requesting module configured to request, via the communication connection, the client device to execute the target function using the call parameter.
[0007] In a fourth aspect of the present disclosure, an apparatus for plug-in processing is provided. The apparatus includes an interaction information sending module configured to, in response to receiving interaction information of a user, send the interaction information to a server device; a second connection establishing module configured to establish, based on a target communication connection mode, a corresponding communication connection with the server device; a call parameter receiving module configured to receive, via the communication connection, a call parameter for a target function at the client device from the server device; and a target function executing module configured to execute the target function based on the call parameter.
[0008] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. The instructions, when executed by the at least one processor, cause the electronic device to perform the method of the first aspect or the method of the second aspect.
[0009] In a sixth aspect of the present disclosure, a computer-readable storage medium is provided. The medium has stored thereon computer-executable instructions that, when executed by a processor, implement the method of the first aspect or the method of the second aspect.
[0010] In a seventh aspect of the present disclosure, a computer-executable instruction product is provided. The product includes computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method of the first aspect or the method of the second aspect according to the present disclosure.
[0011] It should be understood that the description in this section is not intended to define key or essential features of embodiments of the present disclosure, and is not used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS
[0012] The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:
[0013] FIG. 1 illustrates a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;
[0014] FIG. 2 illustrates a flow diagram of a signaling flow for plug-in processing according to some embodiments of the present disclosure;
[0015] FIG. 3 illustrates a flow diagram of a signaling flow for plug-in processing according to some embodiments of the present disclosure;
[0016] FIG. 4 illustrates a flow diagram of a signaling flow for plug-in processing according to some embodiments of the present disclosure;
[0017] FIG. 5 illustrates a flow diagram of a process for plug-in processing according to some embodiments of the present disclosure;
[0018] FIG. 6 illustrates a flow diagram of a process for plug-in processing according to some embodiments of the present disclosure;
[0019] FIG. 7 illustrates an exemplary structural block diagram of an apparatus for plug-in processing according to some embodiments of the present disclosure;
[0020] FIG. 8 illustrates an exemplary structural block diagram of an apparatus for plug-in processing according to some embodiments of the present disclosure; and
[0021] FIG. 9 illustrates a block diagram of an electronic device that can implement one or more embodiments of the present disclosure. DETAILED DESCRIPTION
[0022] Embodiments of the present disclosure will be described more fully hereinafter with reference to the accompanying drawings. While certain embodiments of the present disclosure will be shown and described, it will be understood that the present disclosure can be embodied in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and fully convey the scope of the present disclosure to those skilled in the art. It will be appreciated that the drawings described are for illustration only and should not be construed as limiting the scope of the present disclosure.
[0023] In the description of embodiments of the present disclosure, the term "including" and its derivatives, shall be understood as comprising but not limited to. The term "based on" shall be understood as "based, at least in part, on." The term "one embodiment" or "an embodiment" shall be understood as "at least one embodiment." The term "some embodiments" shall be understood as "at least some embodiments." Other explicit or implicit definitions can also be included below.
[0024] In this document, unless explicitly stated otherwise, performing a step "in response to" an action means performing the step after the action, but can include one or more intervening steps.
[0025] It can be understood that the data involved in the technical solutions of the present disclosure (including but not limited to the data itself, the obtaining, use, storage or deletion of the data) should comply with the requirements of the relevant laws and regulations and relevant provisions.
[0026] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type of information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the relevant user and the authorization of the relevant user should be obtained through appropriate means according to the relevant laws and regulations, wherein the relevant user can include any type of right subject, such as an individual, an enterprise or a group.
[0027] For example, in response to receiving the active request of the user, a prompt information is sent to the relevant user to explicitly prompt the relevant user that the operation requested to be performed will need to obtain and use the information of the relevant user, so that the relevant user can voluntarily choose whether to provide the information to the software or hardware such as the electronic device, the application program, the server or the storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.
[0028] As an optional but non-limiting implementation manner, in response to receiving the active request of the relevant user, the prompt information can be sent to the relevant user in the form of a pop-up window, and the prompt information can be presented in the form of text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide information to the electronic device.
[0029] It can be understood that the above notification and user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.
[0030] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that the corresponding output can be generated for a given input after the training is completed. The generation of the model can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. The neural network model is an example of a model based on deep learning. In this document, "model" can also be referred to as "machine learning model", "learning model", "machine learning network" or "learning network", and these terms are used interchangeably herein.
[0031] FIG. 1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. The environment 100 involves client devices 120, 170, server devices 110, 160, a machine learning model 130, and a plugin service 140.
[0032] As shown in FIG. 1, the client device 120 can be deployed with a digital assistant 121, with which a user 150 can interact through the client device 120 or an attached device of the client device 130. The digital assistant 121 is provided to assist the user 150 in various task processing needs in different applications, scenarios. The digital assistant 121 typically has intelligent conversation and task processing capabilities. In the course of interaction with the digital assistant 121, the user 150 inputs interaction messages (e.g., text, speech, image, video, or other modalities of conversation content), and the digital assistant 121 provides reply messages in response to the user inputs. Typically, the digital assistant 121 is capable of supporting the user 150 to input questions in a natural language manner, and to perform tasks and provide replies based on understanding of the natural language inputs and logical reasoning capabilities.
[0033] In some embodiments, the digital assistant 121 supports the use of plugins 141. Each plugin 141 is capable of providing one or more functionalities of an application. Such plugins 141 include, but are not limited to, one or more of the following: a search plugin, a contact plugin, a message plugin, a document plugin, a table plugin, a mail plugin, a calendar plugin, a schedule plugin, a task plugin, and the like.
[0034] In embodiments of the present disclosure, the plugin service 140 provides a creation, publishing, saving, and application environment of the plugins 141 for the users 150 and 180. The plugin service 140 can be deployed with, for example, a database through which the created plugins 141 are saved. For example, the plugin service 140 can have saved therein a plurality of plugins 141, including plugin 141-1, plugin 141-2, …, plugin 141-N.
[0035] The server device 160 can be deployed with a plugin creation platform, which can provide a set of tools required for creation of the plugins 141. The plugin creation platform can support visual development of the plugins 141. The plugin creation platform can be any suitable platform that supports user development interfaces. For example, the client device 170 can be deployed with a client program of the plugin creation platform. The client program can support the user 180 to interact with the plugin creation platform, which can interact with the plugin service 140 to implement creation of the plugins 141.
[0036] The server device 110 can be deployed with a plug-in application platform that can interact with the plug-in service 140 to implement the invocation of the plug-in 141. After the plug-in 141 is created, the user 150 can input a conversation message in a conversation window of the digital assistant 121, and the digital assistant 121 can request the plug-in service 140 to assist in invoking the plug-in 141 based on the plug-in definition of the plug-in 141, and obtain feedback information from the plug-in 141, determine a reply message based on the feedback information and present to the user in the conversation window.
[0037] In some embodiments, the digital assistant 121 can utilize the machine learning model 130 to support the interaction with the user 150. For example, the digital assistant 121 can utilize one or more machine learning models 130 to provide a question and answer service to the user 150.
[0038] In some embodiments, the one or more machine learning models 130 can be constructed based on a language model (LM). The machine learning model used is a content generative model that is capable of generating a corresponding output based on a model input.
[0039] In some embodiments, the language model based machine learning model is capable of processing a model input in a text modality (e.g., natural language and / or machine language) and / or a non-text modality (e.g., image, voice, video, etc.), and is capable of generating a desired output according to the model input and a prompt word. The prompt word here is used to guide the machine learning model to generate an output that can address a user demand indicated by the model input. In an application scenario for supporting user conversation, a user input can be provided to the machine learning model 130 as at least a part of the model input (other parts can include the prompt word). The user input is considered as a question. Based on the model output, a corresponding answer can be generated to be provided to the user 150.
[0040] In some embodiments, the client device 120 communicates with the server device 110 to implement the provision of services of the digital assistant 121. As shown in FIG. 1, the server device 130 can invoke the machine learning model 130 to support the human-computer conversation function between the digital assistant and the user 150 based on the output of the machine learning model 130.
[0041] The client devices 120, 170 can be any type of mobile terminal, fixed terminal, or portable terminal including a mobile handset, a tablet computer, a laptop computer, a notebook computer, a subnotebook computer, an ultrabook computer, a netbook computer, a media player, a television receiver, a radio broadcast receiver, an electronic book device, a game device, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In some embodiments, the client devices 120, 170 can also be capable of supporting any type of interface for user interaction (such as suitable “wearable” circuitry, etc.). The client device 120 and the client device 170 can be implemented as the same client device or as different client devices.
[0042] The server devices 110, 160 can be various types of computing systems / servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in a cloud environment, etc. The server devices 110, 160 can be implemented based on a cloud environment, for example. The server device 110 and the server device 160 can be implemented as the same server device or as different server devices.
[0043] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only, without implying any limitation on the scope of the present disclosure.
[0044] As mentioned previously, with the development of information technology, various client devices can provide various services to people in work and life, etc. For example, an application providing services can be deployed in a client device. The client device or the application can provide a digital assistant type function to a user to assist the user in using the client device or the application.
[0045] To improve the functional diversity of the digital assistant, some developers configure the digital assistant to support the use of plugins, each of which can provide one or more functions. The digital assistant can invoke a corresponding plugin to provide services to the user according to the user’s needs.
[0046] Traditionally, the digital assistant usually provides services to the user based on the feedback information of the plugin, but the plugin usually cannot operate the client device, which limits the functional expansion and service capability of the digital assistant to some extent.
[0047] In view of this, according to an embodiment of the present disclosure, an improved solution for plug-in processing is provided. According to the solution of the present disclosure, in response to receiving interaction information of a user from a client device, a call to a predetermined plug-in is determined based on the interaction information, the predetermined plug-in being configured to request execution of at least one function at the client device; a call parameter for a target function in the at least one function and a target communication connection mode with the client device are determined based on the interaction information; a communication connection is established with the client device based on the target communication connection mode; and the client device is requested to execute the target function via the communication connection using the call parameter.
[0048] In this way, in the plug-in call scenario, the plug-in can be used to request the client device to execute the target function, which is conducive to expanding the use function and service capability of the digital assistant.
[0049] Some example embodiments of the present disclosure will be described below with continuous reference to the accompanying drawings.
[0050] FIG. 2 shows a flowchart of a signaling flow 200 for plug-in processing according to some embodiments of the present disclosure. The signaling flow 200 involves a client device 120, a server device 110, a gateway 210, a machine learning model 130, and a plug-in service 140. For ease of discussion, the signaling flow 200 will be described with reference to the environment 100 of FIG. 1.
[0051] In an embodiment of the present disclosure, as shown in the signaling flow 200, the client device 120 sends (202) the interaction information to the server device 110 in response to receiving the interaction information of the user 150. The server device 110 provides (206) the interaction information to the machine learning model 130 in response to receiving (204) the interaction information of the user 150 from the client device 120. The machine learning model 130 receives (208) the interaction information provided by the server device 110, determines a response 214 to the interaction information, and feeds back (212) the response 214 to the server device 110.
[0052] The interaction information herein can include various types of information input by the user 150, such as text information, voice information, image information, video information, or information of other modalities. Before the client device 120 sends the interaction information to the server device 110, the client device 120 can establish a communication connection with the server device 110, such as a long connection or a short connection conforming to the Transmission Control Protocol (TCP) protocol, and send the interaction information to the server device 110 via the established communication connection.
[0053] Alternatively or additionally, the service-side device 110 receives the interaction information from the client-side device, can construct a model input of the machine learning model 130 based on the interaction information, and provide the model input to the machine learning model 130. The machine learning model 130 can generate a response 214 to the interaction information based on the model input.
[0054] Here, the machine learning model 130 can determine whether a predetermined plug-in needs to be invoked to request execution of at least one function at the client-side device 120 based on the model input. If the machine learning model 130 determines that the predetermined plug-in does not need to be invoked, a response to the interaction information can be generated. For example, the machine learning model 130 can generate a response text to the question of the user 150. At this time, the service-side device 110 can feed back the response to the client-side device 120. If the machine learning model 130 determines that the predetermined plug-in (e.g., at least one of the plug-ins 141-1, 141-2, …, 141-N) needs to be invoked to request execution of at least one function at the client-side device 120, the machine learning model 130 can determine an invocation parameter for a target function of the at least one function and a target communication connection mode with the client-side device 120 based on the interaction information.
[0055] Here, the at least one function at the client-side device 120 can include various functions that the client-side device 120 is capable of performing, such as volume adjustment of the client-side device 120, screen brightness adjustment, network control, turning on or off Bluetooth, turning on or off a hotspot, device mode adjustment, opening a specific application of the client-side device 120, etc., and the specific type of the at least one function at the client-side device 120 is not limited here. The target function can be any one of the various functions that the client-side device 120 is capable of performing, such as turning on or off Bluetooth, turning on or off a hotspot, etc. Accordingly, the predetermined plug-in here is configured to request the client-side device 120 to perform at least one of the various functions that the client-side device 120 is capable of performing, e.g., the predetermined plug-in can be configured to request the client-side device 120 to perform a volume adjustment function, a screen brightness adjustment function, etc.
[0056] Alternatively or additionally, the invocation parameter can include a function identifier, one or more input parameters, and one or more output parameters. The input parameter here can include information required to be carried to perform the target function, and the output parameter can include information generated or returned after the target function is performed. The function identifier can be used to identify a target function module (e.g., a system call interface, an application program interface, a function, etc.) for performing the target function at the client-side device.
[0057] Alternatively or additionally, the target communication connection mode is used to indicate a communication connection mode between the predetermined plug-in and the client device 120. The target communication connection mode can include, but is not limited to, a long connection and a short connection conforming to a TCP protocol, and the like. Of course, the above communication connection modes are only exemplary, and in actual application, the communication connection mode can be flexibly selected according to actual needs.
[0058] In an embodiment of the present disclosure, as shown in the signaling flow 200, the server device 110 receives (216) the response 214 from the machine learning model 130. The server device 110 determines (218) whether the response 214 indicates the call to the predetermined plug-in. If the response 214 includes the call indication to the predetermined plug-in, the call parameter to the target function in the at least one function, and the target communication connection mode with the client device 120, the server device 110 sends (220) the call request to the plug-in service 140.
[0059] Alternatively or additionally, the server device 110 can determine whether the response 214 contains the call indication to the predetermined plug-in. If the response 214 contains the call indication to the predetermined plug-in, the server device 110 can determine that the response 214 indicates the call to the predetermined plug-in. If the response 214 does not contain the call indication to the predetermined plug-in, it can be determined that the response 214 does not indicate the call to the predetermined plug-in. The server device 110 can also determine whether the response 214 contains the call parameter to the target function and / or the target communication connection mode with the client device 120. If the response 214 contains the call parameter and / or the target communication connection mode, it can be determined that the response 214 indicates the call to the predetermined plug-in. If the response does not contain the call parameter and the target communication connection mode, it can be determined that the response 214 does not indicate the call to the predetermined plug-in. It can be understood that the call indication here is not necessarily required. In actual application, the call to the predetermined plug-in can also be identified by other means.
[0060] Alternatively or additionally, the call request can include the call parameter of the target function and the communication identifier of the target communication connection mode (such as an identifier for identifying a long connection or a short connection). The call request can also include a plug-in identifier (such as a plug-in name, a plug-in number, a plug-in address, and the like) for identifying the predetermined plug-in.
[0061] In an embodiment of the present disclosure, as shown in the signaling flow 200, the plug-in service 140 determines the target communication connection mode between the predetermined plug-in and the client device 120 in response to receiving (222) the call request from the server device 110. If the target communication connection mode indicates a long connection mode, the plug-in service 140 establishes (228) a long connection (sometimes also referred to herein as a "first communication connection") corresponding to the long connection mode between the gateway 201 and the client device 120.
[0062] Alternatively or additionally, the plug-in service 140 can determine the target communication connection mode between the predetermined plug-in and the client device 120 based on the communication identifier in the invocation request. The plug-in service 140 can invoke the predetermined plug-in before the long connection is established, and then establish the long connection between the plug-in service 140 and the client device 120. The plug-in service 140 can also invoke the predetermined plug-in after the long connection is established.
[0063] In an embodiment of the disclosure, as shown in the signaling flow 200, the plug-in service 140 invokes (226) the predetermined plug-in to perform one or more communications with the client device 120 via the long connection using the invocation parameters, to request the client device 120 to perform the target function. The client device 120 receives the invocation parameters for the target function at the client device 120 from the plug-in service 140 via the long connection. The client device 120 performs the target function based on the invocation parameters.
[0064] Alternatively or additionally, the plug-in service 140 can perform one communication with the client device 120 using the invocation parameters to complete the performance of the target function. The plug-in service 140 can also perform multiple communications with the client device 120 using the invocation parameters to complete the performance of the target function.
[0065] Exemplarily, as shown in FIG. 2, the plug-in service 140 can invoke the predetermined plug-in to send (230) a message a to the client device 120 via the long connection, which can include at least part of the invocation parameters, and which can also include other contents. The client device 120 performs (234) an operation a in response to receiving (232) the message a from the plug-in service 140. Then, the client device 120 sends (236) a feedback a to the plug-in service 140 via the long connection, which can include the execution result of the operation a. Of course, the feedback a can also include other information.
[0066] The plug-in service 140 can send (240) a message b to the client device 120 in response to receiving (238) the feedback a from the client device 120, which can also include at least part of the invocation parameters. The client device 120 can perform (244) an operation b in response to receiving (242) the message b from the plug-in service 140. Then, the client device 120 can send (246) a feedback b to the plug-in service 140, which can include the execution result of the operation b. Of course, the feedback b can also include other information.
[0067] It can be understood that the above communication process is only exemplary. In order to request the client device 120 to perform the target function, the plug-in service 140 and the client device 120 can only need to perform one communication, or can need to perform more than two communications.
[0068] In some embodiments, the client device 120 can determine whether it has authorization to perform the target function for the client device. If it is determined that the authorization is available, the target function can be performed based on the invocation parameters. If the authorization is not available, the client device 120 can send a notification to the server device 120, by which the server device 120 is informed that the authorization to perform the target function for the client device 120 is missing. Alternatively, the client device 120 can also present the notification to inform the user 150 that the authorization to perform the target function for the client device 120 is missing, to request the user 150 to perform an authorization operation.
[0069] In some embodiments, if the plug-in service 140 determines that the execution condition of the target function includes that the authorization to perform the target function for the client device 120 is required, the plug-in service 140 can send an authorization request to the client device 120 via the communication connection, the authorization request being used to request the authorization to perform the target function for the client device. The client device 120 can send the authorization to perform the target function for the client device 120 to the plug-in service 140 in response to receiving an authorization operation to perform the target function for the client device 120. The plug-in service 140 can request the client device 120 to perform the target function with the invocation parameters via the communication connection in response to receiving the authorization.
[0070] For example, the message a sent by the plug-in service 140 to the client device 120 can include an authorization request, the client device 120 can present a notification page for informing the user 150 to authorize the target function in response to receiving the message a. The client device 120 can send a feedback a to the plug-in service in response to the authorization operation of the user 150 to perform the target function for the client device 120 (e.g. operation b), the feedback a can include the authorization to perform the target function for the client device 120. The plug-in service 140 can send a message b to the client device 120 in response to receiving the authorization, the message b can include at least part of the invocation parameters. The client device 120 can perform the operation b based on the invocation parameters to implement the target function (e.g. a volume adjustment function, a brightness adjustment function, etc.).
[0071] In embodiments of the disclosure, as shown in signaling flow 200, the client device 120 feeds back the execution result of the target function to the plugin service 140 via the long connection. The plugin service 140 feeds back (250) the execution result of the target function to the server device 110, which can provide (254) the execution result to the machine learning model 130 in response to receiving (252) the execution result. The machine learning model 130 generates a response 262 to the interaction information after receiving (256) the execution result from the server device 110. The machine learning model 130 feeds back (264) the response 262 to the server device 110, which feeds back (266) the response 262 to the client device 120. The client device 120 presents the response 262 in response to receiving (268) the response 262 to the interaction information from the server device 110. The response 262 here indicates at least the execution result of the target function.
[0072] Alternatively or additionally, the client device 120 feeds back one or more execution results of the target function to the plugin service 140 via the long connection. For example, the client device 120 can be configured to perform one or more operations in order to implement the target function, and the client device 120 can be configured to feed back the execution result (e.g., feedback a, feedback b) for each operation to the plugin service 140 after the operation is completed. The client device 120 can also be configured to feed back the execution result of the target function to the plugin service after one or more operations of the target function are completed.
[0073] Alternatively or additionally, the response 262 here can be used to indicate that the target function is completed, the target function fails, or the target function is partially completed. For example, the user 150 can have a conversation with the digital assistant 121 and input a user voice “please increase the volume” to the digital assistant 121. After the plugin service 140 invokes the predetermined plugin to request the client device 120 to complete the volume adjustment, the server device 110 can provide the execution result of the volume adjustment to the machine learning model 130, which can generate a response text or a response voice (e.g., “volume adjustment is completed” or “volume is increased”, etc.), and the server device 110 can feed back the response text or the response voice to the client device 120. The client device 120 can present the response text and / or play the response voice.
[0074] FIG. 3 illustrates a flowchart of a signaling flow 300 for plug-in processing, according to some embodiments of the present disclosure. The signaling flow 300 involves the client device 120, the server device 110, the gateway 210, the machine learning model 130, and the plug-in service 140. It can be appreciated that the signaling flow 300 differs from the signaling flow 200 mainly in that the plug-in service 140 and the client device 120 communicate via short connections, and other processes in the signaling flow 300 are similar to those in the signaling flow 200. Therefore, the processes in the signaling flow 300 that utilize short connection communication will be described in detail below with reference to the environment 100 of FIG. 1, and the other processes can be referred to the description of the signaling flow 200.
[0075] In embodiments of the present disclosure, as shown in the signaling flow 300, the client device 120 sends (302) the interaction information to the server device 110 in response to receiving the interaction information of the user 150. The server device 110 provides (306) the interaction information to the machine learning model 130 in response to receiving (304) the interaction information of the user 150 from the client device 120. The machine learning model 130 receives (308) the interaction information provided by the server device 110, determines a response 314 to the interaction information, and feeds back (312) the response 214 to the server device 110.
[0076] In embodiments of the present disclosure, as shown in the signaling flow 300, the server device 110 receives (316) the response 314 from the machine learning model 130. The server device 110 determines (318) whether the response 314 indicates a call to a predetermined plug-in. If the response 314 includes a call indication to a predetermined plug-in, a call parameter to a target function in the at least one function, and a target communication connection mode with the client device 120, the server device 110 sends (320) a call request to the plug-in service 140.
[0077] In embodiments of the present disclosure, as shown in the signaling flow 300, the plug-in service 140 determines (324) the target communication connection mode with the client device 120 in response to receiving (322) the call request from the server device 110. If the target communication connection mode indicates a short connection mode, the plug-in service 140 establishes a short connection (also referred to as a “second communication connection” herein at times) at least once between the server device 110 and the client device 120. The plug-in service 140 performs a communication with the client device 120 once via each established short connection using the call parameter. The client device 120 performs a communication with the plug-in service 140 once via each established short connection to receive the call parameter from the plug-in service 140. The client device 120 performs the target function based on the call parameter.
[0078] The short connection here can be a short connection conforming to the TCP protocol. The short connection can only perform one communication each time it is established. If the target function requested by the client device 120 only needs to perform one communication, the plug-in service 140 and the client device 120 only need to establish one short connection via the server device 110. If the target function requested by the client device 120 needs to perform multiple communications, the plug-in service 140 and the client device 120 need to establish multiple short connections, each time a short connection is established to perform one communication, and then the established short connection is disconnected.
[0079] For example, as shown in FIG. 3, the plug-in service 140 can establish a short connection via the server device 110 and the client device 120, and the plug-in service 140 sends (328) message c to the client device 120 based on the short connection. The client device 120 performs (332) operation c in response to receiving (330) message c from the plug-in service 140. Then, the client device 120 can establish a short connection again via the server device 110 and the plug-in service 140, and the client device 120 can send (334) feedback c to the plug-in service 140 via the short connection. After the feedback c is sent, the short connection is disconnected again.
[0080] The plug-in service 140 can establish a short connection again with the client device 120 in response to receiving (336) the feedback c, send (338) message d to the client device 120 via the short connection, and then disconnect the short connection again. The client device 120 can perform (342) operation d in response to receiving (340) message d from the plug-in service 140. The client device 120 establishes a short connection again with the plug-in service 140, and sends (344) feedback d to the plug-in service 140 via the short connection.
[0081] In some embodiments, an identifier is included in each message communicated between the server device 110 and the client device 120 via the short connection (also referred to as a "second communication connection" herein), and the identifier is used to identify the current execution of the target function. In this way, in the case where the server device 110 and the client device 120 need to perform multiple communications based on the short connection, the server device 110 and the client device 120 can determine information related to the target function based on the identifier.
[0082] For example, as shown in FIG. 3, the same identifier is included in message c and message d, and after receiving (340) message d, the client device 120 can find the feedback c based on the identifier, and thus obtain the execution result of operation c. Then, the client device 120 can perform operation d based on message d and feedback c. In this way, the continuous execution of the target function can also be ensured in the case of communication via the short connection.
[0083] In embodiments of the present disclosure, as shown in signaling flow 300, the client device 120 feeds back the execution result of the target function to the plugin service 140 via the long connection. The plugin service 140 feeds back (348) the execution result of the target function to the server device 110, which can provide (352) the execution result to the machine learning model 130 in response to receiving (350) the execution result. The machine learning model 130 generates a response 360 to the interaction information after receiving (354) the execution result from the server device 110. The machine learning model 130 feeds back (362) the response 360 to the server device 110, which feeds back (364) the response 360 to the client device 120. The client device 120 presents the response 360 in response to receiving (366) the response 360 to the interaction information from the server device 110. The response 360 here indicates at least the execution result of the target function.
[0084] Alternatively or additionally, to implement the target function, the client device 120 can be configured to perform one or more operations, and the client device 120 can be configured to feed back the execution result for each operation to the plugin service 140 after the operation is performed. The client device 120 can also be configured to feed back the execution result for the target function to the plugin service after all the one or more operations of the target function are performed.
[0085] The plugin service 140 can send the execution result for each operation to the server device 110 after receiving the execution result. The server device 110 can combine the execution results of the one or more operations to form the execution result for the target function based on the identifier after completing the execution of the target function. The execution result for the target function is then provided to the machine learning model 130.
[0086] The plugin service 140 can also combine the execution results of the one or more operations to form the execution result for the target function based on the identifier after completing the execution of the target function. The execution result for the target function is then sent to the server device 110.
[0087] To facilitate understanding of the creation process of the plugin, the creation process of the plugin will be described below with reference to FIG. 4. FIG. 4 shows a flowchart of a signaling flow 400 for plugin processing according to some embodiments of the present disclosure. The signaling flow 400 involves a client device 170, a server device 160, and a plugin service 140. To facilitate discussion, the signaling flow 400 will be described with reference to the environment 100 of FIG. 1.
[0088] In an embodiment of the disclosure, as shown in signaling flow 300, the client device 170 sends (402) a creation request for the plugin to the server device 160 in response to a creation operation of the plugin by the developer (e.g., the user 180). After the server device 160 receives (404) the creation request from the client device 170, the server device 160 requests (406) the plugin service 140 to create (410) the plugin. The plugin service 140 feeds back (412) a creation result of the plugin to the server device 160, and the server device 160 feeds back (416) the creation result of the plugin to the client device 170 in response to receiving (414) the creation result of the plugin.
[0089] Alternatively or additionally, the developer can input information such as a plugin definition to the plugin creation platform via the client device 170 or an attached device of the client device 170. The plugin definition can include, but is not limited to, a name field, an address field, a description field, input parameters, output parameters, a communication identifier, and the like of the plugin. The communication identifier can be used to identify a target communication connection mode adopted by the plugin.
[0090] Alternatively or additionally, the creation request can carry the above-mentioned plugin definition and the like. The server device 160 can also send a plugin creation page to the client device 170 in response to the creation request. The developer can input the above-mentioned plugin definition information to the plugin creation page.
[0091] Alternatively or additionally, the plugin service 140 can be deployed with a database. The plugin service 140 creates the plugin based on the above-mentioned plugin definition and the like, and then the plugin service 140 can save the created plugin in the database for subsequent calling.
[0092] In an embodiment of the disclosure, as shown in signaling flow 300, the client device 170 can send (420) a binding request to the server device 160 in response to the creation result indicating that the plugin is created successfully received (418) by the client device 170. The binding request is used to request the server device 160 to establish a binding relationship between the digital assistant 121 and the plugin.
[0093] In an embodiment of the disclosure, as shown in signaling flow 300, the server device 160 establishes (424) the binding relationship between the digital assistant and the plugin in response to receiving (422) the binding request from the client device 170. The server device 160 can feed back (430) a binding result of the binding relationship to the client device 170. The client device 170 can present the binding result in response to receiving (432) the binding result.
[0094] Alternatively or additionally, after the plug-in service 140 completes the creation of the plug-in, the plug-in service 140 can generate a plug-in number for identifying the plug-in, and the plug-in service 140 can send the plug-in number to the client device 170 through the server device 160. For example, the plug-in number can be included in the creation result of the plug-in. The client device 170 can send a binding request to the server device 160 based on the plug-in number. The server device 160 can establish a binding relationship between the corresponding plug-in and the digital assistant 121 based on the plug-in number, so that the digital assistant 121 can call the plug-in to provide services for the user.
[0095] In summary, according to the embodiments of the present disclosure, the plug-in can be used to request the client device to perform the target function, which is conducive to expanding the use function and service capability of the digital assistant.
[0096] FIG. 5 shows a flowchart of a process 500 for plug-in processing according to some embodiments of the present disclosure. The process 500 can be implemented at the server device 110.
[0097] At block 510, in response to receiving the interaction information of the user from the client device, it is determined based on the interaction information that a call to a predetermined plug-in is requested, the predetermined plug-in being configured to request execution of at least one function at the client device.
[0098] At block 520, based on the interaction information, a call parameter of a target function in the at least one function and a target communication connection mode with the client device are determined.
[0099] At block 530, based on the target communication connection mode, a communication connection with the client device is established.
[0100] At block 540, via the communication connection, the client device is requested to perform the target function using the call parameter.
[0101] In some embodiments, requesting the client device to perform the target function using the call parameter includes: sending an authorization request to the client device via the communication connection, the authorization request being used to request authorization for the client device to perform the target function; and in response to receiving the authorization for the client device to perform the target function, requesting the client device to perform the target function using the call parameter via the communication connection.
[0102] In some embodiments, the target communication connection mode indicates a long connection mode, and wherein requesting the client device to perform the target function using the call parameter includes: using the call parameter to perform one or more communications with the client device via a first communication connection corresponding to the long connection mode to complete the execution of the target function.
[0103] In some embodiments, the target communication connection mode indicates a short connection mode, and wherein establishing the communication connection with the client device comprises: based on the short connection mode, establishing a second communication connection between the service end device and the client device at least once; and wherein requesting the client device to perform the target function with the invocation parameter comprises: via each established second communication connection, performing one communication with the client device with the invocation parameter to complete the performance of the target function.
[0104] In some embodiments, each message communicated between the service end device and the client device via the second communication connection comprises an identifier, the identifier being used to identify the current performance of the target function.
[0105] In some embodiments, the process 500 further comprises: receiving a performance result of the target function from the client device; and sending a response to the interaction information to the client device, the response indicating at least the performance result of the target function.
[0106] FIG. 6 shows a flowchart of a process 600 for question and answer processing, according to some embodiments of the present disclosure. The process 600 can be implemented at the client device 120.
[0107] At block 610, in response to receiving the interaction information of the user, sending the interaction information to the service end device.
[0108] At block 620, establishing a corresponding communication connection with the service end device based on the target communication connection mode.
[0109] At block 630, receiving an invocation parameter of the target function at the client device from the service end device via the communication connection.
[0110] At block 640, performing the target function based on the invocation parameter.
[0111] In some embodiments, performing the target function based on the invocation parameter comprises: determining whether there is an authorization for the client device to perform the target function; and if there is the authorization, performing the target function based on the invocation parameter.
[0112] In some embodiments, the process 600 further comprises: receiving an authorization request from the service end device via the communication connection, the authorization request being used to request an authorization for the client device to perform the target function; and in response to receiving the authorization operation for the client device to perform the target function, sending an authorization for the client device to perform the target function to the service end device.
[0113] In some embodiments, the target communication connection mode indicates a long connection mode, and wherein receiving the invocation parameter of the target function at the client device from the service end device comprises: via a first communication connection corresponding to the long connection mode, performing one or more communications with the service end device to receive the invocation parameter from the service end device.
[0114] In some embodiments, the target communication connection mode indicates a short connection mode, and wherein establishing, with the service-end device, a corresponding communication connection based on the target communication connection mode comprises: based on the short connection mode, establishing a second communication connection between the client-end device and the service-end device at least once; and wherein receiving, from the service-end device, the invocation parameter of the target function at the client-end device comprises: performing, with the service-end device, a communication via each established second communication connection to receive the invocation parameter from the service-end device.
[0115] In some embodiments, an identifier is contained in each message communicated between the service-end device and the client-end device via the second communication connection, the identifier being used to identify the current execution of the target function.
[0116] In some embodiments, the process 600 further comprises: feeding back, to the service-end device, an execution result of the target function; and presenting, in response to receiving, from the service-end device, a response to the interaction information, the response indicating at least the execution result of the target function.
[0117] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above method or process. FIG. 7 shows an exemplary structural block diagram of an apparatus 700 for plug-in processing, according to some embodiments of the present disclosure. The apparatus 700 can be implemented as or included in the service-end device 110. Various modules / components in the apparatus 700 can be implemented by hardware, software, firmware, or any combination thereof.
[0118] As shown in FIG. 7, the apparatus 700 includes an interaction information receiving module 710, a communication mode determining module 720, a first connection establishing module 730, and a target function requesting module 740. The interaction information receiving module 710 is configured to, in response to receiving interaction information of a user from a client-end device, determine, based on the interaction information, an invocation of a predetermined plug-in configured to request execution of at least one function at the client-end device. The communication mode determining module 720 is configured to determine, based on the interaction information, an invocation parameter of a target function in the at least one function and a target communication connection mode with the client-end device. The first connection establishing module 730 is configured to establish, based on the target communication connection mode, a communication connection with the client-end device. The target function requesting module 740 is configured to request, via the communication connection, the client-end device to execute the target function with the invocation parameter.
[0119] In some embodiments, the target function requesting module 740 is further configured to: send, to the client-end device via the communication connection, an authorization request for requesting authorization of the client-end device to execute the target function; and in response to receiving the authorization of the client-end device to execute the target function, request, via the communication connection, the client-end device to execute the target function with the invocation parameter.
[0120] In some embodiments, the target communication connection mode indicates a long connection mode, and the target function request module 740 is further configured to perform one or more communications with the client device via the first communication connection corresponding to the long connection mode, using the invocation parameter, to complete the execution of the target function.
[0121] In some embodiments, the target communication connection mode indicates a short connection mode, and the first connection establishment module 730 is further configured to establish a second communication connection between the server device and the client device at least once based on the short connection mode, and the target function request module 740 is further configured to perform one communication with the client device via each established second communication connection, using the invocation parameter, to complete the execution of the target function.
[0122] In some embodiments, an identifier is contained in each message communicated between the server device and the client device via the second communication connection, the identifier being used to identify the current execution of the target function.
[0123] In some embodiments, the apparatus 700 further includes an execution result receiving module configured to receive an execution result of the target function from the client device, and send a response to the client device for the interaction information, the response at least indicating the execution result of the target function.
[0124] It should be understood that one or more steps in the above methods can be performed by a suitable electronic device or combination of electronic devices. Such an electronic device or combination of electronic devices may, for example, include the server device 110 in FIG. 1.
[0125] Embodiments of the present disclosure also provide a corresponding apparatus for implementing the above methods or processes. FIG. 8 shows an exemplary structural block diagram of an apparatus 800 for plug-in processing, according to some embodiments of the present disclosure. The apparatus 800 can be implemented as or included in the client device 120. The various modules / components in the apparatus 800 can be implemented by hardware, software, firmware, or any combination thereof.
[0126] As shown in FIG. 8, the apparatus 800 includes an interaction information sending module 810, a second connection establishment module 820, an invocation parameter receiving module 830, and a target function execution module 840. The interaction information sending module 810 is configured to send interaction information to the server device in response to receiving the interaction information of the user. The second connection establishment module 820 is configured to establish a corresponding communication connection with the server device based on a target communication connection mode. The invocation parameter receiving module 830 is configured to receive an invocation parameter for a target function at the client device from the server device via the communication connection. The target function execution module 840 is configured to execute the target function based on the invocation parameter.
[0127] In some embodiments, the target function execution module 840 is further configured to: determine whether there is an authorization for the client device to perform the target function; and if there is the authorization, perform the target function based on the invocation parameter.
[0128] In some embodiments, the apparatus 800 further comprises: an authorization request receiving module configured to receive, via the communication connection, an authorization request from the server device, the authorization request being used to request an authorization for the client device to perform the target function; and in response to receiving the authorization operation for the client device to perform the target function, send, to the server device, an authorization for the client device to perform the target function.
[0129] In some embodiments, the target communication connection mode indicates a long connection mode, and the invocation parameter receiving module 830 is further configured to: perform one or more communications with the server device via a first communication connection corresponding to the long connection mode, to receive the invocation parameter from the server device.
[0130] In some embodiments, the target communication connection mode indicates a short connection mode, and the second connection establishing module 820 is further configured to: based on the short connection mode, establish the second communication connection between the client device and the server device at least once; and the invocation parameter receiving module 830 is further configured to: perform a communication with the server device via each established second communication connection, to receive the invocation parameter from the server device.
[0131] In some embodiments, an identifier is contained in each message communicated between the server device and the client device via the second communication connection, the identifier being used to identify the current execution of the target function.
[0132] In some embodiments, the apparatus 800 further comprises: an execution result feedback module configured to feed back, to the server device, an execution result of the target function; and in response to receiving a response for the interaction information from the server device, present the response, the response at least indicating the execution result of the target function.
[0133] It should be understood that one or more steps in the above methods can be performed by a suitable electronic device or combination of electronic devices. Such an electronic device or combination of electronic devices may, for example, include the client device 120 in FIG. 1.
[0134] The units and / or modules included in the apparatus 700 and the apparatus 800 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units and / or modules can be implemented using software and / or firmware, e.g., machine-executable instructions stored on a storage medium. In addition to or alternatively, some or all of the units and / or modules in the apparatus 700 and the apparatus 800 can be implemented at least partially by one or more hardware logic components. As an example and not by way of limitation, example types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.
[0135] FIG. 9 illustrates a block diagram of an electronic device 900 in which one or more embodiments of the disclosure can be implemented. It should be understood that the electronic device 900 illustrated in FIG. 9 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 900 illustrated in FIG. 9 can be used to implement the server device 110 or the client device 120 of FIG. 1, and the electronic device 900 can also be used to implement the apparatus 700 of FIG. 7 or the apparatus 800 of FIG. 8.
[0136] As shown in FIG. 9, the electronic device 900 is in the form of a general electronic device. Components of the electronic device 900 can include, but are not limited to, one or more processors or processing units 910, a memory 920, a storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. The processing unit 910 can be a real or virtual processor and capable of executing various processing according to programs stored in the memory 920. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device 900.
[0137] The electronic device 900 typically includes a number of computer storage media. Such media can be any available media that is accessible by the electronic device 900 and includes both volatile and non-volatile media, removable and non-removable media. The memory 920 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 930 can be a removable or non-removable media and can include machine-readable media, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data and that can be accessed by the electronic device 900.
[0138] The electronic device 900 can further include additional detachable / non-detachable, volatile / non-volatile storage media. Although not shown in FIG. 9, a disk drive for reading from or writing to a detachable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a detachable, non-volatile optical disk (e.g., a CD-ROM) can be provided. In these cases, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 920 can include a computer program product 925 having one or more program modules configured to carry out the various methods or acts of the various embodiments of the present disclosure.
[0139] The communication unit 940 enables communication with other electronic devices through communication media. Additionally, the functionality of the components of the electronic device 900 can be implemented in a single computing cluster or a plurality of computer machines capable of communicating over a communication connection. As such, the electronic device 900 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network nodes in the networking environment.
[0140] The input device 950 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 960 can be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 900 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc., one or more devices that enable a user to interact with the electronic device 900, or any devices (e.g., a network card, a modem, etc.) that enable the electronic device 900 to communicate with one or more other electronic devices, through the communication unit 940, as needed. Such communication can be carried out via an input / output (I / O) interface (not shown).
[0141] According to an example implementation of the present disclosure, a computer readable storage medium having computer executable instructions stored thereon is provided, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.
[0142] Various aspects of the disclosure are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It can be evident, however, that such aspect(s) can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the description.
[0143] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0144] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some, as newer, implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0146] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.
Claims
1. A method for plug-in processing, applied to a server device, comprising: in response to receiving interaction information of a user from a client device, determining, based on the interaction information, a call to a predetermined plug-in configured to request execution of at least one function at the client device; determining, based on the interaction information, a call parameter to a target function in the at least one function and a target communication connection mode with the client device; establishing a communication connection with the client device based on the target communication connection mode; and requesting the client device to execute the target function with the call parameter via the communication connection. 2.The method of claim 1, wherein requesting the client device to execute the target function with the call parameter comprises: sending an authorization request to the client device via the communication connection, the authorization request being used to request authorization for the client device to execute the target function; and in response to receiving the authorization for the client device to execute the target function, requesting the client device to execute the target function with the call parameter via the communication connection. 3.The method of claim 1, wherein the target communication connection mode indicates a long connection mode, and wherein requesting the client device to execute the target function with the call parameter comprises: performing one or more communications with the client device with the call parameter via a first communication connection corresponding to the long connection mode to complete execution of the target function. 4.The method of claim 1, wherein the target communication connection mode indicates a short connection mode, and wherein establishing a communication connection with the client device comprises: based on the short connection mode, establishing a second communication connection between the server device and the client device at least once; and wherein requesting the client device to execute the target function with the call parameter comprises: performing a communication with the client device with the call parameter via each established second communication connection to complete execution of the target function. 5.The method of claim 4, wherein an identifier is contained in each message communicated between the server device and the client device via the second communication connection, the identifier being used to identify the current execution of the target function. 6.The method of claim 1, further comprising: receiving an execution result of the target function from the client device; and sending a response to the interaction information to the client device, the response indicating at least the execution result of the target function. 7.A method for plug-in processing, applied to a client device, comprising: in response to receiving interaction information of a user, sending the interaction information to a server device; establishing a corresponding communication connection with the server device based on a target communication connection mode; receiving a call parameter to a target function at the client device from the server device via the communication connection; and executing the target function based on the call parameter. 8. The method of claim 7, wherein performing the target function based on the invocation parameter comprises: determining whether there is an authorization for the client device to perform the target function; and if there is the authorization, performing the target function based on the invocation parameter.
9. The method of claim 7, further comprising: receiving, from the service end device via the communication connection, an authorization request for requesting an authorization for the client device to perform the target function; and in response to receiving an authorization operation for the client device to perform the target function, sending, to the service end device, an authorization for the client device to perform the target function.
10. The method of claim 7, the target communication connection mode indicates a long connection mode, and wherein receiving, from the service end device, an invocation parameter for a target function at the client device comprises: performing one or more communications with the service end device via a first communication connection corresponding to the long connection mode to receive the invocation parameter from the service end device.
11. The method of claim 7, the target communication connection mode indicates a short connection mode, and wherein establishing a corresponding communication connection with the service end device based on the target communication connection mode comprises: based on the short connection mode, establishing a second communication connection between the client device and the service end device at least once; and wherein receiving, from the service end device, an invocation parameter for a target function at the client device comprises: performing a communication with the service end device via each established second communication connection to receive the invocation parameter from the service end device.
12. The method of claim 11, wherein an identifier is included in each message communicated between the service end device and the client device via the second communication connection, the identifier identifying a current execution of the target function.
13. The method of claim 7, further comprising: feeding back, to the service end device, an execution result of the target function; and in response to receiving, from the service end device, a response for the interaction information, presenting the response indicating at least the execution result of the target function.
14. An apparatus for plug-in processing, comprising: an interaction information receiving module configured to, in response to receiving interaction information of a user from a client device, determine an invocation for a predetermined plug-in based on the interaction information, the predetermined plug-in configured to request execution of at least one function at the client device; a communication mode determining module configured to determine, based on the interaction information, an invocation parameter for a target function of the at least one function and a target communication connection mode with the client device; a first connection establishing module configured to establish a communication connection with the client device based on the target communication connection mode; and a target function requesting module configured to request, via the communication connection, the client device to perform the target function with the invocation parameter.
15. An apparatus for plug-in processing, comprising: The interaction information sending module is configured to send, in response to receiving the interaction information of the user, the interaction information to the server device; The second connection establishing module is configured to establish a corresponding communication connection with the server device based on a target communication connection mode; The calling parameter receiving module is configured to receive, via the communication connection, a calling parameter of a target function at the client device from the server device; And The target function executing module is configured to execute the target function based on the calling parameter. 16.An electronic device, comprising: at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1-6 or 7-13. 17.A computer-readable storage medium having computer-executable instructions stored thereon that are executable by a processor to implement the method according to any one of claims 1-6 or 7-13. 18.A computer-executable instruction product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1-6 or 7-13.
Citation Information
Patent Citations
Plug-in calling method and device
CN108989437A
Test method and device and computer readable storage medium
CN111338865A
Large model plug-in calling method and device, equipment and medium
CN117112065A
Plug-in UI rendering method and device, electronic equipment and storage medium
CN117806758A
Large model plug-in calling method and device, electronic equipment and storage medium
CN118132164A