Call interaction method, system, device, equipment, storage medium and program product

Abstract

The invention relates to a call interaction method, system and device, equipment, a storage medium and a program product, and relates to the field of wireless communication technologies and terminals. The method comprises the following steps: in response to an artificial intelligence function starting instruction for a current call, sending a model capability request to an opposite end of the current call through a data transmission channel in the current call based on a first enhanced call module; the model capability request is used for downloading an end-side model by the opposite end based on a second enhanced call module and feeding back corresponding model capability information; receiving model capability information of the end side model sent by the opposite end through the data transmission channel; obtaining model input parameters determined according to the model capability information; receiving a model generation result sent by the opposite end through the data transmission channel; and the model generation result is obtained by reasoning the end side model according to the model input parameters. By adopting the method, the low-performance equipment can be ensured to obtain real-time and stable end-side AI service in a call scene.

Description

Technical Field

[0001] This application relates to the field of wireless communication and terminal technology, and in particular to a call interaction method, system, apparatus, network device, computer-readable storage medium, and computer program product. Background Technology

[0002] In the fields of wireless communication and terminal technology, with the development of artificial intelligence, edge models have gradually become a key technical support for improving user experience in mobile calling scenarios. Traditionally, in mobile calling scenarios, the complete edge model is stored directly on the mobile terminal to provide users with a more intelligent and convenient interactive experience.

[0003] However, the inference operations of edge models require real-time access to the chip computing power and memory resources of the terminal device they reside on. In the mobile terminal market, there are still many low-performance devices with low chip computing power and small memory. Such low-performance devices cannot meet the inference requirements of edge models and it is difficult to obtain real-time and stable edge AI (Artificial Intelligence) services in call scenarios using edge models. Summary of the Invention

[0004] Therefore, it is necessary to provide a call interaction method, system, device, network equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0005] Firstly, this application provides a call interaction method, the method comprising:

[0006] In response to an instruction to enable AI functionality for the current call, a model capability request is sent to the peer in the current call through the data transmission channel of the current call, based on the first enhanced call module; the model capability request is used by the peer to download the end-side model based on the second enhanced call module and return the corresponding model capability information.

[0007] The system receives model capability information of the end-side model sent by the peer through the data transmission channel.

[0008] Obtain the model input parameters determined based on the model capability information;

[0009] The model generation result sent by the peer is received through the data transmission channel; the model generation result is obtained by the end-side model based on the model input parameters.

[0010] In one embodiment, receiving the model capability information of the end-side model sent by the peer through the data transmission channel includes:

[0011] Based on the first chip module, the model capability information sent by the second chip module of the peer end is received through the data transmission channel, and the model capability information is sent to the first enhanced call module.

[0012] The model capability information is used by the first enhanced call module to determine the model input parameters and send the model input parameters to the first chip module.

[0013] In one embodiment, the step of responding to an AI function activation command for the current call, based on a first enhanced call module, sending a model capability request to the other end of the current call via the data transmission channel in the current call, includes:

[0014] In response to the instruction to enable the artificial intelligence function for the current call, a model capability request is sent to the first chip module based on the first enhanced call module;

[0015] Based on the first chip module, the model capability request is forwarded to the second chip module of the other end through the data transmission channel of the current call.

[0016] In one embodiment, the model capability request is further used for the second chip module to send a first notification to the second enhanced call module; the first notification is used for the second enhanced call module to switch to an enhanced call mode based on the end-side model and send a model download request to the model platform; the model download request is used for the model platform to send an end-side model list to the second enhanced call module.

[0017] The second enhanced call module is configured to respond to user selection information for the terminal model list, download the corresponding terminal model from the model platform, and send a second notification to the second chip module; the second chip module is configured to determine, based on the second notification, that the terminal model is in an enabled state and obtain the model capability information of the terminal model, and send the model capability information to the first chip module through the data transmission channel.

[0018] In one embodiment, the method further includes:

[0019] Based on the first chip module, the model input parameters are sent to the second chip module at the other end through the data transmission channel; the second chip module is used to send the model input parameters to the end-side model through the second enhanced call module; the end-side model is used to infer the model generation result based on the model input parameters, and send the model generation result to the second chip module through the second enhanced call module.

[0020] Based on the first chip module, the model generation result sent by the second chip module of the peer end is received through the data transmission channel, and the model generation result is sent to the first enhanced call module.

[0021] Secondly, this application also provides a call interaction method, the method comprising:

[0022] In response to a model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module and the corresponding model capability information is fed back; the model capability request is obtained by the peer in response to an AI function activation command for the current call and based on the first enhanced call module;

[0023] The model capability information is sent to the peer through the data transmission channel in the current call; the model capability information is used by the peer to determine the model input parameters.

[0024] The model input parameters sent by the peer are received through the data transmission channel;

[0025] Obtain the model generation result obtained by the edge model based on the model input parameters;

[0026] The model generation result is sent to the peer through the data transmission channel.

[0027] Thirdly, this application also provides a call interaction system, including: a first end and a second end of the current call;

[0028] The first end is configured to respond to an AI function activation command for the current call, and based on a first enhanced call module, send a model capability request to the second end through the data transmission channel of the current call; the model capability request is used by the second end to download a terminal-side model based on a second enhanced call module and return the corresponding model capability information; receive the model capability information of the terminal-side model sent by the second end through the data transmission channel; obtain model input parameters determined based on the model capability information; and receive the model generation result sent by the second end through the data transmission channel; the model generation result is obtained by the terminal-side model based on the model input parameters.

[0029] The second terminal is configured to respond to the model capability request sent by the first terminal, download the terminal-side model based on the second enhanced call module and feed back the corresponding model capability information; send the model capability information to the first terminal through the data transmission channel; receive the model input parameters sent by the first terminal through the data transmission channel; obtain the model generation result obtained by the terminal-side model based on the model input parameters; and send the model generation result to the first terminal through the data transmission channel.

[0030] Fourthly, this application also provides a call interaction device, the device comprising:

[0031] The first response module is used to respond to the AI ​​function activation command for the current call, and based on the first enhanced call module, send a model capability request to the other end of the current call through the data transmission channel in the current call; the model capability request is used by the other end to download the terminal model based on the second enhanced call module and feed back the corresponding model capability information.

[0032] The first receiving module is used to receive the model capability information of the end-side model sent by the peer through the data transmission channel;

[0033] The first acquisition module is used to acquire model input parameters determined based on the model capability information;

[0034] The second receiving module is used to receive the model generation result sent by the peer through the data transmission channel; the model generation result is obtained by the end-side model based on the model input parameters.

[0035] Fifthly, this application also provides a call interaction device, the device comprising:

[0036] The second response module is used to respond to the model capability request sent by the peer in the current call, download the terminal-side model based on the second enhanced call module, and feed back the corresponding model capability information; the model capability request is obtained by the peer in response to the artificial intelligence function activation instruction for the current call and based on the first enhanced call module.

[0037] The first sending module is used to send the model capability information to the peer through the data transmission channel in the current call; the model capability information is used by the peer to determine the model input parameters.

[0038] The third receiving module is used to receive the model input parameters sent by the peer through the data transmission channel;

[0039] The second acquisition module is used to acquire the model generation result obtained by the end-side model based on the model input parameters;

[0040] The second sending module is used to send the model generation result to the peer through the data transmission channel.

[0041] Sixthly, this application also provides a network device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0042] In response to an instruction to activate AI functionality for the current call, a model capability request is sent to the peer in the current call via the data transmission channel of the first enhanced call module. This model capability request is used by the peer to download a terminal-side model based on the second enhanced call module and provide corresponding model capability information. The system receives the model capability information of the terminal-side model sent by the peer via the data transmission channel; obtains model input parameters determined based on the model capability information; and receives the model generation result sent by the peer via the data transmission channel. The model generation result is obtained by the terminal-side model based on the model input parameters.

[0043] Alternatively, in response to a model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module, and the corresponding model capability information is fed back. The model capability request is obtained by the peer in response to an AI function activation command for the current call and based on the first enhanced call module. The model capability information is sent to the peer through the data transmission channel in the current call. The model capability information is used by the peer to determine model input parameters. The peer receives the model input parameters sent by the peer through the data transmission channel. The model generation result obtained by the terminal-side model based on the model input parameters is obtained. The model generation result is sent to the peer through the data transmission channel.

[0044] Seventhly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0045] In response to an instruction to activate AI functionality for the current call, a model capability request is sent to the peer in the current call via the data transmission channel of the first enhanced call module. This model capability request is used by the peer to download a terminal-side model based on the second enhanced call module and provide corresponding model capability information. The system receives the model capability information of the terminal-side model sent by the peer via the data transmission channel; obtains model input parameters determined based on the model capability information; and receives the model generation result sent by the peer via the data transmission channel. The model generation result is obtained by the terminal-side model based on the model input parameters.

[0046] Alternatively, in response to a model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module, and the corresponding model capability information is fed back. The model capability request is obtained by the peer in response to an AI function activation command for the current call and based on the first enhanced call module. The model capability information is sent to the peer through the data transmission channel in the current call. The model capability information is used by the peer to determine model input parameters. The peer receives the model input parameters sent by the peer through the data transmission channel. The model generation result obtained by the terminal-side model based on the model input parameters is obtained. The model generation result is sent to the peer through the data transmission channel.

[0047] Eighthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0048] In response to an instruction to activate AI functionality for the current call, a model capability request is sent to the peer in the current call via the data transmission channel of the first enhanced call module. This model capability request is used by the peer to download a terminal-side model based on the second enhanced call module and provide corresponding model capability information. The system receives the model capability information of the terminal-side model sent by the peer via the data transmission channel; obtains model input parameters determined based on the model capability information; and receives the model generation result sent by the peer via the data transmission channel. The model generation result is obtained by the terminal-side model based on the model input parameters.

[0049] Alternatively, in response to a model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module, and the corresponding model capability information is fed back. The model capability request is obtained by the peer in response to an AI function activation command for the current call and based on the first enhanced call module. The model capability information is sent to the peer through the data transmission channel in the current call. The model capability information is used by the peer to determine model input parameters. The peer receives the model input parameters sent by the peer through the data transmission channel. The model generation result obtained by the terminal-side model based on the model input parameters is obtained. The model generation result is sent to the peer through the data transmission channel.

[0050] The aforementioned call interaction method, system, device, network equipment, computer-readable storage medium, and computer program product, in response to an instruction to activate the artificial intelligence function for the current call, firstly, based on a first enhanced call module, sends a model capability request to the other end of the current call through the data transmission channel in the current call. The model capability request is used by the other end to download the end-side model based on a second enhanced call module and provide corresponding model capability information. Then, the system receives the end-side model capability information sent by the other end through the data transmission channel. Next, it obtains the model input parameters determined based on the model capability information. Finally, it receives the model generation result sent by the other end through the data transmission channel. The model generation result is obtained by the end-side model based on the model input parameters. Compared to traditional methods, this solution, in response to an AI function activation command for the current call, initiates a model capability request to the other end of the call. This dynamically migrates the inference task corresponding to the AI ​​function activation command to the other end, which has stronger computing power. The inference task is then executed by the edge model on the other end, and the corresponding model generation results are obtained from the other end. Based on the collaborative capabilities between the two ends of the current call, this solution addresses the issue of the low-performance end being unable to support the edge model. Without increasing hardware costs, it provides edge model inference capabilities to low-performance devices in call scenarios, enabling them to obtain real-time and stable edge AI services. This significantly improves the interactive experience for users of low-performance devices in call scenarios, thus democratizing edge AI services. Furthermore, this solution transmits model capability requests, input parameters, and inference results between the two ends of the current call through the data transmission channel within the current call. This eliminates the need to establish additional connections or consume additional network bandwidth, reducing communication latency and complexity and ensuring the real-time response of AI services. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is an application environment diagram of a call interaction method in one embodiment;

[0053] Figure 2 This is a flowchart illustrating a call interaction method in one embodiment;

[0054] Figure 3 This is a flowchart illustrating the call interaction method in another embodiment;

[0055] Figure 4This is a zero-layer architecture diagram of a call interaction system in one embodiment;

[0056] Figure 5 Here is a zero-level timing flowchart of a call interaction system in one embodiment;

[0057] Figure 6 Here is a single-layer UML class diagram of a call interaction system in one embodiment;

[0058] Figure 7 This is a structural block diagram of a call interaction device in one embodiment;

[0059] Figure 8 This is a structural block diagram of the call interaction device in another embodiment;

[0060] Figure 9 This is a diagram of the internal structure of a network device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various objects, but these objects are not limited by these terms. These terms are only used to distinguish the first object from the second object. The term "comprising" and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion.

[0063] The call interaction method provided in this application embodiment can be applied to, for example... Figure 1 The application environment shown illustrates this. In this scenario, the first end of the call communicates with the second end via a network. The first or second end of the call can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses.

[0064] In one exemplary embodiment, such as Figure 2 As shown, a call interaction method is provided, which can be applied to... Figure 1 The first end in the call, where the first end is the local end of the current call, can include the following steps:

[0065] Step 201: In response to the instruction to enable the artificial intelligence function for the current call, based on the first enhanced call module, a model capability request is sent to the other end of the current call through the data transmission channel in the current call; the model capability request is used by the other end to download the end-side model based on the second enhanced call module and return the corresponding model capability information.

[0066] Artificial intelligence (AI) can refer to the intelligence exhibited by systems created by humans. It can perform complex tasks that typically require human intelligence, such as learning, reasoning, problem-solving, language understanding, pattern recognition, and environmental perception, through machines or software. The AI ​​activation command is used to request the activation of AI functions. The first enhanced call module can refer to the module on the local end of the current call used to enhance the call; the second enhanced call module can refer to the module on the remote end of the current call used to enhance the call. Enhanced call can refer to a series of services that enhance the user experience for individuals and enterprises by improving the system and innovating services based on the operator's IMS (Internet Protocol Multimedia Subsystem) calls. The edge model is obtained by deploying machine learning models on terminal devices, enabling the model to perform intelligent calculations at the source of data generation and processing. Edge models are characterized by high efficiency, multifunctionality, and wide applicability.

[0067] For example, the local end of the current call is a low-performance mobile phone with low chip computing power and small memory, which cannot meet the inference computing requirements of the edge model. The other end of the current call is a high-performance mobile phone with sufficient chip computing power and memory to meet the inference computing requirements of the edge model. In response to the user's input or click operation on the local end of the current call for AI functions, the local end receives an AI activation command for the current call. Based on the first enhanced call module included in the local end, it sends a model capability request to the other end of the current call through the data transmission channel in the current call. The other end responds to the model capability request, downloads the edge model based on the second enhanced call module included in the other end, and feeds back the corresponding model capability information to the local end.

[0068] Step 202: Receive the model capability information of the end-side model sent by the other end through the data transmission channel.

[0069] The model capability information of the end-side model sent by the peer is used to indicate the capability status of the peer's end-side model. The model capability information of the end-side model sent by the peer may include, but is not limited to: what models the peer currently has available and the types of these available models (such as large language models, text-to-graph models, and graphics processing models).

[0070] For example, the local device, based on the first chip module, receives model capability information of the end-side model transparently transmitted by the second chip module of the other end through the data transmission channel in the current call, and then transmits the model capability information received by the first chip module to the first enhanced call module. The first enhanced call module, in response to receiving the model capability information, activates the AI ​​function of the local device.

[0071] Step 203: Obtain the model input parameters determined based on the model capability information.

[0072] Among them, model input parameters can refer to parameter information used to input to the end-side model of the other end.

[0073] For example, this terminal determines the model input parameters based on the model capability information through its first enhanced call module.

[0074] Step 204: Receive the model generation result sent by the peer through the data transmission channel; the model generation result is obtained by the end-side model based on the model input parameters.

[0075] Among them, the model generation result refers to the result obtained by the terminal model based on the model input parameters, and the model generation result can represent the execution result of the inference task corresponding to the AI ​​function of the current terminal in the call.

[0076] For example, the on-device model is a large language model; the model input parameters include the user's prompt word `userPrompt`, system prompt word `systemPrompt`, and auxiliary prompt word `assistantPrompt`, which are input by the user on the local device during the current call, indicating the inference requirements corresponding to the AI ​​function the user wants to activate on the local device. The large language model generates the corresponding model generation result based on the above model input parameters. The model generation result includes the intermediate generation result `progressString` and the final generation result `normalOutput`.

[0077] In this embodiment, the model generation results (such as user questions) can be displayed in text form on the visual interface of the mini-program. In the above call interaction method, in response to the instruction to enable the artificial intelligence function for the current call, a model capability request is first sent to the other end of the current call through the data transmission channel in the current call, based on the first enhanced call module. The model capability request is used by the other end to download the terminal-side model based on the second enhanced call module and return the corresponding model capability information. Then, the model capability information of the terminal-side model sent by the other end is received through the data transmission channel. Next, the model input parameters determined according to the model capability information are obtained. Finally, the model generation result sent by the other end is received through the data transmission channel. The model generation result is obtained by the terminal-side model based on the model input parameters. Compared to traditional methods, this solution, in response to an AI function activation command for the current call, initiates a model capability request to the other end of the call. This dynamically migrates the inference task corresponding to the AI ​​function activation command to the other end, which has stronger computing power. The inference task is then executed by the edge model on the other end, and the corresponding model generation results are obtained from the other end. Based on the collaborative capabilities between the two ends of the current call, this solution addresses the issue of the low-performance end being unable to support the edge model. Without increasing hardware costs, it provides edge model inference capabilities to low-performance devices in call scenarios, enabling them to obtain real-time and stable edge AI services. This significantly improves the interactive experience for users of low-performance devices in call scenarios, thus democratizing edge AI services. Furthermore, this solution transmits model capability requests, input parameters, and inference results between the two ends of the current call through the data transmission channel within the current call. This eliminates the need to establish additional connections or consume additional network bandwidth, reducing communication latency and complexity and ensuring the real-time response of AI services.

[0078] In an exemplary embodiment, step 202, receiving the model capability information of the end-side model sent by the peer through the data transmission channel, may include:

[0079] Based on the first chip module, the system receives model capability information sent by the second chip module at the other end through the data transmission channel, and sends the model capability information to the first enhanced communication module; wherein, the model capability information is used by the first enhanced communication module to determine the model input parameters and send the model input parameters to the first chip module.

[0080] In this context, the first chip module can refer to the chip module on the local end of the current call. The second chip module can refer to the chip module on the remote end of the current call. The data transmission channel can refer to the DC (Data Channel) channel between the first and second chip modules, which is the channel in the IMS network used to transmit additional interactive data (such as model capability requests, model download requests, model input parameters, and model generation results) in addition to the basic voice / video media stream channels. Model input parameters can refer to the parameter information used to input to the remote end's end-side model.

[0081] For example, the local end, based on the first chip module, receives model capability information sent by the remote end's second chip module through the DC channel, and transparently transmits the model capability information to the first enhanced communication module; the local end, based on the first enhanced communication module, determines the model input parameters according to the model capability information and sends the model input parameters to the first chip module; the local end, based on the first chip module, sends the model input parameters to the second chip module through the DC channel. In response to receiving the model input parameters, the remote end's second chip module transmits the model input parameters to the second enhanced communication module; the second enhanced communication module, in response to receiving the model input parameters, sends the model input parameters to the local model.

[0082] In an exemplary embodiment, step 201, in response to an instruction to enable AI functionality for the current call, sending a model capability request to the other end of the current call via the data transmission channel in the current call, based on the first enhanced call module, may include:

[0083] In response to the command to activate the artificial intelligence function for the current call, a model capability request is sent to the first chip module based on the first enhanced call module; based on the first chip module, the model capability request is forwarded to the second chip module of the other end through the data transmission channel of the current call.

[0084] For example, data interaction between the local and remote ends of the current call is achieved through the first chip module, the second chip module, and the DC channel between the first and second chip modules. In response to an instruction to enable artificial intelligence functions for the current call, the local end generates a model capability request based on the first enhanced call module and sends the model capability request to the first chip module. Based on the first chip module, the local end forwards the model capability request to the second chip module of the remote end through the DC channel.

[0085] In one embodiment, the model capability request is further used by the second chip module to send a first notification to the second enhanced call module; the first notification is used by the second enhanced call module to switch to the enhanced call mode based on the edge model and send a model download request to the model platform; the model download request is used by the model platform to send an edge model list to the second enhanced call module; wherein, the second enhanced call module is used to respond to the user selection information for the edge model list, download the corresponding edge model from the model platform, and send a second notification to the second chip module; the second chip module is used to determine that the edge model is in the enabled state and obtain the model capability information of the edge model according to the second notification, and send the model capability information to the first chip module through the data transmission channel.

[0086] The first notification can refer to a notification instruction used to instruct the second enhanced call module to enable its end-side model capability, causing the second enhanced call module to switch to the end-side model-based enhanced call mode and send a model download request to the model platform. The second notification can refer to a notification instruction used to inform the local end of the current call that the end-side model capability of the second enhanced call module is now enabled. The user selection information for the end-side model list can refer to the information generated when the user selects an end-side model from the end-side model list through the local or remote end of the current call.

[0087] For example, in response to the received model capability request, the peer of the current call sends a first notification to the second enhanced call module based on the second chip module; in response to the received first notification, the second enhanced call module switches its working mode to the enhanced call mode based on the edge model and sends a model download request to the model platform; in response to the received model download request, the model platform sends an edge model list to the second enhanced call module; in response to the user selection information for the edge model list, the second enhanced call module downloads the corresponding edge model from the model platform and sends a second notification to the second chip module; in response to the received second notification, the second chip module determines that the edge model is in the enabled state and obtains the model capability information of the edge model, and transmits the model capability information to the first chip module through the data transmission channel.

[0088] In one exemplary embodiment, the method of this application embodiment may further include an interactive reasoning step, which may include:

[0089] Based on the first chip module, model input parameters are sent to the second chip module at the other end through a data transmission channel; the second chip module is used to send model input parameters to the end-side model through the second enhanced communication module; the end-side model is used to infer the model generation result based on the model input parameters and send the model generation result to the second chip module through the second enhanced communication module; based on the first chip module, the model generation result sent by the second chip module at the other end is received through a data transmission channel and sent to the first enhanced communication module.

[0090] For example, the local end of the current call, based on the first chip module, sends model input parameters to the second chip module of the other end through the DC channel; the second chip module, in response to receiving the model input parameters, forwards the received model input parameters to the second enhanced call module, and then forwards the model input parameters to the end-side model through the second enhanced call module; the end-side model, in response to receiving the model input parameters, infers the corresponding model generation result based on the model input parameters and sends the model generation result to the second enhanced call module; the second enhanced call module, in response to receiving the model generation result, returns the model generation result to the second chip module, and then transmits the model generation result to the first chip module through the second chip module; the first chip module, in response to receiving the model generation result, transmits the model generation result to the first enhanced call module.

[0091] In one exemplary embodiment, such as Figure 3 As shown, a call interaction method is also provided, which can be applied to... Figure 1 The second end in the call, where the second end is the local end of the current call, can include the following steps:

[0092] Step 301: In response to the model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module and the corresponding model capability information is fed back; the model capability request is obtained by the peer in response to the artificial intelligence function activation instruction for the current call and based on the first enhanced call module.

[0093] For example, in response to an AI function activation command for the current call, the peer end of the current call sends a model capability request to the first chip module based on the first enhanced call module. Based on the first chip module, the model capability request is forwarded to the second chip module of the local end of the current call via the data transmission channel of the current call. In response to the model capability request sent by the peer end of the current call, the local end of the current call sends a first notification to the second enhanced call module based on the second chip module. The first notification is used for the second enhanced call module to switch to an enhanced call mode based on an edge-side model and send a model download request to the model platform. The model download request is used for the model platform to distribute an edge-side model list to the second enhanced call module. The second enhanced call module, in response to user selection information for the edge-side model list, downloads the corresponding edge-side model from the model platform and sends a second notification to the second chip module. The second chip module, based on the second notification, determines that the edge-side model is enabled and obtains the model capability information of the edge-side model.

[0094] Step 302: Send model capability information to the other end through the data transmission channel in the current call; the model capability information is used by the other end to determine the model input parameters.

[0095] For example, the local end of the current call, based on the second chip module, sends model capability information to the first chip module of the other end through the data transmission channel of the current call; in response to receiving the model capability information, the first chip module of the other end sends the model capability information to the first enhanced call module of the other end; the other end determines the model input parameters based on the model capability information through its first enhanced call module.

[0096] Step 303: Receive the model input parameters sent by the other end through the data transmission channel.

[0097] For example, the peer of the current call sends model input parameters to the first chip module of the peer based on the first enhanced call module; the first chip module is used to send model input parameters to the second chip module of the local end through the data transmission channel; the second chip module is used to send model input parameters to the end-side model through the second enhanced call module.

[0098] Step 304: Obtain the model generation result obtained by the end-side model based on the model input parameters.

[0099] For example, the local end of the current call, based on the terminal-side model, infers the model generation result according to the received model input parameters, and sends the model generation result to the second chip module through the second enhanced call module.

[0100] Step 305: Send the model generation results to the other end through the data transmission channel.

[0101] For example, the local end of the current call, based on the second chip module, sends the model generation result to the first chip module of the other end through the data transmission channel; the first chip module is used to send the model generation result to the first enhanced call module.

[0102] Currently, for low-performance mobile phones, due to their limited chip computing power and small memory, issues such as high inference latency, surged device power consumption, and poor application stability may occur when using edge-side models for inference operations. To address this, in an exemplary embodiment, a call interaction system is provided, the system including: Figure 1 The first and second ends of the current call are shown.

[0103] The first end, in response to an AI function activation command for the current call, sends a model capability request to the second end via the data transmission channel of the current call, based on the first enhanced call module. The model capability request is used by the second end to download the end-side model based on the second enhanced call module and provide corresponding model capability information. The second end receives the end-side model capability information sent by the second end via the data transmission channel; obtains the model input parameters determined based on the model capability information; and receives the model generation result sent by the second end via the data transmission channel. The model generation result is obtained by the end-side model through inference based on the model input parameters.

[0104] The second end is used to respond to the model capability request sent by the first end, download the terminal-side model based on the second enhanced call module and feed back the corresponding model capability information; send the model capability information to the first end through the data transmission channel; receive the model input parameters sent by the first end through the data transmission channel; obtain the model generation result obtained by the terminal-side model based on the model input parameters; and send the model generation result to the first end through the data transmission channel.

[0105] For example, such as Figure 4 As shown, the overall code architecture of the call interaction system is divided into three layers from top to bottom: the model platform layer, the APP (Application) layer, and the AIDL (Android Interface Definition Language) layer. The main functions of each layer are as follows:

[0106] Model Platform Layer: This layer mainly includes the model platform. After receiving the enhanced call SDK request, the model platform will distribute the corresponding model according to the request.

[0107] APP Layer: This layer includes the Enhanced Call SDK (Software Development Kit) module and the client-side model module. The Enhanced Call SDK module is responsible for communication with the upper-layer large model platform and the lower-layer AIDL layer's IMS DC AIDL (Android Interface Definition Language for Data Channels Based on IMS Network) module, receiving content generated by the client-side model and client-side generated content transmitted by the peer's IMS DC. The client-side model module is responsible for providing AI capabilities to the Enhanced Call SDK module. The Enhanced Call SDK module refers to both the first and second Enhanced Call modules.

[0108] AIDL layer: This layer mainly implements the communication function with the chip module's IMS DC Service (a data channel service based on the IMS network).

[0109] AIDL refers to a tool for implementing inter-process communication in Android applications.

[0110] For example, such as Figure 5 As shown, at the UML (Unified Modeling Language) level of the code design, the overall code architecture of the call interaction system is divided into a mini-program process module, a main process module, a client-side model module, a model platform download module, and an IMS DC (IMS network-based data channel) chip interface module. The main functions of each module are as follows:

[0111] The Mini Program process module refers to the Enhanced Call SDK module. This module is mainly responsible for distributing and transmitting the interactive commands to be sent, and ultimately passing the commands to the main process module. Here, "Mini Program" refers to the mini program used during the call. Taking WeChat as the main process module as an example, the mini program opened through WeChat during a WeChat call is the Mini Program process module.

[0112] Main process module: It is mainly responsible for managing the various interface modules and converting the received terminal model instructions into specific logical operations.

[0113] The edge model module is mainly responsible for generating the AI ​​function-related data required by the user and passing the data to the main process module.

[0114] Model platform download module: mainly responsible for downloading client-side models. Users can choose the type of model to download according to their needs.

[0115] The IMS DC chip interface module is mainly responsible for communicating with the model platform, sending data related to the receiving end model to the model platform in order to download the end model from the model platform.

[0116] For example, such as Figure 6 As shown, the overall timing diagram of the call interaction system mainly consists of the enhanced call SDK module (i.e., the first enhanced call module of the first end) and the first chip module of UEA (User Equipment A), and the second chip module, enhanced call SDK module (i.e., the second enhanced call module of the second end), terminal model module, and terminal platform of UEB (User Equipment B). Here, UEA refers to a low-performance device that cannot deploy and use the terminal model, i.e., the first end in the aforementioned call interaction system; UEB refers to a high-performance mobile phone that supports the deployment and use of the terminal model, i.e., the second end in the aforementioned call interaction system. Figure 6 As shown, the overall timing diagram call logic of the call interaction system is as follows:

[0117] 1. UEA users enable AI features during calls.

[0118] For example, when a UEA user makes a call with UEB, the input or click operations performed on the UEA for AI functions cause the UEA to receive an AI activation command for the current call.

[0119] 2. The UEA's enhanced call SDK module sends a model capability request to the first chip module.

[0120] For example, in response to the instruction to enable the artificial intelligence function for the current call, UEA sends a model capability request to UEB through the data transmission channel of the current call based on the first enhanced call module; wherein, the model capability request is used by UEB to download the end-side model based on the second enhanced call module and return the corresponding model capability information.

[0121] 3. UEA requests end-side model capabilities from UEB through the DC channel.

[0122] The DC channel refers to the DC channel in the IMS network.

[0123] For example, UEA forwards the model capability request to the second chip module of UEB through the DC channel between the first chip module and the second chip module, based on the first chip module.

[0124] 4. The UEB notification enhances the call SDK module's ability to enable end-side modeling.

[0125] 5. UEB requests the model platform to download the model.

[0126] 6. The UEB model platform distributes the end-side model to the enhanced call SDK module.

[0127] For example, when the second chip module of the UEB receives a model capability request, it sends a first notification to the second enhanced call module; the first notification is used for the second enhanced call module to switch to the enhanced call mode based on the end-side model and send a model download request to the model platform; the model download request is used for the model platform to send an end-side model list to the second enhanced call module.

[0128] 7. The second enhanced call module sends feedback to the model platform regarding the user's choice to download a specific model.

[0129] For example, the second enhanced call module of UEB obtains user selection information for the end-side model list and sends the user selection information to the model platform.

[0130] 8. The selected end-side model is distributed by the model platform.

[0131] For example, the second enhanced call module of the UEB downloads the corresponding end-side model from the model platform in response to user selection information for the end-side model list.

[0132] 9. The UEB notifies the UEA that the edge model is enabled.

[0133] For example, the second enhanced call module of the UEB sends a second notification to the second chip module; the second chip module is used to determine that the end-side model is in the enabled state and obtain the model capability information of the end-side model according to the second notification.

[0134] 10. UEB transmits its model capability information to UEA through the DC channel.

[0135] Among them, the model capability information is used to indicate the capability status of the UEB's edge-side model.

[0136] For example, the second chip module of UEB sends model capability information to the first chip module of UEA through the DC channel.

[0137] 11. The first chip module transmits the UEB model capability information to the enhanced call SDK module.

[0138] For example, the first chip module sends model capability information to the first enhanced call module.

[0139] 12. The enhanced call SDK module sends model input parameters to the first chip module.

[0140] For example, the first enhanced communication module of UEA determines the model input parameters based on the model capability information and sends the model input parameters to the first chip module of UEA.

[0141] 13. The first chip module sends model input parameters to the UEB through the DC channel.

[0142] For example, based on the first chip module, model input parameters are sent to the second chip module of the UEB via a data transmission channel.

[0143] 14. The second chip module transmits model input parameters to the enhanced call SDK module.

[0144] 15. The UEB enhanced call SDK module sends model input parameters to the end-side model.

[0145] 16. After generating the model, the UEB terminal model is returned to the enhanced call SDK module.

[0146] For example, the second chip module transmits model input parameters to the second enhanced call module; the second enhanced call module sends model input parameters to the end-side model of the UEB; the end-side model of the UEB is used to infer the model generation result based on the model input parameters and send the model generation result to the second enhanced call module.

[0147] 17. The UEB Enhanced Call SDK module returns the model generation results to the chip.

[0148] For example, the second enhanced call module of the UEB sends the model generation result to the second chip module.

[0149] 18. UEB transmits the model generation results of UEB to UEA through the DC channel.

[0150] For example, the second chip module of UEB sends the model generation results to the first chip module of UEA through the DC channel.

[0151] 19. The UEA chip transmits the model generation results to the enhanced call SDK module.

[0152] For example, the first chip module of UEA sends the model generation result to the first enhanced call module.

[0153] In this embodiment, the solution transmits input / output commands related to the edge model (such as model capability requests, model download requests, etc.) through the DC channel in the IMS network. This enables low-performance devices to perform edge model inference using the high-performance mobile phone of the other end of the call in a call scenario. While avoiding anomalies such as high inference latency, surge in device power consumption, and poor application stability, this solution provides edge model inference capabilities for low-performance devices in call scenarios, meets the user's needs for using edge models during the call, and provides users with functions such as intent recognition and intelligent translation during the call, thereby improving the call experience for users using low-performance devices.

[0154] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0155] Based on the same inventive concept, this application also provides a call interaction device for implementing the call interaction method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more call interaction device embodiments provided below can be found in the limitations of the call interaction method described above, and will not be repeated here.

[0156] In one exemplary embodiment, such as Figure 7 As shown, a call interaction device is provided, which can be applied to... Figure 1 The first end in the call, where the first end serves as the local end of the current call, may include the following modules:

[0157] The first response module 701 is used to respond to the instruction to enable artificial intelligence functions for the current call, and based on the first enhanced call module, to send a model capability request to the other end of the current call through the data transmission channel in the current call; the model capability request is used by the other end to download the terminal model based on the second enhanced call module and feed back the corresponding model capability information.

[0158] The first receiving module 702 is used to receive the model capability information of the end-side model sent by the other end through the data transmission channel.

[0159] The first acquisition module 703 is used to acquire model input parameters determined based on model capability information;

[0160] The second receiving module 704 is used to receive the model generation results sent by the peer through the data transmission channel; the model generation results are obtained by the end-side model based on the model input parameters.

[0161] In an exemplary embodiment, the first receiving module 702 is further configured to receive model capability information sent by the second chip module at the other end through a data transmission channel based on the first chip module, and send the model capability information to the first enhanced communication module; wherein, the model capability information is used by the first enhanced communication module to determine model input parameters and send the model input parameters to the first chip module.

[0162] In an exemplary embodiment, the first response module 701 is further configured to, in response to an instruction to enable artificial intelligence functions for the current call, send a model capability request to the first chip module based on the first enhanced call module; and, based on the first chip module, forward the model capability request to the second chip module at the other end through the data transmission channel of the current call.

[0163] In an exemplary embodiment, the model capability request is further used by the second chip module to send a first notification to the second enhanced call module; the first notification is used by the second enhanced call module to switch to the enhanced call mode based on the edge model and send a model download request to the model platform; the model download request is used by the model platform to issue an edge model list to the second enhanced call module; wherein, the second enhanced call module is used to respond to the user selection information for the edge model list, download the corresponding edge model from the model platform, and send a second notification to the second chip module; the second chip module is used to determine that the edge model is in the enabled state and obtain the model capability information of the edge model according to the second notification, and send the model capability information to the first chip module through the data transmission channel.

[0164] In one exemplary embodiment, the call interaction device may further include an interactive inference module. The interactive inference module is configured to send model input parameters to a second chip module at the other end via a data transmission channel, based on the first chip module; the second chip module is configured to send the model input parameters to a terminal-side model via a second enhanced call module; the terminal-side model is configured to infer a model generation result based on the model input parameters and send the model generation result to the second chip module via the second enhanced call module; and based on the first chip module, receive the model generation result sent by the second chip module at the other end via the data transmission channel and send the model generation result to the enhanced call module.

[0165] In one exemplary embodiment, such as Figure 8 As shown, a data transmission device is also provided, which can be applied to... Figure 1 The second end, which serves as the local end of the current call, may include the following modules:

[0166] The second response module 801 is used to respond to the model capability request sent by the peer in the current call, download the terminal model based on the second enhanced call module and feed back the corresponding model capability information; the model capability request is obtained by the peer in response to the artificial intelligence function activation instruction for the current call and based on the first enhanced call module.

[0167] The first sending module 802 is used to send model capability information to the other end through the data transmission channel in the current call; the model capability information is used by the other end to determine the model input parameters.

[0168] The third receiving module 803 is used to receive model input parameters sent by the other end through the data transmission channel.

[0169] The second acquisition module 804 is used to acquire the model generation result obtained by the end-side model based on the model input parameters.

[0170] The second sending module 805 is used to send the model generation results to the other end through the data transmission channel.

[0171] Each module in the aforementioned call interaction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in the network device, or stored in software within the memory of the network device, so that the processor can invoke and execute the corresponding operations of each module.

[0172] In one exemplary embodiment, a network device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the network device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a call interaction method. The display unit of the network device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the network device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the network device, or external keyboards, touchpads, or mice, etc.

[0173] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the network device to which the solution of this application is applied. Specific network devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0174] In one exemplary embodiment, a network device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described call interaction method.

[0175] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described call interaction method.

[0176] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described call interaction method.

[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0178] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0179] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A call interaction method, characterized in that, The method includes: In response to an instruction to enable AI functionality for the current call, a model capability request is sent to the peer in the current call through the data transmission channel of the current call, based on the first enhanced call module; the model capability request is used by the peer to download the end-side model based on the second enhanced call module and return the corresponding model capability information. The system receives model capability information of the end-side model sent by the peer through the data transmission channel. Obtain the model input parameters determined based on the model capability information; The model generation result sent by the peer is received through the data transmission channel; the model generation result is obtained by the end-side model based on the model input parameters.

2. The method according to claim 1, characterized in that, The step of receiving the model capability information of the end-side model sent by the peer through the data transmission channel includes: Based on the first chip module, the system receives the model capability information sent by the second chip module of the peer end through the data transmission channel, and sends the model capability information to the first enhanced call module; wherein, the model capability information is used by the first enhanced call module to determine the model input parameters and send the model input parameters to the first chip module.

3. The method according to claim 1, characterized in that, The response to the AI ​​function activation command for the current call, based on the first enhanced call module, involves sending a model capability request to the other end of the current call through the data transmission channel in the current call, including: In response to the instruction to enable the artificial intelligence function for the current call, a model capability request is sent to the first chip module based on the first enhanced call module; Based on the first chip module, the model capability request is forwarded to the second chip module of the other end through the data transmission channel of the current call.

4. The method according to claim 3, characterized in that, The model capability request is further used by the second chip module to send a first notification to the second enhanced call module; the first notification is used by the second enhanced call module to switch to the enhanced call mode based on the end-side model and send a model download request to the model platform; the model download request is used by the model platform to send an end-side model list to the second enhanced call module. The second enhanced call module is configured to respond to user selection information for the terminal model list, download the corresponding terminal model from the model platform, and send a second notification to the second chip module; the second chip module is configured to determine, based on the second notification, that the terminal model is in an enabled state and obtain the model capability information of the terminal model, and send the model capability information to the first chip module through the data transmission channel.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Based on the first chip module, the model input parameters are sent to the second chip module at the other end through the data transmission channel; the second chip module is used to send the model input parameters to the end-side model through the second enhanced call module; the end-side model is used to infer the model generation result based on the model input parameters, and send the model generation result to the second chip module through the second enhanced call module. Based on the first chip module, the model generation result sent by the second chip module of the peer end is received through the data transmission channel, and the model generation result is sent to the first enhanced call module.

6. A call interaction method, characterized in that, The method includes: In response to a model capability request sent by the peer in the current call, the terminal-side model is downloaded based on the second enhanced call module and the corresponding model capability information is fed back; the model capability request is obtained by the peer in response to an AI function activation command for the current call and based on the first enhanced call module; The model capability information is sent to the peer through the data transmission channel in the current call; the model capability information is used by the peer to determine the model input parameters. The model input parameters sent by the peer are received through the data transmission channel; Obtain the model generation result obtained by the edge model based on the model input parameters; The model generation result is sent to the peer through the data transmission channel.

7. A call interaction system, characterized in that, include: The first and second ends of the current call; The first end is configured to respond to an AI function activation command for the current call, and based on the first enhanced call module, send a model capability request to the second end through the data transmission channel of the current call; the model capability request is used by the second end to download the end-side model based on the second enhanced call module and return the corresponding model capability information; receive the model capability information of the end-side model sent by the second end through the data transmission channel; and obtain the model input parameters determined based on the model capability information. The model generation result sent by the second end is received through the data transmission channel; The model generation result is obtained by the end-side model based on the model input parameters; The second end is used to respond to the model capability request sent by the first end, download the terminal-side model based on the second enhanced call module, and feed back the corresponding model capability information; The model capability information is sent to the first end through the data transmission channel; The model input parameters sent by the first end are received through the data transmission channel; Obtain the model generation result obtained by the end-side model based on the model input parameters; The model generation result is sent to the first end through the data transmission channel.

8. A call interaction device, characterized in that, The device includes: The first response module is used to respond to the AI ​​function activation command for the current call, and based on the first enhanced call module, send a model capability request to the other end of the current call through the data transmission channel in the current call; the model capability request is used by the other end to download the terminal model based on the second enhanced call module and feed back the corresponding model capability information. The first receiving module is used to receive the model capability information of the end-side model sent by the peer through the data transmission channel; The first acquisition module is used to acquire model input parameters determined based on the model capability information; The second receiving module is used to receive the model generation result sent by the peer through the data transmission channel; the model generation result is obtained by the end-side model based on the model input parameters.

9. A call interaction device, characterized in that, The device includes: The second response module is used to respond to the model capability request sent by the peer in the current call, download the terminal-side model based on the second enhanced call module, and feed back the corresponding model capability information; the model capability request is obtained by the peer in response to the artificial intelligence function activation instruction for the current call and based on the first enhanced call module. The first sending module is used to send the model capability information to the peer through the data transmission channel in the current call; the model capability information is used by the peer to determine the model input parameters. The third receiving module is used to receive the model input parameters sent by the peer through the data transmission channel; The second acquisition module is used to acquire the model generation result obtained by the end-side model based on the model input parameters; The second sending module is used to send the model generation result to the peer through the data transmission channel.

10. A network device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 5 or claim 6.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 5 or claim 6.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method described in any one of claims 1 to 5 or claim 6.

Images