Call establishment method and device, communication equipment, storage medium and program product

Abstract

The invention relates to a call establishment method and device, communication equipment, a storage medium and a program product. The method comprises the following steps: an augmented call SDK in a first terminal receives an augmented reality AR call establishment request sent by a first application in the first terminal; based on the AR call establishment request, performing signaling interaction with an enhanced call SDK in the second terminal to establish a network multimedia system data channel IMSDC with a first application in the second terminal, and then obtaining video call window data to render a video call window in a call interface; and controlling a camera in the first terminal to be turned on so as to obtain AR data, wherein the AR data comprises frame data output by the camera; and based on the AR data, rendering a call interface, and sending a multimedia stream to an enhanced call SDK in the second terminal through the IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to a first application in the second terminal. By adopting the method, the establishment of the AR call can be efficiently realized.

Description

Technical Field

[0001] This application relates to the field of wireless communication technology, and in particular to a call establishment method, apparatus, communication device, storage medium, and program product. Background Technology

[0002] AR (Augmented Reality) calling refers to 5G augmented calling with AR capabilities. It uses computer technology to overlay virtual information onto the real environment of the call, achieving a combination of virtual and real effects. How to establish an AR calling architecture on the 5G augmented calling end-user side to ensure the stable and efficient deployment and operation of AR calls is a technical problem that urgently needs to be solved by existing technologies.

[0003] Currently, AR calling services are typically provided by integrating AR SDKs (Software Development Kits), third-party applications, chips, and interfaces into mini-programs. These new modules and interfaces increase the processing logic code of mini-programs, making mini-program development and maintenance much more difficult, and thus reducing the efficiency of AR call establishment. Summary of the Invention

[0004] Therefore, it is necessary to provide a call establishment method, apparatus, communication device, storage medium, and program product to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a call establishment method, applied to an enhanced call software development tool (SDK) in a first terminal, comprising:

[0006] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0007] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a Network Multimedia System Data Channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0008] The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera.

[0009] Based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0010] In one embodiment, controlling the camera in the first terminal to turn on includes:

[0011] An AR call activation instruction is sent to the chip in the first terminal via the AIDL interface, wherein the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

[0012] In one embodiment, the AR call activation instruction is further used to instruct the chip to return parameter information to the enhanced call SDK in the first terminal; the parameter information includes a surface object; the method further includes:

[0013] During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

[0014] In one embodiment, the AR data further includes third-party data; the method further includes:

[0015] Based on the call content, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

[0016] In one embodiment, acquiring the video call window data includes:

[0017] The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

[0018] In one embodiment, the method further includes:

[0019] In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

[0020] In one embodiment, the method further includes:

[0021] The system receives a camera off notification from the video application in the first terminal via the third AIDL interface; wherein the camera off notification is used to notify the enhanced call SDK in the first terminal that the camera has been turned off.

[0022] Secondly, this application also provides a call establishment apparatus, comprising an enhanced call SDK configured in a first terminal, including:

[0023] The request receiving module is used to receive the request to establish an augmented reality (AR) call sent by the first application in the first terminal.

[0024] The data acquisition module is used to interact with the Enhanced Call SDK in the second terminal based on the AR call request, and after establishing a network multimedia system data channel (IMSDC) with the first application in the second terminal, acquire video call window data to render the video call window in the call interface; and,

[0025] The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera.

[0026] The call setup module is used to render the call interface based on the AR data, and send a multimedia stream to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0027] Thirdly, this application also provides a communication device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0028] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0029] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a Network Multimedia System Data Channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0030] The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera.

[0031] Based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0032] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0033] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0034] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a Network Multimedia System Data Channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0035] The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera.

[0036] Based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0037] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0038] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0039] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a Network Multimedia System Data Channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0040] The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera.

[0041] Based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0042] The aforementioned call establishment method, apparatus, communication device, storage medium, and program product, by introducing an enhanced call SDK, after receiving an AR call establishment request sent by a first application in a first terminal, interacts with the enhanced call SDK in a second terminal based on the AR call establishment request to establish an IMSDC with the first application in the second terminal, obtains video call window data to render the video call window in the call interface; and controls the camera in the first terminal to open to obtain AR data, wherein the AR data includes frame data output by the camera; furthermore, based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via the IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal. This solution, by introducing an enhanced call SDK responsible for interacting with the first application, realizes the establishment of AR calls between the first application in the two terminals. Compared with the traditional solution based on integrating an AR SDK into the first application, this reduces development complexity, improves maintenance stability, and enhances the deployment and operation efficiency of AR calls on the terminal side, enabling efficient establishment of AR calls. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is an application environment diagram of a call establishment method in one embodiment;

[0045] Figure 2 This is a flowchart illustrating a call establishment method in one embodiment;

[0046] Figure 3 This is an application scenario diagram provided in one embodiment;

[0047] Figure 4 A signaling diagram for establishing an AR call process between a first terminal and a second terminal in one embodiment;

[0048] Figure 5 This is a signaling diagram showing the AR call interface displayed on the first terminal side in one embodiment;

[0049] Figure 6 This is a structural block diagram of a call establishment device in one embodiment;

[0050] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The call establishment method provided in this application embodiment can be applied to, for example, Figure 1 The application environment shown is illustrated. Both the first and second terminals integrate an Enhanced Call SDK (Software Development Kit), an AR SDK (Augmented Reality Software Development Kit), and a first application. The Enhanced Call SDK 101 is a set of development tools designed to enhance traditional voice call functionality, primarily helping developers quickly integrate richer and smarter call capabilities into their applications. The Enhanced Call SDK and AR SDK 102 are integrated into the same application. The AR SDK is a set of tools, interfaces, and resources for developing augmented reality applications, helping developers more efficiently build AR-enabled applications; therefore, the Enhanced Call SDK also has the ability to call AR. The first application 103 is an application with AR call functionality. The first and second terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, etc.

[0053] In one exemplary embodiment, such as Figure 2 As shown, a call establishment method is provided, which can be applied to... Figure 1 Taking the enhanced call SDK101 of the first terminal as an example, the explanation includes the following steps:

[0054] S201, Receive the request to establish an augmented reality (AR) call sent by the first application in the first terminal.

[0055] The AR call request is used to request the establishment of an AR call, which is a video call that incorporates AR elements.

[0056] Optionally, during video or voice calls conducted through the first application, users on both the first and second terminals can trigger an AR call request by clicking the button provided on the call interface to initiate an AR call. In this case, the first application will send an AR call request to the augmented call SDK.

[0057] It should be noted that the AR call establishment request sent by the first application is received through ARUseCase (Augmented Reality Use Case).

[0058] S202, based on the request to establish an AR call, signaling interaction is performed with the enhanced call SDK in the second terminal to establish a network multimedia system data channel IMSDC with the first application in the second terminal, then video call window data is obtained to render the video call window in the call interface, and the camera in the first terminal is controlled to open to obtain AR data.

[0059] AR data is used to build the AR call interface. In this embodiment, AR data includes frame data output by the camera. The IP Multimedia Subsystem Data Channel (IMSDC) enables real-time interactive information transmission based on traditional audio and video communication, supporting the transmission of multimedia data such as images, emoticons, menus, and AR. Video call window data refers to the data used to construct and display the call interface, including the raw data of the video streams from the first and second terminals, window layout parameters (such as screen size and position), and status information (such as call duration and network quality icon). This data is used to ensure the normal operation of video calls, monitor call quality, and provide a better user experience.

[0060] Optionally, the enhanced call SDK will conduct real-time signaling communication with the enhanced call SDK in the second terminal based on the AR call request. After the signaling interaction is completed, the first terminal and the first application of the second terminal will establish a bidirectional data transmission link under the IMSDC framework based on the negotiated SDP parameters through the session boundary controller or the IP multimedia subsystem of the core network. That is, the IMSDC is established. This channel must meet the requirements of low latency and high reliability (especially the real-time fusion of virtual content and real images in AR scenes, which is sensitive to latency).

[0061] Furthermore, after IMSDC is established, the enhanced call SDKs of the first and second terminals can transmit video call window data in real time through IMSDC. The enhanced call SDK of the first terminal can then receive the video call window data transmitted by the enhanced call SDK of the second terminal, and parse, decode, and render the received video call window data, ultimately presenting a complete video call window (including the local screen, the other party's screen, and interactive controls, etc.) in the call interface. For example, the video screen of the second terminal is displayed according to the negotiated resolution, while the video screen of the first terminal floats in a small window.

[0062] Upon receiving a request to establish an AR call, the augmented call SDK of the first terminal will automatically or with user authorization turn on the camera of the first terminal through a hardware interface. The real-time images captured by the camera (such as the user's facial expressions, images of the surrounding environment, etc.) will be transmitted to the augmented call SDK as AR data.

[0063] It should be noted that the enhanced call SDK and the chip cannot communicate directly; they require intermediaries such as cross-process interfaces and system services. Building upon this, another method to control the camera in the first terminal can be to send an "Enable AR Call" instruction to the chip in the first terminal via the AIDL (AI Language Interface). The chip, integrated into both the first and second terminals, directly interacts with hardware such as the camera and is responsible for camera activation control. The "Enable AR Call" instruction directs the chip to activate the camera in the first terminal. Specifically, the AIDL interface, through a predefined method signature, allows the enhanced call SDK to send commands to the chip. AIDL specifies the parameter format of the "Enable AR Call" instruction, ensuring that the instruction sent by the enhanced call SDK can be correctly parsed by the chip-side services. Upon receiving the "Enable AR Call" instruction, the chip verifies its validity, such as verifying user authorization. If verification is successful, it allocates low-level hardware resources to directly control the camera's power supply and sensor activation, and returns the execution result to the upper layer.

[0064] S203, based on AR data, renders the call interface, and sends a multimedia stream to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

[0065] The multimedia stream may include AR video stream, AR metadata, and audio stream from the first terminal, and this multimedia stream is used by the first application in the second terminal to display the AR screen of the first terminal.

[0066] Optionally, since AR data includes frame data output from the camera, and the number of frames output by the camera generally includes user facial expressions and environmental images, the ARUseCase in the augmented call SDK will start the ARSurfaceRender (augmented reality display carrier rendering) rendering task to perform real-time calculations on AR, determine the coordinate position of virtual content in the real scene, and perform key point recognition on dynamic targets such as user faces and gestures, so that the virtual content changes synchronously with real actions, and superimpose virtual models, special effects, etc. onto the real picture through the graphics rendering engine, and drive the dimensional rendering of the call interface. The call interface not only includes basic call elements, but also needs to integrate AR function entry points so that the first application in the first terminal displays the AR call interface.

[0067] Furthermore, the enhanced calling SDK on the first terminal generates a multimedia stream based on the content displayed on the call interface. After encoding and compressing the multimedia stream, it is sent to the enhanced calling SDK on the second terminal via IMSDC. Upon receiving the multimedia stream, the enhanced calling SDK on the second terminal decodes it and sends it to the first application on the second terminal. The first application then simultaneously displays the AR visuals.

[0068] In the above call establishment method, by introducing an enhanced call SDK, after receiving an AR call establishment request sent by the first application in the first terminal, signaling interaction is performed with the enhanced call SDK in the second terminal based on the AR call establishment request. After establishing an IMSDC with the first application in the second terminal, video call window data is obtained to render the video call window in the call interface; and the camera in the first terminal is controlled to open to obtain AR data, wherein the AR data includes frame data output by the camera; furthermore, based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal through the IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal. The above solution, by introducing an enhanced call SDK responsible for interacting with the first application, realizes the establishment of AR calls between the first application in the two terminals. Compared with the traditional solution based on integrating an AR SDK in the first application, it reduces the development complexity, improves the stability of maintenance, and enhances the deployment and operation efficiency of AR calls on the terminal side, and can efficiently realize the establishment of AR calls.

[0069] Furthermore, the AR call activation indicator also instructs the chip to return parameter information to the enhanced call SDK in the first terminal. This parameter information includes a Surface object. The Surface object represents a rectangular area on the screen of the first terminal and can be used to draw images, text, or other graphic elements. In this embodiment, the Surface object is used to display AR images. After acquiring AR data, during a call between the first application in the first terminal and the first application in the second terminal, AR data is written to the Surface object so that AR images can be displayed on the screen of the first terminal.

[0070] Furthermore, to enhance user experience and enrich AR call content, AR data also includes third-party data; this includes, but is not limited to, data from third-party applications, such as location data from location applications. Based on this, when a user needs to acquire and display third-party data, they can trigger a data acquisition request through a function button provided on the call interface. For example, based on the call content, third-party data can be acquired from a second application on the first terminal via a second AIDL interface. The second AIDL interface is the interface for interacting with the second application.

[0071] Building upon this, a third AIDL interface can be introduced between the enhanced call SDK and the video application in the first terminal to obtain video call window data. For example, the third AIDL interface can be used to interact with the video application in the first terminal to obtain video call window data. Here, the video application is the one that provides the call video parameters.

[0072] Furthermore, when the user does not need to conduct AR calls, they can initiate a request to close the AR call via the "Close AR Call" function button provided in the call interface. Specifically, in response to the request to close the AR call sent by the first application in the first terminal, the AR function is turned off, and a "Close AR Call" instruction is sent to the chip in the first terminal through the first AIDL interface; wherein, the "Close AR Call" instruction is used to instruct the chip to turn off the camera in the first terminal. Optionally, in response to the request to close the AR call sent by the first application in the first terminal, the first enhancement SDK will send a "Close AR Call" instruction to the chip in the first terminal through the first AIDL interface to instruct the chip to turn off the camera in the first terminal, stop capturing camera images, and release virtual model rendering resources to release the AR call.

[0073] Furthermore, the system receives a camera-off notification from the video application in the first terminal via the third AIDL interface. This camera-off notification informs the enhanced calling SDK in the first terminal that the camera is off. When the video application receives the user's camera-off command, it sends a camera-off notification to the enhanced calling SDK via the third AIDL interface to inform the enhanced calling SDK in the first terminal that the camera is off.

[0074] It should be noted that the call establishment method provided in this application embodiment uses an enhanced call SDK introduced in the first terminal and the second terminal to replace the first application in interacting with the chip, the second application, the video application, etc., without integrating modules such as the chip, the second application, and the video application into the first application.

[0075] To intuitively describe the process of establishing an AR call between the first terminal and the second terminal before the first application, in one embodiment, a mini-program is used as the first application and a calling app is used as the video application to describe the AR call establishment process. Figure 3 The illustration shows an application scenario provided by an embodiment of this application. AIDL includes ARCall AIDL and AROEM AIDL, where ARCall AIDL is the first AIDL interface and AROEM AIDL is the third AIDL interface. Figure 3 On the basis of, such as Figure 4 The diagram shown illustrates the signaling process for establishing an AR call between the first and second terminals. (Combined with...) Figure 4 The specific implementation process is as follows:

[0076] S401, the augmented call SDK in the first terminal receives an AR call establishment request sent by the first application in the first terminal.

[0077] S402, the enhanced call SDK in the first terminal interacts with the enhanced call SDK in the second terminal based on the request to establish an AR call.

[0078] S404, the enhanced call SDK in the first terminal establishes an IMSDC with the first application in the second terminal.

[0079] S405, the enhanced call SDK in the first terminal interacts with the video application in the first terminal through the third AIDL interface to obtain video call window data and render the video call window in the call interface.

[0080] S406, the enhanced call SDK in the first terminal sends an instruction to enable AR call to the chip in the first terminal through the first AIDL interface.

[0081] The AR call activation indicator is used to instruct the chip to activate the camera in the first terminal.

[0082] S407, the chip in the first terminal controls the camera in the first terminal to turn on.

[0083] S408, the enhanced calling SDK in the first terminal acquires AR data through the camera.

[0084] AR data includes frame data output from the camera.

[0085] S409 renders the call interface based on AR data and sends a multimedia stream to the enhanced call SDK in the second terminal via IMSDC.

[0086] S410, the enhanced call SDK in the second terminal sends a multimedia stream to the first application in the second terminal.

[0087] The specific processes of S401-S410 described above can be found in the description of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0088] Furthermore, such as Figure 5 The diagram shown is a signaling diagram of the AR call interface displayed on the first terminal side according to an embodiment of this application. Combined with... Figure 5 The specific implementation process is as follows:

[0089] S501, the first application initiates a request to establish an augmented reality (AR) call to the augmented call SDK of the first terminal.

[0090] S502, the enhanced call SDK of the first terminal interacts with the video application in the first terminal through the third AIDL interface to obtain video call window data.

[0091] S503, the enhanced call SDK of the first terminal sends an instruction to enable AR call to the chip in the first terminal through the first interface definition language AIDL interface, and obtains AR data.

[0092] The AR call activation indicator is used to instruct the chip to activate the camera in the first terminal.

[0093] S504, the chip of the first terminal returns a Surface object to the enhanced call SDK.

[0094] S505, the enhanced call SDK of the first terminal writes AR data to the Surface object of the AR SDK.

[0095] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0096] Based on the same inventive concept, this application also provides a call establishment apparatus for implementing the call establishment method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more call establishment apparatus embodiments provided below can be found in the limitations of the call establishment method described above, and will not be repeated here.

[0097] In one exemplary embodiment, such as Figure 6 As shown, a call setup device 600 is provided, which is an enhanced call SDK configured in a first terminal, including: a request receiving module 610, a data acquisition module 620, and a call setup module 630, wherein:

[0098] The request receiving module 610 is used to receive the request to establish an augmented reality (AR) call sent by the first application in the first terminal.

[0099] The data acquisition module 620 is used to interact with the augmented call SDK in the second terminal based on the request to establish an AR call, and after establishing a network multimedia system data channel IMSDC with the first application in the second terminal, acquire video call window data to render the video call window in the call interface; and control the camera in the first terminal to open to acquire AR data; wherein, the AR data includes frame data output by the camera.

[0100] The call setup module 630 is used to render the call interface based on AR data, and send a multimedia stream to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal can send the multimedia stream to the first application in the second terminal.

[0101] The aforementioned call establishment device, by introducing an enhanced call SDK, after receiving an AR call establishment request sent by a first application in a first terminal, interacts with the enhanced call SDK in a second terminal based on the AR call establishment request. After establishing an IMSDC with the first application in the second terminal, it acquires video call window data to render the video call window in the call interface; and controls the camera in the first terminal to open to acquire AR data, including frame data output by the camera. Further, based on the AR data, it renders the call interface and sends a multimedia stream to the enhanced call SDK in the second terminal via the IMSDC, enabling the enhanced call SDK in the second terminal to send the multimedia stream to the first application in the second terminal. This solution, by introducing an enhanced call SDK responsible for interacting with the first application, establishes an AR call between the first application in both terminals. Compared to traditional solutions that integrate an AR SDK into the first application, this reduces development complexity, improves maintenance stability, and enhances the deployment and operational efficiency of AR calls on the terminal side, enabling efficient AR call establishment.

[0102] In one embodiment, the data acquisition module 620 is used for:

[0103] The system sends an AR call activation instruction to the chip in the first terminal via the AIDL interface (first interface definition language); the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

[0104] In one embodiment, the AR call activation indicator is further configured to instruct the chip to return parameter information to the augmented call SDK in the first terminal; the parameter information includes a surface object; the data acquisition module 620 is further configured to:

[0105] During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

[0106] In one embodiment, the AR data also includes third-party data; the call establishment device 600 further includes: a data acquisition unit, used for:

[0107] Based on the content of the call, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

[0108] In one embodiment, the data acquisition module 620 is further configured to:

[0109] The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

[0110] In one embodiment, the call setup device 600 further includes a call termination module, used for:

[0111] In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

[0112] In one embodiment, the call termination module is further configured to:

[0113] The system receives a camera shutdown notification from the video application in the first terminal via the third AIDL interface; the camera shutdown notification is used to notify the enhanced call SDK camera in the first terminal that it has been turned off.

[0114] Each module in the aforementioned call establishment device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the communication device in hardware form or independent of it, or stored in the memory of the communication device in software form, so that the processor can call and execute the corresponding operations of each module.

[0115] In one exemplary embodiment, a communication device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 7 As shown, the communication device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a call establishment method.

[0116] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0117] In one exemplary embodiment, a communication device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0118] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0119] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a network multimedia system data channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0120] The camera in the first terminal is turned on to acquire AR data; the AR data includes frame data output by the camera.

[0121] Based on AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal can send the multimedia stream to the first application in the second terminal.

[0122] In one embodiment, when the processor executes a computer program to control the camera in the first terminal to turn on, the following steps are also performed:

[0123] The system sends an AR call activation instruction to the chip in the first terminal via the AIDL interface (first interface definition language); the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

[0124] In one embodiment, the AR call activation indicator is further used to instruct the chip to return parameter information to the augmented call SDK in the first terminal; the parameter information includes a surface object; the processor also performs the following steps when executing the computer program:

[0125] During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

[0126] In one embodiment, the AR data also includes third-party data; the processor, while executing the computer program, also performs the following steps:

[0127] Based on the content of the call, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

[0128] In one embodiment, when the processor executes a computer program to obtain video call window data, it also performs the following steps:

[0129] The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

[0130] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0131] In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

[0132] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0133] The system receives a camera shutdown notification from the video application in the first terminal via the third AIDL interface; the camera shutdown notification is used to notify the enhanced call SDK camera in the first terminal that it has been turned off.

[0134] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0135] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0136] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a network multimedia system data channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0137] The camera in the first terminal is turned on to acquire AR data; the AR data includes frame data output by the camera.

[0138] Based on AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal can send the multimedia stream to the first application in the second terminal.

[0139] In one embodiment, when the processor executes a computer program to control the camera in the first terminal to turn on, the following steps are also performed:

[0140] The system sends an AR call activation instruction to the chip in the first terminal via the AIDL interface (first interface definition language); the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

[0141] In one embodiment, the AR call activation indicator is further used to instruct the chip to return parameter information to the augmented call SDK in the first terminal; the parameter information includes a surface object; the processor also performs the following steps when executing the computer program:

[0142] During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

[0143] In one embodiment, the AR data also includes third-party data; the processor, while executing the computer program, also performs the following steps:

[0144] Based on the content of the call, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

[0145] In one embodiment, when the processor executes a computer program to obtain video call window data, it also performs the following steps:

[0146] The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

[0147] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0148] In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

[0149] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0150] The system receives a camera shutdown notification from the video application in the first terminal via the third AIDL interface; the camera shutdown notification is used to notify the enhanced call SDK camera in the first terminal that it has been turned off.

[0151] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0152] Receive an augmented reality (AR) call request sent by the first application in the first terminal;

[0153] Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a network multimedia system data channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and...

[0154] The camera in the first terminal is turned on to acquire AR data; the AR data includes frame data output by the camera.

[0155] Based on AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal can send the multimedia stream to the first application in the second terminal.

[0156] In one embodiment, when the processor executes a computer program to control the camera in the first terminal to turn on, the following steps are also performed:

[0157] The system sends an AR call activation instruction to the chip in the first terminal via the AIDL interface (first interface definition language); the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

[0158] In one embodiment, the AR call activation indicator is further used to instruct the chip to return parameter information to the augmented call SDK in the first terminal; the parameter information includes a surface object; the processor also performs the following steps when executing the computer program:

[0159] During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

[0160] In one embodiment, the AR data also includes third-party data; the processor, while executing the computer program, also performs the following steps:

[0161] Based on the content of the call, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

[0162] In one embodiment, when the processor executes a computer program to obtain video call window data, it also performs the following steps:

[0163] The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

[0164] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0165] In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

[0166] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0167] The system receives a camera shutdown notification from the video application in the first terminal via the third AIDL interface; the camera shutdown notification is used to notify the enhanced call SDK camera in the first terminal that it has been turned off.

[0168] It should be noted that the data involved in this application (including but not limited to video call window data and AR data) are all information and data that have been fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0169] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0170] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0171] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for establishing a call, characterized in that, The method, which involves an enhanced call software development kit (SDK) applied to a first terminal, includes: Receive an augmented reality (AR) call request sent by the first application in the first terminal; Based on the AR call request, signaling interaction is performed with the Enhanced Call SDK in the second terminal to establish a Network Multimedia System Data Channel (IMSDC) with the first application in the second terminal. Then, video call window data is obtained to render the video call window in the call interface; and... The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera. Based on the AR data, the call interface is rendered, and a multimedia stream is sent to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

2. The method according to claim 1, characterized in that, The step of controlling the camera in the first terminal to turn on includes: An AR call activation instruction is sent to the chip in the first terminal via the AIDL interface, wherein the AR call activation instruction is used to instruct the chip to activate the camera in the first terminal.

3. The method according to claim 2, characterized in that, The AR call activation instruction is also used to instruct the chip to return parameter information to the enhanced call SDK in the first terminal; The parameter information includes a surface object; the method further includes: During a call between the first application on the first terminal and the first application on the second terminal, AR data is written to the Surface object.

4. The method according to any one of claims 1-3, characterized in that, The AR data also includes third-party data; the method further includes: Based on the call content, third-party data is obtained from the second application in the first terminal through the second AIDL interface.

5. The method according to any one of claims 1-3, characterized in that, The acquisition of video call window data includes: The system interacts with the video application in the first terminal via the third AIDL interface to obtain video call window data.

6. The method according to any one of claims 1-3, characterized in that, The method further includes: In response to a request to disable AR calls sent by a first application in the first terminal, the AR function is disabled, and an instruction to disable AR calls is sent to the chip in the first terminal via a first AIDL interface; wherein, the instruction to disable AR calls is used to instruct the chip to disable the camera in the first terminal.

7. The method according to claim 6, characterized in that, The method further includes: The system receives a camera off notification from the video application in the first terminal via the third AIDL interface; wherein the camera off notification is used to notify the enhanced call SDK in the first terminal that the camera has been turned off.

8. A call setup device, characterized in that, An enhanced call SDK configured in a first terminal, the device comprising: The request receiving module is used to receive the request to establish an augmented reality (AR) call sent by the first application in the first terminal. The data acquisition module is used to interact with the Enhanced Call SDK in the second terminal based on the AR call request, and after establishing a network multimedia system data channel (IMSDC) with the first application in the second terminal, acquire video call window data to render the video call window in the call interface; and, The camera in the first terminal is turned on to acquire AR data; wherein the AR data includes frame data output by the camera. The call setup module is used to render the call interface based on the AR data, and send a multimedia stream to the enhanced call SDK in the second terminal via IMSDC, so that the enhanced call SDK in the second terminal sends the multimedia stream to the first application in the second terminal.

9. A communication device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images