Video call method and device, terminal and storage medium

Abstract

The embodiment of the invention discloses a video call method and device, a terminal and a storage medium, and belongs to the technical field of video calls. The method comprises the following steps: performing video stream extraction on a first video call interface to obtain a home terminal video stream and at least one path of opposite terminal video stream; processing the home terminal video stream and the opposite terminal video stream based on the display configuration information of the third terminal to obtain a target video stream, a target video call picture corresponding to the target video stream including a home terminal video picture and an opposite terminal video picture, and the target video picture meeting a display demand represented by the display configuration information; and sending the target video stream to a third terminal to enable the third terminal to display a second video call interface based on the target video stream, the second video call interface including the target video call picture. By adopting the scheme provided by the embodiment of the invention, the display effect of the video call interface on the interconnection terminal can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of video calling, and in particular to a video calling method, device, terminal and storage medium. Background Art

[0002] With the development of communication technology, the functions of terminals are becoming more and more abundant. Users can realize video chat through terminals, and the same user can also make video calls through multiple terminals at the same time.

[0003] In order to enable the same user to make video calls through multiple terminals at the same time, during the video call, the video call content on one terminal (such as a mobile phone) is often projected to other connected terminals (such as a car computer) for display. Users can conduct video chats through the video call content displayed on other connected terminals.

[0004] However, when the user performs other operations on the terminal sending the projection content, the projection will be interrupted, and the screen sizes of the terminal sending the projection content and the interconnected terminal receiving the projection content are often inconsistent, resulting in poor display of the video call content on the interconnected terminal receiving the projection content. Summary of the invention

[0005] The embodiments of the present application provide a video call method, device, terminal and storage medium, which are helpful to improve the display effect of the video call interface on the interconnected terminal. The technical solution is as follows:

[0006] On the one hand, an embodiment of the present application provides a video call method, which is used for a first terminal, and the method includes:

[0007] Extracting a video stream from a first video call interface to obtain a local video stream and at least one peer video stream, wherein the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal;

[0008] Processing the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream, wherein a target video call screen corresponding to the target video stream includes a local video screen and an opposite video screen, and the target video screen meets the display requirements represented by the display configuration information;

[0009] The target video stream is sent to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

[0010] On the other hand, an embodiment of the present application provides a video calling method, which is used by a third terminal, and the method includes:

[0011] Providing display configuration information of the third terminal to the first terminal;

[0012] Receive a target video stream sent by the first terminal, the target video call screen corresponding to the target video stream meets the display requirements represented by the display configuration information, the target video stream is obtained by the first terminal processing the local video stream and the opposite video stream based on the display configuration information, the local video stream and the opposite video stream are obtained by the first terminal performing video stream extraction on a first video call interface, the target video call screen includes the local video screen and the opposite video screen, and the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal;

[0013] A second video call interface is displayed based on the target video stream, wherein the second video call interface includes the target video call screen.

[0014] On the other hand, an embodiment of the present application provides a video call device, the device comprising:

[0015] A video stream extraction module, configured to extract a video stream from a first video call interface to obtain a local video stream and at least one peer video stream, wherein the first video call interface is an interface for a video call between the first terminal and at least one second terminal;

[0016] A video stream processing module, configured to process the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream, wherein the target video call screen corresponding to the target video stream includes the local video screen and the opposite video screen, and the target video screen meets the display requirements represented by the display configuration information;

[0017] The video stream sending module sends the target video stream to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

[0018] On the other hand, an embodiment of the present application provides a video call device, the device comprising:

[0019] a configuration information sending module, providing display configuration information of the third terminal to the first terminal;

[0020] a video stream receiving module for receiving a target video stream sent by the first terminal, wherein the target video stream corresponds to a target video call screen that meets the display requirements represented by the display configuration information, wherein the target video stream is obtained by the first terminal processing a local video stream and a peer video stream based on the display configuration information, wherein the local video stream and the peer video stream are obtained by the first terminal performing video stream extraction on a first video call interface, wherein the target video call screen includes a local video screen and a peer video screen, and wherein the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal;

[0021] The interface display module displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

[0022] On the other hand, an embodiment of the present application provides a terminal, comprising a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the video call method as described in any of the above aspects.

[0023] On the other hand, an embodiment of the present application provides a computer-readable storage medium, wherein at least one program is stored in the computer-readable storage medium, and the at least one program is loaded and executed by a processor to implement the video call method as described in any of the above aspects.

[0024] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the video call method provided in any of the above aspects.

[0025] In the embodiment of the present application, the first video interface is an interface for a first terminal to conduct a video call with at least one second terminal. By extracting the video stream from the first video call interface, the local video stream and at least one opposite video stream can be obtained. The third terminal is an interconnected terminal of the first terminal. In order to enable the third terminal to display a better video call interface, the local video stream and the opposite video stream are processed using the display configuration information of the third terminal to obtain a target video stream that meets the display requirements of the third terminal; the target video stream is sent to the interconnected third terminal through the first terminal, so that the third terminal displays the second video call interface, which includes a target video call screen corresponding to the target video stream. Since the target video stream meets the display requirements of the third terminal, it helps to improve the display effect of the video call content on the third terminal side. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0027] Figure 1 is a schematic diagram of an implementation environment shown in an exemplary embodiment of the present application;

[0028] Figure 2 A flowchart of a video calling method provided by an exemplary embodiment of the present application is shown;

[0029] Figure 3 is a flow chart of a video stream extraction method shown in an exemplary embodiment of the present application;

[0030] Figure 4 is a schematic diagram of a first video call interface shown in an exemplary embodiment of the present application;

[0031] Figure 5 A flowchart of extracting a peer video stream provided by another exemplary embodiment of the present application is shown;

[0032] Figure 6 A schematic diagram of picture processing provided by another exemplary embodiment of the present application is shown;

[0033] Figure 7 A flowchart of a video stream processing method provided by another exemplary embodiment of the present application is shown;

[0034] Figure 8 A flowchart of generating a target video stream after configuration update provided by another exemplary embodiment of the present application is shown;

[0035] Fig. 9 A schematic diagram showing an update of a second video call interface provided by another exemplary embodiment of the present application is shown;

[0036] Fig.10 A schematic diagram of an answering prompt provided by an exemplary embodiment of the present application is shown;

[0037] Fig.11 A flowchart of a video calling method provided by another exemplary embodiment of the present application is shown;

[0038] Fig.12 A structural block diagram of a video call device provided by an exemplary embodiment of the present application is shown;

[0039] Fig.13A structural block diagram of a video call device provided by another exemplary embodiment of the present application is shown;

[0040] Fig.14 A schematic diagram of the structure of a terminal provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] It should be understood that the "several" mentioned in this article refers to one or more, and "multiple" refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0043] The following is a brief introduction to the terms involved in the embodiments of the present application:

[0044] Video Streaming: Video streaming refers to the transmission of video data. Video streaming technology is a technology for real-time transmission and sharing of video content located on the network. Video streaming can be processed as a stable and continuous stream through the network, allowing users to watch video content instantly without downloading it or caching it. Video streaming technology is often used in scenarios such as video calls, interactive games and live broadcasts.

[0045] The video call method provided in the embodiment of the present application can be applied to the scenario where the video call content is displayed on the Internet terminal. Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment includes a first terminal 110, a second terminal 120, a third terminal 130 and a cloud 140.

[0046] The first terminal 110 and the second terminal 120 are electronic devices with video call functions, and the electronic devices may be smart phones, tablet computers, wearable devices, personal computers, etc. The third terminal 130 may be interconnected with the first terminal 110 through a communication protocol, so that the first terminal 110 and the third terminal 130 can transmit data to each other, wherein the third terminal 130 is an electronic device with a video call content display requirement, and the electronic device may be a vehicle-mounted terminal (vehicle device), a smart phone, a tablet computer, a wearable device, a personal computer, etc., which is not limited in this embodiment.

[0047] The cloud 140 can provide a video call background service for the first terminal 110 and the second terminal 120. For example, the first terminal 110 and the second terminal 120 are installed with a video call application, and the cloud 140 is a background server of the video call application, which is used to transmit video streams between the video call terminals.

[0048] In the embodiments of this specification, the first terminal 110 and the third terminal 130 may be directly or indirectly connected via wired or wireless communication, which is not limited in the present disclosure. The first terminal 110 and the second terminal 120 may be directly or indirectly connected to the cloud 140 via wired or wireless communication, which is also not limited in the present disclosure.

[0049] For the convenience of description, the following description is made by taking the first terminal 110 and the second terminal 120 as smartphones used by users, the third terminal 130 as a vehicle device used by users, and the cloud 140 as a cloud server as an example.

[0050] In some embodiments, based on the video call background service provided by the cloud 140, the local user can make a video call with the second terminal 120 used by the other user through the first terminal 110. Figure 1 As shown, the local user is in a driving environment and it is inconvenient to use the first terminal 110 , so after a video call is established between the first terminal 110 and the second terminal 120 , the video call content is processed and displayed in the video display interface of the third terminal 130 .

[0051] The following describes an embodiment of a video call method of the present application with the first terminal as the execution subject. Figure 2 Please refer to Figure 2 , which shows a flowchart of a video calling method provided by an exemplary embodiment of the present application, the method comprising:

[0052] Step 201, extracting a video stream from a first video call interface to obtain a local video stream and at least one opposite video stream, wherein the first video call interface is an interface for a first terminal to conduct a video call with at least one second terminal.

[0053] The first video call interface is a display interface corresponding to a video application used for video calling on the first terminal when the first terminal is in a video call with the second terminal, and the first video interface is used to display the video call content between the first terminal and at least one second terminal. In an optional embodiment, the first terminal can establish a video call with at least one second terminal, and the first terminal displays the first video call interface to display the video call content between the first terminal and at least one second terminal.

[0054] In one possible implementation, the first terminal records the first video call interface to obtain a recorded screen, and the first terminal extracts the video stream based on the recorded screen to obtain a local video stream and at least one opposite-end video stream, wherein the local video stream is video data including a local user's video screen, and the opposite-end video stream is video data including at least one opposite-end user's video screen.

[0055] In another possible implementation, the first terminal records the first video call interface to obtain a recorded image, the first terminal extracts the local video stream corresponding to the first terminal based on the recorded image, and the first terminal obtains at least one opposite-end video stream corresponding to the second terminal from the cloud that provides video call background services.

[0056] In another possible implementation, the first terminal records the first video call interface to obtain a recorded image, and the first terminal extracts at least one opposite-end video stream corresponding to the second terminal based on the recorded image, and the first terminal uses the video stream captured by the camera of the first terminal as the local video stream. Optionally, since the first terminal is interconnected with the third terminal through a communication protocol, if the third terminal has a camera function, the first terminal can also use the video stream captured by the camera of the third terminal as the local video stream.

[0057] Step 202, based on the display configuration information of the third terminal, the local video stream and the opposite video stream are processed to obtain a target video stream, wherein the target video call screen corresponding to the target video stream includes the local video screen and the opposite video screen, and the target video screen meets the display requirements represented by the display configuration information.

[0058] In some embodiments, the display configuration information may include configuration information such as screen display size information, screen display position information, and screen resolution that characterize the display requirements of the third terminal. In order for the third terminal to display the target video call content that meets the requirements of size, position, and clarity, the first terminal needs to process the local video stream and the peer video stream based on the display configuration information of the third terminal to obtain the target video stream corresponding to the target video call screen, wherein the video stream processing performed by the first terminal on the local video stream and the peer video stream includes but is not limited to: cropping the corresponding screen of the video stream, blurring the corresponding part of the video stream, and splicing the corresponding screen of the video stream, which is not limited in the embodiments of the present application.

[0059] Step 203: Send the target video stream to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, and the second video call interface includes the target video call screen.

[0060] In some embodiments, after the first terminal obtains the target video stream that meets the display requirements of the third terminal, the first terminal sends the target video stream to the third terminal interconnected with the first terminal, wherein the first terminal and the third terminal are interconnected using a communication protocol, and the communication protocol supports video data transmission between the first terminal and the third terminal. For example, when the first terminal is a smart phone and the third terminal is a vehicle-mounted device, the first terminal and the third terminal can perform video data transmission through mobile vehicle-mounted interconnection protocols such as Carlink, Carlife, CarPlay, and HiCar. The first terminal sends the target video stream to the third terminal interconnected with the first terminal so that the third terminal displays a second video call interface, and the second video interface is used to display the video call content between the first terminal and at least one second terminal, wherein the second call interface includes a target video call screen corresponding to the target video stream, and the target video call screen includes a local video screen corresponding to the local user, and a peer video screen corresponding to the peer user.

[0061] In a possible implementation, the first terminal sends the target video stream to the third terminal by streaming transmission, wherein the format of the media file corresponding to the target video stream supports streaming transmission and playback, for example, the media format of the file corresponding to the target video stream is RM format (Real Media file format, streaming format), MOV format (Movie to Quick Time Movie, film format), ASF (Advanced Streaming Format, advanced streaming format), SWF (Shock Wave Flash, vector-based Flash animation format), etc. By streaming transmission, the first terminal can send the target video stream to the third terminal, thereby realizing continuous and real-time transmission of video data.

[0062] To summarize, in the embodiment of the present application, the first terminal can obtain the local video stream and at least one opposite-end video stream by extracting the video stream from the first video call interface, and the first terminal processes the local video stream and the opposite-end video stream using the display configuration information of the third terminal, thereby obtaining a target video stream that meets the display requirements of the third terminal; the first terminal sends the target video stream to the interconnected third terminal, so that the third terminal can display the second video call interface containing the target video call screen; since the target video stream meets the display requirements of the third terminal, it helps to improve the display effect of the video call content on the third terminal side.

[0063] During a video call, the first terminal first extracts the video stream from the video call interface to obtain the local video stream and at least one peer video stream. Figure 3 , which shows a flowchart of a video stream extraction method provided by an exemplary embodiment of the present application, the method comprising:

[0064] Step 301: determine the video stream collected by the camera of the first terminal or the third terminal as the local video stream.

[0065] In some embodiments, when a first terminal makes a video call with a second terminal, the first terminal collects a video image of a local user through a camera of the first terminal, so the first terminal obtains a video stream collected by the camera of the first terminal and determines the video stream as a local video stream.

[0066] In a possible implementation, when a first terminal conducts a video call with a second terminal, if a third terminal interconnected with the first terminal has a camera function, the first terminal can capture a video image of the local user through the camera of the third terminal, and the first terminal obtains a video stream captured by the camera of the third terminal and sent by the third terminal, and determines the video stream as the local video stream.

[0067] Step 302: extract at least one peer video stream from the first video call interface based on the local video stream.

[0068] In some embodiments, when the first terminal is unable to obtain the peer video stream corresponding to at least one second terminal from the cloud-based background service responsible for the video call, the first terminal can identify the peer video stream in the first video call interface based on the local video stream that can be directly obtained, and then extract the peer video stream from the first video call interface.

[0069] In a possible implementation manner, the first terminal may extract at least one peer video stream from the first video call interface based on the local video stream in the following manner:

[0070] 1. Get a screenshot of the first video call interface.

[0071] In a possible implementation, the first terminal performs a screenshot operation to obtain a screenshot of the first video call interface at the current moment.

[0072] In another possible implementation, the first terminal records the first video call interface to obtain a recorded screen, and selects a screen corresponding to a certain historical moment in the recorded screen as a screenshot of the first video call interface.

[0073] 2. Perform a screen extraction operation on the screenshot to obtain window screens corresponding to each video call window in the first video call interface.

[0074] In a possible implementation, the first terminal uses OpenCV (Open Source Computer Vision Library) combined with image extraction algorithms such as feature extraction algorithms and face recognition algorithms to perform image extraction operations on the screenshot to obtain window images corresponding to each video call window in the first video call interface.

[0075] 3. Determine the peer video window in the video call window based on the image similarity between the local video image and the window image corresponding to the local video stream.

[0076] In some embodiments, when the first terminal obtains a screenshot of the first video call interface, the first terminal obtains a frame image of the local video stream at the corresponding time based on the corresponding screenshot time of the screenshot, and compares the screen similarity between the frame image and the window screen. Since the frame image is the video image corresponding to the local user, when the screen similarity between the window screen and the frame image is high, the corresponding window of this window screen is more likely to be the local video window, and vice versa, when the screen similarity between the window screen and the frame image is low, the video call window corresponding to this window screen is more likely to be the other party video window.

[0077] In a possible implementation, the first terminal calculates the image similarity between the window image and the frame image. When the image similarity is lower than the target threshold, the first terminal determines the video call window corresponding to the window image as the peer video window. For example, the first terminal makes a video call with two second terminals. At this time, the first video call interface includes three windows, a local video window corresponding to the first terminal and peer video windows corresponding to the two second terminals. The first terminal obtains a screenshot of the first video call interface, extracts three window images, and compares the three window images with the frame image obtained by the first terminal. The image similarities are 0.4, 0.3, and 0.8, and the target threshold is set to 0.6. The first terminal determines the video windows corresponding to the window images with image similarities of 0.4 and 0.3 as peer video windows.

[0078] 4. Extract the video stream of the peer video window in the first video call interface to obtain the peer video stream.

[0079] In some embodiments, the first terminal records the first video call interface to obtain a recorded screen, the first terminal captures the opposite end screen in the opposite end video window from the recorded screen, and the first terminal generates an opposite end video stream based on the continuously captured opposite end screens. The screen recording operation is that the first terminal continuously captures the set area (first video call interface) at a certain time interval to obtain multiple continuous recorded screens, the first terminal captures multiple continuous recorded screens according to the determined opposite end video window to obtain multiple continuous captured screens, and finally integrates all the captured screens at corresponding time intervals to obtain the opposite end video stream.

[0080] In a possible implementation, the first terminal may use tools such as OpenCV, FFmpeg (Fast Forward Mpeg, a multimedia video processing tool) and Python to extract the video stream of the peer video window in the first video call interface to obtain the peer video stream.

[0081] In summary, in the embodiment of the present application, the first terminal determines the video stream captured by the camera of the first terminal as the local video stream, and extracts at least one opposite-end video stream from the first video call interface based on the local video stream. By calculating the screen similarity between the screenshot screen corresponding to the first video stream and each window screen in the first video call interface, the opposite-end video window is determined by using the screen similarity, and the video stream is extracted to obtain the opposite-end video stream, which can improve the extraction accuracy of the opposite-end video stream.

[0082] In a possible application scenario, when a local user makes a video call with a single opposite user, the first video call interface includes a full-screen window and a single small window, as shown in Figure 4, which shows a schematic diagram of a first video call interface provided by an exemplary embodiment of the present application, and the first video call interface includes: a full-screen window 410 and a small window 420. The small window 420 is located on the upper layer of the full-screen window 410, wherein the picture displayed by the full-screen window 410 occupies the entire first video call interface, and the picture size displayed by the small window 420 is the same as the picture size displayed by the full-screen window. The full-screen window 410 and the small window 420 are used to display the video call screen between the local user and the opposite user. The first terminal can extract the video stream of the first video interface including the full-screen window 410 and the single small window 420 to obtain the local video stream and the opposite video stream.

[0083] In the case where the first video call interface includes a full-screen window and a single small window, the first terminal extracts the video stream from the first video interface to obtain the local video stream and the peer video stream, which can be achieved through the following steps: Figure 5 As shown, it shows a flowchart of extracting a peer video stream provided by an exemplary embodiment of the present application, and the process includes:

[0084] Step 501, obtaining a screenshot of the first video call interface.

[0085] The method for obtaining the screenshot of the first video call interface can refer to the above step 302, which will not be repeated here.

[0086] Step 502: Determine the screen similarity between the local video screen and the window screen corresponding to the full-screen window.

[0087] In some embodiments, when the first terminal obtains a screenshot of the first video call interface, the first terminal obtains a frame image of the corresponding time of the local video stream based on the corresponding screenshot time of the screenshot, and the first terminal compares the picture similarity between the frame image and the window picture corresponding to the full-screen window. Since the window picture of the full-screen window occupies the entire first video call interface, the screenshot of the first video call interface can be regarded as the window picture corresponding to the full-screen window. In addition, the small window is located on the upper layer of the full-screen window, and the window size of the full-screen window is larger than the window size of the small window. Therefore, in the screenshot, the window picture content displayed by the full-screen window is more than the window picture content displayed by the small window. Since the frame image is the video image corresponding to the local user, when the picture similarity between the window picture corresponding to the full-screen window (i.e., the screenshot picture) and the frame image is high, the full-screen window is more likely to be the local video window. Conversely, when the picture similarity between the screenshot picture and the frame image is low, the small window is more likely to be the opposite video window.

[0088] Step 503: When the picture similarity is greater than the similarity threshold, the small window is determined as the peer video window.

[0089] In a possible implementation, the first terminal calculates the image similarity between the window image corresponding to the full-screen window and the frame image, and when the image similarity is higher than the target threshold, the first terminal determines the small window as the opposite-end video window. For example, the first terminal makes a video call with a second terminal, and the first video call interface includes a full-screen window and a single small window. The first terminal obtains a screenshot of the first video call interface, and the first terminal compares the screenshot with the frame image obtained by the first terminal. The image similarity is 0.8, and the target threshold is set to 0.6. The first terminal determines the small window as the opposite-end video window.

[0090] Step 504: When the picture similarity is less than the similarity threshold, the full-screen window is determined as the peer video window.

[0091] In a possible implementation, the first terminal calculates the image similarity between the window image corresponding to the full-screen window and the frame image, and when the image similarity is lower than the target threshold, the first terminal determines the small window as the peer video window. For example, the first terminal makes a video call with a second terminal, and the first video call interface includes a full-screen window and a single small window. The first terminal obtains a screenshot of the first video call interface, and compares the screenshot with the frame image obtained by the first terminal. The image similarity is 0.4, and the target threshold is set to 0.6. The first terminal determines the full-screen window as the peer video window.

[0092] Step 505: extract the video stream of the peer video window in the first video call interface to obtain the peer video stream.

[0093] In a possible implementation, the video stream extraction method for the peer video window in the first video call interface may be obtained in the following manner:

[0094] 1. When the video window of the other end is a full-screen window and there is a small window on the upper layer of the full-screen window, the full-screen window is extracted to obtain the full-screen window image.

[0095] In some embodiments, when there is a small window on the upper layer of the full-screen window, the full-screen window image extracted by the first terminal includes not only the video window image displayed in the full-screen window, but also the video window image displayed in the small window.

[0096] 2. Perform image processing on the image area corresponding to the small window in the full-screen window image. The image processing is used to eliminate other video images in the full-screen window image.

[0097] In some embodiments, since the full-screen window image includes both the video window image displayed in the full-screen window and the video window image displayed in the small window, when the full-screen window is a window of the other end, in order to improve the display effect of the video stream of the other end, the first terminal eliminates other video images displayed in the corresponding area of ​​the small window above the full-screen window.

[0098] In a possible implementation, Figure 6 As shown, it shows a schematic diagram of picture processing provided by an exemplary embodiment of the present application, including a full-screen window 610 and a small window 620. The first terminal performs blur processing on the picture area corresponding to the small window, for example, the first terminal processes the picture area corresponding to the small window 620 through the blur filter of FFMpeg, crops the video picture corresponding to the picture area of ​​the small window to obtain the small window picture, blurs the small window picture, and then pastes the blurred video picture onto the corresponding picture area of ​​the small window. At this time, the full-screen window displays the video picture of the other end, and the small window on the upper layer of the full-screen window displays the blurred picture.

[0099] In another possible implementation, the first terminal can also perform AI completion on the picture area corresponding to the small window. For example, the video picture in the small window picture area is first cropped or blurred, and based on the machine learning image completion algorithm, the information of the missing pixels is predicted through the pixels near the small window area and the partial image pixels displayed in the full-screen window. In this way, while eliminating other video pictures corresponding to the small window area, the video picture displayed in the full-screen window corresponding to the video stream of the other end is completed.

[0100] 3. Generate peer video stream based on the full-screen window image after image processing.

[0101] In a possible implementation, the full-screen window image processed by the first terminal does not include other video information, and the opposite-end video image information included in the opposite-end video stream generated based on the full-screen window image after the image processing is more accurate.

[0102] In summary, in the embodiment of the present application, when a local user and a single opposite user conduct a video call, the first video call interface may include a full-screen window and a single small window, and the full-screen window and the single small window are used to display the video call content between the local user and the opposite user. The first terminal can determine the window that currently displays the video screen corresponding to the opposite user by comparing the screen similarity between the screenshot screen and the window screen corresponding to the full-screen window, thereby extracting the opposite video stream. Moreover, when the full-screen window is the opposite video window, the first terminal first processes the screen displayed in the full-screen window, and before extracting the opposite video stream, eliminates the video screen corresponding to the local user in the opposite video stream. The opposite video stream extracted by the first terminal only includes the video screen corresponding to the opposite user, thereby improving the extraction quality of the opposite video stream.

[0103] During a video call, after obtaining the local video stream and at least one other video stream, the first terminal needs to process the local video stream and at least one other video stream to obtain a target video stream that meets the display requirements of the third terminal, so that the third terminal can subsequently display the target video call interface based on the target video stream. Please refer to Figure 7 , which shows a flowchart of a video stream processing method provided by an exemplary embodiment of the present application, the method comprising:

[0104] Step 701 : based on the local screen size information and the opposite screen size information in the display configuration information, the local video stream and the opposite video stream are cropped or scaled.

[0105] In some embodiments, the display configuration information includes local screen size information and opposite end screen size information, which represents the third terminal's requirements for the local video screen display size and the opposite end video screen display size. However, the local video stream and the opposite end video stream obtained by the first terminal may not meet the local video screen display size and the opposite end video screen display size requirements, so the local video stream and the opposite end video stream need to be cropped or scaled.

[0106] In one possible implementation, since video is a time series of images, video scaling is the scaling of each frame of the image, which is essentially image scaling. In order to prevent the image from being distorted during the scaling process, a pixel map is used for scaling, and the nearest neighbor interpolation algorithm, linear interpolation algorithm, bilinear interpolation algorithm and other methods are combined to achieve size scaling of the local video stream and the peer video stream.

[0107] In one possible implementation, video is a time sequence of images, so video cropping is to crop each frame of the image, which is essentially image cropping. The first terminal can perform AI recognition on the images corresponding to the local video stream and the opposite video stream, and identify the face areas of the local user and the opposite user. The first terminal crops the local video stream and the opposite video stream based on the identified face areas and display configuration information, so that the cropped local video stream and the opposite video stream can retain the user's facial information.

[0108] Step 702: Based on the local screen position information and the opposite screen position information in the display configuration information, the local video stream and the opposite video stream after the size scaling processing are synthesized to obtain a target video stream.

[0109] In some embodiments, the display configuration information includes the position information of the local screen and the position information of the opposite screen, which represent the positions of the local video screen and the opposite video screen in the display interface of the third terminal. After the first terminal scales the local video stream and the opposite video stream obtained by the first terminal, the first terminal needs to synthesize the two streams into one video stream (target video stream) according to the corresponding position information of the two, so that the target video screen corresponding to the synthesized target video stream includes the local video screen and the opposite video screen, and the display position of the local video screen and the opposite video screen meet the display requirements of the third terminal.

[0110] To summarize, in the embodiment of the present application, based on the display configuration requirements of the third terminal, the first terminal needs to process the local video stream and the opposite video stream, adjust the screen sizes corresponding to the local video stream and the opposite video stream respectively, scale the local video screen and the opposite video screen, and combine the local video screen and the opposite video screen to generate the opposite video stream. Since the target video stream meets the display requirements of the third terminal, the subsequent third terminal can display the target video screen with better effect based on the target video stream.

[0111] In a possible application scenario, after obtaining the local video stream and at least one opposite video stream, the first terminal can send the local video stream and at least one opposite video stream to a third terminal, so that the third terminal processes the two based on its own display configuration information to obtain the target video stream.

[0112] In some embodiments, the first terminal sends a local video stream and at least one opposite video stream to a third terminal. The third terminal processes the local video stream and the opposite video stream based on display configuration information to obtain a target video stream, and displays a second video call interface based on the target video stream.

[0113] In a possible implementation, the format of the media file corresponding to the local video stream supports streaming transmission and playback, for example, the media format of the file corresponding to the target video stream is RM, MOV, ASF, SWF, etc. The first terminal uses streaming transmission to send the local video stream and the opposite video stream to the third terminal. The third terminal processes the local video stream and the opposite video stream based on the display configuration information, and the process of obtaining the target video stream can refer to the above steps 701 and 702, which will not be repeated here.

[0114] In the above embodiment, the first terminal sends the local video stream and at least one opposite video stream to the third terminal, so that the third terminal generates a target video stream based on the local video stream and the opposite video stream, thereby displaying the second video call interface, making the third terminal more flexible in obtaining the target video stream.

[0115] During a video call, after the first terminal sends a target video stream to the third terminal so that the third terminal displays a second video call interface based on the target video stream, the user may switch the display windows of the local video screen and the opposite video screen in the second video call interface on the third terminal through the first video call interface displayed on the first terminal according to his or her own needs. Therefore, when receiving a window switching operation in the first video call interface, the first terminal needs to re-execute the step of extracting the video stream from the first video call interface to obtain the local video stream and at least one opposite video stream.

[0116] In some embodiments, the window switching operation in the first video call interface received by the first terminal may be a trigger operation performed by the user on the first terminal, and optionally, the trigger operation includes but is not limited to gesture triggering, voice triggering, and clicking on the display windows of the local video screen and the opposite video screen. Optionally, the first video call interface may also include a window switching control, and the window switching operation may be triggered by the user clicking on the window switching control.

[0117] In another possible application scenario, after the first terminal sends the target video stream to the third terminal so that the third terminal displays the second video call interface based on the target video stream, the user can switch the display windows of the local video screen and the opposite video screen in the second video call interface on the third terminal according to their own needs through the second video call interface displayed on the third terminal. Therefore, when the first terminal receives the window switching instruction sent by the third terminal, the first terminal needs to re-execute the step of extracting the video stream from the first video call interface to obtain the local video stream and at least one opposite video stream.

[0118] In some embodiments, when the third terminal receives a window switching operation within the second video call interface, the first terminal receives a window switching instruction sent by the third terminal; based on the window switching instruction, the first terminal re-executes the step of extracting the video stream from the first video call interface to obtain the local video stream and at least one opposite-end video stream.

[0119] The window switching operation received by the third terminal may be a trigger operation performed by the user on the third terminal, and optionally, the trigger operation includes but is not limited to gesture triggering, voice triggering, and click triggering on the display windows of the local video screen and the opposite video screen. The second video call interface may also include a window switching control, and the window switching operation may be triggered by the user clicking on the window switching control.

[0120] In the above embodiment, after the first terminal sends the target video stream to the third terminal so that the third terminal displays the second video call interface based on the target video stream, the user can switch windows through the first video call interface displayed on the first terminal or through the second video call interface displayed on the third terminal as needed, change the display windows of the local video screen and the other party video screen in the second video call interface on the third terminal, and increase the flexibility of displaying the video call content.

[0121] In a possible application scenario, after the first terminal sends a target video stream to the third terminal so that the third terminal displays a second video call interface based on the target video stream, the user may adjust the second video call interface displayed by the third terminal according to his or her own needs, such as readjusting the display window sizes of the local video screen and the other party's video screen, or readjusting the position of the video window in the second video display interface. Therefore, the third terminal updates the display configuration information according to the needs, and the first terminal re-acquires the target video stream based on the updated display configuration information.

[0122] Please refer to Figure 8 , which shows a flowchart of generating a target video stream after a configuration update provided by an exemplary embodiment of the present application, the method comprising:

[0123] Step 801: receiving a display configuration update instruction sent by a third terminal, wherein the display configuration update instruction includes updated display configuration information, and the display configuration update instruction is triggered when an interface update instruction for a second video call interface is received.

[0124] In some embodiments, the updated display configuration information may include: size update information of the local video screen and the opposite video screen and position update information of the local video screen and the opposite video screen, which respectively represent the screen display size requirements and screen display position requirements of the local video screen and the opposite video screen, the size update information includes the local video screen update information and the opposite video screen size update information, and the position update information includes the local video screen update information and the opposite video screen position update information.

[0125] Step 802: Process the local video stream and the peer video stream based on the updated display configuration information to obtain a target video stream.

[0126] In some embodiments, the first terminal performs size scaling processing on the local video stream and the opposite video stream based on the local video screen update information and the opposite video screen size update information in the updated display configuration information; based on the local screen position information and the opposite screen position information in the updated display configuration information, the first terminal performs synthesis processing on the local video stream and the opposite video stream after the size scaling processing to obtain the target video stream.

[0127] In a possible application scenario, such as Fig. 9 As shown, it shows a schematic diagram of updating the second video call interface provided by an exemplary embodiment of the present application. The third terminal is a vehicle device. The user uses the video call function during driving, and the second video call interface is displayed on the vehicle device. The display effect before the second video call interface is updated is as follows Fig. 9As shown in (a), the second video call interface 910 is included. At this time, the second video call interface 910 is a full-screen interface. After the vehicle device displays the second video call interface, due to the user's navigation needs, it is necessary to display the navigation screen and the video call interface on the vehicle device at the same time. Therefore, the second video call interface displayed on the third terminal is adjusted according to the user's needs. The effect of the updated second video call interface is shown in 9(b), including the second video call interface 910 and the navigation interface 920. Among them, the target video screen in the second video call interface 910 is smaller than that in the vehicle device. Fig. 9 (a), it can be seen that the sizes and positions of the local video screen and the opposite video screen in the target video screen are adjusted.

[0128] In the above embodiment, after the first terminal sends the target video stream to the third terminal so that the third terminal displays the second video call interface based on the target video stream, the first terminal receives the display configuration update instruction sent by the third terminal, and uses the updated display configuration information to process the local video stream and the opposite video stream to obtain the target video stream, so that the third terminal re-displays the second video call interface based on the target video stream to meet the user's need to adjust the display effect of the second video call interface, making the video call method applicable to more scenarios, thereby increasing the universality of the video call method.

[0129] In a possible application scenario, before a first terminal makes a video call with a second terminal, the first terminal may generate a call prompt message based on a video call answering request sent by the second terminal, and the first terminal sends the call prompt message to a third terminal so that a call prompt pop-up window is displayed on a device interface of the third terminal.

[0130] In some embodiments, the call prompt information may include identity information of a peer user using the second terminal. The first terminal sends the identity information of the peer user to the third terminal so that the third terminal displays a call prompt pop-up window, wherein the call prompt pop-up window is used to display the identity information of the peer user.

[0131] In some embodiments, the call prompt pop-up window may include an answer control and a hang-up control. After the first terminal sends the call prompt information to the third terminal so that the device interface of the third terminal displays the call prompt pop-up window, when the first terminal receives the answer instruction, the first terminal extracts the video stream from the first video call interface to obtain the local video stream and at least one opposite video stream. The answer instruction is issued by the third terminal when a trigger operation is received for the answer control. Correspondingly, when the first terminal receives the hang-up instruction, the first terminal ends the video call, and the hang-up instruction is issued by the third terminal when a trigger operation is received for the hang-up control.

[0132] Optionally, the first terminal may generate call prompt information based on the video call answering request sent by the second terminal, and display a call prompt pop-up window on the device interface of the first terminal. Accordingly, after the first terminal displays the call prompt pop-up window based on the call prompt information, when the first terminal receives the answering instruction, the first terminal extracts the video stream from the first video call interface to obtain the local video stream and at least one opposite video stream. The answering instruction is issued by the first terminal when a trigger operation is received for the answering control. Accordingly, when the first terminal receives a hang-up instruction, the first terminal ends the video call. The hang-up instruction is issued by the first terminal when a trigger operation is received for the hang-up control.

[0133] like Fig.10 As shown, it shows a schematic diagram of answering prompt provided by an exemplary embodiment of the present application, including: a device interface 1010 of a third terminal and a video call answering pop-up window 1020, wherein an answering control 1030 and a hang-up control 1040 are arranged on the upper layer of the video call answering pop-up window 1020. Before the first terminal makes a video call with the second terminal, the first terminal can generate a call prompt message according to the video call answering request sent by the second terminal, and send the call prompt message to the third terminal so that the device interface 1010 of the third terminal displays the video call answering pop-up window 1020. When the user triggers the answering control 1030, the first terminal receives the answering instruction sent by the third terminal, and the first terminal extracts the video stream of the first video call interface. When the user triggers the hang-up control 1040, the first terminal receives the hang-up instruction sent by the third terminal, and the first terminal ends the video call with the second terminal.

[0134] The following describes an embodiment of a video call method of the present application with the third terminal as the execution subject. Fig.11 As shown, it shows a flowchart of a video calling method provided by another exemplary embodiment of the present application, the method comprising:

[0135] Step 1101: Provide display configuration information of a third terminal to a first terminal.

[0136] In some embodiments, upon receiving a display configuration request from the first terminal, the third terminal sends display configuration information to the first terminal, where the display configuration information includes the video screen size information of the local terminal, the video screen size information of the opposite terminal, the video screen position information of the local terminal, and the screen position information of the opposite terminal. Optionally, the display configuration information may also include other display requirement information, such as the video screen resolution of the local terminal and the video screen resolution of the opposite terminal.

[0137] Step 1102, receiving a target video stream sent by the first terminal, the target video call screen corresponding to the target video stream meets the display requirements represented by the display configuration information, the target video stream is obtained by the first terminal processing the local video stream and the opposite video stream based on the display configuration information, the local video stream and the opposite video stream are obtained by the first terminal extracting the video stream of the first video call interface, the target video call screen includes the local video screen and the opposite video screen, and the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal.

[0138] In some embodiments, the first terminal and the third terminal are interconnected using a communication protocol to support video data transmission between the first terminal and the third terminal. For example, when the first terminal is a smart phone and the third terminal is a vehicle-mounted device, the first terminal and the third terminal can transmit video data through mobile-vehicle interconnection protocols such as Carlink, Carlife, CarPlay, and HiCar.

[0139] In a possible implementation, the third terminal receives the target video stream sent by the first terminal in a streaming manner, wherein the format of the media file corresponding to the target video stream supports streaming and playback, for example, the media format of the file corresponding to the target video stream is RM, MOV, ASF, SWF, etc. Using streaming, the third terminal receives the target video stream sent by the first terminal.

[0140] Step 1103: Display a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

[0141] In some embodiments, since the target video stream is obtained by processing the local video stream and the opposite video stream based on the display configuration information of the third terminal, the target video call screen and the target video screen corresponding to the target video stream meet the display requirements represented by the display configuration information, and the screen size and screen position of the local video screen and the opposite video screen in the target video screen meet the display requirements of the third terminal.

[0142] Please refer to Fig.12 , which shows a structural block diagram of a video call device provided by an exemplary embodiment of the present application, the device includes:

[0143] The video stream extraction module 1201 is used to extract the video stream from the first video call interface to obtain the local video stream and at least one peer video stream, wherein the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal;

[0144] The video stream processing module 1202 is used to process the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream, wherein the target video call screen corresponding to the target video stream includes the local video screen and the opposite video screen, and the target video screen meets the display requirements represented by the display configuration information;

[0145] The first video stream sending module 1203 is used to send the target video stream to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, and the second video call interface includes the target video call screen.

[0146] Optionally, the video stream extraction module 1201 is further used for:

[0147] Determine a video stream captured by a camera of the first terminal or the third terminal as the local video stream;

[0148] At least one of the opposite-end video streams is extracted from the first video call interface based on the local-end video stream.

[0149] Optionally, the video stream extraction module 1201 is further used for:

[0150] Obtaining a screenshot of the first video call interface;

[0151] Performing a screen extraction operation on the screenshot screen to obtain a window screen corresponding to each video call window in the first video call interface;

[0152] Determining the peer video window in the video call window based on the picture similarity between the local video picture corresponding to the local video stream and the window picture;

[0153] The video stream of the peer video window in the first video call interface is extracted to obtain the peer video stream.

[0154] Optionally, the video stream extraction module 1201 is further used for:

[0155] Recording the first video call interface to obtain a recorded screen;

[0156] Intercepting the opposite end picture in the opposite end video window from the screen recording picture;

[0157] The peer video stream is generated based on the continuously captured peer screens. Optionally, the first video call interface includes a full-screen window and a single small window, the small window is located on an upper layer of the full-screen window, and the video stream extraction module 1201 is further used to:

[0158] Determine the picture similarity between the local video picture and the window picture corresponding to the full-screen window;

[0159] In the case where the picture similarity is greater than a similarity threshold, determining the small window as the peer video window;

[0160] In the case that the picture similarity is less than a similarity threshold, the full-screen window is determined as the peer-end video window.

[0161] Optionally, the video stream extraction module 1201 is further used for:

[0162] When the opposite end video window is a full-screen window and there is a small window on the upper layer of the full-screen window, extracting the picture of the full-screen window to obtain the full-screen window picture;

[0163] Performing image processing on the image area corresponding to the small window in the full-screen window image, wherein the image processing is used to eliminate other video images in the full-screen window image;

[0164] The peer video stream is generated based on the full-screen window image after image processing.

[0165] Optionally, the video stream processing module 1202 is further configured to:

[0166] Based on the local end screen size information and the opposite end screen size information in the display configuration information, the local end video stream and the opposite end video stream are subjected to size cropping or scaling processing;

[0167] Based on the local screen position information and the opposite screen position information in the display configuration information, the local video stream and the opposite video stream after the size scaling process are synthesized to obtain the target video stream.

[0168] Optionally, the device further includes a window switching module, configured to:

[0169] When a window switching operation is received in the first video call interface, the step of extracting the video stream from the first video call interface is re-executed to obtain the local video stream and at least one opposite video stream.

[0170] Optionally, the device further includes a configuration update module, configured to:

[0171] receiving a display configuration update instruction sent by the third terminal, wherein the display configuration update instruction includes the updated display configuration information, and the display configuration update instruction is triggered when an interface update instruction for the second video call interface is received;

[0172] The local video stream and the opposite video stream are processed based on the updated display configuration information to obtain the target video stream.

[0173] Optionally, the device further includes a second video stream sending module, configured to:

[0174] The local video stream and at least one opposite-end video stream are sent to the third terminal, and the third terminal is used to process the local video stream and the opposite-end video stream based on the display configuration information to obtain the target video stream, and display the second video call interface based on the target video stream.

[0175] Please refer to Fig.13 , which shows a structural block diagram of a video call device provided by another exemplary embodiment of the present application, the device comprising:

[0176] The configuration information sending module 1301 provides the display configuration information of the third terminal to the first terminal;

[0177] The video stream receiving module 1302 receives a target video stream sent by the first terminal, the target video call screen corresponding to the target video stream meets the display requirements represented by the display configuration information, the target video stream is obtained by the first terminal processing the local video stream and the opposite video stream based on the display configuration information, the local video stream and the opposite video stream are obtained by the first terminal performing video stream extraction on the first video call interface, the target video call screen includes the local video screen and the opposite video screen, and the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal;

[0178] The interface display module 1303 displays a second video call interface based on the target video stream, where the second video call interface includes the target video call screen.

[0179] Optionally, the device further includes an update instruction sending module, configured to:

[0180] When receiving the interface update instruction for the second video call interface, a display configuration update instruction is sent to the first terminal, so that the first terminal processes the local video stream and the opposite video stream based on the updated display configuration information to obtain the target video stream.

[0181] In summary, in the embodiment of the present application, by extracting the video stream from the first video call interface, the local video stream and at least one opposite video stream can be obtained, and the local video stream and the opposite video stream are processed using the display configuration information of the third terminal, so as to obtain the target video stream that meets the display requirements of the third terminal; the first terminal sends the target video stream to the interconnected third terminal, so that the third terminal displays the second video call interface, which includes the target video call screen. Since the target video stream meets the display requirements of the third terminal, the display effect of the video call content is better. This allows the interconnected terminal (third terminal) of the first terminal to display a video call interface with a better effect.

[0182] It should be noted that: the device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0183] Please refer to Fig.14 , which shows a block diagram of a terminal provided by an exemplary embodiment of the present application. The terminal 1400 may be a smart phone, a tablet computer, a wearable device, etc. The terminal 1400 in the present application may include one or more of the following components: a processor 1410, a memory 1420, a display screen 1430, and a camera 1440.

[0184] The processor 1410 may include one or more processing cores. The processor 1410 uses various interfaces and lines to connect the various parts of the entire terminal 1400, and executes various functions and processes data of the terminal 1400 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1420, and calling data stored in the memory 1420. Optionally, the processor 1410 can be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 1410 can integrate one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display 1430; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1410, but may be implemented by a separate chip.

[0185] The memory 1420 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1420 includes a non-transitory computer-readable storage medium. The memory 1420 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1420 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as image data, audio data), etc., created according to the use of the terminal 1400.

[0186] The display screen 1430 is a component for displaying images. The display screen 1430 can be a full screen, a special-shaped screen, a double-sided screen, a folding screen, a flexible screen, etc., which is not limited in this embodiment. In addition to having a display function, the display screen 1430 can also have a touch function, that is, the display screen 1430 is a touch display screen.

[0187] The camera 1440 is a component for collecting video streams. Optionally, the camera 1440 can be an RGB camera or a depth camera. In the embodiment of the present application, cameras 1440 (i.e., a front camera and a rear camera) are provided on both sides of the display screen 1430, and the camera 1440 is used to collect video streams on either side of the display screen 1430.

[0188] In addition, those skilled in the art can understand that the structure of the terminal 1400 shown in the above drawings does not constitute a limitation on the terminal 1400, and the terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, the terminal 1400 also includes a radio frequency circuit, a sensor, an audio circuit, a wireless fidelity (WiFi) component, a power supply, a Bluetooth component, and other components, which will not be described in detail here.

[0189] An embodiment of the present application further provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is loaded and executed by the processor to implement the video call method in the virtual relationship development application as described in the above embodiments.

[0190] The embodiment of the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the video call method described in the above embodiment.

[0191] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0192] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A video calling method, characterized in that: The method is used for a first terminal, and the method includes: Extracting a video stream from a first video call interface to obtain a local video stream and at least one peer video stream, wherein the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal; Processing the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream, wherein a target video call screen corresponding to the target video stream includes a local video screen and an opposite video screen, and the target video screen meets the display requirements represented by the display configuration information; The target video stream is sent to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

2. The method according to claim 1, characterized in that The step of extracting the video stream from the first video call interface to obtain the local video stream and at least one peer video stream includes: Determine a video stream captured by a camera of the first terminal or the third terminal as the local video stream; At least one of the opposite-end video streams is extracted from the first video call interface based on the local-end video stream.

3. The method according to claim 2, characterized in that The extracting at least one peer video stream from the first video call interface based on the local video stream includes: Obtaining a screenshot of the first video call interface; Performing a screen extraction operation on the screenshot screen to obtain a window screen corresponding to each video call window in the first video call interface; Determining the peer video window in the video call window based on the picture similarity between the local video picture corresponding to the local video stream and the window picture; The video stream of the peer video window in the first video call interface is extracted to obtain the peer video stream.

4. The method according to claim 3, characterized in that The extracting the video stream of the peer video window in the first video call interface to obtain the peer video stream includes: Recording the first video call interface to obtain a recorded screen; Intercepting the opposite end picture in the opposite end video window from the screen recording picture; The peer-end video stream is generated based on the continuously captured peer-end images.

5. The method according to claim 3, characterized in that: The first video call interface includes a full-screen window and a single small window, and the small window is located on an upper layer of the full-screen window; The determining the opposite-end video window in the video call window based on the picture similarity between the local-end video picture corresponding to the local-end video stream and the window picture includes: Determine the picture similarity between the local video picture and the window picture corresponding to the full-screen window; In the case where the picture similarity is greater than a similarity threshold, determining the small window as the peer video window; In the case that the picture similarity is less than a similarity threshold, the full-screen window is determined as the peer-end video window.

6. The method according to claim 3, characterized in that The extracting the video stream of the peer video window in the first video call interface to obtain the peer video stream includes: When the opposite end video window is a full-screen window and there is a small window on the upper layer of the full-screen window, extracting the picture of the full-screen window to obtain the full-screen window picture; Performing image processing on the image area corresponding to the small window in the full-screen window image, wherein the image processing is used to eliminate other video images in the full-screen window image; The peer video stream is generated based on the full-screen window image after image processing.

7. The method according to claim 1, characterized in that The processing of the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream includes: Based on the local end screen size information and the opposite end screen size information in the display configuration information, the local end video stream and the opposite end video stream are subjected to size cropping or scaling processing; Based on the local screen position information and the opposite screen position information in the display configuration information, the local video stream and the opposite video stream after the size scaling process are synthesized to obtain the target video stream.

8. The method according to claim 1, characterized in that The method also includes: when a window switching operation is received in the first video call interface, re-executing the step of extracting the video stream from the first video call interface to obtain a local video stream and at least one opposite video stream.

9. The method according to claim 1, characterized in that: The method further comprises: receiving a display configuration update instruction sent by the third terminal, wherein the display configuration update instruction includes the updated display configuration information, and the display configuration update instruction is triggered when an interface update instruction for the second video call interface is received; The local video stream and the opposite video stream are processed based on the updated display configuration information to obtain the target video stream.

10. The method according to claim 1, characterized in that The first terminal is a mobile terminal, and the third terminal is a vehicle-mounted terminal.

11. The method according to claim 1, characterized in that: The method further comprises: The local video stream and at least one opposite-end video stream are sent to the third terminal, and the third terminal is used to process the local video stream and the opposite-end video stream based on the display configuration information to obtain the target video stream, and display the second video call interface based on the target video stream.

12. A video calling method, characterized in that: The method is used for a third terminal, and the method includes: Providing display configuration information of the third terminal to the first terminal; Receive a target video stream sent by the first terminal, the target video call screen corresponding to the target video stream meets the display requirements represented by the display configuration information, the target video stream is obtained by the first terminal processing the local video stream and the opposite video stream based on the display configuration information, the local video stream and the opposite video stream are obtained by the first terminal performing video stream extraction on a first video call interface, the target video call screen includes the local video screen and the opposite video screen, and the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal; A second video call interface is displayed based on the target video stream, wherein the second video call interface includes the target video call screen.

13. The method according to claim 12, characterized in that The method further comprises: When receiving the interface update instruction for the second video call interface, a display configuration update instruction is sent to the first terminal, so that the first terminal processes the local video stream and the opposite video stream based on the updated display configuration information to obtain the target video stream.

14. A video call device, characterized in that: The device comprises: A video stream extraction module, configured to extract a video stream from a first video call interface to obtain a local video stream and at least one peer video stream, wherein the first video call interface is an interface for a video call between the first terminal and at least one second terminal; A video stream processing module, configured to process the local video stream and the opposite video stream based on the display configuration information of the third terminal to obtain a target video stream, wherein the target video call screen corresponding to the target video stream includes the local video screen and the opposite video screen, and the target video screen meets the display requirements represented by the display configuration information; The video stream sending module sends the target video stream to the third terminal, so that the third terminal displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

15. A video call device, characterized in that: The device comprises: a configuration information sending module, providing display configuration information of the third terminal to the first terminal; a video stream receiving module for receiving a target video stream sent by the first terminal, wherein the target video stream corresponds to a target video call screen that meets the display requirements represented by the display configuration information, wherein the target video stream is obtained by the first terminal processing a local video stream and a peer video stream based on the display configuration information, wherein the local video stream and the peer video stream are obtained by the first terminal performing video stream extraction on a first video call interface, wherein the target video call screen includes a local video screen and a peer video screen, and wherein the first video call interface is an interface for the first terminal to conduct a video call with at least one second terminal; The interface display module displays a second video call interface based on the target video stream, wherein the second video call interface includes the target video call screen.

16. A terminal, characterized in that: The terminal includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the video calling method as described in any one of claims 1 to 11, or the video calling method as described in any one of claims 12 to 13.

17. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the video calling method according to any one of claims 1 to 11, or the video calling method according to any one of claims 12 to 13.

18. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device implements the video call method as described in any one of claims 1 to 11, or the video call method as described in any one of claims 12 to 13.

Images