A video call method and display device

Abstract

The application provides a video call method and a display device. The method comprises: receiving a push message sent by a server, wherein the push message comprises a second session ID, and the push message is sent by the server after receiving a second video call request sent by a second mobile terminal; and sending a second call success message comprising a first session ID to the server, so that the second mobile terminal performs a call through a room mapped by the first session ID and the display device, wherein the first session ID corresponds to a video call being performed by the display device when the push message is received.

Description

[0001] This application is a divisional application of Chinese patent application filed on February 28, 2020, with application number 202010129731.1 and title "A video call method and display device".

[0002] Chinese patent application No. 202010129731.1 claims priority to Chinese patent application No. 201910502717.9 filed with the Chinese Patent Office on June 11, 2019, entitled "A video call method and display device". Technical Field

[0003] This application relates to display technology. More specifically, it relates to a video call method and a display device. Background Technology

[0004] Currently, camera-equipped devices have made social interactions more interconnected. When one device initiates a video call, the server forwards the call request to the other device. The other device then clicks a control to indicate acceptance of the call request, and the two devices can then establish a video call. It is evident that the current video call setup process requires user intervention on the other device, making the process cumbersome. Summary of the Invention

[0005] In a first aspect, this application provides a video call method, the method comprising: receiving a push message sent by a server, wherein the push message includes a second session ID, and the push message is sent by the server after receiving a second video call request sent by a second mobile terminal; when the display device is not conducting a video call, determining whether the camera corresponding to the display device is occupied; and when the camera is not occupied, automatically establishing a video call with the second mobile terminal based on the second session ID.

[0006] Secondly, this application provides a video call method, the method comprising: receiving a push message sent by a server, wherein the push message includes a second session ID, and the push message is sent by the server after receiving a second video call request sent by a second mobile terminal; determining whether a camera corresponding to a display device is occupied; automatically establishing a video call with the second mobile terminal when the camera is not occupied; and feeding back a message indicating that the camera is occupied to the second mobile terminal when the camera is occupied by an application other than the video call application.

[0007] Thirdly, this application provides a display device including a controller for performing the method described in any of the above-mentioned methods. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 The diagram illustrates an operational scenario between a display device and a control device according to an embodiment.

[0010] Figure 2 The diagram illustrates, by way of example, a hardware configuration block diagram of the control device 100 according to an embodiment;

[0011] Figure 3 The diagram illustrates, by way of example, a hardware configuration block diagram of a display device 200 according to an embodiment;

[0012] Figure 4 The example shown is based on Figure 3 Hardware architecture block diagram of display device 200;

[0013] Figure 5 The diagram above exemplarily illustrates a functional configuration of a display device 200 according to an embodiment;

[0014] Figure 6a The diagram above exemplarily illustrates a software configuration schematic of a display device 200 according to an embodiment;

[0015] Figure 6b The diagram illustrates, by way of example, a configuration diagram of an application in a display device 200 according to an embodiment;

[0016] Figure 7 The diagram illustrates, by way of example, a user interface in a display device 200 according to an embodiment;

[0017] Figure 8a The diagram above illustrates a user interface after the display device automatically establishes a call process, according to one embodiment.

[0018] Figure 8b The example shown in the middle Figure 8a A schematic diagram of the user interface of the mobile terminal after the display device automatically establishes a call process under the described embodiment;

[0019] Figure 8c The example shown in the middle Figure 8a A schematic diagram of the user interface in which the display device switches from a small window display to a full-screen display under the described embodiment;

[0020] Figure 9a The diagram above illustrates a user interface after the display device automatically establishes a call process, according to another implementation method.

[0021] Figure 9b The example shown in the middle Figure 9a A schematic diagram of the user interface of the mobile terminal after the display device automatically establishes a call process under the described embodiment;

[0022] Figure 10a The diagram illustrates a process flow under one implementation method.

[0023] Figure 10b The diagram above illustrates a flowchart of the process before sending an authentication request in another implementation method.

[0024] Figure 10c The diagram illustrates, exemplarily, the process after sending an authentication request under another implementation method; Figure 11 This is a schematic diagram illustrating the structure of a video call system according to some exemplary embodiments;

[0025] Figure 12 This is a flowchart of a video call method provided in one embodiment of this application;

[0026] Figure 13 This is a flowchart of a video call method provided in another embodiment of this application;

[0027] Figure 14 This is a flowchart of a video call method provided in one embodiment of this application;

[0028] Figure 15 This is a flowchart of a video call method provided in another embodiment of this application;

[0029] Figure 16 This is a structural block diagram of a video call device provided in one embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0031] Video calling is an important means of social interaction. Once a video call process is established between two devices, a video call can be made. The device that initiates the video call request is the initiating device, and the device that receives the video call request is the invited device. The video calling method provided in this application is not limited by hardware and can be used in devices without dual-chip systems as well as in dual-chip dual-system devices. For ease of understanding of dual-chip dual-system devices, a detailed description of dual-chip dual-system devices is provided.

[0032] The concepts involved in this application will first be explained with reference to the accompanying drawings. It should be noted that the following explanation of each concept is only to make the content of this application easier to understand and does not imply any limitation on the scope of protection of this application.

[0033] The term "module" as used in some embodiments of this application may refer to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0034] The term "remote control" as used in some embodiments of this application refers to a component of an electronic device (such as the display device disclosed in this application) that can wirelessly control the electronic device over a relatively short distance. This component typically connects to the electronic device using infrared and / or radio frequency (RF) signals and / or Bluetooth, and may also include functional modules such as WiFi, wireless USB, Bluetooth, and motion sensors. For example, a handheld touch remote control replaces most of the physical built-in hard buttons in a typical remote control device with a user interface on a touchscreen.

[0035] The term "gesture" as used in some embodiments of this application refers to user behavior that uses a change in hand shape or hand movement to express an expected idea, action, purpose, and / or result.

[0036] The term "hardware system" as used in some embodiments of this application can refer to a physical component with computing, control, storage, input, and output functions, composed of mechanical, optical, electrical, and magnetic devices such as integrated circuits (ICs) and printed circuit boards (PCBs). In various embodiments of this application, the hardware system is also commonly referred to as a motherboard or chip.

[0037] The term "operating system" as used in some embodiments of this application may refer to a computer system presented to the user after the processor reads code from the memory, such as "Android OS", "Mac OS", "Windows OS", etc.

[0038] Figure 1 The diagram illustrates an operational scenario between a display device and a control device according to an embodiment. Figure 1 As shown, the user can operate the display device 200 through the control device 100.

[0039] The control device 100 can be a remote control 100A, which can communicate with the display device 200 via infrared, Bluetooth, ZigBee, or other short-range communication methods to control the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, or control panel input. For example, users can input corresponding control commands through volume up / down buttons, channel control buttons, up / down / left / right movement buttons, voice input buttons, menu buttons, and power buttons on the remote control to achieve the function of controlling the display device 200.

[0040] The control device 100 can also be a smart device, such as a mobile terminal 100B, tablet computer, computer, laptop computer, etc., which can communicate with the display device 200 through a local area network (LAN), wide area network (WAN), wireless local area network (WLAN), or other networks, and control the display device 200 through an application corresponding to the display device 200. For example, the display device 200 can be controlled using an application running on the smart device. This application can provide the user with various controls through an intuitive user interface (UI) on the screen associated with the smart device.

[0041] For example, both the mobile terminal 100B and the display device 200 can install software applications, enabling them to connect and communicate via network communication protocols, thereby achieving one-to-one control operations and data communication. For instance, the mobile terminal 100B and the display device 200 can establish a control command protocol, synchronizing the remote control keyboard to the mobile terminal 100B. By controlling the user interface on the mobile terminal 100B, the display device 200 can be controlled. Alternatively, audio and video content displayed on the mobile terminal 100B can be transmitted to the display device 200 for synchronized display.

[0042] like Figure 1As shown, the display device 200 can also communicate with the server 300 via various communication methods. In various embodiments of this application, the display device 200 may be allowed to communicate with the server 300 via a local area network, a wireless local area network, or other networks. The server 300 may provide the display device 200 with various content and interactive features.

[0043] For example, display device 200 interacts by sending and receiving information, as well as with an Electronic Program Guide (EPG), receiving software updates, or accessing a remotely stored digital media library. Server 300 can be a group or multiple groups, and can be one or more types of servers. Other network services, such as video-on-demand and advertising services, are provided through server 300.

[0044] The display device 200 can be, in one sense, a liquid crystal display (LCD), an OLED (Organic Light Emitting Diode) display, or a projection display device; in another sense, it can be a smart TV or a display system consisting of a monitor and a set-top box. The specific type, size, and resolution of the display device are not limited, but those skilled in the art will understand that the display device 200 can be modified in terms of performance and configuration as needed.

[0045] In addition to providing broadcast television reception functionality, display device 200 may also include smart network television functionality that provides computer support. Examples include IPTV, smart TV, Internet Protocol TV (IPTV), etc. In some embodiments, the display device may not have broadcast television reception functionality.

[0046] like Figure 1 The display device may be connected to or equipped with a camera to display the images captured by the camera on the display interface of this display device or other display devices, thereby enabling interactive chat between users. Specifically, the images captured by the camera can be displayed in full screen, half screen, or any selectable area on the display device.

[0047] As an optional connection method, the camera is connected to the back cover of the monitor via a connecting plate and is fixedly installed on the upper center of the back cover of the monitor. As an installable method, it can be fixedly installed in any position on the back cover of the monitor, as long as its image acquisition area is not blocked by the back cover. For example, the image acquisition area is oriented in the same direction as the display device.

[0048] As an alternative connection method, the camera can be raised and lowered to the display back cover via a connecting plate or other conceivable connector. The connector is equipped with a lifting motor. When the user wants to use the camera or an application needs to use the camera, it is raised above the display. When the camera is not in use, it can be recessed into the back cover to protect the camera from damage.

[0049] As one embodiment, the camera used in this application can be 16 megapixels to achieve ultra-high-definition display. In practical use, cameras with higher or lower resolutions than 16 megapixels can also be used.

[0050] When a camera is installed on a display device, the content displayed in different application scenarios can be integrated in various ways, thereby achieving functions that traditional display devices cannot achieve.

[0051] In some embodiments, a user can video chat with at least one other user while watching a video program. The video program can be presented as a background image, and the video chat window is displayed on top of the background image. Figuratively speaking, this feature can be called "watch and chat simultaneously".

[0052] In some embodiments, in a "watch and chat" scenario, at least one video chat can be conducted across devices while watching live or online videos.

[0053] In some embodiments, users can video chat with at least one other user while learning within an educational application. For example, students can interact remotely with teachers while studying content in an educational application. This feature can be figuratively called "learning while chatting."

[0054] In some embodiments, users can video chat with other players while playing card games. For example, players can remotely interact with other players when entering a game application to participate in a game. Figuratively speaking, this feature can be called "watch and play".

[0055] In some embodiments, the game scene and video footage are merged, and the portraits in the video footage are cut out and displayed in the game screen to improve the user experience.

[0056] In some embodiments, in motion-sensing games (such as ball games, boxing games, running games, dancing games, etc.), human postures and movements are acquired through cameras, limb detection and tracking, and detection of key points of human skeleton data are performed. These data are then integrated with animations in the game to realize games in scenarios such as sports and dance.

[0057] In some embodiments, users can interact with at least one other user via video and voice within a karaoke app. This feature can be figuratively called "singing while watching." Preferably, when at least one user enters the app in a chat context, multiple users can collaboratively record a song.

[0058] In some embodiments, users can open their local camera to capture images and videos; figuratively, this function can be called "looking in a mirror".

[0059] In some embodiments, additional functions may be added or the aforementioned functions may be removed. This application does not specifically limit the functions of the display device.

[0060] Figure 2 The diagram illustrates a configuration block diagram of the control device 100 according to an exemplary embodiment. Figure 3 As shown, the control device 100 includes a controller 110, a communicator 130, a user input / output interface 140, a memory 190, and a power supply 180.

[0061] The control device 100 is configured to control the display device 200 and to receive user input commands, converting the commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200. For example, if the user operates the channel up / down keys on the control device 100, the display device 200 will respond with the channel up / down operation.

[0062] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be equipped with various applications to control the display device 200 according to user needs.

[0063] In some embodiments, such as Figure 1 As shown, the mobile terminal 100B or other smart electronic devices can perform similar functions to the control device 100 after installing the application on the control display device 200. For example, users can install the application and use the various function keys or virtual buttons on the graphical user interface provided on the mobile terminal 100B or other smart electronic devices to achieve the functions of the physical buttons on the control device 100.

[0064] The controller 110 includes a processor 112, RAM 113 and ROM 114, a communication interface, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0065] Under the control of the controller 110, the communicator 130 enables communication of control signals and data signals with the display device 200. For example, it sends received user input signals to the display device 200. The communicator 130 may include at least one of the following communication modules: WIFI module 131, Bluetooth module 132, NFC module 133, etc.

[0066] User input / output interface 140, wherein the input interface includes at least one of the following: microphone 141, touchpad 142, sensor 143, button 144, etc. For example, the user can realize the user command input function through voice, touch, gesture, pressing and other actions. The input interface converts the received analog signal into a digital signal, and the digital signal into a corresponding command signal, and sends it to the display device 200.

[0067] The output interface includes an interface for sending received user commands to the display device 200. In some embodiments, this can be an infrared interface or a radio frequency (RF) interface. For example, with an infrared signal interface, the user input command needs to be converted into an infrared control signal according to an infrared control protocol, and then sent to the display device 200 via an infrared transmitting module. Similarly, with an RF signal interface, the user input command needs to be converted into a digital signal, then modulated according to an RF control signal modulation protocol, and finally sent to the display device 200 via an RF transmitting terminal.

[0068] In some embodiments, the control device 100 includes at least one of a communicator 130 and an output interface. The communicator 130, configured in the control device 100, may be a module such as WIFI, Bluetooth, or NFC, which can encode user input commands via WIFI, Bluetooth, or NFC protocols and send them to the display device 200.

[0069] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller 110. The memory 190 can also store various control signal commands input by the user.

[0070] The power supply 180 provides power to the various components of the control device 100 under the control of the controller 110. It may include a battery and related control circuitry.

[0071] Figure 3 The diagram illustrates a hardware configuration block diagram of the hardware system in the display device 200 according to an exemplary embodiment.

[0072] When adopting a dual hardware system architecture, the structural relationships of the hardware systems can be... Figure 3As shown. For ease of description, one hardware system in the dual hardware system architecture will be referred to as the first hardware system or system A, or chip A, and the other hardware system will be referred to as the second hardware system or system N, or chip N. Chip A includes its controller and various interfaces, while chip N includes its controller and various interfaces. Chip A and chip N can each have a relatively independent operating system installed. The operating systems of chip A and chip N can communicate with each other through a communication protocol. For example, the framework layer of the operating system of chip A and the framework layer of the operating system of chip N can communicate to transmit commands and data, thus creating two independent but interconnected subsystems in the display device 200.

[0073] like Figure 3 As shown, chip A and chip N can connect, communicate, and be powered through multiple different types of interfaces. The interface types between chip A and chip N can include at least one of the following: General-purpose input / output (GPIO), USB, HDMI, UART, etc. Chip A and chip N can use one or more of these interfaces for communication or power transfer. For example... Figure 3 As shown, in the dual hardware system architecture, the N chip can be powered by an external power supply, while the A chip can be powered by the N chip without an external power supply.

[0074] In addition to the interface used to connect to the N chip, the A chip may also include interfaces for connecting other devices or components, such as... Figure 3 The image shows the MIPI interface, Bluetooth interface, etc., used for connecting a camera.

[0075] Similarly, in addition to the interface for connecting to the N chip, the N chip may also include a VBY interface for connecting the display TCON (Timer Control Register), an i2S interface for connecting the power amplifier (AMP) and speaker; as well as an IR / Key interface, a USB interface, a Wi-Fi interface, a Bluetooth interface, an HDMI interface, a Tuner interface, etc.

[0076] The following is combined Figure 4 The dual hardware system architecture of this application will be further explained. It should be noted that... Figure 4 This is merely an illustrative example of the dual-hardware system architecture of this application and does not imply any limitation on the application. In practical applications, both hardware systems may include more or fewer hardware or interfaces as needed.

[0077] Figure 4 The example shown is based on Figure 3 Hardware architecture block diagram of display device 200. (See diagram below.) Figure 4 As shown, the hardware system of the display device 200 may include an A chip and an N chip, as well as modules connected to the A chip or the N chip through various interfaces.

[0078] The N chip may include at least one of the following: a tuner / demodulator 220, a communicator 230, an external device interface 250, a controller 210, a memory 290, a user input interface, a video processor 260-1, an audio processor 260-2, a display 280, an audio output interface 272, and a power supply. In other embodiments, the N chip may also include more or fewer modules.

[0079] The tuner / demodulator 220 is used to amplify, mix, and resonate the received broadcast television signals via wired or wireless means, thereby demodulating the audio and video signals carried in the frequency of the television channel selected by the user from multiple wireless or wired broadcast television signals, as well as additional information (such as EPG data signals). Depending on the television signal broadcasting standard, the signal path of the tuner / demodulator 220 can be various, such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, or internet broadcasting; and depending on the modulation type, the signal adjustment method can be digital modulation or analog modulation; and depending on the type of received television signal, the tuner / demodulator 220 can demodulate analog signals and / or digital signals.

[0080] The tuner 220 is also used to respond to the television channel frequency selected by the user and the television signal carried by that frequency, as selected by the user and controlled by the controller 210.

[0081] In some other exemplary embodiments, the tuner 220 may also be in an external device, such as an external set-top box. In this way, the set-top box outputs television audio and video signals after modulation and demodulation, which are then input to the display device 200 via the external device interface 250.

[0082] The communicator 230 is a component used to communicate with external devices or external servers according to various communication protocol types. For example, the communicator 230 may include a WIFI module 231, a Bluetooth communication protocol module 232, a wired Ethernet communication protocol module 233, and other network communication protocol modules or near-field communication protocol modules such as an infrared communication protocol module.

[0083] The display device 200 can establish a connection for control signals and data signals with an external control device or content providing device via a communicator 230. For example, the communicator can receive control signals from the remote controller 100 according to the controller's control.

[0084] The external device interface 250 is a component that provides data transmission between the N-chip controller 210 and the A-chip and other external devices. The external device interface can connect to external devices such as set-top boxes, gaming devices, and laptops via wired / wireless connections, and can receive data from external devices such as video signals (e.g., motion pictures), audio signals (e.g., music), and additional information (e.g., EPG).

[0085] The external device interface 250 may include one or more of the following: an HDMI terminal 251, a Composite Video Blanking Synchronization (CVBS) terminal 252, an analog or digital component terminal 253, a Universal Serial Bus (USB) terminal 254, and a Red, Green, and Blue (RGB) terminal (not shown in the figure). This application does not limit the number or type of external device interfaces.

[0086] The controller 210 controls the operation of the display device 200 and responds to user operations by running various software control programs (such as operating systems and / or various applications) stored in the memory 290.

[0087] like Figure 4 As shown, the controller 210 includes a read-only memory (RAM) 213, a random access memory (ROM) 214, a graphics processor (GPU) 216, a CPU processor (CPU) 212, a communication interface 218, and a communication bus. The RAM 213, ROM 214, GPU 216, CPU processor 212, and communication interface 218 are connected to each other via the bus.

[0088] ROM 213 is used to store various system startup instructions. For example, when a power-on signal is received, the display device 200 starts up, the CPU processor 212 executes the system startup instructions in the ROM, and copies the temporary data generated by the operating system stored in memory 290 to RAM 214 to run or start the operating system. After the operating system has started, the CPU processor 212 copies the temporary data generated by various application programs in memory 290 to RAM 214, and then runs or starts various application programs.

[0089] The graphics processor 216 is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. It includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes. It also includes a renderer that generates various objects based on the ALU's results and displays the rendered results on the monitor 280.

[0090] CPU processor 212 is used to execute operating system and application instructions stored in memory 290, as well as various interactive instructions received from external input, to execute various applications, data, and content, so as to ultimately display and play various audio and video content.

[0091] In some exemplary embodiments, the CPU processor 212 may include multiple processors. These multiple processors may include a main processor and multiple or one sub-processors. The main processor is used to perform some operations of the display device 200 in a pre-power-on mode and / or to display a screen in normal mode. The multiple or one sub-processors are used to perform an operation in a standby mode or other state.

[0092] The communication interface may include a first interface 218-1 to an nth interface 218-n. These interfaces may be network interfaces that are connected to external devices via a network.

[0093] The controller 210 can control the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to be displayed on the display 280, the controller 210 can perform operations related to the object selected by the user command.

[0094] The object can be any of the optional objects, such as a hyperlink or an icon. Operations related to the selected object include, for example, displaying links to hyperlinked pages, documents, images, etc., or executing operations corresponding to the icon. User commands for selecting UI objects can be input via various input devices connected to the display device 200 (e.g., mouse, keyboard, touchpad, etc.) or voice commands corresponding to spoken commands by the user.

[0095] The memory 290 includes various software modules for driving and controlling the display device 200. These software modules stored in the memory 290 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0096] The basic module is a low-level software module used for signal communication between various hardware components in the display device 200 and for sending processing and control signals to higher-level modules. The detection module is a management module used to collect various information from various sensors or user input interfaces, perform digital-to-analog conversion, and analyze and manage the data.

[0097] For example: the voice recognition module includes a voice parsing module and a voice command database module. The display control module is used to control the display 280 to display image content, and can be used to play multimedia image content and UI interface information. The communication module is used for control and data communication with external devices. The browser module is used to perform data communication with browser servers. The service module is used to provide various services and applications.

[0098] Meanwhile, the memory 290 is also used to store received external data and user data, images of various items in various user interfaces, and visual renderings of the focus object.

[0099] The user input interface is used to send user input signals to the controller 210, or to transmit signals output from the controller to the user. For example, a control device (e.g., a mobile terminal or remote control) can send user input signals such as power switch signals, channel selection signals, and volume adjustment signals to the user input interface, which then forwards them to the controller; or, the control device can receive output signals such as audio, video, or data output from the user input interface after processing by the controller, and display the received output signals or output the received output signals as audio or vibration.

[0100] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 280, and the user input interface receives the user input commands through the graphical user interface (GUI). Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0101] The video processor 260-1 is used to receive video signals and perform video data processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signal, so as to obtain a video signal that can be directly displayed or played on the monitor 280.

[0102] For example, the video processor 260-1 includes a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, a display formatting module, etc.

[0103] The demultiplexing module is used to demultiplex the input audio and video data streams. For example, if the input is MPEG-2, the demultiplexing module will demultiplex it into video signals and audio signals.

[0104] The video decoding module is used to process the demultiplexed video signal, including decoding and scaling.

[0105] Image compositing modules, such as image synthesizers, are used to overlay and blend GUI signals generated by a graphics generator based on user input or its own data with scaled video images to generate displayable image signals.

[0106] The frame rate conversion module is used to convert the frame rate of the input video, such as converting the frame rate of an input video of 24Hz, 25Hz, 30Hz, or 60Hz to 60Hz, 120Hz, or 240Hz. The input frame rate can be related to the source video stream, and the output frame rate can be related to the display's refresh rate. Inputs in common formats are implemented using methods such as frame interpolation.

[0107] The display formatting module is used to change the signal output by the frame rate conversion module into a signal that conforms to a display format, such as converting the signal output by the frame rate conversion module to output RGB data signals.

[0108] In some embodiments, the video processor and the graphics processor can be integrated into a single chip. In other embodiments, the functional modules of the graphics processor and the video processor can be configured as needed.

[0109] Display 280 is used to receive image signals input from video processor 260-1 and to display video content, images, and a menu control interface. Display 280 includes a display component for presenting the image and a driving component for driving the image display. The displayed video content may be video from a broadcast signal received by tuner 220, or video content input from a communicator or external device interface. Display 220 also displays the user interface (UI) generated in display device 200 and used to control display device 200.

[0110] Depending on the type of display 280, it may also include a driving component for driving the display. Alternatively, if the display 280 is a projection display, it may also include a projection device and a projection screen.

[0111] The audio processor 260-2 is used to receive audio signals, perform decompression and decoding according to the standard encoding and decoding protocol of the input signals, as well as audio data processing such as noise reduction, digital-to-analog conversion and amplification, to obtain an audio signal that can be played in the speaker 272.

[0112] The audio output interface 270 is used to receive audio signals output by the audio processor 260-2 under the control of the controller 210. The audio output interface may include a speaker 272, or an external audio output terminal 274 for outputting to an external device, such as an external audio terminal or headphone output terminal.

[0113] In some other exemplary embodiments, the video processor 260-1 may include one or more chips. The audio processor 260-2 may also include one or more chips.

[0114] In addition, in some other exemplary embodiments, the video processor 260-1 and the audio processor 260-2 may be separate chips or integrated with the controller 210 into one or more chips.

[0115] The power supply, under the control of the controller 210, provides power to the display device 200 from an external power source. The power supply may be a built-in power circuit installed inside the display device 200, or it may be an external power source, such as a power interface within the display device 200 that provides an external power supply.

[0116] Similar to the N chip, such as Figure 4 As shown, chip A may include a controller 310, a communicator 330, a detector 340, and a memory 390. In some embodiments, it may also include a user input interface, a video processor, an audio processor, a display, and an audio output interface. In some embodiments, a power supply independently powering chip A may also be present.

[0117] The communicator 330 is a component used to communicate with external devices or external servers according to various communication protocol types. For example, the communicator 330 may include a WIFI module 331, a Bluetooth communication protocol module 332, a wired Ethernet communication protocol module 333, and other network communication protocol modules or near-field communication protocol modules such as an infrared communication protocol module.

[0118] The communicator 330 of chip A and the communicator 230 of chip N also interact with each other. For example, the WiFi module 231 in the N chip hardware system is used to connect to an external network and establish network communication with external servers, etc. The WiFi module 331 in chip A is used to connect to the WiFi module 231 in the N chip hardware system, but does not establish a direct connection with external networks, etc. Chip A connects to the external network through chip N. Therefore, for the user, a display device as described in the above embodiment only displays a WiFi account.

[0119] Detector 340 is a component of display device A chip used to collect signals from the external environment or interact with the external environment. Detector 340 may include a light receiver 342, a sensor for collecting ambient light intensity, which can adapt to changes in display parameters by collecting ambient light; it may also include an image acquisition device 341, such as a camera, which can be used to collect external environmental scenes and to collect user attributes or user interaction gestures, which can adaptively change display parameters and recognize user gestures to achieve interactive functions with the user.

[0120] The external device interface 350 provides a component for data transmission between the controller 310 and the N chip or other external devices. The external device interface can connect to external devices such as set-top boxes, gaming devices, and laptops via wired or wireless connections.

[0121] The controller 310 controls the operation of the display device 200 and responds to user operations by running various software control programs (such as installed third-party applications) stored in the memory 390 and by interacting with the N chip.

[0122] like Figure 4 As shown, the controller 310 includes a read-only memory (ROM) 313, a random access memory (RAM) 314, a graphics processor 316, a CPU processor 312, a communication interface 318, and a communication bus. The ROM 313, RAM 314, graphics processor 316, CPU processor 312, and communication interface 318 are connected to each other via the bus.

[0123] ROM 313 is used to store various system startup instructions. CPU processor 312 executes the system startup instructions in ROM, copying the temporary data of the operating system stored in memory 390 to RAM 314 to run or start the operating system. After the operating system has started, CPU processor 312 copies the temporary data of various applications in memory 390 to RAM 314, and then runs or starts various applications.

[0124] CPU processor 312 is used to execute operating system and application instructions stored in memory 390, and to communicate, transmit and interact with N chip, such as signals, data and instructions, and to execute various applications, data and content according to various interactive instructions received from external input, so as to ultimately display and play various audio and video content.

[0125] The communication interface may include a first interface 318-1 to an nth interface 318-n. These interfaces may be network interfaces that are connected to external devices via a network, or network interfaces that are connected to the N chip via a network.

[0126] The controller 310 can control the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to be displayed on the display 280, the controller 210 can perform operations related to the object selected by the user command.

[0127] The graphics processor 316 is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. It includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes. It also includes a renderer that generates various objects based on the ALU's results and displays the rendered results on the monitor 280.

[0128] Both the graphics processor 316 of chip A and the graphics processor 216 of chip N can generate various graphical objects. The difference is that if application 1 is installed on chip A and application 2 is installed on chip N, when the user is in the interface of application 1 and performs user input commands within application 1, the graphical object is generated by the graphics processor 316 of chip A. When the user is in the interface of application 2 and performs user input commands within application 2, the graphical object is generated by the graphics processor 216 of chip N.

[0129] Figure 5 The diagram illustrates a functional configuration of a display device according to an exemplary embodiment.

[0130] like Figure 5 As shown, the memory 390 of chip A and the memory 290 of chip N are used to store the operating system, applications, content, and user data, respectively. Under the control of the controller 310 of chip A and the controller 210 of chip N, they drive the system operation of the display device 200 and respond to various user operations. The memory 390 of chip A and the memory 290 of chip N may include volatile and / or non-volatile memory.

[0131] For the N chip, memory 290 is specifically used to store the running program of the controller 210 in the display device 200, as well as various applications built into the display device 200, various applications downloaded by the user from external devices, various graphical user interfaces related to the applications, various objects related to the graphical user interfaces, user data information, and various internal data supporting the applications. Memory 290 is used to store system software such as the operating system (OS) kernel, middleware, and applications, as well as input video and audio data, and other user data.

[0132] The memory 290 is specifically used to store drivers and related data for the video processor 260-1 and audio processor 260-2, display 280, communication interface 230, tuner / demodulator 220, input / output interface, etc.

[0133] In some embodiments, memory 290 may store software and / or programs representing an operating system (OS), including, for example, a kernel, middleware, application programming interfaces (APIs), and / or applications. Exemplarily, the kernel may control or manage system resources, or functions implemented by other programs (such as the middleware, APIs, or applications), and the kernel may provide interfaces to allow middleware and APIs, or applications, to access controllers to control or manage system resources.

[0134] For example, memory 290 includes a broadcast receiving module 2901, a channel control module 2902, a volume control module 2903, an image control module 2904, a display control module 2905, an audio control module 2906, an external command recognition module 2907, a communication control module 2908, an optical receiving module 2909, a power control module 2910, an operating system 2911, and other application programs 2912, a browser module, etc. Controller 210 executes various functions such as broadcast television signal reception and demodulation, television channel selection control, volume selection control, image control, display control, audio control, external command recognition, communication control, optical signal reception, power control, a software control platform supporting various functions, and a browser function by running various software programs stored in memory 290.

[0135] The memory 390 includes various software modules for driving and controlling the display device 200. These software modules stored in the memory 390 include: a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules. Since the memory 390 and memory 290 have similar functions, relevant details can be found in memory 290 and will not be elaborated upon here.

[0136] For example, memory 390 includes an image control module 3904, an audio control module 2906, an external command recognition module 3907, a communication control module 3908, an optical receiving module 3909, an operating system 3911, and other application programs 3912, a browser module, etc. Controller 210 executes various functions such as image control, display control, audio control, external command recognition, communication control, optical signal receiving, power control, a software control platform supporting various functions, and a browser function by running various software programs stored in memory 290.

[0137] The difference lies in that the external instruction recognition module 2907 of the N chip and the external instruction recognition module 3907 of the A chip can recognize different instructions.

[0138] For example, since image receiving devices such as cameras are connected to chip A, the external command recognition module 3907 of chip A may include an image recognition module 2907-1. The image recognition module 3907-1 stores an image database. When the camera receives an external image command, it establishes a correspondence with the command in the image database to control the display device. Similarly, since voice receiving devices and remote controls are connected to chip N, the external command recognition module 2907 of chip N may include a voice recognition module 2907-2. The image recognition module 2907-2 stores a voice database. When voice receiving devices receive external voice commands, they establish a correspondence with the command in the voice database to control the display device. Likewise, the control device 100, such as a remote control, is connected to chip N, and the button command recognition module interacts with the control device 100.

[0139] Figure 6a The diagram illustrates a configuration block diagram of the software system in the display device 200 according to an exemplary embodiment.

[0140] For N chips, such as Figure 6a As shown, the operating system 2911 includes execution software for handling various basic system services and for performing hardware-related tasks.

[0141] In some embodiments, a portion of the operating system kernel may include a suite of software for managing display device hardware resources and providing services to other programs or software code.

[0142] In other embodiments, a portion of the operating system kernel may include one or more device drivers. A device driver can be a set of software code within the operating system that helps operate or control devices or hardware associated with the display device. The driver may contain code that operates video, audio, and / or other multimedia components. Examples include drivers for displays, cameras, Flash, WiFi, and audio.

[0143] The accessibility module 2911-1 is used to modify or access the application to enable accessibility of the application and operability of its displayed content.

[0144] The communication module 2911-2 is used for connecting to other peripherals via relevant communication interfaces and communication networks.

[0145] User interface module 2911-3 provides objects for displaying the user interface, which can be accessed by various applications to enable user operability.

[0146] Control application 2911-4 is used to control process management, including runtime applications, etc.

[0147] The event transmission system 2914 can be implemented within the operating system 2911 or in the application 2912. In some embodiments, it is implemented both within the operating system 2911 and in the application 2912, and is used to listen for various user input events and to implement a set of or more predefined operations based on the identification results of various events or sub-events that represent various events.

[0148] Among them, the event listening module 2914-1 is used to listen for user input interface input events or sub-events.

[0149] The event recognition module 2914-1 is used to input various event definitions into various user input interfaces, identify various events or sub-events, and transmit them to the processor to execute one or more sets of corresponding processing programs.

[0150] Here, an event or sub-event refers to the input detected by one or more sensors in the display device 200, as well as the input from external control devices (such as control device 100). Examples include various voice input sub-events, gesture recognition sub-events, and sub-events of remote control button commands input from the control device. For instance, one or more sub-events in a remote control include various forms, including but not limited to one or a combination of button presses (up / down / left / right), confirmation button, and button press / hold. It also includes operations of non-physical buttons, such as moving, pressing, and releasing.

[0151] The interface layout management module 2913 directly or indirectly receives user input events or sub-events from the event transmission system 2914, and uses them to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, as well as the size or position and hierarchy of containers and other execution operations related to the interface layout.

[0152] Since the operating system 3911 of chip A and the operating system 2911 of chip N have similar functions, please refer to the operating system 2911 for relevant details, and will not be repeated here.

[0153] like Figure 6b As shown, the application layer of the display device contains various applications that can be executed on the display device 200.

[0154] The application layer 2912 of the N chip may include, but is not limited to, one or more applications, such as video-on-demand applications, application centers, and games. The application layer 3912 of the A chip may include, but is not limited to, one or more applications, such as live TV applications and media center applications. It should be noted that the applications included on the A chip and the N chip are determined based on the operating system and other design considerations, and this invention does not require specific limitations or divisions regarding the applications included on the A chip and the N chip.

[0155] A live TV application can provide live television from various signal sources. For example, a live TV application can provide television signals from cable television, terrestrial broadcasting, satellite services, or other types of live TV services. Furthermore, the live TV application can display the video of the live TV signal on a display device 200.

[0156] Video-on-demand (VOD) applications can provide video from various storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, VOD can come from cloud storage servers or from local hard drive storage containing pre-stored video programs.

[0157] Media Center applications are applications that provide playback of various multimedia content. For example, Media Center provides services that allow users to access various images or audios, unlike live TV or video-on-demand.

[0158] The application center provides a storage space for various applications. An application can be a game, an application, or other applications related to a computer system or other device but capable of running on a display device. The application center obtains these applications from various sources, stores them in local storage, and then makes them available for execution on the display device 200.

[0159] In some embodiments, the operating system includes a kernel, a database, a framework layer, and an application layer, where all applications are applications within the application layer.

[0160] Figure 7 The diagram illustrates a user interface in a display device 200 according to an exemplary embodiment. Figure 7 As shown, the user interface includes multiple view display areas, exemplified by a first view display area 201 and a playback screen 202, wherein the playback screen includes a layout of one or more different items. The user interface also includes a selector indicating the selected item, which can be moved by user input to change the selected item.

[0161] It should be noted that multiple view display areas can present different levels of display content. For example, the first view display area can display video chat content, while the second view display area can display application-layer content (such as web videos, VOD demonstrations, application screens, etc.).

[0162] Optionally, different view display areas have different display priorities, and the display priority of view display areas with different priorities is different. For example, the system layer has a higher priority than the application layer. When the user uses the selector and switches screens in the application layer, the display of the system layer's view display area is not obscured. Also, when the size and position of the application layer's view display area change according to the user's selection, the size and position of the system layer's view display area are not affected.

[0163] It can also display the same level of screen. In this case, the selector can switch between the first view display area and the second view display area, and when the size and position of the first view display area change, the size and position of the second view display area can change accordingly.

[0164] Because chip A and chip N may each have their own independent operating system installed, the display device 200 contains two independent but interconnected subsystems. For example, both chip A and chip N can independently install Android and various apps, enabling each chip to perform certain functions and allowing chip A and chip N to work together to achieve a specific function.

[0165] This application provides a video call method that allows an authorized account to directly establish a video connection with a display device without requiring operation from the display device (when acting as an invited device), thus simplifying the video call establishment process. The solution provided in this application can be used not only with dual-chip or multi-chip display devices, but also with single-chip display devices.

[0166] In some embodiments, the display device that is the invited party in a video call is equipped with a camera, and the device that initiates the video call can be a mobile terminal or another display device. In some embodiments, the display device is a smart TV or a system consisting of a monitor and a set-top box.

[0167] In some embodiments, taking a mobile terminal as the initiator of the video request, the mobile terminal has a corresponding APP installed, such as at least one of: Juhaokan, Xiaojuzaijia, and Haijian. The mobile terminal sending a video call request can be achieved by the user opening the corresponding APP and clicking the corresponding video function control within the APP. The mobile terminal then sends a video call request to the server. The video call request includes at least one of the unique identifier of the display device or the account signal of the TV, as well as account information on the mobile terminal. The video call request is used to request the display device to transmit image information acquired by the camera and sound information acquired by the microphone on the display device to the mobile terminal. The unique identifier of the display device or the account signal of the TV can serve as the identification information of the display device, i.e., the first identification information. The account information on the mobile terminal can serve as the identification information representing the mobile terminal, i.e., the second identification information.

[0168] After receiving a video call request, the server sends a push message to the TV based on the first identification information. The push message includes the first identification information and / or the second identification information. The push message is used to trigger the display device to determine the legitimacy of the mobile terminal.

[0169] In some embodiments, after receiving the video call request, the server sends a push message to the TV based on the TV's account and / or the unique identifier of the display device. The push message includes at least one of the following: the unique identifier of the display device, the TV's account, or the account information on the mobile terminal.

[0170] In some embodiments, the server may perform authentication before sending a push message to the display device, so that the terminal can directly establish a call process based on the push message. Specifically, the authentication process includes verifying whether the unique identifier of the display device or the account signal on the TV in the video call request authorizes the account logged in on the mobile terminal to directly establish a call request, and then sending a push message and an authentication success message.

[0171] In response to the push message, the display device automatically initiates a video call process.

[0172] In some embodiments, the automatic establishment of a video call by the display device includes: the display device sending an authentication request to the server based on the display device's unique identifier and / or the currently logged-in account; the server responding to the authentication request determining whether the mobile terminal is authorized by the television device according to a preset correspondence; and the display device automatically establishing the video call process in response to the successful authentication message.

[0173] In some embodiments, when the display device determines that the account on the TV terminal and / or the unique identifier of the display device in the push request matches the currently logged-in account or the unique identifier of the current display device, it automatically establishes the video call process in response to the authentication success message.

[0174] In some embodiments, the server only sends a push message when authentication is successful, and does not send one when authentication fails. Therefore, the display device can directly determine, based on the push message, that the account on the TV terminal and / or the unique identifier of the display device in the push request matches the currently logged-in account or the unique identifier of the current display device, and automatically establish the video call process.

[0175] In some embodiments provided in this application, the first video application installed in the display device (e.g., Xiaojujiazai) may also include a second video application (e.g., Haijian), and may also include other applications that use a camera (e.g., a game application whose main function is card games, or a camera application whose main function is taking pictures, or a mirror application).

[0176] During the use of this function, the mobile terminal first needs to obtain authorization from the display device. This authorization process can involve the display device presenting a QR code that allows the mobile terminal to obtain the display device ID. After scanning the code, the mobile terminal obtains the display device ID and displays it on the UI. After the user selects to bind the device on the mobile terminal, the server binds the current account information on the mobile device with the display device ID. In some embodiments, the display device ID can be replaced with the account information logged in on the display device. The server binds the current account information on the mobile device and the account information logged in on the display device. In some embodiments, the QR code can simultaneously contain the account information logged in on the display device and the display device ID.

[0177] In some embodiments, during the authorization process for the display device, it may be required that the account information logged in on the display device is the same as the account information logged in on the mobile terminal before binding the display device's ID.

[0178] In some embodiments, the authentication correspondence established after binding can be stored on a server or in a display device. The authentication correspondence may include at least one of the following: the correspondence between the current account information on the mobile device and the ID of the display device, the correspondence between the current account information on the mobile device and the account information logged in on the display device, and the correspondence between the account information logged in on the display device and the ID of the display device and the current account information on the mobile device.

[0179] like Figure 10a As shown, the video call process includes:

[0180] S001: The mobile terminal sends a video call request;

[0181] In some embodiments, the video call request includes first identification information and second identification information. The video call request is used to request the display device to transmit image information acquired by the camera on the display device to the mobile terminal. The first identification information is identification information representing the invited display terminal in the video call, and the second identification information is identification information representing the mobile terminal that initiated the video call.

[0182] In some embodiments, during the authorization process, a mapping relationship between the first identification information and the second identification information is established, and the server has pre-recorded this mapping relationship.

[0183] In some embodiments, the first identification information may be at least one of the unique identifier of the display device (device ID1) or the first account information (customer ID1) stored on the mobile terminal; the second identification information may be the currently logged-in account information (customer ID2) or the unique device identifier of the mobile terminal (device ID2).

[0184] The first identification information may be the display device ID and / or display device account information stored on the mobile terminal during the video call authorization process, or it may be the display device ID and / or display device account information manually entered by the user.

[0185] The video call request also includes a first video application identifier (e.g., the identifier of the "Xiaoju Zaijia" function) so that the display device can determine which function needs to be invoked to establish the video call process.

[0186] In some embodiments, the display device is a television, and the mobile terminal is a mobile phone or a tablet computer.

[0187] In some embodiments, the display device sends a video call request to the server, and the server informs the display device that a video call request has been made. The display device and the mobile terminal are on different networks, that is, not on the same local area network (LAN). For example, the television is on the home LAN, while the mobile phone is on a data network or another LAN.

[0188] S002: After receiving the video call request, the server sends a push message to the TV according to the first identification information;

[0189] In some embodiments, the push message includes first identification information and / or second identification information, and the push message is used to trigger a display device to determine the legitimacy of the mobile terminal.

[0190] In some embodiments, determining the legitimacy of the mobile terminal means determining whether the display device authorizes the mobile terminal to directly connect to a video call.

[0191] The push message is used to inform the display device of a video call access request and trigger the display device to determine the legitimacy of the mobile terminal.

[0192] In some embodiments, the push message includes second identification information to trigger a validity check on the display device.

[0193] In some embodiments, the push message further includes first identification information, enabling the display device to determine whether its own identifier matches the first identification information and trigger a legality check;

[0194] In some embodiments, the push message includes first identification information and second identification information, so that the display device can determine whether its own identification matches the first identification information and trigger a legality judgment at the same time.

[0195] In some embodiments, a push message may be sent to the TV based on the first identification information (i.e., the account on the TV and / or the unique identifier of the display device), the push message including at least one of the unique identifier of the display device (deviceID1), the account on the TV (customer ID1), and the account information on the mobile terminal (customer ID2).

[0196] In some embodiments, the push message may further include an identifier of a first application function, which is used to enable the display device to determine which function's call process should be established.

[0197] In some embodiments, the first identification information includes first account information (i.e., account information on the TV). A persistent connection exists between the running display device and the system. This persistent connection is established based on the account logged in on the display device. The server determines whether a persistent connection exists corresponding to the TV account information (customer ID 1) uploaded by the mobile phone. If it exists, the connection continues; otherwise, the server reports that the mobile terminal device is offline. The server pushes a message to the device corresponding to the TV account based on the persistent connection. The pushed message includes at least one of the above parameters (first identification information and second identification information).

[0198] Correspondingly, if the display device is in operation (e.g., display device A) and there is a long connection between the display device and the server, the server determines the corresponding display device (e.g., display device A) based on the account information (customer ID 1) uploaded by the mobile phone to the TV terminal and sends the generated push message to the display (e.g., display device A). If the display device (e.g., display device A) is in a powered-off or standby state, there is no long connection between the display device and the server, and the server cannot find the long connection corresponding to the account information (customer ID 1) uploaded by the mobile phone to the TV terminal. Therefore, it sends a message to the mobile terminal indicating that the display device corresponding to the account information (customer ID 1) on the TV terminal is offline.

[0199] In some embodiments, the server determines whether the display device is online based on the polling information of the display device. If it is online, the process continues; if it is offline, the server sends a notification to the mobile terminal device that it is offline.

[0200] In some embodiments, the display device automatically establishes a call process in response to the push message.

[0201] In some embodiments, the display device automatically sends a confirmation message to the server in response to the push message; the server allocates a video call room address, the display device receives the call room address sent by the server in response to the confirmation message; and initiates the video call according to the call room address.

[0202] In some embodiments, after receiving a video call request from a mobile terminal, the server determines whether the mobile terminal represented by the call request has been authorized by the requested display device based on a pre-stored authorization relationship. If the server confirms authorization, it sends a push message and an authentication success message to the display device, and the display device automatically establishes a call process. If the server confirms that there is no authorization, it sends a rejection message to the mobile terminal.

[0203] In some embodiments, the method further includes S003: the display device receives a push message and sends an authentication request to the server when the first identification information corresponds to the identification information of the display device;

[0204] In some embodiments, the display device responds to the push message and determines the legitimacy of the mobile terminal;

[0205] The determination of the legitimacy of the mobile terminal includes: the display device determining whether the first identification information and the identification information of the display device correspond based on the first identification information; if they correspond, the display device sends an authentication request to the server based on the identification information of the display device (including the unique identifier of the display device and / or the currently logged-in account) and / or the first identification information.

[0206] In some embodiments, the above steps further include: after receiving the push message, the display device determines whether the unique identifier (device ID1) of the display device in the push message matches the unique identifier (device ID3) of the display device and / or whether the account (customer ID1) on the TV matches the account (customer ID3) logged in on the display device, and then sends an authentication request to the server based on the determination result. In some embodiments, when the unique identifier and / or the account on the TV match, the display device sends an authentication request to the server. In some embodiments, the authentication request includes the unique identifier (device ID3) of the display device and / or the currently logged-in account (customer ID3) and / or the first identifier information. Since the unique identifier (device ID1) of the display device in the push message matches the unique identifier (device ID3) of the display device and / or the account (customer ID1) on the TV matches the account (customer ID3) logged in on the display device, the authentication request may also include the unique identifier (device ID1) of the display device or the account (customer ID1) on the TV in the push message.

[0207] In some embodiments, the display device includes a "Xiaoju Zaijia" call function and a "Hijian" call function. The above steps further include: the display device receiving the push message; the display device determining whether the push message corresponds to "Xiaoju Zaijia" based on the identifier of the "Xiaoju Zaijia" function; if so, the display device determining whether the TV account and TV ID carried in the push message are consistent with the currently logged-in account and device ID of the device; if not, establishing a call according to the "Hijian" call process. In "Hijian" calls, the user on the display terminal needs to select "answer" before establishing a call connection. The "Xiaoju Zaijia" function, however, does not require user operation on the display terminal to directly establish a call connection. "Xiaoju Zaijia" (Application A) and "Hijian" (Application B) are two functions of the same application software package.

[0208] The method also includes determining whether the TV account and TV ID carried in the push message are consistent with the account and device ID currently logged in by the device: if they are inconsistent, they are ignored; if they are consistent, an authentication request is sent to the server.

[0209] In some embodiments, the authentication request includes the display device ID and / or the account currently logged into the display device. It may also include the mobile terminal's device ID and account information. The server determines the authentication based on the above information and a preset authorization mapping relationship.

[0210] In some embodiments, the method further includes S004: in response to the authentication request, the server determines, according to the authentication request, whether the mobile terminal represented by the second identifier is authorized by the display device represented by the first identifier.

[0211] In some embodiments, if the authorization is passed, a message indicating successful authentication is sent to the display device.

[0212] In some embodiments, the method further includes: if the authentication is not successful, sending a message indicating authentication failure to the display device (television), wherein the display device feeds back a message indicating no authorization to the mobile terminal through the server, and the mobile terminal presents prompt information indicating no authorization in a manner of an interface, voice, or text.

[0213] If the authentication is successful, a message indicating successful authentication is sent to the display device.

[0214] In some embodiments, the authentication request includes the identifier information of the display device and / or the first identifier information, and the server determines, according to a pre-stored authorization relationship, that the mobile terminal represented by the second identifier is authorized by the display device represented by the first identifier.

[0215] In some embodiments, the display device determines whether it is in a call state. If it is in the call state, it determines whether the ongoing call is a Xiao Ju Zai Jia function call or a Hai Jian function call. If it is the Hai Jian function call, the display terminal feeds back a terminal busy message to the mobile terminal through the server. If the ongoing call is the Xiao Ju Zai Jia function call, the session ID is sent to the server, and the server sends the session ID to the mobile end, so that the mobile end automatically joins the session corresponding to the session ID. For example: the corresponding scenario is that when a second mobile end uses the Xiao Ju Zai Jia function to establish a call with the display terminal, a first mobile terminal initiates a call request to the same display terminal. Since the first mobile terminal is authorized by the television, the first mobile terminal can directly join the ongoing call established by the second mobile terminal and the display terminal. The viewport in the second mobile terminal changes from two (the television end viewport and the local viewport of the second mobile terminal) to three (the television end viewport, the local viewport of the second mobile terminal, and the viewport of the first mobile terminal). The first mobile terminal directly displays three viewports (the television end viewport, the local viewport of the first mobile terminal, and the viewport of the second mobile terminal). The television end changes from one viewport (the viewport of the second mobile terminal) to two viewports (the viewport of the first mobile terminal and the viewport of the second mobile terminal).

[0216] In some embodiments, authentication can be performed before the server sends the call request to the display terminal (TV). After receiving the first request from the mobile terminal, the server determines, based on pre-stored record information, whether the display terminal represented by the first identification information authorizes the mobile terminal represented by the second identification information to directly establish a call. Upon successful authentication, a push message is sent to the server. Upon receiving the push message, the display device again determines, based on the first identification information, whether the first identification information and the display device's identification information correspond. If they correspond, a call thread is established; otherwise, the push message is discarded. This method of server-side authentication and further determination by the TV based on the first identification information increases the accuracy of direct call establishment and reduces the probability of erroneous operations.

[0217] In some embodiments, the method further includes S005: the display device automatically establishes the call process in response to the authentication success message.

[0218] In some embodiments, in response to an authentication failure message, the display device causes the server to send a call rejection message to the mobile terminal.

[0219] In some embodiments, after receiving a successful authentication message, the display device automatically establishes a call process and initiates a call with the mobile terminal, avoiding the inconvenience of requiring user intervention to complete the call.

[0220] In some embodiments, after receiving a successful authentication message, the display device automatically returns a confirmation message to automatically establish a call process; the server allocates a call room address for this call process and sends the call room address to both the mobile terminal and the display device, so that the display device and the mobile terminal establish a video call based on the call room address.

[0221] In some embodiments, establishing a call process includes establishing a video call link and creating a floating window on the display of the display device as a small window that only displays the image data of the mobile terminal, without creating a window that displays the local video of the terminal, such as... Figure 8a As shown. The small window method refers to creating a non-full-screen display window. The floating window occupies an area smaller than the display interface area, and is set to not receive focus. In some embodiments, the small window is created on the OSD overlay. In this case, in response to the creation of the video process, the mobile terminal simultaneously displays the mobile terminal's local video window and the window displaying the video from the display device on the interface, such as... Figure 8b As shown.

[0222] In some embodiments, the display device provides a switching interface (e.g., a user triggers a "Settings" button via a remote control to switch the small window to full-screen mode). The switching interface can be a control on the display interface or a key value of a specific button on the remote control. Switching the floating window of the small window to full-screen mode cancels the floating window in small window mode and launches the thread of the display device's local video window, displaying both the local video window and the window displaying the mobile terminal image data in full-screen mode. Full-screen mode means that the layers containing the local window displaying the local image and the window displaying the mobile terminal image data occupy the entire display screen of the display device. The local image refers to the image acquired by the device through its camera. The display device's local video window and the mobile terminal video window are simultaneously displayed on the display device's interface. Figure 8c As shown.

[0223] In some embodiments, the local window corresponding to full-screen mode and the newly created first window (the peer window) are two windows built on the video layer, used to display image data acquired by the local camera and image data sent by the peer device (mobile terminal), respectively. Both windows can receive focus, allowing the user to control the volume or image quality separately.

[0224] In some embodiments, when the display shows the local window and the first window in the video layer, a user input is received to switch to a smaller window; in response to the user input to switch to a smaller window, the local window and the first window displayed in the video layer are canceled, and a second window and a third window are created in the floating layer, wherein the second window is used by the user to play video data captured by the camera, and the third window is used to play video data sent by the mobile terminal.

[0225] In some embodiments, the display device provides two video call functions, such as "Xiaoju Zaijia" (first functional module or first application) and "Hijian" (second functional module or second application). Therefore, in some embodiments: the display device determines whether it is in a call state; the display device determines whether it is in a call state based on whether a video call link exists. If so, it determines whether the application corresponding to the call is "Hijian" or "Xiaoju Zaijia". If it is "Hijian", it sends the terminal busy information to the server, and the server sends the terminal busy message to the mobile terminal. If it is "Xiaoju Zaijia", it proves that a video call link already exists, so it sends the terminal busy information to the server and sends the session ID to the server. The server receives polling transmission information (session ID and the address of the current call room (the server address corresponding to the session ID)) and sends it to the mobile terminal. The mobile terminal automatically answers and joins the session. In response to the mobile terminal joining, the display device adds a call window corresponding to the mobile terminal in the call interface, such as... Figure 9aAs described above, at this time, all windows in the mobile terminal meeting session are displayed on the mobile terminal's interface, including the mobile terminal's local preview window, the display device's video window, and the video window of the other party in the existing call, such as... Figure 9b As shown.

[0226] In some embodiments, if not in a video call state, the display device determines whether the camera is being used by another program. If so, the display device prompts the user on the screen whether to accept the video call request. If not accepted, the device sends a busy message to the server, which then sends the busy message to the mobile terminal. If accepted, the display device exits the application using the camera, such as a card game application (stops the process), automatically connects the call, and displays only the mobile video in a small window. If the camera is not being used by another application, the display device automatically establishes the call process, establishes a video link, and displays the mobile video call interface in a small window on the terminal.

[0227] In some embodiments, the display device receives a push message sent by a server after receiving a video call request from the mobile terminal; when the mobile terminal is authorized by the display device, the display device automatically establishes a call process in response to the push message; when the mobile terminal is not authorized by the display device, the display device rejects the video call in response to the push message.

[0228] Furthermore, the push message sent by the receiving server includes: a push message sent by the display device receiving the server and an authentication success message, wherein the authentication success message is sent by the server in response to receiving a video call request initiated by the mobile terminal, and the authentication success message indicates that the server determines that the mobile terminal has been authorized by the display device.

[0229] Furthermore, when the mobile terminal obtains authorization from the display device, the step of automatically establishing a call process in response to the push message includes: the display device determining, based on the successful authentication message, that the mobile terminal has obtained authorization from the display device; and the display device automatically establishing a call process to realize the transmission of audio and video data between the display device and the mobile terminal.

[0230] Furthermore, the automatic call establishment process includes: the display device automatically sending a confirmation message to the server; the display device receiving the call room address sent by the server in response to the confirmation message; and the display device initiating the video call based on the call room address.

[0231] Furthermore, the push message includes a first identifier, and the video call request includes a first identifier and a second identifier. The first identifier is used to represent the display device invited in the video call, and the second identifier is used to represent the mobile terminal that initiated the video call. Receiving the push message and authentication success message sent by the server includes: the display device receiving the push message sent by the server; the display device sending an authentication request to the server in response to the first identifier in the push message being the same as the identifier of the display device; and the display device receiving an authentication success message sent by the server. The authentication success message is sent by the server in response to the authentication request when it determines, based on a pre-stored authorization relationship, that the mobile terminal represented by the second identifier has been authorized by the display device represented by the first identifier.

[0232] Furthermore, the automatic call establishment process includes: the display device determining whether the camera is occupied; if the camera is occupied, determining the application currently occupying the camera; if the application currently occupying the camera is the video call application, then controlling the server to send the address of the current call room to the mobile terminal so that the mobile terminal can directly join the current call based on the address of the call room; if the application currently occupying the camera is another application, controlling the server to send a message indicating that the camera of the final display device is occupied to the mobile terminal; if the camera is not occupied, the call process is automatically established.

[0233] In some embodiments, the display device receives a push message sent by a server after receiving a video call request from the mobile terminal. When the mobile terminal is authorized by the display device, the display device automatically establishes a call process in response to the push message and controls the display to show the peer window instead of the local window, wherein the peer window refers to the window used to play image data sent by the mobile terminal, and the local window refers to the window used to play image data captured by the camera. When the mobile terminal is not authorized by the display device, the display device rejects the video call in response to the push message.

[0234] Furthermore, the push message sent by the receiving server includes: a push message sent by the display device receiving server and an authentication success message, wherein the authentication success message is sent by the server in response to receiving a video call request initiated by the mobile terminal, and the authentication success message indicates that the server determines that the mobile terminal has been authorized by the display device.

[0235] Furthermore, when the mobile terminal obtains authorization from the display device, in response to the push message, automatically establishing a call process and controlling the display to show the peer window and not the local window includes: the display device determining, based on the successful authentication message, that the mobile terminal has obtained authorization from the display device; the display device automatically establishing a call process to realize the transmission of audio and video data between the display device and the mobile terminal; and creating the peer window and not the local window on the display interface.

[0236] In some embodiments, the peer window includes a floating window created on an overlay of the display interface, the area of ​​which is smaller than the display area of ​​the screen.

[0237] In some embodiments, the display device receives a switching operation input by the user; in response to the switching operation input by the user, the display device cancels the floating window and creates a local video window and a first window in the video layer, wherein the first window is set as the peer window.

[0238] In some embodiments, the server sends a push message to the display device based on a video call request received from the mobile terminal, so that the display device automatically establishes a call process in response to the push message when the mobile terminal is authorized by the display device; and rejects the video call in response to the push message when the mobile terminal is not authorized by the display device.

[0239] Furthermore, the server sending a push message to the display device based on the received video call request from the mobile terminal includes: receiving the video call request from the mobile terminal, wherein the first call request includes a first identifier and a second identifier, the first identifier being used to identify the display device invited in the video call, and the second identifier being used to identify the mobile terminal initiating the video call; sending a push message and an authentication success message to the display device based on the second identifier, wherein the authentication success message is determined based on the first identifier, the second identifier, and a pre-stored authorization relationship, and the authentication success message indicates that the mobile terminal has received an authorization push message from the display device.

[0240] Furthermore, the server receives an automatic confirmation message from the display device; the server sends the call room address to both the mobile terminal and the display device, so that the display device automatically establishes a video call with the mobile terminal.

[0241] Furthermore, the server sending a push message and an authentication success message to the display device based on the second identifier includes: the server querying whether an authorization relationship exists between the first identifier and the second identifier in a pre-stored authorization relationship; if an authorization relationship exists, the server sends a push message and an authentication success message to the display device; if an authorization relationship does not exist, the server rejects the video call request.

[0242] Furthermore, the step of sending a push message and an authentication success message to the display device based on the second identifier includes: the server sending a push message to the display device based on the first identifier, the push message containing the first identifier and the second identifier; the server receiving an authentication request sent by the display device, the authentication request including the first identifier, the authentication request being sent by the display device when the first identifier and the display device's identifier are the same; the server querying a pre-stored authorization relationship based on the first identifier and the second identifier to see if an authorization relationship exists between the first identifier and the second identifier; if it exists, the server sending an authentication success message to the display device; if it does not exist, the server sending an authentication failure message to the display device.

[0243] Please refer to Figure 11 This illustration shows a schematic diagram of a video call system according to an embodiment of this application. The video call system includes a first mobile terminal 1110, a second mobile terminal 1120, a smart TV 1130, and a server 1140. The first mobile terminal 1110 establishes a connection with the server 1140 via a wired or wireless network, the second mobile terminal 1120 establishes a connection with the server 1140 via a wired or wireless network, and the smart TV 1130 establishes a connection with the server 1140 via a wired or wireless network.

[0244] The first mobile terminal 1110 is a device with video calling capabilities, such as a smartphone.

[0245] The second mobile terminal 1120 is a device with video calling capabilities, such as a smartphone.

[0246] Server 1140 is a backend server, which can be a single server or a server cluster or cloud computing center consisting of multiple servers.

[0247] In this embodiment, during a first video call between a first mobile terminal 1110 and a smart TV 1130 via a server 1140, a second mobile terminal 1120 requests a second video call with the smart TV 1130, thereby joining the first video call. The number of both the first mobile terminal 1110 and the second mobile terminal 1120 is at least one. Figure 11The example shown is a first mobile terminal 1110 and a second mobile terminal 1120.

[0248] Please refer to Figure 12 It illustrates a flowchart of a video call method provided in one embodiment of this application, which can be applied to... Figure 11 The video call system shown. The video call method includes:

[0249] In step S201, when the first mobile terminal is making a first video call with the smart TV through the server, the second mobile terminal sends a first call request to the server, which is used to request a second video call with the smart TV.

[0250] In step S202, the server sends a first call request to the smart TV according to the first call request. The first call request is used to instruct the second mobile terminal to call the smart TV.

[0251] In step S203, the smart TV obtains the first call information of the first video call according to the first call request, and sends the first call information to the server. The first call information is the identification information of the first video call.

[0252] In step S204, the server sends the first call information to the second mobile terminal.

[0253] Step 205: The second mobile terminal accesses the first video call based on the first call information.

[0254] In summary, the video call method provided in this application embodiment allows the second mobile terminal to initiate a second video call with the smart TV if the first mobile terminal needs to do so during a first video call with the smart TV via the server. The server then sends a first call request to the smart TV, which in turn sends the first call information of the first video call to the server. The server then sends the first call information to the second mobile terminal, thus instructing the second mobile terminal to access the first video call. This ensures that the second mobile terminal can access the first video call even if it is not invited by the first mobile terminal.

[0255] Please refer to Figure 13 It illustrates a flowchart of a video call method according to another embodiment of this application, which can be applied to... Figure 11 The video call system shown. The video call method includes:

[0256] Step 301: The first mobile terminal sends a second call request to the server, which is used to request a first video call with the smart TV.

[0257] Prior to step 301, the method further includes: the first mobile terminal sending a second terminal identifier of the first mobile terminal and a TV identifier of the smart TV to the server; the server determining whether the first mobile terminal has been authorized by the smart TV based on the second terminal identifier and the TV identifier; if the first mobile terminal has been authorized by the smart TV, the server sending a third confirmation message to the first mobile terminal; and the first mobile terminal triggering the execution of step 301 based on the third confirmation message.

[0258] Step 302: The server generates first call information based on the second call request and sends the first call information to the smart TV and the first mobile terminal respectively.

[0259] When the server includes a load balancing network element and a first service network element, step 302 can be implemented as follows:

[0260] The load balancing network element allocates a first service network element to the first mobile terminal according to the second call request, the second call request including the second terminal identifier of the first mobile terminal and the TV identifier of the smart TV;

[0261] The first service network element generates a second call request based on the second call request, sends the second call request to the smart TV, the second call request includes the first call information, the second terminal identifier and the TV identifier, and sends the first call information to the first mobile terminal.

[0262] Step 303: The smart TV sends a call answer success message to the server based on the first call information.

[0263] When the server includes a load balancing network element and a first service network element, step 303 can be implemented as follows:

[0264] The smart TV sends the second terminal identifier and the TV identifier to the server. The server uses the second terminal identifier and the TV identifier to determine whether the first mobile terminal is authorized by the smart TV. If the first mobile terminal is authorized by the smart TV, it sends a second confirmation message to the smart TV. The smart TV sends a successful answer message to the first service network element based on the second confirmation message.

[0265] Step 304: The server sends a success message to the first mobile terminal.

[0266] When the server includes a load balancing network element and a first service network element, step 304 can be implemented as follows:

[0267] The first service network element will send a message indicating successful reception to the first mobile terminal.

[0268] Step 305: The first mobile terminal accesses the first video call based on the first call information.

[0269] Step 306: The smart TV connects to the first video call based on the first call information.

[0270] Step 307: When the first mobile terminal is making a first video call with the smart TV through the server, the second mobile terminal sends a first call request to the server. The first call request is used to request a second video call with the smart TV.

[0271] Step 308: The server sends a first call request to the smart TV based on the first call request. The first call request is used to instruct the second mobile terminal to call the smart TV.

[0272] When the server includes a load balancing network element, a first service network element, and a second service network element, and the first mobile terminal makes a first video call with a smart TV through the first service network element, step 308 can be implemented as follows:

[0273] The load balancing network element allocates a second service network element to the second mobile terminal according to the first call request. The first call request includes the first terminal identifier of the second mobile terminal and the TV identifier of the smart TV.

[0274] The second service network element generates a first call request based on the first call request and sends the first call request to the smart TV. The first call request includes the second call information of the second video call, the first terminal identifier, and the TV identifier.

[0275] Step 309: The smart TV obtains the first call information of the first video call based on the first call request, and sends the first call information to the server.

[0276] When the server includes a load balancing network element, a first service network element, and a second service network element, and the first mobile terminal makes a first video call with the smart TV through the first service network element, step 309 can be implemented as follows:

[0277] The smart TV sends a first terminal identifier and a TV identifier to the server. The server uses the first terminal identifier and the TV identifier to determine whether the second mobile terminal is authorized by the smart TV. If the second mobile terminal is authorized by the smart TV, it sends a first confirmation message to the smart TV. The smart TV obtains the first call information based on the first confirmation message and sends the first call information to the second service network element.

[0278] Step 310: The server sends the first call information to the second mobile terminal.

[0279] When the server includes a load balancing network element, a first service network element, and a second service network element, and the first mobile terminal makes a first video call with the smart TV through the first service network element, step 310 can be implemented as follows: the second service network element sends the first call information to the second mobile terminal.

[0280] Step 311: The second mobile terminal accesses the first video call based on the first call information.

[0281] In this embodiment, the method further includes:

[0282] The second service network element sends the second call information to the load balancing network element;

[0283] The load balancing network element sends the second call information to the second mobile terminal;

[0284] The second mobile terminal sends a first polling request to the second serving network element based on the second call information;

[0285] The second service network element sends the first polling response to the second mobile terminal, and the first polling response includes the first call information;

[0286] The second mobile terminal receives the first polling response and obtains the first call information from the first polling response.

[0287] In summary, the video call method provided in this application embodiment allows the second mobile terminal to initiate a second video call with the smart TV if the first mobile terminal needs to do so during a first video call with the smart TV via the server. The server then sends a first call request to the smart TV, which in turn sends the first call information of the first video call to the server. The server then sends the first call information to the second mobile terminal, thus instructing the second mobile terminal to access the first video call. This ensures that the second mobile terminal can access the first video call even if it is not invited by the first mobile terminal.

[0288] Steps 301-306 describe the process of establishing a first video call between the first mobile terminal and the smart TV. The following will illustrate this process. Figure 14 The implementation process is described below. The server's load balancing network element is CSR (Call Signaling Router), the service network element is CSE (Call Signaling Engine), and the server also includes a system API (Application Programming Interface) and a messaging platform.

[0289] (1) Check: The first mobile terminal sends a request to the API, carrying the device ID (deviceid) and client ID (customerid) of the first mobile terminal, as well as the deviceid and customerid of the smart TV.

[0290] (2) Ret OK: The API checks whether the deviceid of the smart TV's customerid has authorized the video call of the deviceid of the first mobile terminal's customerid. If authorized, it returns an OK message. If not authorized, it returns an unauthorized message. If an unauthorized message is returned, the call ends. If an OK message is returned, step (3) is executed.

[0291] (3)homelnvite: The first mobile terminal initiates a call request to the CSR of the video call.

[0292] (4) homeInvite: When the CSR receives a call request, it selects a CSE and forwards the call request to that CSE.

[0293] (5) Task: When CSE receives a call request, it assigns a unique session ID for the call and sends a push request to the messaging platform. The push request carries the unique session ID, the URL address of CSE, the customer ID of the smart TV, the device ID and customer ID of the first mobile terminal, and the caller ID.

[0294] (6) Response: CSE returns information to CSR, carrying a unique session ID and the URL address of the service network element CSE.

[0295] (7) Response: CSR forwards the information returned by CSE to the first mobile terminal.

[0296] (12)(14)(16)homeRing: The first mobile terminal receives the URL address of the CSE, then polls the CSE and waits for the smart TV to answer.

[0297] (8) Message: The message platform sends a message to the smart TV corresponding to the deviceid of the customer.

[0298] (9) Receivetask: The messaging platform returns a message push result to the CSE, which includes whether the message was pushed to the smart TV. If the smart TV is not online, the CSE ends the call. The first mobile terminal receives a reply indicating the call has ended during polling in (12), (14), and (16). The first mobile terminal ends the call.

[0299] (10) Check: The smart TV sends a request to the API, which carries the smart TV's deviceid, customerid, the first mobile terminal's deviceid, and customerid.

[0300] (11) Ret OK: The API checks whether the deviceid of the customerid of the first mobile terminal has been authorized by the deviceid of the customerid of the smart TV. If authorized, it returns an OK message. If not authorized, it returns an unauthorized message.

[0301] (13) homeRing busy-type=0: The smart TV obtains a unique session ID, the CSE's URL address, the smart TV's customerid, the first mobile terminal's deviceid, customerid, and number from the message push. If unauthorized, an unauthorized message is sent to the CSE. The first mobile terminal receives a reply indicating unauthorized call in the polling of (12)(14)(16). The first mobile terminal ends the call.

[0302] (15) homeReceive OK: If authorized, the smart TV sends a successful answer message to the CSE. The first mobile terminal receives a successful answer reply during the polling in (12)(14)(16).

[0303] (17) HomeTalking: After the first mobile terminal enters the call, it joins the unique room mapped by the unique session ID, conducts video calls in the room, and polls the CSE URL for heartbeat messages to ensure that the call continues.

[0304] (18) HomeTalking: After the smart TV enters the call, it joins the unique room mapped by the unique session ID, conducts video calls in the room, and polls the CSE URL address for heartbeat messages to ensure that the call continues.

[0305] Steps 307-311 describe the process of the second mobile terminal accessing the first video call, which will be explained below. Figure 15 The implementation process is described below. The server uses a load balancing network element (CSR) and a service network element (CSE), and also includes a system API and a messaging platform.

[0306] (1) HomeTalking: After the first mobile terminal enters the call, it joins the unique room mapped by the unique session ID session1, conducts video calls in the room, and polls the CSE's url1 address for heartbeat messages to ensure that the call continues.

[0307] (2) HomeTalking: After the smart TV enters the call, it joins the unique room mapped by the unique session ID session1, conducts video calls in the room, and polls the CSE's url1 address for heartbeat messages to ensure that the call continues.

[0308] (3) Check: The second mobile terminal sends a request to the API, carrying the deviceid and customerid of the first mobile terminal, and the deviceid and customerid of the smart TV.

[0309] (4) Ret OK: The API checks whether the smart TV's customerid deviceid has authorized the second mobile terminal's customerid deviceid for video calling. If authorized, it returns an OK message. If not authorized, it returns an unauthorized message. If the unauthorized message is returned, the call ends. If the OK message is returned, step (5) is executed.

[0310] (5) homeInvite: The second mobile terminal initiates a call request to the CSR of the video call.

[0311] (6) homeInvite: When the CSR receives a call request, it selects a CSE and forwards the call request to that CSE.

[0312] (7) Task: When CSE receives a call request, it assigns a unique session ID session2 to the call and sends a push request to the messaging platform. The push request carries the unique session ID session2, CSE's URL2 address, smart TV's customerid, second mobile terminal's deviceid and customerid, and the caller's number.

[0313] (8) Response: CSE returns information to CSR, carrying the unique session ID session2 and CSE's URL2 address.

[0314] (9) Response: The CSR forwards the information returned by the CSE to the second mobile terminal.

[0315] (15)homeRing msg: The second mobile terminal receives the url2 address of the CSE, and then polls the CSE, waiting for the smart TV to answer.

[0316] (10) Message: The message platform sends a message to the deviceid corresponding to the customer of the called smart TV.

[0317] (11) Receive task: The messaging platform returns a message push result to the CSE, which includes whether the message was pushed to the called party. If the smart TV is not online, the CSE ends the call. The second mobile terminal receives a reply indicating the end of the call in the polling of (15)(homeRingmsg). The second mobile terminal ends the call.

[0318] (12) Check: The smart TV sends a request to the API, which carries the smart TV's deviceid, customerid, the second mobile terminal's deviceid, and customerid.

[0319] (13) Ret OK: The API checks whether the deviceid of the customerid of the second mobile terminal has been authorized by the deviceid of the customerid of the called smart TV. If authorized, it returns an OK message. If not authorized, it returns an unauthorized message.

[0320] (14) homeRingis-busy=1busy_type=1msg: The smart TV obtains the unique session ID session2, the CSE's url2 address, the smart TV's customerid, the second mobile terminal's deviceid and customerid, and the caller ID from the push notification. If unauthorized, an unauthorized message is sent to the CSE. The second mobile terminal receives a reply indicating that the call is unauthorized in the polling of (15). The second mobile terminal ends the call. If authorized, a successful answer message is sent to the CSE, which contains the unique session ID session1 of the call and the url1 address of the CSE. The second mobile terminal receives a successful answer message in the polling of (15).

[0321] (16)homeReceive: The second mobile terminal obtains the unique session ID session1 and the URL1 address of the CSE in the current call from the polling message of (15), and sends a request to join the call to the URL1 address of the CSE. The request carries the unique session ID session1, the device ID and customer ID of the second mobile terminal and the calling number.

[0322] (17)homeTalking: After the second mobile terminal enters the call, it joins the unique room mapped by the unique session ID session1, conducts video calls in the room, and polls the heartbeat message to the url1 address of CSE to ensure that the call continues.

[0323] (18) HomeTalking: The smart TV continues the call, continues the video call in the unique room mapped by the unique session ID session1, polls the heartbeat message to the url1 address of CSE to ensure that the call continues, and obtains the information of newly joined members synchronously through polling messages.

[0324] (19) HomeTalking: The first mobile terminal continues the call and continues to conduct video calls in the unique room mapped by the unique session ID session1. It polls the heartbeat message to the url1 address of CSE to ensure that the call continues and obtains the information of newly joined members synchronously through polling messages.

[0325] Please refer to Figure 16 This diagram illustrates a structural block diagram of a video call device according to an embodiment of this application, which can be applied to a server. The video call device includes:

[0326] The receiving module 610 is used to receive a first call request sent by the second mobile terminal when the first mobile terminal makes a first video call with the smart TV through the server. The first call request is used to request to make a second video call with the smart TV.

[0327] The sending module 620 is used to send a first call request to the smart TV according to the first call request. The first call request is used to instruct the second mobile terminal to call the smart TV.

[0328] The receiving module 610 is also used to receive the first call information of the first video call sent by the smart TV according to the first call request;

[0329] The sending module 620 is also used to send the first call information to the second mobile terminal, the first call information being used to instruct the second mobile terminal to access the first video call.

[0330] In some embodiments, a display device includes: a camera;

[0331] A display for showing a peer window and / or a local window, wherein the peer window is a window for playing image data sent by the mobile terminal, and the local window is a window for playing image data captured by the camera;

[0332] Processor, the processor being used for:

[0333] The system receives push messages from the server, which are sent by the server after receiving a video call request from the mobile terminal. When the mobile terminal is authorized by the display device, it automatically establishes a call process in response to the push message and controls the display to show the other party's window instead of the local window. When the mobile terminal is not authorized by the display device, it rejects the video call in response to the push message.

[0334] Furthermore, the processor is also used for:

[0335] When the peer window is displayed on the monitor but the local window is not, receive user input to switch the window;

[0336] In response to a user input switching operation, the floating window is canceled, and a local video window and a first window are created in the video layer, wherein the first window is set as the peer window.

[0337] Furthermore, the processor is also used for:

[0338] When the monitor displays the local window and the first window on the video layer, receive user input to switch between small windows;

[0339] In response to the user's input to switch small windows, the local window and the first window displayed in the video layer are canceled, and a second window and a third window are created in the floating layer. The second window is used by the user to play the video data captured by the camera, and the third window is used to play the video data sent by the mobile terminal.

[0340] In some embodiments, a display device includes: a microphone; a speaker; and a camera;

[0341] A display for showing a peer window and / or a local window, wherein the peer window is a window for playing image data sent by the mobile terminal, and the local window is a window for playing image data captured by the camera;

[0342] Processor, the processor being used for:

[0343] The server receives a push message, which is sent by the server after receiving a video call request from the mobile terminal.

[0344] When the mobile terminal obtains authorization from the display device, in response to the push message, it controls the display to show the peer window and not the local window, controls the speaker to output the audio data sent by the mobile terminal, controls the camera to collect image data for sending to the mobile terminal, and controls the microphone to collect audio data for sending to the mobile terminal.

[0345] When the mobile terminal does not obtain authorization from the display device, it rejects the video call in response to the push message.

[0346] In summary, the video call device provided in this application embodiment allows the second mobile terminal to initiate a second video call with the smart TV if the first mobile terminal needs to do so during the first video call between the first mobile terminal and the smart TV via the server. The server then sends a first call request to the smart TV, which in turn sends the first call information of the first video call to the server. The server then sends the first call information to the second mobile terminal, thus instructing the second mobile terminal to access the first video call. In this way, the second mobile terminal can access the first video call even if it is not invited by the first mobile terminal.

[0347] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0348] It should be understood that the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate, for example, to allow implementation in orders other than those given in the embodiments illustrated or described in this application.

[0349] Furthermore, the terms “including” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0350] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of some embodiments of this application.

Claims

1. A video call method, applied to a display device, characterized in that, The method includes: The server receives a push message, wherein the push message includes a second session ID, and the push message is sent by the server after receiving a second video call request from a second mobile terminal. When the display device is not engaged in a video call and the camera is not occupied, a video call is automatically established with the second mobile terminal based on the second session ID; When the display device is not conducting a video call and the camera is occupied by another application, a prompt is displayed on the display screen of the display device asking the user whether to answer the video call of the second mobile terminal; if the user answers, the other application occupying the camera is exited, and a video call is automatically established with the second mobile terminal based on the second session ID; When the display device is conducting a video call, it sends a first session ID corresponding to the current video call to the server so that the second mobile terminal can join the current video call through the room mapped by the first session ID.

2. The method as described in claim 1, characterized in that, After establishing a video call with the second mobile terminal, the method further includes: The video call interface of the second mobile terminal is presented in a small window.

3. A video call method, applied to a display device, characterized in that, The method includes: The server receives a push message, wherein the push message includes a second session ID, and the push message is sent by the server after receiving a second video call request from a second mobile terminal. Determine whether the camera corresponding to the display device is occupied; When the camera is not occupied, a video call is automatically established with the second mobile terminal based on the second session ID. When the camera is occupied by a video call application, the first session ID corresponding to the current video call is sent to the server so that the second mobile terminal can join the current video call through the room mapped by the first session ID.

4. The method as described in claim 3, characterized in that, When the camera is occupied by an application other than the video call application, a message indicating that the camera is occupied is sent to the second mobile terminal.

5. A display device, characterized in that, include: Memory, used to store software control programs; A controller for running the software control program to perform the method according to any one of claims 1-4.

Images