Call screen projection method and terminal equipment

Abstract

The invention provides a call screen projection method and terminal equipment, and the method comprises the steps: obtaining the equipment state information of second terminal equipment through first terminal equipment under the condition that a selection operation of a user on the second terminal equipment in a screen projection list is detected, and carrying out the screen projection through the first terminal equipment under the condition that the equipment state information indicates that hardware equipment of the second terminal equipment can be used for screen projection. A first terminal device and a second terminal device create a first data transmission channel and a second data transmission channel, the first terminal device executes a screen projection operation, and transmits call data with the second terminal device by using the first data transmission channel and the second data transmission channel, so that circulation of services among different devices can be realized by using a screen projection technology; and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a call projection method and terminal device. Background Art

[0002] Screen projection technology can display the screen of one terminal device on the screen of another terminal device. Currently, screen projection is mostly used in the viewing process of pictures and videos. For example, users can project pictures or videos on their mobile phones to devices with larger screens such as TVs and projectors to improve the viewing experience.

[0003] With the development of terminal technology and the improvement of people's living standards, users still have the demand to cast calls. Calls include video calls and voice calls. The current call casting technology can only project the call interface of one terminal device (generally a small-screen device) onto another terminal device (generally a large-screen device). Taking mobile phones and TVs as an example, the user is using the mobile phone to make a call and projects the call interface onto the TV. During the call, the user is still using the phone's camera, microphone, speaker and other hardware devices. On the one hand, this screen casting method does not reasonably utilize the hardware resources of the terminal device (i.e., the large-screen device) being cast. On the other hand, this screen casting method requires the user to hold the mobile phone, and the user experience is poor. Summary of the invention

[0004] The present application provides a call screen projection method and terminal device, which can use screen projection technology to realize the flow of services between different devices, rationally utilize the hardware resources of the terminal device being projected, and thus improve the user experience.

[0005] In the first aspect, a call screen projection method is provided, which is applied to a communication system including a first terminal device and a second terminal device, wherein a Bluetooth wireless communication connection is established between the first terminal device and the second terminal device, or the user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network. The method may include: when detecting that the user selects the second terminal device in the screen projection list, the first terminal device obtains the device status information of the second terminal device, and the device status information is used to indicate whether the hardware device of the second terminal device can be used for screen projection. When the device status information indicates that the hardware device of the second terminal device can be used for screen projection, the first terminal device and the second terminal device create a first data transmission channel and a second data transmission channel, the first data transmission channel is used to transmit the call data collected by the second terminal device through some devices in the hardware device, and the second data transmission channel is used to transmit the call data obtained by the first terminal device through the network. The first terminal device performs the screen projection operation, and uses the first data transmission channel and the second data transmission channel to transmit the call data with the second terminal device.

[0006] The call screen projection method provided by the present application, when a user's selection operation on a second terminal device in a screen projection list is detected, the first terminal device obtains the device status information of the second terminal device, and when the device status information indicates that some of the hardware devices of the second terminal device can be used for screen projection, the first terminal device and the second terminal device create a first data transmission channel and a second data transmission channel, the first terminal device performs the screen projection operation, and uses the first data transmission channel and the second data transmission channel to transmit call data to the second terminal device, which can use screen projection technology to realize the flow of services between different devices, thereby improving the user experience.

[0007] The above device status information can also be understood as whether the second terminal device is equipped with a hardware device for screen projection, and / or whether the hardware device of the second terminal device is occupied. The above device status information can also be understood as whether the second terminal device supports distributed screen projection.

[0008] For the video call service, if the second terminal device is equipped with a camera, a microphone and a speaker, and / or the camera, the microphone and the speaker of the second terminal device are not occupied, the hardware device of the second terminal device can be used for screen projection, that is, the second terminal device supports distributed screen projection.

[0009] The above-mentioned device status information may include a device identifier and a distributed status identifier, wherein the device identifier is used to indicate the second terminal device, and the distributed identifier is used to indicate whether the hardware device can be used for screen projection, that is, whether it supports distributed screen projection.

[0010] The device identifier may be a device name, a network address, or other identifier that can be used to identify the second terminal device, which is not limited in this application. The device name may be a name preset when the second terminal device leaves the factory, or a name modified by the user, which is not limited in this application. The network address may include an Internet Protocol (IP) address and / or a Media Access Control (MAC) address.

[0011] The distributed state identifier can be represented by English characters, and the distributed state identifier of the second terminal device can also be represented by a character string. The present application does not limit the specific form of the distributed state identifier of the second terminal device.

[0012] Taking user A using the first terminal device to make a video call with user B as an example, the first data transmission channel is used by the second terminal device to send the image data of user A collected by the second terminal device through the camera and the voice data of user A collected by the second terminal device through the microphone to the first terminal device; the second data transmission channel is used by the first terminal device to send the image data of user B and the voice data of user B obtained by the first terminal device through the network to the second terminal device.

[0013] Taking user A using the first terminal device to have a voice call with user B as an example, the first data transmission channel is used for the second terminal device to send the voice data of user A collected by the second terminal device through a microphone to the first terminal device; the second data transmission channel is used for the first terminal device to send the voice data of user B obtained by the first terminal device through the network to the second terminal device.

[0014] In combination with the first aspect, in certain implementations of the first aspect, when a user selection operation on a second terminal device in the projection list is detected, the first terminal device obtains device status information of the second terminal device, including two possible implementations.

[0015] In a first possible implementation, a Bluetooth wireless communication connection is established between a first terminal device and a second terminal device. In response to a user clicking a screen projection control in a drop-down menu, the first terminal device scans terminal devices within a preset range that have established wireless communication connections with the first terminal device, and displays a list of projection devices, which includes an identifier of the second terminal device. In response to a user selecting the second terminal device, the first terminal device obtains device status information of the second terminal device via Bluetooth.

[0016] The device status information of the second terminal device is broadcast by the second terminal device through the Bluetooth of the second terminal device.

[0017] In this way, the first terminal device and the second terminal device do not need to be connected to the local area network, and screen projection can be performed only through Bluetooth, which reduces the power consumption of the first terminal device and the second terminal device to a certain extent.

[0018] In a second possible implementation, the user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network. In response to the user clicking on the control center control, the first terminal device scans the terminal devices that are logged in to the same account as the first terminal device and are in the same local area network, and displays a list of projection devices, which includes the identifier of the second terminal device; in response to the user selecting the second terminal device, the first terminal device obtains the device status information of the second terminal device through the information synchronization module.

[0019] It should be understood that the second terminal device can store the device status information of the second terminal device in the information synchronization module of the second terminal device, so the first terminal device can interact with the information synchronization module of the second terminal device through the information synchronization module of the first terminal device to obtain the device status information of the second terminal device.

[0020] In this way, because the first terminal device and the second terminal device are logged in to the same device account, there is no need to authenticate the second terminal device through the first device virtualization module, which reduces the screen projection connection time to a certain extent and improves the user experience.

[0021] The above two possible implementation methods both enable the first terminal device to obtain the device status information of the second terminal device, and then display the name of the connectable device to the user. When the screen projection cannot be initiated from the drop-down menu interface due to reading errors or other reasons, the user can initiate the screen projection from the control center interface. When the screen projection cannot be initiated from the control center interface, the user can initiate the screen projection from the drop-down menu interface, thereby improving the user experience.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the second data transmission channel includes two possible implementations.

[0023] In a first possible implementation, when both the first terminal device and the second terminal device support the new protocol, a second data transmission channel (which may be referred to as a communication service management magic link channel) is created based on the new protocol, wherein the single data transmission amount supported by the new protocol is greater than the single data transmission amount supported by the old protocol.

[0024] In this way, the data transmission efficiency between the first terminal device and the second terminal device can be improved by increasing the single data transmission amount.

[0025] In a second possible implementation, when the first terminal device and / or the second terminal device does not support the new protocol, a second data transmission channel (which may be referred to as a socket channel for short) is created based on the old protocol, wherein the single data transmission amount supported by the new protocol is greater than the single data transmission amount supported by the old protocol.

[0026] In this way, for devices that support the new protocol (referred to as new devices for short), the data transmission channel between the multi-screen interaction module created based on the new protocol and the screen projection manager can be used to transmit the business data obtained by the first terminal device through the network. For devices that support the old protocol (referred to as old devices for short), the data transmission channel between the multi-screen interaction module created based on the old protocol and the screen projection manager can be used to transmit the business data obtained by the first terminal device through the network, thereby ensuring the compatibility of new and old devices.

[0027] In combination with the first aspect, in certain implementations of the first aspect, the above-mentioned determination of whether the protocol supported by the first terminal device and the second terminal device is a new protocol or an old protocol can be made by comparing the version number of the communication service manager with a preset version number.

[0028] Specifically, for the first terminal device, the version number of the communication service manager of the first terminal device is compared with the preset version number. If the version number of the communication service manager of the first terminal device is greater than or equal to the preset version number, it is determined that the first terminal device supports the new protocol; if the version number of the communication service manager of the first terminal device is less than the preset version number, it is determined that the first terminal device does not support the new protocol.

[0029] Specifically, for the second terminal device, the version number of the communication service manager of the first terminal device is compared with the preset version number. If the version number of the communication service manager of the second terminal device is greater than or equal to the preset version number, it is determined that the second terminal device supports the new protocol; if the version number of the communication service manager of the second terminal device is less than the preset version number, it is determined that the second terminal device does not support the new protocol.

[0030] Taking the above call as a video call as an example, the above hardware devices include a camera, a microphone and a speaker.

[0031] In the case where the camera, microphone and speaker can all be used for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device can be used for screen projection. The second terminal device collects the first data through the camera and microphone, and transmits the first data to the first terminal device through the first data transmission channel; the first terminal device receives the first data through the first data transmission channel; the first terminal device sends the second data obtained by the first terminal device through the network to the second terminal device through the second data transmission channel; the second terminal device receives the second data through the second data transmission channel; the second terminal device plays the voice data in the second data through the speaker, and displays the image data in the second data through the display screen of the second terminal device.

[0032] Among them, the camera, microphone and speaker can all be used for screen projection, which can be understood as the second terminal device is equipped with a camera, microphone and speaker, or it can be understood as the camera, microphone and speaker are not occupied.

[0033] It should be understood that in the case that one of the camera, microphone and speaker is not available for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device is not available for screen projection.

[0034] Among them, the fact that one of the camera, microphone and speaker cannot be used for screen projection can be understood as that at least one of the camera, microphone and speaker is not provided on the second terminal device, or it can be understood as that at least one of the camera, microphone and speaker is occupied.

[0035] Taking the above call as a voice call as an example, the above hardware device includes a microphone and a speaker.

[0036] In the case where both the microphone and the speaker can be used for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device can be used for screen projection. The second terminal device collects the first data through the microphone, and transmits the first data to the first terminal device through the first data transmission channel; the first terminal device receives the first data through the first data transmission channel; the first terminal device sends the second data obtained by the first terminal device through the network to the second terminal device through the second data transmission channel; the second terminal device receives the second data through the second data transmission channel; the second terminal device plays the voice data in the second data through the speaker, and displays the image data in the second data through the display screen of the second terminal device.

[0037] Among them, both the microphone and the speaker can be used for screen projection, which can be understood as the second terminal device is equipped with a microphone and a speaker, or it can be understood as the microphone and the speaker are not occupied.

[0038] It should be understood that when the microphone and / or speaker are not available for screen projection, the above-mentioned device status information is information used to indicate that the hardware device of the second terminal device is not available for screen projection.

[0039] Among them, the fact that the microphone and / or speaker cannot be used for screen projection can be understood as the second terminal device is not equipped with a microphone and / or speaker, or it can be understood as the microphone and / or speaker is occupied.

[0040] In this way, the user does not need to hold the first terminal device, and the user's image and sound can be collected through the second terminal device, freeing the user's hands and making it convenient for the user to perform other activities during the video call, such as doing housework, watching short videos using the first terminal device, etc., thereby improving the user experience.

[0041] In a second aspect, the present application provides a terminal device, including a processor, the processor is coupled to a memory, and can be used to execute instructions in the memory to implement the method in any possible implementation of the first aspect. Optionally, the device also includes a memory. Optionally, the device also includes a communication interface, and the processor is coupled to the communication interface.

[0042] In one implementation, the terminal device is a first terminal device. When the apparatus is a first terminal device, the communication interface may be a transceiver, or an input / output interface.

[0043] In another implementation, the terminal device is a chip configured in the first terminal device. When the terminal device is a chip configured in the first terminal device, the communication interface may be an input / output interface.

[0044] In one implementation, the terminal device is a second terminal device. When the apparatus is a second terminal device, the communication interface may be a transceiver, or an input / output interface.

[0045] In another implementation, the terminal device is a chip configured in the second terminal device. When the terminal device is a chip configured in the second terminal device, the communication interface may be an input / output interface.

[0046] In a third aspect, a processor is provided, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation of the first aspect.

[0047] In the specific implementation process, the processor may be a chip, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit may be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit may be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit may be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0048] In a fourth aspect, a terminal device is provided, comprising a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a receiver and transmit signals through a transmitter to execute the method in any possible implementation of the first aspect.

[0049] Optionally, the number of the processors is one or more, and the number of the memories is one or more.

[0050] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0051] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips respectively. This application does not limit the type of memory and the setting method of the memory and the processor.

[0052] It should be understood that the relevant data interaction process, such as sending indication information, can be a process of outputting indication information from a processor, and receiving capability information can be a process of receiving input capability information from a processor. Specifically, the processed output data can be output to a transmitter, and the input data received by the processor can come from a receiver. Among them, the transmitter and the receiver can be collectively referred to as a transceiver.

[0053] The terminal device in the fourth aspect mentioned above can be a chip, and the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0054] In a fifth aspect, a computer program product is provided, comprising: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute a method in any possible implementation of the first aspect.

[0055] In a sixth aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instruction) which, when executed on a computer, enables the computer to execute a method in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0057] Figure 2 A software structure diagram of a terminal device provided in an embodiment of the present application;

[0058] Figure 3 A scene diagram applicable to the embodiments of the present application;

[0059] Figure 4 A schematic flowchart of a call screen projection method provided in an embodiment of the present application;

[0060] Figure 5 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of a display interface of a first terminal device applicable to an embodiment of the present application;

[0062] Figure 7 A schematic diagram of a display interface of another first terminal device applicable to the embodiment of the present application;

[0063] Figure 8 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0064] Fig. 9 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0065] Fig.10 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0066] Fig.11 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0067] Fig.12 A schematic flowchart of another call screen projection method provided in an embodiment of the present application;

[0068] Fig.13 A schematic block diagram of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0070] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially identical functions and effects. For example, the first value and the second value are only used to distinguish between different values, and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0071] It should be noted that in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0072] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0073] In order to better understand the terminal device in the embodiment of the present application, Figure 1 The hardware structure of the terminal device in the embodiment of the present application is described in detail.

[0074] Figure 1 This is a schematic diagram of the structure of the terminal device 100 provided in the embodiment of the present application. Figure 1 As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0075] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0076] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0077] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0078] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0079] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0080] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 100.

[0081] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0082] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0083] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0084] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 100.

[0085] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0086] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal device 100, and can also be used to transmit data between the terminal device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0087] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0088] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal device 100. While the charging management module 140 is charging the battery 142, it may also power the terminal device through the power management module 141.

[0089] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0090] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0091] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0092] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0093] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0094] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0095] In some embodiments, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0096] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0097] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0098] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0099] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0100] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0101] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0102] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0103] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0104] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0105] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0106] The terminal device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0107] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0108] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0109] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0110] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 100 can be provided with at least one microphone 170C. In other embodiments, the terminal device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 100 can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, realization of directional recording function, etc.

[0111] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0112] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 100 detects the touch operation intensity according to the pressure sensor 180A. The terminal device 100 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0113] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0114] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0115] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0116] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in various directions (generally three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0117] The distance sensor 180F is used to measure the distance. The terminal device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0118] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light outward through the light emitting diode. The terminal device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can use the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0119] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket to prevent accidental touch.

[0120] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0121] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 heats the battery 142 to avoid abnormal shutdown of the terminal device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0122] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 100, which is different from the position of the display screen 194.

[0123] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0124] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100.

[0125] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0126] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0127] The SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the terminal device 100 by inserting the SIM card interface 195 or pulling it out from the SIM card interface 195. The terminal device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to realize functions such as calls and data communications. In some embodiments, the terminal device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 100 and cannot be separated from the terminal device 100. The software system of the terminal device 100 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the terminal device 100.

[0128] Figure 2 This is a software structure block diagram of the Android system of the terminal device 100 according to an embodiment of the present application.

[0129] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: application layer, application framework layer, Android runtime and system library, hardware abstraction layer (HAL) layer and kernel layer.

[0130] The application layer can include a series of application packages.

[0131] like Figure 2 As shown, in the embodiment of the present application, the application package includes at least display manager, camera, Bluetooth, WLAN and other application packages.

[0132] The display manager can call the system library and kernel layer according to the user's operation.

[0133] The display manager may also be referred to as a personal computer explorer (also abbreviated as PCE) module, which is not limited in this application.

[0134] It should be understood that Figure 2Just as an example, the application package may also include application packages such as gallery, calendar, call, map, navigation, music, video, short message, third-party application, etc.

[0135] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0136] like Figure 2 As shown, in an embodiment of the present application, the application framework layer includes at least a screen projection manager, a multi-screen interaction module, a security module, a communication service manager, a wireless display module, a display manager, a connection manager, etc.

[0137] The screen casting manager is used to initiate screen casting and make screen casting connections.

[0138] The screen projection manager can also be called the Cast+ module or the CastPlus module, which is not limited in this application.

[0139] The multi-screen interaction module is used to initiate screen projection and to manage the data transmission channels maintained by the kernel layer to achieve data transmission across devices.

[0140] The multi-screen interaction module may also be referred to as an Airsharing module, which is not limited in this application.

[0141] Both the security module and the communication service manager can realize data transmission across devices. The communication service manager supports both the creation of channels using new protocols (i.e., the communication service management magiclink channel) and the creation of channels using old protocols (i.e., socket channels), but the security module only supports the creation of channels using new protocols. In an embodiment of the present application, the video or image data stream captured by the camera of the second terminal device and the sound data stream collected by the microphone of the second terminal device are transmitted to the first terminal device through the communication service manager or the security module.

[0142] The security module may also be referred to as a HiChain module, etc., and this application does not limit this.

[0143] The communication service manager may also be called a smart interconnection module, a MagicLink transmission management service module, etc. This application does not make any specific limitations on this.

[0144] The wireless display module can realize the display of the video call interface across devices. In the embodiment of the present application, the video call interface of the first terminal device is displayed on the second terminal device through the wireless display module.

[0145] The wireless display module may also be referred to as a MiraCastTV module, etc., which is not limited in this application.

[0146] The connection manager can assist the display manager to call the system library and kernel layer according to the user's operation.

[0147] The connection manager may also be referred to as a personal computer assistant (also abbreviated as PCA) module, which is not limited in this application.

[0148] It should be understood that Figure 2 By way of example only, the application framework layer may also include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0149] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0150] Content providers are used to store and retrieve data and make it accessible to applications. The above data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0151] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0152] The phone manager is used to provide communication functions of the terminal device 100, such as management of call status (including connection, disconnection, etc.).

[0153] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0154] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0155] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0156] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0157] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0158] The system library can include multiple functional modules. Figure 2 As shown, in the embodiment of the present application, the system library includes at least: a device virtualization module and a multi-screen interaction module.

[0159] The device virtualization module is used to initiate screen projection and / or make screen projection connections, and is also used to manage the data transmission channels maintained by the communication service manager to achieve data transmission across devices.

[0160] The device virtualization module may also be called a hardware service cross-device sharing module, a distributed mobile sensing development platform (DMSDP), a hardware virtualization module, etc. This application does not make any specific limitations on this.

[0161] It should be understood that the system library may also include: a surface manager, media libraries, a 3D graphics processing library (eg, OpenGL ES), a 2D graphics engine (eg, SGL), and the like.

[0162] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0163] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0164] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0165] A 2D graphics engine is a drawing engine for 2D drawings.

[0166] The kernel layer is used to perform functions such as managing system processes, memory, device drivers and files. In the embodiment of the present application, the kernel layer at least includes: a screen projection manager, a device scanning module, an information synchronization module, a camera driver, a microphone driver and a speaker driver.

[0167] The device scanning module is used to manage the Bluetooth driver.

[0168] The device scanning module may also be referred to as a NearBy module.

[0169] The information synchronization module is used to synchronize the changed information in the information synchronization module with other information synchronization modules in the same local area network, and the information synchronization module is also used to discover surrounding devices. In the embodiment of the present application, the device that logs in the same device account as the first terminal device and is in the same local area network is discovered through the information synchronization module.

[0170] The information synchronization module may also be referred to as a Profile module, which is not limited in this application.

[0171] Combine the following Figure 3 , taking the first terminal device as a mobile phone and the second terminal device as a smart TV as an example, the application scenario of this application is described in detail.

[0172] Screen projection technology can display the screen of one terminal device on the screen of another terminal device. Currently, screen projection is mostly used in the viewing process of pictures and videos. For example, users can project pictures or videos on their mobile phones to devices with larger screens such as TVs and projectors to improve the viewing experience.

[0173] With the development of terminal technology and the improvement of people's living standards, users still have the demand to cast calls. Calls include video calls and voice calls. The current call casting technology can only project the call interface of one terminal device (generally a small-screen device) onto another terminal device (generally a large-screen device). Taking mobile phones and TVs as an example, the user is using the mobile phone to make a call and projects the call interface onto the TV. During the call, the user is still using the phone's camera, microphone, speaker and other hardware devices. On the one hand, this screen casting method does not reasonably utilize the hardware resources of the terminal device (i.e., the large-screen device) being cast. On the other hand, this screen casting method requires the user to hold the mobile phone, and the user experience is poor.

[0174] Taking video calls as an example, due to the limited popularity of video call applications on smart screens and the limited number of users, users currently mainly use small-screen devices such as mobile phones and tablets to make video calls, and then use the large screen of large-screen devices such as smart TVs to experience large-screen services through screen projection. Specifically, when user A uses a mobile phone to make a video call with user B, he can select a smart TV for screen projection, and display the video call interface on the mobile phone screen (that is, the screen of user B) on the screen of the smart TV, and at the same time play the sound of the video call played on the mobile phone (that is, the voice of user B) through the speakers of the smart TV, such as Figure 3 As shown, user A can use the smart TV to watch user B's image and play user B's audio, thereby enjoying the large-screen experience of the video call service.

[0175] However, the video call service is still provided by the mobile phone. In order for user B to see user A's screen, user A needs to be within the screen acquisition range of the mobile phone's camera. Since the screen acquisition range of the mobile phone camera is small, user A needs to hold the mobile phone and aim the mobile phone camera at the target object that he wants to show to user B, which does not provide a good user experience. Especially for scenarios such as multi-person meetings and family gatherings where multiple users need to use one camera for video calls at the same time, using the mobile phone's camera will cause users to be very crowded and the user experience is poor.

[0176] When user A uses a mobile phone to make a voice call with user B by means of screen projection, the voice call service is still provided by the mobile phone. In order for user A to hear user B's voice, user A needs to be within the range of the mobile phone's speaker. Since the volume of the mobile phone's speaker is low, user A needs to stay close to the mobile phone, which greatly restricts the user's activities. Especially in a noisy external environment, it is difficult for the user to hear the sound played by the mobile phone speaker, and the user experience is poor.

[0177] In view of this, the present application uses screen projection technology to realize the flow of services between different devices, and can transfer the services provided by one terminal device to another terminal device, so that the other terminal device can continue to provide corresponding services to the user. The above services can specifically be voice calls, video calls, picture displays and other services. For video call services, user A can use the camera of the smart TV to collect the picture data of user A, and can use the microphone of the smart TV to collect the sound data of user A; for voice call services, user A can use the speakers of the smart TV to play the voice of user B; for video playback services, user A can use the speakers of the smart TV to play the video sound, which improves the user experience.

[0178] Combine the following Figures 4 to 12, the call screen projection method provided by this application is described in detail.

[0179] Figure 4 A schematic flow chart of a call projection method 400 provided in an embodiment of the present application. The method 400 is applied to a communication system including a first terminal device and a second terminal device, where a Bluetooth wireless communication connection is established between the first terminal device and the second terminal device, or a user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network. The hardware structures of the first terminal device and the second terminal device can be as follows: Figure 1 As shown, the software structures of the first terminal device and the second terminal device can be as follows Figure 2 The method 400 may include:

[0180] S401. When detecting a user's selection operation on a second terminal device in a screen projection list, the first terminal device obtains device status information of the second terminal device, where the device status information is used to indicate whether a hardware device of the second terminal device can be used for screen projection.

[0181] The above device status information can also be understood as whether the second terminal device is equipped with a hardware device for screen projection, and / or whether the hardware device of the second terminal device is occupied. The above device status information can also be understood as whether the second terminal device supports distributed screen projection.

[0182] Exemplarily, for the video call service, if the second terminal device is provided with a camera, a microphone and a speaker, and / or the camera, the microphone and the speaker of the second terminal device are not occupied, the hardware device of the second terminal device can be used for screen projection, that is, the second terminal device supports distributed screen projection.

[0183] The above-mentioned device status information may include a device identifier and a distributed status identifier, wherein the device identifier is used to indicate the second terminal device, and the distributed identifier is used to indicate whether the hardware device can be used for screen projection, that is, whether it supports distributed screen projection.

[0184] The device identifier may be a device name, a network address, or other identifier that can be used to identify the second terminal device, which is not limited in this application. Among them, the device name may be a name preset when the second terminal device leaves the factory, or a name modified by the user, which is not limited in this application. The network address may include an Internet Protocol (IP) address and / or a Media Access Control (MAC) address. Exemplarily, the device identifier is the second terminal device.

[0185] The distributed status identifier can be represented by English characters. Exemplarily, if the second terminal device supports distributed screen projection, the distributed status identifier of the second terminal device may be true; if the second terminal device does not support distributed screen projection, the distributed status identifier of the second terminal device is false. The distributed status identifier of the second terminal device can also be represented by a string. Exemplarily, if the second terminal device supports distributed screen projection, the distributed status identifier of the second terminal device may be 1; if the second terminal device does not support distributed screen projection, the distributed status identifier of the second terminal device is 0. This application does not limit the specific form of the distributed status identifier of the second terminal device.

[0186] S402. When the device status information indicates that the hardware device of the second terminal device can be used for screen projection, the first terminal device and the second terminal device create a first data transmission channel and a second data transmission channel. The first data transmission channel is used to transmit call data collected by the second terminal device through some devices in the hardware device, and the second data transmission channel is used to transmit call data obtained by the first terminal device through the network.

[0187] Taking user A using the first terminal device to make a video call with user B as an example, the first data transmission channel is used by the second terminal device to send the image data of user A collected by the second terminal device through the camera and the voice data of user A collected by the second terminal device through the microphone to the first terminal device; the second data transmission channel is used by the first terminal device to send the image data of user B and the voice data of user B obtained by the first terminal device through the network to the second terminal device.

[0188] Taking user A using the first terminal device to have a voice call with user B as an example, the first data transmission channel is used for the second terminal device to send the voice data of user A collected by the second terminal device through a microphone to the first terminal device; the second data transmission channel is used for the first terminal device to send the voice data of user B obtained by the first terminal device through the network to the second terminal device.

[0189] S403: The first terminal device performs a screen projection operation, and transmits call data to the second terminal device using the first data transmission channel and the second data transmission channel.

[0190] In the call projection method provided in the embodiment of the present application, when a user selects a second terminal device in the projection list, the first terminal device obtains device status information of the second terminal device. When the device status information indicates that some of the hardware devices of the second terminal device can be used for projection, the first terminal device and the second terminal device create a first data transmission channel and a second data transmission channel. The first terminal device performs the projection operation and uses the first data transmission channel and the second data transmission channel to transmit call data to the second terminal device. Screen projection technology can be used to realize the flow of services between different devices, thereby improving user experience.

[0191] In the above S401, when it is detected that the user has performed a selection operation on the second terminal device in the screen projection list, the first terminal device obtains the device status information of the second terminal device, including two possible implementation methods.

[0192] In a first possible implementation, a Bluetooth wireless communication connection is established between a first terminal device and a second terminal device. In response to a user clicking a screen projection control in a drop-down menu, the first terminal device scans terminal devices within a preset range that have established wireless communication connections with the first terminal device, and displays a list of projection devices, which includes an identifier of the second terminal device. In response to a user selecting the second terminal device, the first terminal device obtains device status information of the second terminal device via Bluetooth.

[0193] The device status information of the second terminal device is broadcast by the second terminal device through the Bluetooth of the second terminal device.

[0194] In this way, the first terminal device and the second terminal device do not need to be connected to the local area network, and screen projection can be performed only through Bluetooth, which reduces the power consumption of the first terminal device and the second terminal device to a certain extent.

[0195] In a second possible implementation, the user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network. In response to the user clicking on the control center control, the first terminal device scans the terminal devices that are logged in to the same account as the first terminal device and are in the same local area network, and displays a list of projection devices, which includes the identifier of the second terminal device; in response to the user selecting the second terminal device, the first terminal device obtains the device status information of the second terminal device through the information synchronization module.

[0196] It should be understood that the second terminal device can store the device status information of the second terminal device in the information synchronization module of the second terminal device, so the first terminal device can interact with the information synchronization module of the second terminal device through the information synchronization module of the first terminal device to obtain the device status information of the second terminal device.

[0197] In this way, because the first terminal device and the second terminal device are logged in to the same device account, there is no need to authenticate the second terminal device through the first device virtualization module, which reduces the screen projection connection time to a certain extent and improves the user experience.

[0198] The above two possible implementation methods both enable the first terminal device to obtain the device status information of the second terminal device, and then display the name of the connectable device to the user. When the screen projection cannot be initiated from the drop-down menu interface due to reading errors or other reasons, the user can initiate the screen projection from the control center interface. When the screen projection cannot be initiated from the control center interface, the user can initiate the screen projection from the drop-down menu interface, thereby improving the user experience.

[0199] As an optional embodiment, the second data transmission channel in the above S402 includes two possible implementation methods.

[0200] In a first possible implementation, when both the first terminal device and the second terminal device support the new protocol, a second data transmission channel (also referred to as a communication service management magic link channel) is created based on the new protocol, wherein the single data transmission amount supported by the new protocol is greater than the single data transmission amount supported by the old protocol.

[0201] In this way, the data transmission efficiency between the first terminal device and the second terminal device can be improved by increasing the single data transmission amount.

[0202] In a second possible implementation, when the first terminal device and / or the second terminal device does not support the new protocol, a second data transmission channel (also referred to as a socket channel) is created based on the old protocol, wherein the single data transmission amount supported by the new protocol is greater than the single data transmission amount supported by the old protocol.

[0203] Exemplarily, the old protocol may be a socket protocol.

[0204] In this way, for devices that support the new protocol (referred to as new devices for short), the data transmission channel between the multi-screen interaction module created based on the new protocol and the screen projection manager can be used to transmit the business data obtained by the first terminal device through the network. For devices that support the old protocol (referred to as old devices for short), the data transmission channel between the multi-screen interaction module created based on the old protocol and the screen projection manager can be used to transmit the business data obtained by the first terminal device through the network, thereby ensuring the compatibility of new and old devices.

[0205] As an optional embodiment, the above-mentioned determination of whether the protocol supported by the first terminal device and the second terminal device is a new protocol or an old protocol may be made by comparing the version number of the communication service manager with a preset version number.

[0206] Specifically, for the first terminal device, the version number of the communication service manager of the first terminal device is compared with the preset version number. If the version number of the communication service manager of the first terminal device is greater than or equal to the preset version number, it is determined that the first terminal device supports the new protocol; if the version number of the communication service manager of the first terminal device is less than the preset version number, it is determined that the first terminal device does not support the new protocol.

[0207] Specifically, for the second terminal device, the version number of the communication service manager of the first terminal device is compared with the preset version number. If the version number of the communication service manager of the second terminal device is greater than or equal to the preset version number, it is determined that the second terminal device supports the new protocol; if the version number of the communication service manager of the second terminal device is less than the preset version number, it is determined that the second terminal device does not support the new protocol.

[0208] Taking the above call as a video call as an example, the above hardware devices include a camera, a microphone and a speaker.

[0209] In the case where the camera, microphone and speaker can all be used for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device can be used for screen projection. The above S403 can specifically be: the second terminal device collects the first data through the camera and microphone, and transmits the first data to the first terminal device through the first data transmission channel; the first terminal device receives the first data through the first data transmission channel; the first terminal device sends the second data obtained by the first terminal device through the network to the second terminal device through the second data transmission channel; the second terminal device receives the second data through the second data transmission channel; the second terminal device plays the voice data in the second data through the speaker, and displays the image data in the second data through the display screen of the second terminal device.

[0210] Among them, the camera, microphone and speaker can all be used for screen projection, which can be understood as the second terminal device is equipped with a camera, microphone and speaker, or it can be understood as the camera, microphone and speaker are not occupied.

[0211] It should be understood that in the case that one of the camera, microphone and speaker is not available for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device is not available for screen projection.

[0212] Among them, the fact that one of the camera, microphone and speaker cannot be used for screen projection can be understood as that at least one of the camera, microphone and speaker is not provided on the second terminal device, or it can be understood as that at least one of the camera, microphone and speaker is occupied.

[0213] Taking the above call as a voice call as an example, the above hardware device includes a microphone and a speaker.

[0214] In the case where both the microphone and the speaker can be used for screen projection, the above device status information is information used to indicate that the hardware device of the second terminal device can be used for screen projection. The above S403 can specifically be: the second terminal device collects the first data through the microphone, and transmits the first data to the first terminal device through the first data transmission channel; the first terminal device receives the first data through the first data transmission channel; the first terminal device sends the second data obtained by the first terminal device through the network to the second terminal device through the second data transmission channel; the second terminal device receives the second data through the second data transmission channel; the second terminal device plays the voice data in the second data through the speaker, and displays the image data in the second data through the display screen of the second terminal device.

[0215] Among them, both the microphone and the speaker can be used for screen projection, which can be understood as the second terminal device is equipped with a microphone and a speaker, or it can be understood as the microphone and the speaker are not occupied.

[0216] It should be understood that when the microphone and / or speaker are not available for screen projection, the above-mentioned device status information is information used to indicate that the hardware device of the second terminal device is not available for screen projection.

[0217] Among them, the fact that the microphone and / or speaker cannot be used for screen projection can be understood as the second terminal device is not equipped with a microphone and / or speaker, or it can be understood as the microphone and / or speaker is occupied.

[0218] In this way, the user does not need to hold the first terminal device, and the user's image and sound can be collected through the second terminal device, freeing the user's hands and making it convenient for the user to perform other activities during the video call, such as doing housework, watching short videos using the first terminal device, etc., thereby improving the user experience.

[0219] The following is a description of the call screen projection method provided by this application in conjunction with the interaction process between the modules involved in the process of a user using the first terminal device to project the screen to the second terminal device. Among them, the first terminal device includes a display manager, a connection manager, a first device virtualization module, a multi-screen interaction module, a first device scanning module, a first communication service manager, a first security module, a first information synchronization module and a first kernel layer. The second terminal device includes: a wireless display module, a second device virtualization module, a screen projection manager, a second device scanning module, a second communication service manager, a second security module, a second information synchronization module and a second kernel layer.

[0220] The process of a user using a first terminal device to project a screen to a second terminal device can be divided into three stages: device discovery, device connection, and service data transmission. Figure 5 , explaining the interaction process between the modules involved in each stage.

[0221] Figure 5 A schematic flowchart of a call screen projection method 500 provided in an embodiment of the present application.

[0222] 1. Device discovery phase.

[0223] After the second terminal device is turned on, the wireless display module and the second device virtualization module start running. The second device virtualization module passes the detected distributed status identifier of the second terminal device to the second device scanning module layer by layer through the wireless display module and the screen projection manager, and calls the second terminal device's Bluetooth broadcast device identifier and the distributed status identifier of the second terminal device through the second device scanning module. At the same time, the second device virtualization module directly passes the distributed status identifier to the second information synchronization module, and the second information synchronization module stores the distributed status identifier of the second terminal device.

[0224] Among them, the distributed state identifier of the second terminal device is a state identifier used to indicate whether the second terminal device supports distributed screen projection. For video call services, if the second terminal device can detect a camera, a microphone, and a speaker, the second terminal device supports distributed screen projection; if the second terminal device cannot detect at least one of the hardware devices such as a camera, a microphone, and a speaker, the second terminal device does not support distributed screen projection. For voice call services, if the second terminal device can detect a microphone and a speaker, the second terminal device supports distributed screen projection; if the second terminal device cannot detect a microphone and / or a speaker, the second terminal device does not support distributed screen projection. The second terminal device being able to detect the hardware device can be understood as the second terminal device being provided with the hardware device, and the second terminal device being unable to detect the hardware device can be understood as the second terminal device not being provided with the hardware device. Optionally, the second terminal device being able to detect the hardware device can also be understood as the second terminal device being provided with the hardware device, and the hardware device not being called by other services, and the second terminal device being unable to detect the hardware device can also be understood as the second terminal device not being provided with the hardware device and / or the hardware device being called by other services. The distributed state identifier of the second terminal device can be represented by English characters. Exemplarily, if the second terminal device supports distributed screen projection, the distributed status identifier of the second terminal device may be true; if the second terminal device does not support distributed screen projection, the distributed status identifier of the second terminal device is false. The distributed status identifier of the second terminal device may also be represented by a string. Exemplarily, if the second terminal device supports distributed screen projection, the distributed status identifier of the second terminal device may be 1; if the second terminal device does not support distributed screen projection, the distributed status identifier of the second terminal device is 0. This application does not limit the specific form of the distributed status identifier of the second terminal device.

[0225] The device identification of the second terminal device may be the device name of the second terminal device, or the network address of the second terminal device, or other identification that can be used to identify the second terminal device. The device name of the second terminal device may be a name preset when the second terminal device leaves the factory, or a name modified by the user, which is not limited in this application. Exemplarily, the device name of the second terminal device is My Device. The network address of the second terminal device may include an Internet Protocol (IP) address and / or a Media Access Control (MAC) address.

[0226] It should be understood that the device information of the second terminal device broadcast by the Bluetooth of the second terminal device can only be received by devices that are within the Bluetooth communication range of the second terminal device, and the device information of the second terminal device broadcast by the second information synchronization module can only be received through device self-discovery by devices that have logged in to the same device account as the second terminal device and are in the same local area network as the second terminal device.

[0227] Taking the first terminal device as a mobile phone and the second terminal device as a smart TV as an example, when the user uses the first terminal device to initiate screen projection, there are two possible implementation methods: initiating screen projection from the drop-down menu interface and initiating screen projection from the control center interface.

[0228] In a first possible implementation, a Bluetooth wireless communication connection is established between the first terminal device and the second terminal device. The user uses the first terminal device to initiate screen projection from a drop-down menu interface. In response to the user's click operation on the screen projection control (such as Figure 6 As shown in a), the display manager passes the scanning instruction to the first device scanning module through the multi-screen interaction module, and calls the Bluetooth of the first terminal device through the first device scanning module to scan the devices (such as the second terminal device) within the Bluetooth communication range of the first terminal device, and displays the names of devices that can be connected for screen projection reported layer by layer by the Bluetooth of the first terminal device through the first device scanning module and the multi-screen interaction module in the form of a pop-up window. At the same time, the multi-screen interaction module stores the distributed status identifier reported by the Bluetooth of the first terminal device through the first device scanning module.

[0229] In a second possible implementation, a user can use the first terminal device to initiate screen projection from the control center interface when the first terminal device and the second terminal device have logged in to the same device account and the first terminal device and the second terminal device are in the same local area network. In response to the user's click operation on the control center control, the display manager passes the scanning instruction to the first information synchronization module layer by layer through the connection manager and the first device virtualization module. The first information synchronization module performs device self-discovery on the device (such as the second terminal device) in the same local area network as the first terminal device, thereby synchronizing information with the device identification and distributed status identification of the second terminal device stored in the second information synchronization module. The first information synchronization module stores the discovered device identification and distributed status identification, and reports the device identification that can be connected for screen projection to the display manager layer by layer through the first device virtualization module and the connection manager. The display manager displays the devices that can be connected for screen projection in the control center interface.

[0230] Both of the above two possible implementation methods enable users to initiate screen projection. When screen projection cannot be initiated from the drop-down menu interface due to reading errors or other reasons, users can initiate screen projection from the control center interface. When screen projection cannot be initiated from the control center interface, users can initiate screen projection from the drop-down menu interface, thereby improving the user experience.

[0231] 2. Device connection phase.

[0232] For the first possible implementation method described above, when the user uses the first terminal device to select the second terminal device from the device list pop-up window for screen projection connection (such as Figure 6 As shown in b), in response to the user's click operation on the device name of the second terminal device, the display manager obtains the distributed state identifier of the second terminal device from the multi-screen interaction module, and determines whether the second terminal device supports distributed screen projection. In the case where the second terminal device supports distributed screen projection, the display manager passes the connection instruction to the first device virtualization module through the connection manager, and the first device virtualization module authenticates the second terminal device. Exemplarily, the first device virtualization module authenticates the second terminal device through a personal identification number (PIN) entered by the user. If the authentication is successful, the first device virtualization module calls the Bluetooth of the first terminal device through the first device scanning module to send a connection request to the second terminal device, requesting the creation of a data transmission channel between the first device virtualization module and the second device virtualization module. Correspondingly, the second terminal device receives the connection request through the Bluetooth of the second terminal device, and passes the connection request to the second device virtualization module through the second device scanning module, forming a data transmission channel between the first device virtualization module and the second device virtualization module. Among them, taking user A using the first terminal device to make a video call with user B as an example, the data transmission channel between the first device virtualization module and the second device virtualization module is used to send the data collected by the camera and microphone of the second terminal device from the second terminal device to the first terminal device, and the first terminal device then sends the data collected by the camera and microphone of the second terminal device to the terminal device of user B (hereinafter referred to as the third terminal device) through the network. In addition, regardless of whether the second terminal device supports distributed screen projection, the display manager also needs to call the multi-screen interaction module to send a connection request to the second terminal device through the first communication service manager, requesting the creation of a data transmission channel between the multi-screen interaction module and the screen projection manager. The second terminal device receives the connection request through the second communication service manager, and passes the connection request to the screen projection manager to form a data transmission channel between the multi-screen interaction module and the screen projection manager. Among them, the data transmission channel between the multi-screen interaction module and the screen projection manager is used to receive data corresponding to the content displayed or played on the screen of the first terminal device.

[0233] In this way, the first terminal device and the second terminal device do not need to be connected to the local area network, and screen projection can be performed only through Bluetooth, which reduces the power consumption of the first terminal device and the second terminal device to a certain extent.

[0234] For the second possible implementation, when the user uses the first terminal device to select the second terminal device from the control center interface for screen projection connection, in response to the user's pinch operation of the icon of the first terminal device and the icon of the second terminal device (such as Figure 7 As shown), the display manager reads the distributed status identifier of the second terminal device stored in the first information synchronization module through the connection manager and the first device virtualization module, and determines whether the second terminal device supports distributed screen projection.

[0235] It should be understood that the interaction processes between the remaining modules in the second possible implementation manner can refer to the description in the first possible implementation manner, and will not be repeated here.

[0236] In this way, because the first terminal device and the second terminal device are logged in to the same device account, there is no need to authenticate the second terminal device through the first device virtualization module, which reduces the screen projection connection time to a certain extent and improves the user experience.

[0237] 3. Business data transmission stage.

[0238] Taking user A using the first terminal device to make a video call with user B as an example, the business data transmission process after a data transmission channel is established between the first terminal device and the second terminal device (it can also be understood as the device connection between the first terminal device and the second terminal device is completed) is explained.

[0239] When user A uses the first terminal device to make a video call with user B, the first kernel layer obtains the data corresponding to the video call interface of user B and the voice of user B displayed on the screen of the first terminal device through the network (also referred to as the business data of user B), and passes it to the multi-screen interaction module, and the multi-screen interaction module sends the business data of user B to the projection manager through the data transmission channel between the multi-screen interaction module and the projection manager. The projection manager passes the received business data of user B to the second kernel layer and the wireless display module respectively. The second kernel layer calls the speaker through the speaker driver to play the voice of user B, and the wireless display module displays the video call interface of user B. If the second device virtualization module detects the camera and the microphone, the second device virtualization module calls the camera through the camera driver to collect the video call interface of user A, and calls the microphone through the microphone driver to collect the voice of user A (also referred to as the business data of user A), and sends the collected business data of user A to the second device virtualization module through the data transmission channel between the first device virtualization module and the second device virtualization module. The second device virtualization module passes the received business data of user A to the first kernel layer, and uses the first kernel layer to send the business data of user A to the third terminal device through the network.

[0240] In this way, the user does not need to hold the first terminal device, and the user's image and sound can be collected through the second terminal device, freeing the user's hands and making it convenient for the user to perform other activities during the video call, such as doing housework, watching short videos using the first terminal device, etc., thereby improving the user experience.

[0241] Combine the following Figure 8 , taking the device identifier of the second terminal device as the device name as an example, Figure 5 The interaction process between the modules involved in the device discovery phase is further described in detail.

[0242] Figure 8 A schematic flowchart of a call screen projection method 800 provided in an embodiment of the present application.

[0243] After the second terminal device is turned on, the wireless display module and the second device virtualization module start running, and the wireless display module passes the initialization instruction to the screen projection manager to initialize the screen projection manager and configure the second terminal device for the working environment of receiving the screen projection. The second device virtualization module obtains the distributed state of the second terminal device by detecting the hardware devices of the second terminal device such as the camera, microphone and speaker, and passes the distributed state identifier to the second information synchronization module, and also passes the distributed state identifier to the second device scanning module layer by layer through the wireless display module and the screen projection manager, and calls the Bluetooth broadcast of the second terminal device through the second device scanning module. The device identifier and distributed state identifier of the second terminal device and the second terminal device are broadcast.

[0244] When the user uses the first terminal device to initiate screen projection from the drop-down menu interface, in response to the user's click operation on the screen projection control, the display manager passes the scan request to the first Bluetooth layer by layer through the multi-screen interaction module and the first device scanning module. The first Bluetooth scans the devices (such as the second terminal device) within the Bluetooth communication range of the first terminal device. The first Bluetooth reports the scanned device name and distributed status identifier to the multi-screen interaction module layer by layer through the first Bluetooth and the first device scanning module for storage, and the multi-screen interaction module reports the device name to the display manager. The display manager displays the device name in a pop-up window.

[0245] When the user uses the first terminal device to initiate screen projection from the control center interface, in response to the user's click operation on the control center control, the display manager calls the first information synchronization module layer by layer through the connection manager and the first device virtualization module. The first information synchronization module performs device self-discovery on devices (such as the second terminal device) in the same local area network as the first terminal device, thereby synchronizing information with the device name and distributed status identifier of the second terminal device stored in the second information synchronization module. The first information synchronization module stores the discovered device name and distributed status identifier, and reports the discovered device name to the display manager layer by layer through the first device virtualization module and the connection manager. The display manager displays the device name.

[0246] Combine the following Fig. 9 , taking the second terminal device providing a video call service to a user as an example, Figure 5 In the first possible implementation method (i.e., when the user uses the first terminal device to initiate screen projection from the drop-down menu interface), the interaction process between the modules involved in the device connection phase is further described in detail.

[0247] Fig. 9 The present invention provides a schematic flow chart of a call screen projection method 900 provided in an embodiment of the present application. The method 900 is executed by a first terminal device and includes:

[0248] S901. When a user uses a first terminal device to select a second terminal device from a device list pop-up window for screen projection connection, in response to a user clicking on the device name of the second terminal device, the display manager obtains a distributed status identifier of the second terminal device stored in the multi-screen interaction module.

[0249] S902. The display manager determines whether the second terminal device supports distributed screen projection based on the distributed state identifier of the second terminal device.

[0250] If the distributed state of the second terminal device indicates that it supports distributed screen projection (i.e., the second terminal device can detect the camera, microphone, and speaker), perform the following steps:

[0251] S903: The display manager pulls up the first device virtualization module through the connection manager to authenticate the second terminal device.

[0252] Exemplarily, the second terminal device is authenticated by a personal identification number (PIN) input by a user.

[0253] S904: The first device virtualization module determines whether the second terminal device passes the authentication.

[0254] If the authentication is successful, S905 is executed, where the display manager pulls up the first device virtualization module through the connection manager, and creates a data transmission channel between the first device virtualization module and the second device virtualization module.

[0255] Among them, taking user A using the first terminal device to make a video call with user B as an example, the data transmission channel between the first device virtualization module and the second device virtualization module is used to send the image data of user A collected by the camera of the second terminal device and the sound data of user A collected by the microphone of the second terminal device.

[0256] S906. The display manager pulls up the multi-screen interaction module and creates a data transmission channel between the multi-screen interaction module and the screen projection manager.

[0257] Among them, taking user A using the first terminal device to make a video call with user B as an example, the data transmission channel between the multi-screen interaction module and the screen projection manager is used to receive the picture data of user B and the sound data of user B obtained by the first terminal device from the third terminal device through the network.

[0258] If the authentication fails, the device cannot be connected, and S907 is executed to stop screen projection.

[0259] If the distributed state of the second terminal device is marked as not supporting distributed screen projection (i.e., the second terminal device cannot detect at least one of the hardware devices such as the camera, microphone, and speaker), method 900 directly executes S906 after the above S902. At this time, when user A uses the first terminal device to make a video call with user B, user A's picture data is collected through the camera of the first terminal device, user A's voice data is collected through the microphone of the first terminal device, user B's picture is displayed through the screen of the second terminal device, and user B's voice is played through the speaker of the second terminal device.

[0260] Combine the following Fig.10 ,right Figure 5 In the second possible implementation method (i.e., when the user uses the first terminal device to initiate screen projection from the control center interface), the interaction process between the modules involved in the device connection phase is further described in detail.

[0261] Fig.10 The present invention provides a schematic flow chart of a call screen projection method 1000 provided in an embodiment of the present application. The method 1000 is executed by a first terminal device and includes:

[0262] S1001. When a user uses a first terminal device to select a second terminal device from a control center interface for screen projection connection, in response to the user's pinching operation on the icon of the first terminal device and the icon of the second terminal device, the display manager reads the distributed status identifier of the second terminal device stored in the first information synchronization module through the connection manager and the first device virtualization module.

[0263] S1002. The display manager determines whether the reading is successful and whether the second terminal device supports distributed screen projection.

[0264] If the distributed state identifier of the second terminal device returned by the first information synchronization module indicates that distributed screen projection is supported, execute S1003, and the display manager pulls up the first device virtualization module through the connection manager to create a data transmission channel between the first device virtualization module and the second device virtualization module.

[0265] S1004. The display manager pulls up the multi-screen interaction module and creates a data transmission channel between the multi-screen interaction module and the screen projection manager.

[0266] If the distributed status of the second terminal device returned by the first information synchronization module indicates that distributed screen projection is not supported, method 1000 directly executes S1004 after the above S1002.

[0267] It should be understood that if the display manager fails to read the second terminal device distributed state identifier stored in the first information synchronization module, for example, the second terminal device distributed state identifier returned by the first information synchronization module is a null value, the device connection cannot be performed.

[0268] Combine the following Fig.11 ,right Figures 4 to 10 The data transmission channel between the multi-screen interaction module and the screen projection manager is further explained in detail.

[0269] For the call screen projection scenario, in order to ensure the unified planning of physical channels and the coexistence processing of multiple services, and considering that the switching of transmission channels involves the modification of both ends of the first terminal device and the second terminal device, which is easy to cause compatibility issues between new and old devices, the embodiment of the present application implements the data transmission channel between the two multi-screen interactive modules and the screen projection manager through the communication service manager and the security module. Specifically, it is divided into a new channel (i.e., magiclink channel) and an old channel (i.e., socket channel). The final effect is that as long as one of the first terminal device and the second terminal device is an old device, data transmission is carried out through the socket channel. Only when both the first terminal device and the second terminal device are new devices, data transmission is carried out through the magiclink channel.

[0270] Fig.11 A schematic flowchart of a call screen projection method 1100 provided in an embodiment of the present application.

[0271] When the display manager pulls up the multi-screen interaction module and creates a data transmission channel between the multi-screen interaction module and the projection manager, the first device virtualization module sends a connection request to the multi-screen interaction module, and the connection request carries the device unique identifier of the second terminal device. Based on the device unique identifier of the second terminal device, the multi-screen interaction module uses the first communication service manager to send a request to obtain the version number and device identifier of the communication service manager to the second terminal device, and obtains the version number of the first communication service manager stored in the first communication service manager. The second terminal device sends the version number and device identifier of the second communication service manager of the second terminal device to the first terminal device through the second communication service manager. The first terminal device passes the version number and device identifier of the second communication service manager of the second terminal device to the multi-screen interaction module through the first communication service manager. The multi-screen interaction module determines whether the version number of the first communication service manager and the version number of the second communication service manager included in the device information of the second terminal device are both higher than or equal to the preset communication service manager version number.

[0272] If the version number of the first communication service manager and the version number of the second communication service manager are both higher than or equal to the preset communication service manager version number (it can also be understood that the first terminal device and the second terminal device are both new devices), if the user casts the screen from the drop-down menu interface, the multi-screen interaction module uses the new protocol to initiate a connection to the second terminal device through the first communication service manager, and creates a new channel between the first communication service manager and the second communication service manager, which can also be understood as a new data transmission channel between the multi-screen interaction module and the projection manager (i.e., magiclink channel). If the user casts the screen from the control center interface, the multi-screen interaction module uses the new protocol to initiate a connection to the second terminal device through the first security module, and creates a new channel between the first security module and the second security module (i.e., magiclink channel).

[0273] If the second communication service manager version number and / or the first communication service manager version number is lower than the preset communication service manager version number (it can also be understood that the first terminal device and / or the second terminal device is an old device), no matter the user casts the screen from the drop-down menu interface or the control center interface, the multi-screen interaction module uses the old protocol to initiate a connection to the second terminal device through the first communication service manager, and creates a channel between the first communication service manager and the second communication service manager, which can also be understood as a data transmission channel (i.e., a socket channel) between the old multi-screen interaction module and the projection manager. Exemplarily, the old protocol can be a socket protocol.

[0274] Compared with the old protocol, the new protocol supports a larger single data transmission volume and higher data transmission efficiency.

[0275] It should be understood that after the wireless display module initializes the screen projection manager, the screen projection manager can register and callback the second communication service manager and the second security module, so that the screen projection manager can know whether the protocol used by the multi-screen interaction module is the old protocol or the new protocol, so that the screen projection manager can use the same protocol to establish a data transmission channel with the multi-screen interaction module.

[0276] In summary, there are two possible implementations of the data transmission channel between the multi-screen interaction module and the screen projection manager. Taking the case where user A uses the first terminal device to make a video call with user B who uses the third terminal device, and the second terminal device provides video call service for user A as an example, in the first possible implementation, the data transmission channel between the multi-screen interaction module and the screen projection manager is created using the old protocol, and is used for the new first terminal device or the old first terminal device to send user B's video call service data to the old second terminal device, or for the old first terminal device to send user B's video call service data to the new second terminal device; in the second possible implementation, the data transmission channel between the multi-screen interaction module and the screen projection manager is created using the new protocol, and is used for the new first terminal device to send user B's video call service data to the new second terminal device.

[0277] In this way, two types of data transmission channels can be created between the multi-screen interaction module and the projection manager according to the version number of the communication service manager. For new devices, the new data transmission channel between the multi-screen interaction module and the projection manager can be used to transmit the business data obtained by the first terminal device from the network device. For old devices, the data transmission channel between the multi-screen interaction module and the projection manager can be used to transmit the business data obtained by the first terminal device from the network device, thereby ensuring the compatibility of new and old devices.

[0278] Combine the following Fig.12 For example, when user A uses the first terminal device to make a video call with user B, and the second terminal device provides a video call service for user A, Figures 7 to 11 An overall description of the data transfer process involved is given.

[0279] Fig.12 A schematic flowchart of a call screen projection method 1200 provided in an embodiment of the present application.

[0280] After the second terminal device is turned on, the method 1200 includes S1201 to S1210.

[0281] S1201. The wireless display module passes the initialization instruction to the screen projection manager.

[0282] S1202. The screen projection manager creates a session control for the channel between the first communication service manager and the second communication service manager, and performs a registration callback.

[0283] The session control includes the session control of the socket channel and the session control of the magiclink channel.

[0284] S1203. The screen projection manager monitors the conversation with the second communication service manager.

[0285] The session monitoring includes monitoring the sessions of the socket channel and monitoring the sessions of the magiclink channel.

[0286] S1204. The screen projection manager creates a session control of the magiclink channel between the first security module and the second security module, and performs a registration callback.

[0287] S1205. The screen projection manager monitors the session of the second security module.

[0288] It should be understood that the order of S1202 and S1203 and S1204 and S1205 is not limited.

[0289] S1206. The second device virtualization module writes the distributed state identifier of the second terminal device into the second information synchronization module.

[0290] S1207. The second device virtualization module transmits the distributed state identifier of the second terminal device to the wireless display module.

[0291] It should be understood that the order of S1207 and S1206 is not limited.

[0292] S1208. The wireless display module transmits the distributed status identifier of the second terminal device to the screen projection manager.

[0293] S1209. The screen projection manager transmits the distributed status identifier of the second terminal device to the second device scanning module.

[0294] It should be understood that the screen projection manager can pass the distributed status identifier of the second terminal device to the second device scanning module by setting parameters in the second device scanning module.

[0295] Exemplarily, the screen projection manager sets the lowest bit parameter in the screen projection capability set array mBleCustData[1] to 1, thereby informing the second device scanning module that the second terminal device can use the camera, microphone, and speaker.

[0296] Optionally, the screen projection manager can also register and monitor the second device scanning module.

[0297] S1210. The second device scanning module calls the second Bluetooth broadcast device name and distributed status identifier of the second terminal device through the second Bluetooth driver.

[0298] The name of the second terminal device may be a name preset when the second terminal device leaves the factory, or a name modified by the user, and this application does not limit this.

[0299] Exemplarily, the name of the second terminal device may be the second terminal device of user A.

[0300] When user A initiates screen projection from the drop-down menu interface using the first terminal device, method 1200 also includes S1211 to S1214.

[0301] S1211. In response to user A's click operation on the screen projection control, the display manager passes the scanning instruction to the multi-screen interaction module.

[0302] S1212: The first device scanning module calls the first Bluetooth through the first Bluetooth driver to receive the device name and distributed state identifier of the second terminal device broadcast by the second Bluetooth.

[0303] S1213. The first device scanning module transmits the device name of the second terminal device to the display manager through the multi-screen interaction module, and transmits the distributed state identifier of the second terminal device to the multi-screen interaction module for storage.

[0304] S1214: The display manager displays the device name of the second terminal device in a pop-up window.

[0305] When user A uses the first terminal device to select the second terminal device from the device list pop-up window for screen projection connection, method 1200 also includes S1215.

[0306] S1215. In response to user A clicking on the device name of the second terminal device, the display manager determines whether the second terminal device supports distributed screen projection by reading the distributed status identifier of the second terminal device stored in the first device scanning module.

[0307] In the case where the second terminal device supports distributed screen projection, method 1200 also includes S1216.

[0308] S1216. The display manager pulls up the first device virtualization module through the connection manager to authenticate the second terminal device.

[0309] Exemplarily, the above authentication may be: the user enters the PIN code of the second terminal device in a password input pop-up window of the first terminal device.

[0310] When the authentication and verification is successful, method 1200 further includes S1217.

[0311] S1217: The display manager pulls up the first device virtualization module through the connection manager, and creates a data transmission channel between the first device virtualization module and the second device virtualization module.

[0312] The data transmission channel between the first device virtualization module and the second device virtualization module is used to transmit the call data collected by the camera and / or microphone of the second terminal device.

[0313] It should be understood that if the authentication and authorization fails, the user cannot use the first terminal device to project the screen to the second terminal device.

[0314] S1218. The first device virtualization module pulls up the multi-screen interaction module and creates a data transmission channel between the multi-screen interaction module and the screen projection manager.

[0315] The data transmission channel between the above-mentioned multi-screen interaction module and the screen projection manager is used to transmit the call data of user B obtained by the first terminal device from the network device.

[0316] When the second terminal device does not support distributed screen projection, method 1200 directly executes S1218 after S1215.

[0317] In a possible implementation, the above S1218 specifically includes S1219 to S1225.

[0318] S1219. The first device virtualization module transmits the connection request to the multi-screen interaction module, wherein the connection request carries the unique device identifier of the second terminal device.

[0319] S1220. The multi-screen interaction module obtains the version number of the first communication service manager and the version number of the second communication service manager by calling the first communication service manager.

[0320] S1221. The first communication service manager transmits the version number of the first communication service manager and the version number of the second communication service manager to the multi-screen interaction module.

[0321] S1222. The multi-screen interaction module determines whether the version number of the first communication service manager and the version number of the second communication service manager are both higher than or equal to the version number of the preset communication service manager.

[0322] When the version number of the first communication service manager and the version number of the second communication service manager are both higher than or equal to the preset version number of the communication service manager, if user A uses the first terminal device to initiate screen projection from the drop-down menu interface, method 1200 also includes S1223; if user A uses the first terminal device to initiate screen projection from the control center interface, S1223 can be replaced by S1224.

[0323] S1223. The multi-screen interaction module uses the new protocol to call the first communication service manager to initiate a connection to the second communication service manager.

[0324] S1224. The multi-screen interactive module uses the new protocol to call the first security module to initiate a connection to the second security module.

[0325] When the version number of the first communication service manager is lower than the version number of the preset communication service manager, and / or the version number of the second communication service manager is lower than the version number of the preset communication service manager, regardless of whether user A uses the first terminal device to initiate screen projection from the drop-down menu interface or uses the first terminal device to initiate screen projection from the control center interface, S1223 can be replaced by S1235.

[0326] S1235. The multi-screen interaction module uses the old protocol to call the first communication service manager to initiate a connection to the second communication service manager.

[0327] When user A uses the first terminal device to initiate screen projection from the control center interface, the above S1211 to S1216 can be replaced by S1226 to S1229.

[0328] S1226. In response to user A's click operation on the control center control, the first information synchronization module and the second information synchronization module perform device self-discovery, and the first information synchronization module stores the device name and distributed status identifier of the second terminal device.

[0329] S1227. The first information synchronization module transmits the device name of the second terminal device to the display manager.

[0330] S1228: The display manager displays the device name of the second terminal device.

[0331] When user A uses the first terminal device to select the second terminal device from the device list pop-up window for screen projection connection, method 1200 also includes S1229.

[0332] S1229. In response to user A's pinching operation on the icon of the first terminal device and the icon of the second terminal device, the display manager determines whether the second terminal device supports distributed screen projection.

[0333] When the second terminal device supports distributed screen projection, method 1200 directly executes S1217 after S1221.

[0334] When the second terminal device does not support distributed screen projection, method 1200 directly executes S1218 after S1221.

[0335] When user A uses the first terminal device to make a video call with user B, method 1200 also includes: S1230 to S1232.

[0336] S1230. The multi-screen interaction module reads the call data of user B in the first kernel layer, and transmits the call data of user B to the screen projection manager through the data transmission channel between the multi-screen interaction module and the screen projection manager.

[0337] It should be understood that the call data of user B is obtained by the first terminal device from the terminal device of user B through the network using the first kernel layer.

[0338] S1231: The screen projection manager transmits the received call data of user B to the wireless display module. The wireless display module displays the video call interface of user B based on the call data of user B.

[0339] S1232: The screen projection manager transmits the received call data of user B to the second kernel layer. The second kernel layer calls the speaker to play the voice of user B through the speaker driver based on the call data of user B.

[0340] It should be understood that S1231 and S1232 do not limit the order.

[0341] If the second terminal device supports distributed screen projection, method 1200 also includes S1233 to S1235.

[0342] S1233. The second device virtualization module calls the camera through the camera driver and calls the microphone through the microphone driver to collect the call data of user A.

[0343] S1234: The second device virtualization module sends the collected call data of user A to the first device virtualization module through the data transmission channel between the first device virtualization module and the second device virtualization module. Correspondingly, the first device virtualization module receives the call data of user A.

[0344] S1235. The first device virtualization module calls the first kernel layer to send the received call data of user A to the terminal device of user B through the network.

[0345] Combination of the above Figures 4 to 12 , describes in detail the call screen projection method of the embodiment of the present application, and then combines Fig.13 , describe in detail the terminal device of an embodiment of the present application.

[0346] Fig.13A schematic block diagram of a terminal device 1300 provided in an embodiment of the present application. The terminal device 1300 includes a processor 1301, a communication interface 1302, and a memory 1303. Among them, the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through an internal connection path, the memory 1303 is used to store instructions, and the processor 1301 is used to execute the instructions stored in the memory 1303. The communication interface 1302 can be used to send signals to other terminal devices (such as the processor 1301 or the touch screen of the terminal device), and can also be used to receive signals from other terminal devices (such as the memory 1303). Exemplarily, the communication interface 1302 reads the instructions stored in the memory 1303 and sends the instructions to the processor 1301.

[0347] It should be understood that the terminal device 1300 can be specifically the first terminal device or the second terminal device in the above embodiment, and can be used to execute the various steps and / or processes corresponding to the first terminal device or the second terminal device in the above method embodiment. Optionally, the memory 1303 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1301 may be used to execute instructions stored in the memory, and when the processor 1301 executes instructions stored in the memory, the processor 1301 is used to execute the various steps and / or processes of the above method embodiment.

[0348] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0349] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0350] The present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the method shown in the above method embodiment.

[0351] The present application also provides a computer program product, which includes a computer program (also referred to as code or instruction). When the computer program runs on a computer, the computer can execute the method shown in the above method embodiment.

[0352] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0353] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0354] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0355] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0356] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0357] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0358] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, which should be included in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Claims

1. A method for projecting screen during a call, characterized in that: Applied to a communication system including a first terminal device and a second terminal device, a Bluetooth wireless communication connection is established between the first terminal device and the second terminal device, or a user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network, the method comprising: When detecting a selection operation of the user on the second terminal device in the screen projection list, the first terminal device acquires device status information of the second terminal device, where the device status information is used to indicate whether the hardware device of the second terminal device can be used for screen projection; When the device status information indicates that the hardware device of the second terminal device can be used for screen projection, the first terminal device and the second terminal device create a first data transmission channel and a second data transmission channel, the first data transmission channel is used to transmit call data collected by the second terminal device through some of the hardware devices, and the second data transmission channel is used to transmit call data obtained by the first terminal device through the network; The first terminal device performs a screen projection operation and uses the first data transmission channel and the second data transmission channel to transmit call data with the second terminal device.

2. The method according to claim 1, characterized in that A Bluetooth wireless communication connection is established between the first terminal device and the second terminal device, and the method further includes: In response to a user clicking on a screen projection control in the drop-down menu, the first terminal device scans terminal devices within a preset range that have established wireless communication connections with the first terminal device, and displays the screen projection device list, which includes an identifier of the second terminal device; The first terminal device acquires the device status information of the second terminal device, including: In response to the user's selection operation on the second terminal device, the first terminal device obtains the device status information of the second terminal device via Bluetooth.

3. The method according to claim 2, characterized in that The method further comprises: The second terminal device broadcasts device status information of the second terminal device via Bluetooth.

4. The method according to claim 1, characterized in that: The user logs in to the same account on the first terminal device and the second terminal device, and the first terminal device and the second terminal device are in the same local area network, and the method further includes: In response to a user clicking operation on a control center control, the first terminal device scans terminal devices that are logged into the same account as the first terminal device and are in the same local area network, and displays the screen projection device list, wherein the screen projection device list includes an identifier of the second terminal device; The first terminal device acquires the device status information of the second terminal device, including: In response to the user's selection operation on the second terminal device, the first terminal device obtains the device status information of the second terminal device through the information synchronization module.

5. The method according to claim 4, characterized in that The method further comprises: The second terminal device stores the device status information of the second terminal device in an information synchronization module of the second terminal device; The first terminal device obtains the device status information of the second terminal device through the information synchronization module, including: The first terminal device interacts with the information synchronization module of the second terminal device through the information synchronization module of the first terminal device to obtain the device status information of the second terminal device.

6. The method according to any one of claims 1 to 5, characterized in that The first terminal device and the second terminal device establish a second data transmission channel, including: In the case where both the first terminal device and the second terminal device support the new protocol, the second data transmission channel is created based on the new protocol, and a single data transmission amount supported by the new protocol is greater than a single data transmission amount supported by the old protocol.

7. The method according to any one of claims 1 to 5, characterized in that The first terminal device and the second terminal device establish a second data transmission channel, including: When the first terminal device and / or the second terminal device does not support the new protocol, the second data transmission channel is created based on the old protocol, and the single data transmission amount supported by the new protocol is greater than the single data transmission amount supported by the old protocol.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: Based on the version number of the communication service manager of the first terminal device, determining whether the first terminal device supports the new protocol; Based on the version number of the communication service manager of the second terminal device, it is determined whether the second terminal device supports the new protocol.

9. The method according to claim 8, characterized in that The determining, based on the version number of the communication service manager of the first terminal device, whether the first terminal device supports the new protocol includes: Comparing the version number of the communication service manager of the first terminal device with a preset version number; If the version number of the communication service manager of the first terminal device is greater than or equal to the preset version number, it is determined that the first terminal device supports the new protocol.

10. The method according to claim 9, characterized in that The determining, based on the version number of the communication service manager of the first terminal device, whether the first terminal device supports the new protocol includes: If the version number of the communication service manager of the first terminal device is smaller than the preset version number, it is determined that the first terminal device does not support the new protocol.

11. The method according to any one of claims 1 to 10, characterized in that The call is a video call, and the hardware equipment includes a camera, a microphone and a speaker.

12. The method according to claim 11, characterized in that In the case where the camera, the microphone and the speaker can all be used for screen projection, the device status information is information used to indicate that the hardware device of the second terminal device can be used for screen projection.

13. The method according to claim 11 or 12, characterized in that: The transmitting call data with the second terminal device by using the first data transmission channel and the second data transmission channel includes: The second terminal device collects first data through the camera and the microphone, and transmits the first data to the first terminal device through the first data transmission channel; The first terminal device receives the first data through the first data transmission channel; The first terminal device sends second data acquired by the first terminal device through the network to the second terminal device through the second data transmission channel; The second terminal device receives the second data through the second data transmission channel; The method further comprises: The second terminal device plays the voice data in the second data through the speaker, and displays the image data in the second data through the display screen of the second terminal device.

14. A terminal device, characterized in that: include: A processor, wherein the processor is coupled to a memory, wherein the memory is used to store a computer program, and when the processor calls the computer program, the terminal device executes the method according to any one of claims 1 to 13.

15. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 13.

16. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 13.

Images