Hardware equipment calling method and device

Abstract

The invention provides a hardware equipment calling method and device. The hardware equipment calling efficiency can be improved. The method comprises the following steps: when first terminal equipment and second terminal equipment establish equipment interconnection for the first time, the first terminal equipment sends a first request for requesting an equipment configuration file of the second terminal equipment to a cloud server, and the first terminal equipment and the second terminal equipment log in the same account; wiFi switches and Bluetooth switches of the first terminal device and the second terminal device are in an on state; the first terminal device receives a device configuration file of a second terminal device from the cloud server; under the condition that the first terminal equipment needs to call the first hardware equipment of the second terminal equipment, the first terminal equipment judges whether the equipment configuration file of the second terminal equipment comprises the hardware equipment capability information of the first hardware equipment or not; and if yes, the first terminal equipment calls the first hardware equipment by using the hardware equipment capability information of the first hardware equipment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method and apparatus for calling a hardware device. Background Art

[0002] For terminal devices that support interconnection services, such as mobile phones, tablet computers, and laptops, users can interconnect the first terminal device and the second terminal device through the interconnection service, so that the hardware devices of the second terminal device, such as cameras, microphones, and speakers, can be used on the first terminal device through applications. When the first terminal device calls the first hardware device of the second terminal device, the first terminal device needs to first obtain the hardware device capability information of the first hardware device of the second terminal device. For example, if the first hardware device is a camera, the hardware device capability information of the first hardware device can be information such as the resolution and frame rate of the camera; then the first terminal device uses the hardware device capability information to call the first hardware device. At present, the first terminal device usually obtains the hardware device capability information of the second terminal device from the second terminal device.

[0003] However, although the calling of hardware devices between devices can be realized in the above scenario, how to improve the calling efficiency of hardware devices is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a hardware device calling method and apparatus, which can shorten the time taken by a first terminal device to obtain hardware device capability information of a second terminal device, thereby improving the efficiency of hardware device calling.

[0005] In a first aspect, a hardware device calling method is provided, which is applied to a first terminal device, and the method includes: when the first terminal device and the second terminal device establish device interconnection for the first time, the first terminal device sends a first request to a cloud server for requesting a device configuration file of the second terminal device, a user logs in to the same account on the first terminal device and the second terminal device, and the wireless fidelity WiFi switch and the Bluetooth switch of the first terminal device and the second terminal device are both in the on state; the first terminal device receives the device configuration file of the second terminal device from the cloud server; when the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device; if so, the first terminal device calls the first hardware device of the second terminal device using the hardware device capability information of the first hardware device.

[0006] In the hardware device calling method of the present application, the first terminal device caches a device configuration file of the second terminal device; when the first terminal device needs to call the first hardware device of the second terminal device, if the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, the first terminal device can obtain the hardware device capability information of the first hardware device from the device configuration file of the second terminal device, and then call the first hardware device based on the hardware device capability information of the first hardware device. In such a hardware device calling method, when the first terminal device obtains the device capability information of the first hardware device of the second terminal device, if the device configuration file of the second terminal device cached by the first terminal device includes the device capability information of the first hardware device, the first terminal device does not need to interact with other devices, and can directly obtain it from the cache of the first terminal device. Compared with obtaining the hardware device capability information of the first hardware device from the second terminal device, such a method of obtaining the hardware device capability information does not need to interact with multiple software modules of the application layer of the second terminal device, so that the first terminal device takes a shorter time to obtain the device capability information of the first hardware device of the second terminal device, thereby improving the efficiency of the hardware device calling method.

[0007] It should be understood that the initial establishment of device interconnection refers to the first time that the first terminal device establishes device interconnection with the second terminal device each time the first terminal device switches from a shutdown state to a power-on state, or switches from a state not connected to a LAN to a state connected to a LAN. The device configuration file of the second terminal device may also be referred to as a profile, and the device configuration file of the second terminal device may include information such as the device model of the second terminal device. The cloud server may refer to one or more servers, and the cloud server may transmit data with the first terminal device and the second terminal device respectively. The cloud server may be used to store device configuration files of multiple terminal devices.

[0008] In certain implementations of the first aspect, the method further includes: if the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request to the second terminal device for requesting the hardware device capability information of the first hardware device; and the first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

[0009] It should be understood that when the device configuration file of the second terminal device stored in the first terminal device does not include the hardware device capability information of the first hardware device, the first terminal device needs to obtain the hardware device capability information of the first hardware device from the second terminal device.

[0010] In certain implementations of the first aspect, the first terminal device sends a second request to the second terminal device for requesting hardware device capability information of the first hardware device, including: a device virtualization module of the first terminal device establishes a signaling channel; the device virtualization module sends the second request to the second communication service manager of the second terminal device through the first communication service manager of the first terminal device and using the signaling channel; the first terminal device receives the hardware device capability information from the second terminal device, including: the first communication service manager receives the hardware device capability information from the second communication service manager; the first communication service manager transmits the hardware device capability information to the device virtualization module.

[0011] It should be understood that the first communication service manager and the second communication service manager can realize data transmission between the first terminal device and the second terminal device. Therefore, the device virtualization module of the first terminal device can send a second request to the second communication service manager through the first communication service manager.

[0012] In certain implementations of the first aspect, when the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, including: the device virtualization module of the first terminal device sends a third request for requesting the hardware device information of the first hardware device to the device management service module of the first terminal device, the third request including the device identification of the second terminal device and the identification of the first hardware device; the device management service module obtains the device configuration file of the second terminal device corresponding to the device identification of the second terminal device; the device management service module determines whether the device configuration file of the second terminal device includes information corresponding to the identification of the first hardware device.

[0013] It should be understood that the device management service module of the first terminal device can be used to cache the device configuration file of the second terminal device. The device virtualization module of the first terminal device can determine whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device. In the case where the device virtualization module of the first terminal device determines whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device, the device virtualization module of the first terminal device can obtain the hardware device capability information of the hardware device of the second terminal device from the device management service module of the first terminal device, so that the first terminal device can efficiently determine the hardware device capability information of the first hardware device of the second terminal device.

[0014] In some implementations of the first aspect, the first request includes a device identification of the second terminal device.

[0015] It should be understood that the device identification of the second terminal device may refer to the device model, device name or device ID of the second terminal device. The cloud server may determine the device configuration file of the second terminal device that the first terminal device needs to obtain based on the device identification in the first request.

[0016] In certain implementations of the first aspect, the method further includes: the first terminal device receives updated configuration information from the cloud server, the updated configuration information includes a new device configuration file of the second terminal device or incremental information between the new device configuration file and the device configuration file; the first terminal device updates the device configuration file of the second terminal device based on the updated configuration information.

[0017] It should be understood that when the device configuration file of the second terminal device stored in the cloud server changes, the cloud server can instruct the first terminal device to update the device configuration file of the second terminal device cached by the first terminal device. In this way, when the first terminal device needs to obtain the device configuration information in the device configuration file of the second terminal device, it can obtain the updated device configuration information from the new device configuration file, which can improve the efficiency and accuracy of the first terminal device in obtaining the device configuration information of the second terminal device.

[0018] In certain implementations of the first aspect, before the first terminal device receives updated configuration information from the cloud server, the method also includes: the first terminal device receives an update message from the cloud server, the update message including a device identifier of the second terminal device; in response to the update message, the first terminal device sends a fourth request to the cloud server for requesting a new device configuration file for the second terminal device.

[0019] It should be understood that the fourth request may be the same as or different from the first request, that is, the fourth request may be used to request a new device configuration file of the second terminal device stored in the cloud server, or may be used to request the incremental information between the new device configuration file of the second terminal device and the device configuration file. In the case where the device configuration information of the second terminal device stored in the cloud server changes, the cloud server sends an update message to the first terminal device. In the case where the first terminal device needs to update the device configuration information of the second terminal device, the first terminal device sends the fourth request to the cloud server, so that the first terminal device can control the timing of updating the device configuration file of the second terminal device, which helps to improve the success rate of the first terminal device updating the device configuration file of the second terminal device.

[0020] In certain implementations of the first aspect, updating the device configuration file of the second terminal device includes: when the updated configuration information includes a new device configuration file for the second terminal device, replacing the device configuration file of the second terminal device with the new device configuration file of the second terminal device; or, when the updated configuration information includes the incremental information, combining the incremental information and the device configuration file of the second terminal device to determine the new device configuration file of the second terminal device.

[0021] It should be understood that, in combination with the incremental information and the device configuration file of the second terminal device, the new device configuration file of the second terminal device can be determined specifically as follows: if the incremental information is data information that needs to be newly added, the first terminal device adds the incremental information to the device configuration file of the second terminal device to obtain a new device configuration file; if the incremental information is data information that needs to be deleted, the first terminal device deletes the data information corresponding to the incremental information in the device configuration file of the second terminal device to obtain a new device configuration file; if the incremental information is newly added data information that needs to be changed, the first terminal device changes the data information that needs to be changed in the device configuration file of the second terminal device to obtain a new device configuration file.

[0022] In certain implementations of the first aspect, the first terminal device, without being shut down, disconnects from the second terminal device and then reconnects, and the method further includes: when the first terminal device needs to call a second hardware device of the second terminal device, the first terminal device determines whether a new device configuration file of the second terminal device includes hardware device capability information of the second hardware device; if so, the first terminal device calls the second hardware device of the second terminal device by using the hardware device capability of the second hardware device.

[0023] It should be understood that compared to the device configuration file, when the hardware device capability information of the second hardware device is added to the new device configuration file, such a hardware device calling method enables the first terminal device to obtain the hardware device capability information of the second hardware device from the new device configuration file of the second terminal device cached in the first terminal device, which helps to improve the efficiency of the first terminal device in obtaining the hardware device capability information of the second hardware device of the second terminal device, thereby improving the efficiency of the first terminal device in calling the second hardware device of the second terminal device.

[0024] In some implementations of the first aspect, the first hardware device is at least one of a camera, a microphone, or a speaker.

[0025] In certain implementations of the first aspect, when the first hardware device is a camera, the hardware device capability information includes a resolution and / or a frame rate of the camera.

[0026] It should be understood that when the first terminal device needs to call the camera of the second terminal device, it can determine the configuration supported by the camera based on the resolution and / or frame rate of the camera, so that the user can select a camera with a corresponding configuration in the first terminal device.

[0027] In a second aspect, a hardware device calling method is provided, which is applied to a system including a first terminal device, a second terminal device and a cloud server, a user logs in to the same account on the first terminal device and the second terminal device, and the wireless fidelity WiFi switch and the Bluetooth switch of the first terminal device and the second terminal device are both in the on state, and the method includes: when the first terminal device and the second terminal device establish device interconnection for the first time, the first terminal device sends a first request to the cloud server for requesting a device configuration file of the second terminal device; the cloud server receives the first request from the first terminal device; the cloud server sends the device configuration file of the second terminal device to the first terminal device based on the first request; the first terminal device receives the device configuration file of the second terminal device from the cloud server; when the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device; if so, the first terminal device uses the hardware device capability information of the first hardware device to call the first hardware device of the second terminal device.

[0028] In certain implementations of the second aspect, the method further includes: in a case where it is determined that the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request to the second terminal device for requesting the hardware device capability information of the first hardware device; the second terminal device receives the second request from the first terminal device; and in response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device.

[0029] In certain implementations of the second aspect, the method also includes: when a third terminal device obtains hardware device capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware device capability information of the first hardware device to the cloud server; the cloud server receives the hardware device capability information of the first hardware device, and stores the hardware device capability information of the first hardware device in the device configuration information of the second terminal device.

[0030] In certain implementations of the second aspect, the method further includes: the second terminal device sends incremental information between a new device configuration file of the second terminal device and the device configuration file to the cloud server; the cloud server receives the incremental information from the second terminal device, and determines the new device configuration file of the second terminal device in combination with the incremental information and the device configuration file of the second terminal device; the cloud server sends updated configuration information to the first terminal device, where the updated configuration information includes the new device configuration file of the second terminal device or the incremental information; and the first terminal device updates the device configuration file of the second terminal device based on the updated configuration information to obtain the new device configuration information of the second terminal device.

[0031] In certain implementations of the second aspect, before the cloud server sends the updated configuration information to the first terminal device, the method also includes: the cloud server sends an update message to the first terminal device, the update message including the device identifier of the second terminal device; the first terminal device receives the update message from the cloud server; in response to the update message, the first terminal device sends a fourth request to the cloud server for requesting a new device configuration file for the second terminal device; the cloud server sends the updated configuration information to the first terminal device, including: in response to the fourth request, the cloud server sends the updated configuration information to the first terminal device.

[0032] In certain implementations of the second aspect, the updating of the device configuration file of the second terminal device includes: when the updated configuration information includes a new device configuration file for the second terminal device, replacing the device configuration file of the second terminal device with the new device configuration file of the second terminal device; or, when the updated configuration information includes the incremental information, combining the incremental information and the device configuration file of the second terminal device to determine the new device configuration file of the second terminal device.

[0033] In certain implementations of the second aspect, the first terminal device, without being shut down, disconnects from the second terminal device and then reconnects, and the method further includes: when the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; if so, the first terminal device calls the second hardware device of the second terminal device by using the hardware device capability of the second hardware device.

[0034] In a third aspect, the present application provides a hardware device calling device, including a processor, the processor is coupled to a memory, and can be used to execute instructions in the memory to implement the method in any possible implementation of the first aspect or the second aspect. Optionally, the device also includes a memory. Optionally, the device also includes a communication interface, and the processor is coupled to the communication interface.

[0035] In one implementation, the apparatus is a first terminal device. When the apparatus is a first terminal device, the communication interface may be a transceiver, or an input / output interface.

[0036] In one implementation, the apparatus is a second terminal device. When the apparatus is a second terminal device, the communication interface may be a transceiver, or an input / output interface.

[0037] In one implementation, the device is a cloud server. When the device is a cloud server, the communication interface may be a transceiver, or an input / output interface.

[0038] In another implementation, the device is a chip configured in a terminal device. When the device is a chip configured in a terminal device, the communication interface may be an input / output interface.

[0039] In a fourth aspect, a processor is provided, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation of the first aspect or the second aspect.

[0040] In the specific implementation process, the processor may be a chip, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit may be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit may be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit may be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0041] In a fifth aspect, a processing device is provided, comprising a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a receiver and transmit signals through a transmitter to execute the method in any possible implementation of the first aspect or the second aspect.

[0042] Optionally, the number of the processors is one or more, and the number of the memories is one or more.

[0043] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0044] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips. This application does not limit the type of memory and the setting method of the memory and the processor.

[0045] It should be understood that the relevant data interaction process, such as sending indication information, can be a process of outputting indication information from a processor, and receiving capability information can be a process of receiving input capability information from a processor. Specifically, the processed output data can be output to a transmitter, and the input data received by the processor can come from a receiver. Among them, the transmitter and the receiver can be collectively referred to as a transceiver.

[0046] The processing device in the fifth aspect mentioned above can be a chip. The processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0047] In a sixth aspect, a computer program product is provided, comprising: a computer program (also referred to as code, or instruction), which, when executed, enables a computer to execute a method in any possible implementation of the first aspect or the second aspect.

[0048] In the seventh aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instructions). When the computer-readable storage medium is run on a computer, the computer executes a method in any possible implementation of the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0051] Figure 3 A software structure block diagram of a Windows system of a terminal device according to an embodiment of the present application;

[0052] Figure 4 A software structure block diagram of an Android system of a terminal device according to an embodiment of the present application;

[0053] Figure 5 A flowchart of a hardware device calling method provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of a process of a first terminal device calling a first hardware device of a second terminal device provided in an embodiment of the present application;

[0055] Figure 7 A flowchart of a method for a first terminal device to call a camera of a second terminal device provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of an interface of a notebook computer display device manager provided in an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of an interface for a laptop computer to detect that a mobile phone is online provided in an embodiment of the present application;

[0058] Fig.10 A schematic diagram of another interface of a notebook computer display device manager provided in an embodiment of the present application;

[0059] Fig.11 A schematic diagram of a camera selection interface in a conference application provided in an embodiment of the present application;

[0060] Fig.12 A schematic block diagram of a hardware device calling apparatus provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0062] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially identical functions and effects. For example, the first value and the second value are only used to distinguish between different values, and do not limit their order. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0063] It should be noted that in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplarily" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0064] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0065] Currently, for terminal devices that support Internet services, such as mobile phones, tablet computers, laptops, etc., users can interconnect a first terminal device and a second terminal device through the Internet service, thereby realizing data transmission between the first terminal device and the second terminal device.

[0066] Combine the following Figure 1 , taking the first terminal device as a laptop computer and the second terminal device as a mobile phone as an example, the application scenario of the embodiment of the present application is described in detail.

[0067] Figure 1 Schematic diagram of an application scenario 100 provided in an embodiment of the present application. Figure 1 As shown, when a user conducts a video conference through a video conference application on a laptop computer 101, if it is necessary to shoot experimental equipment, the user can interconnect the laptop computer 101 and the mobile phone 102 so that the laptop computer 101 and the mobile phone 102 can transmit data. In this way, the laptop computer 101 can call the camera of the mobile phone 102, and the video data stream of the experimental equipment shot by the camera of the mobile phone 102 can be transmitted to the laptop computer 101, so as to show the experimental equipment to other people participating in the video conference.

[0068] In order to better understand the terminal device in the embodiment of the present application, Figure 1 The hardware structure of the terminal device in the embodiment of the present application is described in detail.

[0069] Figure 2 This is a schematic diagram of the structure of the terminal device 200 provided in the embodiment of the present application. Figure 2 As shown, the terminal device 200 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0070] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal device 200. In other embodiments of the present application, the terminal device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0071] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0072] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0073] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0074] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0075] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the terminal device 200.

[0076] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0077] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0078] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0079] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the terminal device 200. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the terminal device 200.

[0080] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0081] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the terminal device 200, and can also be used to transmit data between the terminal device 200 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other terminal devices, such as AR devices, etc.

[0082] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the terminal device 200. In other embodiments of the present application, the terminal device 200 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0083] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the terminal device 200. While the charging management module 140 is charging the battery 142, it may also power the terminal device through the power management module 141.

[0084] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0085] The wireless communication function of the terminal device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0086] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0087] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the terminal device 200. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0088] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0089] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 200. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0090] In some embodiments, the antenna 1 of the terminal device 200 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 200 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0091] The terminal device 200 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0092] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 200 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0093] The terminal device 200 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0094] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0095] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 200 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0096] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 200 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0097] Video codecs are used to compress or decompress digital videos. The terminal device 200 may support one or more video codecs. Thus, the terminal device 200 may play or record videos in multiple coding formats, such as moving picture experts group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0098] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0099] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 200. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0100] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 200 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 200 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0101] The terminal device 200 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0102] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0103] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 200 can listen to music or listen to a hands-free call through the speaker 170A.

[0104] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 200 receives a call or voice message, the voice can be received by placing the receiver 170B close to the ear.

[0105] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 170C to input the sound signal into the microphone 170C. The terminal device 200 can be provided with at least one microphone 170C. In other embodiments, the terminal device 200 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 200 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0106] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0107] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 200 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the terminal device 200 detects the touch operation intensity according to the pressure sensor 180A. The terminal device 200 can also calculate the touch position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0108] The gyroscope sensor 180B can be used to determine the motion posture of the terminal device 200. In some embodiments, the angular velocity of the terminal device 200 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 200 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 200 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0109] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 200 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0110] The magnetic sensor 180D includes a Hall sensor. The terminal device 200 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 200 is a flip phone, the terminal device 200 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0111] The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 200 in various directions (generally three axes). When the terminal device 200 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the terminal device and applied to applications such as horizontal and vertical screen switching and pedometers.

[0112] The distance sensor 180F is used to measure the distance. The terminal device 200 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 200 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0113] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 200 emits infrared light outward through the light emitting diode. The terminal device 200 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 200. When insufficient reflected light is detected, the terminal device 200 can determine that there is no object near the terminal device 200. The terminal device 200 can use the proximity light sensor 180G to detect that the user holds the terminal device 200 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and pocket mode automatically unlocks and locks the screen.

[0114] The ambient light sensor 180L is used to sense the ambient light brightness. The terminal device 200 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the terminal device 200 is in a pocket to prevent accidental touch.

[0115] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 200 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0116] The temperature sensor 180J is used to detect temperature. In some embodiments, the terminal device 200 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 200 reduces the performance of the processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 200 heats the battery 142 to avoid abnormal shutdown of the terminal device 200 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the terminal device 200 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

[0117] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the terminal device 200, which is different from the position of the display screen 194.

[0118] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.

[0119] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The terminal device 200 may receive key input and generate key signal input related to user settings and function control of the terminal device 200.

[0120] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0121] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0122] The SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the terminal device 200 by inserting the SIM card interface 195 or pulling it out from the SIM card interface 195. The terminal device 200 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card, etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The terminal device 200 interacts with the network through the SIM card to realize functions such as calls and data communications. In some embodiments, the terminal device 200 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 200 and cannot be separated from the terminal device 200. The software system of the terminal device 200 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the terminal device 200.

[0123] Currently, for terminal devices that support Internet services, such as mobile phones, tablets, laptops, etc., users can interconnect the first terminal device and the second terminal device through the Internet service, thereby using the hardware devices of the second terminal device, such as cameras, microphones, speakers, etc., through applications on the first terminal device.

[0124] Combine the following Figure 3 and Figure 4 , the software structure of the first terminal device and the second terminal device involved in the information acquisition method provided in the embodiment of the present application is described in detail. Figure 3 Taking the first terminal device as a personal computer (PC) and the system of the first terminal device as a Windows system as an example, the software structure thereof is described; Figure 4 The software structure of the second terminal device is described by taking a mobile phone of Android system as an example.

[0125] It should be understood that the first terminal device may be a terminal device of a type other than a personal computer, and the system of the first terminal device may be an Android system; the second terminal device may be a terminal device of a type other than a mobile phone, and the system of the second terminal device may be a Windows system. This application does not specifically limit the device type and operating system type of the first terminal device and the second terminal device.

[0126] Figure 3is a software structure diagram of the first terminal device. The layered architecture of the Windows system divides the software into several layers, and the layers communicate with each other through software interfaces. For example, the application layer and the system layer can interact through the application programming interface (API), and the system layer and the kernel layer can interact through the standard I / O interface. In some embodiments, the Windows system is divided into four layers, which are the application layer, the application framework layer, the system layer, and the kernel layer from top to bottom. The specific functional modules included in each layer are explained below.

[0127] 1. Application layer

[0128] The application layer may include a series of applications, such as music, PC manager, Bluetooth, WIFI, games, videos, third-party applications, cameras, and software development kits (SDKs). In the embodiment of the present application, the application layer also includes: device virtualization SDKs, device managers, etc. In a possible implementation, third-party applications can access the device virtualization SDK and call the device virtualization module of the system layer through the device virtualization SDK.

[0129] The PC manager may include installation packages for multiple applications, and users may install multiple applications for the first terminal device through the PC manager; it is also used to implement system optimization, smart audio-visual, smart interconnection between devices, multi-screen collaboration, and hotspot sharing. The PC manager may also have other names, which are not specifically limited in this application.

[0130] The device manager is used to view and change device properties, change device drivers, configure device settings, and uninstall devices. In the embodiment of the present application, the device manager is also used to implement security authentication and account authentication of devices with the same account, self-discovery and self-organizing networking of devices, data transmission between devices, and access to communication services.

[0131] The above-mentioned third-party applications may include video conferencing, dating and chatting applications, etc.

[0132] 2. Application framework layer

[0133] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0134] like Figure 3As shown, in the embodiment of the present application, the application framework layer includes at least: a device management service module, a communication service manager and a device virtualization module.

[0135] The device management service module is used to manage the device data information (profile) of the first terminal device, which may include various attribute information of the first terminal device, such as device type, etc. In an embodiment of the present application, the device data information of the first terminal device may also include device capability information of the hardware device of the first terminal device, such as the resolution and frame rate of the camera. The device management service is also used to monitor the online device. For example, after the second terminal device is online, the first terminal device can monitor the online second terminal device through the device management service. In addition, the device management service is also used to synchronize the device data information of the second terminal device from the cloud server.

[0136] The communication service manager can realize data transmission between devices. For example, when the first terminal device calls the camera of the second terminal device, the video or image data stream captured by the camera is transmitted through the communication service manager.

[0137] The communication service manager may also be called a smart interconnection module, a MagicLink transmission management service module, etc. This application does not make any specific limitations on this.

[0138] The device virtualization module is used to manage the life cycle of the virtual device driver, control the creation and deletion of the virtual device driver through the virtual bus driver (also known as the insertion and removal of the virtual device driver), and is also used to process requests sent by third-party applications to the virtual device driver, and feed back the processing results to the third-party application. The device virtualization module can interact with other applications through the device virtualization SDK of the application layer. In an embodiment of the present application, the device virtualization module is also used to manage the data channels and signaling channels maintained by the communication service manager to realize data transmission across devices.

[0139] The device virtualization module can also be called a hardware service cross-device sharing module, a distributed mobile sensing development platform (DMSDP), a hardware virtualization module, etc., and this application does not make specific restrictions on this. Correspondingly, the device virtualization SDK can also be called a hardware service cross-device sharing SDK, a DMSDPSDK, a hardware virtualization SDK, etc., and this application embodiment does not make any restrictions on this.

[0140] 3. System layer

[0141] The system layer may include the Windows multimedia framework, etc., which may provide corresponding services for the execution of user application processes.

[0142] The Windows Multimedia Framework includes a variety of Windows functional functions that can be called by applications for developing applications. The Windows Multimedia Framework is also used to connect the application layer and the kernel layer to achieve interaction between the application layer and the kernel layer. Exemplarily, the request sent by the application to the kernel layer can be first transmitted to the Windows Multimedia Framework through the Windows Application Framework (framwork, FWK) API, and then transmitted to the kernel layer through the standard I / O interaction interface.

[0143] 4. Kernel layer

[0144] The kernel layer is the layer between hardware and software. The kernel layer is used to drive the hardware to make it work. The kernel layer may include virtual bus drivers, virtual device drivers, Windows Driver Foundation (WDF) modules, operating system (OS) kernels, etc.

[0145] The virtual device driver can be recognized by the terminal device as a universal serial bus (USB) device and used by the terminal device. For example, a virtual camera device, a virtual microphone (mic) device. If the first terminal device recognizes an available USB device (also called a hardware device), a corresponding virtual device driver can be created for each USB device, and the number of virtual device drivers included in the kernel layer of the first terminal device can be multiple; if the first terminal device does not recognize an available USB device, the number of virtual device drivers included in the kernel layer of the terminal device 200 is 0. In the case where the first terminal device creates at least one virtual device driver, the at least one virtual device driver can be displayed in its device manager.

[0146] The virtual bus driver is used to create or delete virtual device drivers, that is, it is responsible for managing the insertion, removal and communication processes of virtual device drivers.

[0147] Optionally, PC Manager can call the installation script and installation package of the virtual bus driver to install the virtual bus driver.

[0148] The virtual bus driver may also be referred to as a virtual bus controller, a virtual driver bus, etc., and this application does not make any specific limitation on this.

[0149] The OS kernel is used to convert requests from the application layer into data processing instructions and send them to the virtual device driver through the driver development framework.

[0150] The driver development framework is used to implement sending data processing instructions from the OS kernel to the virtual device driver.

[0151] Figure 4 A software structure diagram of a second terminal device provided in an embodiment of the present application. The Android layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the hardware abstraction layer, and the Linux kernel layer.

[0152] 1. Application layer

[0153] The application layer can include a series of application packages. Figure 4 As shown, in the embodiment of the present application, the application layer includes at least application packages such as third-party applications, camera, Bluetooth, WIFI, and device virtualization SDK.

[0154] The third-party application may be a video conferencing application, a dating chat application, or the like.

[0155] Among them, the device virtualization module SDK can also include public service interface, camera service interface, audio service interface and speaker service interface, etc. The interfaces of these services are exposed through the Android interface definition language (AIDL) provided by the interface object abstraction layer. The interface object abstraction layer can be encapsulated by the service interface adaptation layer to achieve cross-process access.

[0156] In an embodiment of the present application, the device virtualization module SDK can call the device virtualization module in the system library through a public service interface.

[0157] 2. Application framework layer

[0158] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0159] like Figure 4 As shown, in the embodiment of the present application, the application framework layer at least includes: a communication service manager, a device management service module, and a device virtualization module, etc. In the embodiment of the present application, when the camera of the second terminal device is called by the first terminal device, the picture data stream captured by the camera is transmitted through the communication service manager.

[0160] In an embodiment of the present application, the device virtualization module can be responsible for interacting with the hardware device, for example, it can turn on the camera, turn off the camera, and obtain the camera video stream; it can also perform image processing and encoding and decoding, for example, it can realize image blurring and encode the image obtained by the camera; the device virtualization module can also manage the data transmission channel of the communication service manager, for example, when the user chooses to pause the call of the camera, the transmission of the data stream through the data transmission channel is paused; in addition, the device virtualization module can also process the command message from the user interface through interaction with the device virtualization module SDK.

[0161] The device virtualization module may also include an interface security management module, a distribution control module, a channel management module, a device management module, a service management module, etc. In the embodiment of the present application, the service management module may include a camera service module, and the camera service module may also include an image processing module and a codec module. The image processing module may realize the virtualization processing of the image, and the codec module may realize the encoding of the image. Optionally, the service management module may also include an audio service module and a speaker service module. The present application does not limit the hardware service modules and functions contained therein.

[0162] 3. System layer

[0163] The system layer includes the system library and the Android runtime. The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function that the Java language needs to call, and the other is the Android core library. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0164] The system library may include multiple functional modules, and at least includes: a graphics processing library, etc.

[0165] The graphics processing library may include a 3D graphics processing library and a 2D graphics processing library. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics processing library can be used to implement the drawing and processing of 2D graphics.

[0166] 4. Hardware Abstraction Layer

[0167] The hardware abstraction layer is an abstracted structure between the Linux kernel layer and the system runtime layer. The hardware abstraction layer can be a package of the hardware driver, providing a unified interface for the call of the upper layer application. The hardware abstraction layer may include a Bluetooth module, a WIFI module, a hardware configuration module, etc. For example, in the embodiment of the present application, the Bluetooth module and the WIFI module in the hardware abstraction layer can shield the specific hardware implementation details of the Bluetooth driver and the WIFI driver in the kernel layer, and the upper layer application can implement the Bluetooth function and the WIFI function by calling the interface without having to know the specific implementation method of the kernel layer hardware driver.

[0168] 5. Linux kernel layer

[0169] The Linux kernel layer is the layer between hardware and software. The Linux kernel layer contains at least display driver, camera driver, memory driver, sensor driver, inter-process communication driver, etc.

[0170] When the first terminal device calls the first hardware device of the second terminal device, the first terminal device needs to first obtain the hardware device capability information of the first hardware device of the second terminal device. For example, the first hardware device is a camera, and the hardware device capability information of the first hardware device may be information such as the resolution and frame rate of the camera. Then, the first terminal device uses the hardware device capability information to call the first hardware device.

[0171] Currently, the first terminal device usually obtains the hardware device capability information of the second terminal device from the second terminal device.

[0172] However, in such a hardware device calling method, each time the first terminal device calls the hardware device of the second terminal device, it needs to interact with multiple modules of the application framework layer of the first terminal device and multiple software modules of the application framework layer of the second terminal device, which makes it take a long time for the first terminal device to obtain the hardware device capability information of the second terminal device, and thus makes the efficiency of the first terminal device calling the hardware device of the second terminal device low. Therefore, although the calling of hardware devices between devices can be realized in the above scenario, how to improve the calling efficiency of hardware devices is an urgent problem to be solved.

[0173] In order to solve the above technical problems, the present application provides a method and apparatus for calling a hardware device, wherein a device configuration file of a second terminal device is stored in a cloud server; after the first terminal device and the second terminal device are interconnected, the first terminal device obtains the device configuration file of the second terminal device from the cloud server; when the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device can obtain the hardware device capability information of the first hardware device from the device configuration file of the second terminal device, and then call the first hardware device based on the hardware device capability information of the first hardware device. In such a hardware device calling method, when the first terminal device obtains the device capability information of the first hardware device of the second terminal device, when the device configuration file of the second terminal device cached by the first terminal device includes the device capability information of the first hardware device, the first terminal device does not need to interact with other devices, and can directly obtain it from the cache of the first terminal device. Compared with obtaining the hardware device capability information of the first hardware device from the second terminal device, such a method for obtaining the hardware device capability information does not need to interact with multiple software modules of the application layer of the second terminal device, so that the first terminal device takes a shorter time to obtain the device capability information of the first hardware device of the second terminal device, thereby improving the efficiency of the hardware device calling method.

[0174] It should be understood that the above-mentioned device configuration file can be uploaded by the second terminal device to the cloud server. If there is a terminal device that obtains the hardware device capability information of the first hardware device of the second terminal device from the second terminal device, the second terminal device can send the hardware device capability information of the first hardware device to the cloud server, and the cloud server stores the hardware device capability information in the device configuration file of the second terminal device, so that other terminal devices can subsequently directly obtain the hardware device capability information of the first hardware device from the cloud server.

[0175] Combine the following Figures 5 to 11 The hardware calling method provided by the present application is described in detail. The embodiment shown in the present application illustrates the hardware device calling method provided by the present application from the perspective of device interaction. The specific form and quantity of each device shown therein are only examples and should not constitute any limitation on the implementation of the method provided by the present application. Below, taking the first terminal device, the second terminal device and the cloud server as the execution subjects as an example, the hardware device calling method of the embodiment of the present application is described in detail.

[0176] It should be understood that the terminal device can be the terminal device itself, or it can be a chip, chip system or processor that supports the terminal device to implement a cross-device hardware control method, or it can be a logic module or software that can implement all or part of the terminal device functions. This application does not make specific limitations on this.

[0177] Figure 5A flowchart of a hardware device calling method 500 provided in an embodiment of the present application. The method 500 is applied to a system including a first terminal device, a second terminal device, and a cloud server. A user logs in to the same account on the first terminal device and the second terminal device, and the wireless fidelity (WiFi) switch and the Bluetooth switch of the first terminal device and the second terminal device are both turned on. The hardware structure of the first terminal device and the second terminal device can be as follows: Figure 1 As shown, the software structure of the first terminal device can be as follows Figure 3 As shown, the software structure of the second terminal device can be as follows Figure 4 shown.

[0178] It should be understood that an account may refer to a personal registration account provided by different terminal device brands to ensure the device security of the user. After the first terminal device and the second terminal device log in to the same account, data sharing between the devices can be achieved. When the first terminal device and the second terminal device log in to the same account, and the WiFi switch and the Bluetooth switch of the first terminal device and the second terminal device are both turned on, the first terminal device can detect the second terminal device.

[0179] The method 500 comprises the following steps:

[0180] S501: When a first terminal device and a second terminal device establish device interconnection for the first time, the first terminal device sends a first request for a device configuration file of the second terminal device to a cloud server. Correspondingly, the cloud server receives the first request from the first terminal device.

[0181] It should be understood that the initial establishment of device interconnection refers to the first time that the first terminal device establishes device interconnection with the second terminal device each time the first terminal device switches from a power-off state to a power-on state, or switches from a state not connected to a LAN to a state connected to a LAN. The device configuration file of the second terminal device may also be referred to as a profile, and the device configuration file of the second terminal device may include information such as the device model of the second terminal device. The cloud server may refer to one or more servers, and the cloud server may transmit data with the first terminal device and the second terminal device respectively. The cloud server may be used to store device configuration files for multiple terminal devices.

[0182] S502: The cloud server sends a device configuration file of the second terminal device to the first terminal device based on the first request. Correspondingly, the first terminal device receives the device configuration file of the second terminal device from the cloud server.

[0183] S503: When the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes hardware device capability information of the first hardware device.

[0184] It should be understood that a hardware device may also be referred to as a USB device, such as a camera. The first hardware device may refer to any one or more hardware devices in the second terminal device. Hardware device capability information may refer to information used to indicate the properties and configuration of a hardware device. The situation in which the first terminal device needs to call the first hardware device of the second terminal device may also be understood as the situation in which the first terminal device needs to create a virtual device driver corresponding to the first hardware device based on the hardware device capability information of the first hardware device; that is, the situation in which the first terminal device needs to call the first hardware device of the second terminal device may refer to any situation in which the first terminal device needs the hardware device capability information of the first hardware device.

[0185] S504: If the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, the first terminal device uses the hardware device capability information of the first hardware device to call the first hardware device of the second terminal device.

[0186] It should be understood that calling the first hardware device of the second terminal device may also refer to creating a virtual device driver corresponding to the first hardware device, that is, calling the first hardware device does not necessarily mean using the first hardware device, and this application does not make specific limitations on this.

[0187] In the hardware device calling method of the present application, the first terminal device caches a device configuration file of the second terminal device; when the first terminal device needs to call the first hardware device of the second terminal device, if the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, the first terminal device can obtain the hardware device capability information of the first hardware device from the device configuration file of the second terminal device, and then call the first hardware device based on the hardware device capability information of the first hardware device. In such a hardware device calling method, when the first terminal device obtains the device capability information of the first hardware device of the second terminal device, if the device configuration file of the second terminal device cached by the first terminal device includes the device capability information of the first hardware device, the first terminal device does not need to interact with other devices, and can directly obtain it from the cache of the first terminal device. Compared with obtaining the hardware device capability information of the first hardware device from the second terminal device, such a method of obtaining the hardware device capability information does not need to interact with multiple software modules of the application layer of the second terminal device, so that the first terminal device takes a shorter time to obtain the device capability information of the first hardware device of the second terminal device, thereby improving the efficiency of the hardware device calling method.

[0188] As an optional embodiment, the method 500 further includes: S505, if the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; correspondingly, the second terminal device receives the second request from the first terminal device. S506, in response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device; correspondingly, the first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

[0189] It should be understood that when the device configuration file of the second terminal device stored in the first terminal device does not include the hardware device capability information of the first hardware device, the first terminal device needs to obtain the hardware device capability information of the first hardware device from the second terminal device.

[0190] In a possible implementation, S505 is specifically implemented in the following manner: the device virtualization module of the first terminal device establishes a signaling channel; the device virtualization module sends a second request to the second communication service manager of the second terminal device through the first communication service manager of the first terminal device and using the signaling channel. S506 is specifically implemented in the following manner: the first communication service manager receives the hardware device capability information from the second communication service manager; the first communication service manager transmits the hardware device capability information to the device virtualization module.

[0191] It should be understood that the first communication service manager and the second communication service manager can realize data transmission between the first terminal device and the second terminal device. Therefore, the device virtualization module of the first terminal device can send a second request to the second communication service manager through the first communication service manager.

[0192] As an optional embodiment, method 500 also includes: when the third terminal device obtains the hardware device capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware device capability information of the first hardware device to the cloud server; the cloud server receives the hardware device capability information of the first hardware device, and stores the hardware device capability information of the first hardware device in the device configuration information of the second terminal device.

[0193] As an optional embodiment, after S506, method 500 also includes: the second terminal device sends the hardware device capability information of the first hardware device to the cloud server; correspondingly, the cloud server receives the hardware device capability information of the first hardware device from the second terminal device, and stores the hardware device capability information of the first hardware device in the device configuration file of the second terminal device.

[0194] It should be understood that the third terminal device can be the first terminal device or other terminal devices. When the third terminal device is the first terminal device, the step of the second terminal device sending the hardware device capability information of the first hardware device to the cloud server can be performed after S505 or S506. After the second terminal device receives a request from other terminal devices or the first terminal device to obtain the hardware device capability information of the hardware device of the second terminal device, the second terminal device sends the hardware device capability information of the hardware device to the cloud server. In this way, at a later moment, when other terminal devices need to obtain the hardware device capability information of the hardware device, they can obtain it from the device configuration file of the second terminal device cached by the other terminal devices, which can improve the efficiency of the other terminal devices in obtaining the hardware device capability information of the hardware device.

[0195] In a possible implementation, the second terminal device sends the hardware device capability information of the first hardware device to the cloud server, including: after the device virtualization module of the second terminal device detects the second request from the first terminal device, sending a first indication to the device management service module of the second terminal device to instruct the hardware device capability information of the first hardware device to be sent to the cloud server; in response to the first indication, the device management service module of the second terminal device sends the hardware device capability information of the first hardware device to the cloud server.

[0196] As an optional embodiment, the method 500 further includes: S507, the second terminal device sends the device configuration information of the second terminal device to the cloud server; the cloud server receives the device configuration information from the second terminal device.

[0197] It should be understood that when the second terminal device accesses the local area network for the first time, or when the device interconnection is established with the third terminal device for the first time, the second terminal device can send the device configuration file of the second terminal device to the cloud server, and the third terminal device can be any terminal device other than the second terminal device. In this way, the cloud server can store the device configuration file of the second terminal device. When the fourth terminal device establishes a device interconnection with the second terminal device for the first time, the fourth terminal device can obtain the device configuration file of the second terminal device from the cloud server, which helps to improve the efficiency of other terminal devices obtaining the device configuration file of the second terminal device. Among them, the fourth terminal device can be a terminal device other than the third terminal device, or it can be a third terminal device. When the fourth terminal device is a third terminal device, the first establishment of a device interconnection between the fourth terminal device and the second terminal device means that after the fourth terminal device establishes a device interconnection with the third terminal device for the first time, it switches from the power-on state to the power-off state; and then switches from the power-off state to the power-on state, and then establishes a device interconnection with the second terminal device for the first time.

[0198] In a possible implementation, S507 is implemented in the following manner: the second terminal device sends the device configuration information of the second terminal device to the cloud server through the device management service.

[0199] As an optional embodiment, method 500 also includes: S508, the second terminal device sends the incremental information between the new device configuration file and the device configuration file of the second terminal device to the cloud server; S509, the cloud server receives the incremental information from the second terminal device, and determines the new device configuration file of the second terminal device in combination with the incremental information and the device configuration file of the second terminal device; S510, the cloud server sends updated configuration information to the first terminal device, and the updated configuration information includes the new device configuration file or incremental information of the second terminal device; S511, the first terminal device updates the device configuration file of the second terminal device based on the updated configuration information, and obtains the new device configuration information of the second terminal device.

[0200] It should be understood that the incremental information may refer to the newly added data information in the new device configuration file compared to the device configuration file; the data information that needs to be deleted; or the changed data information. Optionally, the incremental information may include information for indicating the change mode of the data information in the new device configuration file compared to the device configuration file. For example, if the incremental information includes the data information that needs to be added in the new device configuration file compared to the device configuration file, the incremental information may include 1, which is used to indicate the addition of data information; if the incremental information includes the data information that needs to be deleted in the new device configuration file compared to the device configuration file, the incremental information may include 2, which is used to indicate the addition of data information; if the incremental information includes the data information that is changed in the new device configuration file compared to the device configuration file, the incremental information may include 3, which is used to indicate the changed data information.

[0201] When the device configuration file of the second terminal device stored in the cloud server changes, the cloud server can instruct the first terminal device to update the device configuration file of the second terminal device cached by the first terminal device. In this way, when the first terminal device needs to obtain the device configuration information in the device configuration file of the second terminal device, it can obtain the updated device configuration information from the new device configuration file, which can improve the efficiency and accuracy of the first terminal device in obtaining the device configuration information of the second terminal device.

[0202] As an optional embodiment, before S510, method 500 further includes: S512, the cloud server sends an update message to the first terminal device, the update message includes the device identifier of the second terminal device; correspondingly, the first terminal device receives the update message from the cloud server. S513, in response to the update message, the first terminal device sends a fourth request to the cloud server for requesting a new device configuration file for the second terminal device; correspondingly, the cloud server receives the fourth request from the first terminal device. S510 is specifically implemented in the following manner: in response to the fourth request, the cloud server sends updated configuration information to the first terminal device.

[0203] It should be understood that the fourth request may be the same as or different from the first request, that is, the fourth request may be used to request a new device configuration file of the second terminal device stored in the cloud server, or may be used to request the incremental information between the new device configuration file of the second terminal device and the device configuration file. In the case where the device configuration information of the second terminal device stored in the cloud server changes, the cloud server sends an update message to the first terminal device. In the case where the first terminal device needs to update the device configuration information of the second terminal device, the first terminal device sends the fourth request to the cloud server, so that the first terminal device can control the timing of updating the device configuration file of the second terminal device, which helps to improve the success rate of the first terminal device updating the device configuration file of the second terminal device.

[0204] In a possible implementation, S511 is specifically implemented in the following manner: when the updated configuration information includes a new device configuration file of the second terminal device, the device configuration file of the second terminal device is replaced with the new device configuration file of the second terminal device; or, when the updated configuration information includes incremental information, the new device configuration file of the second terminal device is determined by combining the incremental information and the device configuration file of the second terminal device.

[0205] It should be understood that, in combination with the incremental information and the device configuration file of the second terminal device, the new device configuration file of the second terminal device can be determined specifically as follows: if the incremental information is data information that needs to be newly added, the first terminal device adds the incremental information to the device configuration file of the second terminal device to obtain a new device configuration file; if the incremental information is data information that needs to be deleted, the first terminal device deletes the data information corresponding to the incremental information in the device configuration file of the second terminal device to obtain a new device configuration file; if the incremental information is newly added data information that needs to be changed, the first terminal device changes the data information that needs to be changed in the device configuration file of the second terminal device to obtain a new device configuration file.

[0206] As an optional embodiment, the first terminal device re-establishes a connection with the second terminal device after disconnecting from the second terminal device without shutting down, and method 500 also includes: S514, when the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; S515, if so, the first terminal device uses the hardware device capability of the second hardware device to call the second hardware device of the second terminal device.

[0207] It should be understood that after the first terminal device caches the device configuration information of the second terminal device, the device configuration information of the second terminal device will always be cached in the first terminal device when the first terminal device is not shut down; when the device configuration file of the second terminal device changes during the period when the first terminal device is not shut down, the first terminal device synchronously updates the device configuration file of the second terminal device to obtain a new device configuration file; after the first terminal device caches the new device configuration file of the second terminal device, if the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device can determine whether the new device configuration file includes the hardware device capability information of the second hardware device, and the first terminal device obtains the hardware device capability information of the second hardware device from the new device configuration file.

[0208] Compared to the device configuration file, when the hardware device capability information of the second hardware device is added to the new device configuration file, such a hardware device calling method enables the first terminal device to obtain the hardware device capability information of the second hardware device from the new device configuration file of the second terminal device cached in the first terminal device, which helps to improve the efficiency of the first terminal device in obtaining the hardware device capability information of the second hardware device of the second terminal device, thereby improving the efficiency of the first terminal device in calling the second hardware device of the second terminal device.

[0209] As an optional embodiment, S503 is specifically implemented in the following manner: the device virtualization module of the first terminal device sends a third request for requesting hardware device information of the first hardware device to the device management service module of the first terminal device, and the third request includes the device identification of the second terminal device and the identification of the first hardware device; the device management service module obtains a device configuration file of the second terminal device corresponding to the device identification of the second terminal device; the device management service module determines whether the device configuration file of the second terminal device includes information corresponding to the identification of the first hardware device.

[0210] It should be understood that the device management service module of the first terminal device can be used to cache the device configuration file of the second terminal device. The device virtualization module of the first terminal device can determine whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device. In the case where the device virtualization module of the first terminal device determines whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device, the device virtualization module of the first terminal device can obtain the hardware device capability information of the hardware device of the second terminal device from the device management service module of the first terminal device, so that the first terminal device can efficiently determine the hardware device capability information of the first hardware device of the second terminal device.

[0211] As an optional embodiment, the first request includes a device identification of the second terminal device.

[0212] It should be understood that the device identification of the second terminal device may refer to the device model, device name or device ID of the second terminal device. The cloud server may determine the device configuration file of the second terminal device that the first terminal device needs to obtain based on the device identification in the first request.

[0213] As an optional embodiment, the first hardware device is at least one of a camera, a microphone or a speaker.

[0214] As an optional embodiment, when the first hardware device is a camera, the hardware device capability information includes a resolution and / or a frame rate of the camera.

[0215] It should be understood that when the first terminal device needs to call the camera of the second terminal device, it can determine the configuration supported by the camera based on the resolution and / or frame rate of the camera, so that the user can select a camera with a corresponding configuration in the first terminal device.

[0216] As an optional embodiment, the first terminal device calls the hardware device of the second terminal device, which can be implemented in the following way: the device virtualization module of the first terminal device, through the virtual bus driver of the first terminal device, creates a virtual device driver corresponding to the hardware device of the second terminal device based on the hardware device capability information of the hardware device of the second terminal device; then the first terminal device obtains the data stream running on the hardware device of the second terminal device through the virtual device driver.

[0217] Combine the following Figure 6 , taking the first hardware device as a camera as an example, the process of the first terminal device calling the first hardware device of the second terminal device is described in detail.

[0218] The software structure of the first terminal device can be as follows Figure 3As shown, the software structure of the second terminal device can be as follows Figure 4 For the convenience of description, the device virtualization module of the first terminal device is referred to as the first device virtualization module, and the device virtualization module of the second terminal device is referred to as the second device virtualization module; the communication service manager of the first terminal device is referred to as the first communication service manager, and the communication service manager of the second terminal device is referred to as the second communication service manager; the device management service module of the first terminal device is referred to as the first device management service module, and the device management service module of the second terminal device is referred to as the second device management service module.

[0219] Figure 6 A schematic diagram of a process in which a first terminal device calls a first hardware device of a second terminal device provided in an embodiment of the present application. Figure 6 As shown, in the first process: after the second terminal device is connected to the local area network, the second device management service module can send the device configuration file of the second terminal device to the cloud server. Assume that the device configuration file of the second terminal device does not include camera capability information.

[0220] The second process: After the first terminal device is connected to the local area network, or switched from the shutdown state to the power-on state, when the first device management service module finds the second terminal device that is online for the first time, the first device management service module obtains the device configuration file of the second terminal device from the cloud server.

[0221] The third process: The hardware device call service can refer to any service that the first terminal device needs to call the camera of the second terminal device. For example, a three-party conference application running on the first terminal device needs to call the camera of the second terminal device. When the first device management service module monitors that the second terminal device is online, it can send a notification message to the hardware device call service to notify the hardware device call service that the second terminal device is online. The hardware device call service sends an enable camera to the first device virtualization module, and then the first device virtualization module obtains the capability information of the camera of the second terminal device from the first device management service module, such as the frame rate and resolution of the camera. The first device management service module determines that the device configuration file of the second terminal device does not include the camera capability information, and notifies the first device virtualization module that the camera capability information acquisition failed.

[0222] The fourth process: the first device virtualization module sends a request for camera capability information to the second device virtualization module through the first communication service manager and the second communication service manager in turn; in response to the request, the second device virtualization module collects the camera capability information from the camera service; the second device virtualization module sends the camera capability information to the first device virtualization module through the second communication service manager and the first communication service manager in turn.

[0223] The fifth process: After the second device virtualization module receives a request for camera capability information from other terminal devices for the first time, it instructs the second device management service module to send the camera capability information to the cloud server, and the cloud server stores the camera capability information in the device configuration file of the second terminal device.

[0224] The sixth process: the first device virtualization module transmits the camera capability information to the virtual bus driver, and creates a virtual camera driver through the virtual bus driver using the camera capability information. In this way, the hardware device call service can call the camera of the second terminal device through the virtual camera driver to obtain the video stream from the second terminal device.

[0225] It should be understood that there is no order of precedence between the fifth process and the sixth process, and both can be performed simultaneously, and this application does not make any specific limitation on this.

[0226] Combine the following Figure 7 , a process of the first terminal device calling the camera of the second terminal device is described in detail.

[0227] Figure 7 A flowchart of a method 700 for a first terminal device to call a camera of a second terminal device is provided in an embodiment of the present application. Figure 7 As shown, the method 700 includes the following steps:

[0228] S10. The PC manager instructs the first device virtualization module to enable a camera service.

[0229] S11, the first device virtualization module sends a first message to the first device management service module, the first message obtaining capability information data of the second terminal device from the first device management service module. Correspondingly, the first device management service module receives the first message from the first device virtualization module.

[0230] S12: When the capability information data of the second terminal device is not included in the first device management service module, the first device management service module sends a second message to the first device virtualization module, where the second message is used to indicate that the first device virtualization module has failed to obtain the capability information data of the second terminal device. Correspondingly, the first device virtualization module receives the second message from the first device management service module.

[0231] S13. The first device virtualization module pulls up the second device virtualization module and creates a signaling channel.

[0232] S14, the second device virtualization module sends a third message to the camera framework, the third message is used to obtain capability information data of the camera of the second terminal device from the camera framework. Correspondingly, the camera framework receives the third message from the second device virtualization module.

[0233] S15, the camera framework sends a fourth message to the second device virtualization module, the fourth message including capability information data of the camera of the second terminal device. Correspondingly, the second device virtualization module receives the fourth message from the camera framework.

[0234] S16: The second device virtualization module sends the camera capability information data of the second terminal device to the first device virtualization module through the second communication service manager and the first communication service manager in turn. Correspondingly, the first device virtualization module receives the camera capability information data of the second terminal device from the second device virtualization module.

[0235] S17. The first device virtualization module deletes the signaling channel and the second device virtualization module.

[0236] S18, the first communication service manager sends a fifth message to the first device virtualization module, the fifth message is used to indicate to the first device virtualization module that the capability information data of the camera of the second terminal device has been successfully acquired. Correspondingly, the first device virtualization module receives the fifth message from the first communication service manager.

[0237] S19. The first device virtualization module instructs the virtual bus driver to create a virtual device driver based on the capability information data of the camera of the second terminal device. For example, the capability information data of the camera of the second terminal device includes that the camera supports resolutions of 1080P, 720P, and 480P, and the camera supports frame rates of 30FPS and 15FPS.

[0238] S20: A three-party application starts a conference. The three-party application may be a conference application.

[0239] S21. The third-party application sends an instruction to the virtual device driver to turn on the camera. Optionally, the instruction may include capability information data of the camera to be turned on, such as 480P and 30FPS.

[0240] S22. The first device virtualization module reads a sixth message in the cache message queue of the virtual device driver through the virtual driver bus. The sixth message is used to instruct the first device virtualization module to turn on the camera and includes capability information data of the camera, such as 480P and 30FPS.

[0241] S23: The first device virtualization module pulls up the second device virtualization module and creates a data channel.

[0242] S24, the first device virtualization module sends a camera opening instruction for instructing to open the camera to the second device virtualization module through the first communication service manager and the second communication service manager in sequence. The camera opening instruction may include camera capability information data, such as 480P and 30FPS.

[0243] S25. The second device virtualization module sends a seventh message to the camera framework, where the seventh message is used to instruct the camera framework to turn on the camera and indicate capability information data of the camera, such as 480P and 30FPS.

[0244] S26, the camera framework sends the captured image data to the second device virtualization module. Correspondingly, the second device virtualization module receives the image data from the camera framework.

[0245] S27: The second device virtualization module processes the image data, such as encoding and decoding.

[0246] S28: The second device virtualization module sends the processed image data to the first device virtualization module through the second communication service manager and the first communication service manager in sequence. Correspondingly, the first device virtualization module receives the processed image data from the second device virtualization module.

[0247] S29, the second device virtualization module decodes the processed image data and transmits the decoded image data to the virtual bus driver. Correspondingly, the bus driver reads the decoded image data.

[0248] S30 , driving the bus driver to feed back the decoded image data to the third-party application.

[0249] It should be understood that the specific implementation of S10 to S30 is similar to the steps in method 500, and can be found above, and will not be repeated here.

[0250] It should be understood that the order of execution of the above methods does not necessarily mean the order in which they are executed. The order in which the methods are executed should be determined by their functions and internal logic.

[0251] Combine the following Figures 8 to 11 , taking the first terminal device as a laptop, the second terminal device as a mobile phone model H90, and the first hardware device as the front camera and rear camera of the mobile phone as an example, the interface in the hardware device calling process is illustrated.

[0252] Figure 8 The following is a schematic diagram of an interface of a notebook computer display device manager provided in an embodiment of the present application. Figure 8 As shown, after the virtual bus driver is installed in the notebook computer, the system devices of the device manager in the notebook computer include the virtual bus driver.

[0253] Fig. 9 This is a schematic diagram of an interface for a laptop computer to find that a mobile phone is online, provided in an embodiment of the present application. Fig. 9 As shown, after the device management service module of the laptop computer finds that the mobile phone is online, the device model H90 of the second terminal device can be displayed in the PC housekeeper application in the laptop computer. Optionally, the device management service module can be a software module in the PC housekeeper application.

[0254] Fig.10 Another interface diagram of a notebook computer display device manager provided in an embodiment of the present application. Fig.10 As shown, the available hardware devices of the mobile phone include a front camera and a rear camera, and the device model of the mobile phone is H90. After the virtual bus driver of the laptop creates a virtual device driver corresponding to the front camera and a virtual device driver corresponding to the rear camera, the device manager of the laptop displays virtual devices corresponding to the two virtual device drivers: H90 rear camera 1001 and H90 front camera 1002.

[0255] Fig.11 This is a schematic diagram of a camera selection interface in a conference application provided by an embodiment of the present application. Fig.11 As shown, when the virtual camera driver for the H90 rear camera and the virtual camera driver for the H90 front camera have been installed in the laptop computer, the camera selection interface of the conference application in the laptop computer includes the cameras corresponding to the two virtual camera drivers: H90 rear camera 1101 and H90 front camera 1102.

[0256] Combination of the above Figures 2 to 11 , describes in detail the hardware device calling method of the embodiment of the present application, and the following is combined with Fig.12 , describes in detail the hardware device calling device of the embodiment of the present application.

[0257] Fig.12A schematic block diagram of a hardware device calling device 1200 provided in an embodiment of the present application. The device 1200 includes a processor 1201, a communication interface 1202, and a memory 1203. Among them, the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through an internal connection path, the memory 1203 is used to store instructions, and the processor 1201 is used to execute the instructions stored in the memory 1203. The communication interface 1202 can be used to send signals to other devices (such as the processor 1201 or the touch screen of the terminal device), and can also be used to receive signals from other devices (such as the memory 1203). Exemplarily, the communication interface 1202 reads the instructions stored in the memory 1203 and sends the instructions to the processor 1201.

[0258] It should be understood that the device 1200 can be specifically the first terminal device, the second terminal device or the cloud server in the above-mentioned embodiment, and can be used to execute the various steps and / or processes corresponding to the first terminal device, the second terminal device or the cloud server in the above-mentioned method embodiment. Optionally, the memory 1203 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type. The processor 1201 may be used to execute instructions stored in the memory, and when the processor 1201 executes instructions stored in the memory, the processor 1201 is used to execute the various steps and / or processes of the above-mentioned method embodiment.

[0259] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0260] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor executes the instructions in the memory, and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0261] The present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to implement the method shown in the above method embodiment.

[0262] The present application also provides a computer program product, which includes a computer program (also referred to as code or instruction). When the computer program runs on a computer, the computer can execute the method shown in the above method embodiment.

[0263] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0264] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0265] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0266] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0267] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0268] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0269] The above is only a specific implementation of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application, which should be included in the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be based on the protection scope of the claims.

Claims

1. A method for calling a hardware device, characterized in that: Applied to a first terminal device, the method includes: When the first terminal device and the second terminal device establish device interconnection for the first time, the first terminal device sends a first request for requesting a device configuration file of the second terminal device to the cloud server, a user logs in to the same account on the first terminal device and the second terminal device, and the wireless fidelity WiFi switch and the Bluetooth switch of the first terminal device and the second terminal device are both in the turned-on state; The first terminal device receives a device configuration file of the second terminal device from the cloud server; In the case where the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device; If so, the first terminal device uses the hardware device capability information of the first hardware device to call the first hardware device of the second terminal device.

2. The method according to claim 1, characterized in that The method further comprises: If the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; The first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

3. The method according to claim 2, characterized in that The first terminal device sends a second request for requesting hardware device capability information of the first hardware device to the second terminal device, including: The device virtualization module of the first terminal device establishes a signaling channel; The device virtualization module sends the second request to the second communication service manager of the second terminal device through the first communication service manager of the first terminal device using the signaling channel; The first terminal device receives the hardware device capability information from the second terminal device, including: The first communication service manager receives the hardware device capability information from the second communication service manager; The first communication service manager transmits the hardware device capability information to the device virtualization module.

4. The method according to any one of claims 1 to 3, characterized in that In the case where the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, including: The device virtualization module of the first terminal device sends a third request for requesting hardware device information of the first hardware device to the device management service module of the first terminal device, wherein the third request includes a device identifier of the second terminal device and an identifier of the first hardware device; The device management service module obtains a device configuration file of the second terminal device corresponding to the device identification of the second terminal device; The device management service module determines whether the device configuration file of the second terminal device includes information corresponding to the identifier of the first hardware device.

5. The method according to any one of claims 1 to 4, characterized in that The first request includes the device identification of the second terminal device.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: The first terminal device receives updated configuration information from the cloud server, where the updated configuration information includes a new device configuration file of the second terminal device or incremental information between the new device configuration file and the device configuration file; The first terminal device updates a device configuration file of the second terminal device based on the updated configuration information.

7. The method according to claim 6, characterized in that Before the first terminal device receives the updated configuration information from the cloud server, the method further includes: The first terminal device receives an update message from the cloud server, where the update message includes a device identifier of the second terminal device; In response to the update message, the first terminal device sends a fourth request to the cloud server for requesting a new device configuration file of the second terminal device.

8. The method according to claim 6 or 7, characterized in that: The updating of the device configuration file of the second terminal device includes: In a case where the updated configuration information includes a new device configuration file of the second terminal device, replacing the device configuration file of the second terminal device with the new device configuration file of the second terminal device; or, In a case where the updated configuration information includes the incremental information, a new device configuration file of the second terminal device is determined by combining the incremental information and the device configuration file of the second terminal device.

9. The method according to any one of claims 6 to 8, characterized in that The first terminal device disconnects from the second terminal device and then reconnects with the second terminal device when the first terminal device is not turned off, and the method further includes: In the case where the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; If so, the first terminal device uses the hardware device capability of the second hardware device to call the second hardware device of the second terminal device.

10. The method according to any one of claims 1 to 9, characterized in that The first hardware device is at least one of a camera, a microphone or a speaker.

11. The method according to any one of claims 1 to 10, characterized in that When the first hardware device is a camera, the hardware device capability information includes a resolution and / or a frame rate of the camera.

12. A method for calling a hardware device, characterized in that: Applied to a system including a first terminal device, a second terminal device and a cloud server, a user logs in to the same account on the first terminal device and the second terminal device, and the wireless fidelity WiFi switch and the Bluetooth switch of the first terminal device and the second terminal device are both in the turned-on state, the method includes: When the first terminal device and the second terminal device establish device interconnection for the first time, the first terminal device sends a first request for requesting a device configuration file of the second terminal device to the cloud server; The cloud server receives the first request from the first terminal device; The cloud server sends a device configuration file of the second terminal device to the first terminal device based on the first request; The first terminal device receives a device configuration file of the second terminal device from the cloud server; In the case where the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device; If so, the first terminal device uses the hardware device capability information of the first hardware device to call the first hardware device of the second terminal device.

13. The method according to claim 12, characterized in that The method further comprises: In a case where it is determined that the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; The second terminal device receives the second request from the first terminal device; In response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device.

14. The method according to claim 12 or 13, characterized in that The method further comprises: In the case where the third terminal device obtains the hardware device capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware device capability information of the first hardware device to the cloud server; The cloud server receives the hardware device capability information of the first hardware device, and stores the hardware device capability information of the first hardware device in the device configuration information of the second terminal device.

15. The method according to any one of claims 12 to 14, characterized in that The method further comprises: The second terminal device sends the incremental information between the new device configuration file of the second terminal device and the device configuration file to the cloud server; The cloud server receives the incremental information from the second terminal device, and determines a new device configuration file of the second terminal device by combining the incremental information and the device configuration file of the second terminal device; The cloud server sends update configuration information to the first terminal device, where the update configuration information includes a new device configuration file of the second terminal device or the incremental information; The first terminal device updates the device configuration file of the second terminal device based on the updated configuration information to obtain new device configuration information of the second terminal device.

16. The method according to claim 15, characterized in that Before the cloud server sends the update configuration information to the first terminal device, the method further includes: The cloud server sends an update message to the first terminal device, where the update message includes a device identifier of the second terminal device; The first terminal device receives the update message from the cloud server; In response to the update message, the first terminal device sends a fourth request to the cloud server for requesting a new device configuration file of the second terminal device; The cloud server sends update configuration information to the first terminal device, including: In response to the fourth request, the cloud server sends the updated configuration information to the first terminal device.

17. A hardware device calling device, characterized in that: include: A processor, wherein the processor is coupled to a memory, wherein the memory is used to store a computer program, and when the processor calls the computer program, the device is caused to execute the method according to any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that: Used to store a computer program, the computer program comprising instructions for implementing the method according to any one of claims 1 to 16.

19. A computer program product, characterized in that The computer program product includes a computer program code, and when the computer program code is executed on a computer, the computer is enabled to implement the method according to any one of claims 1 to 16.

Images