Hardware device invocation method and apparatus

Abstract

This application provides a method and apparatus for invoking a hardware device, which can improve the efficiency of hardware device invocation. The method includes: when a first terminal device and a second terminal device initially establish device interconnection, the first terminal device sends a first request to a cloud server to request a device configuration file of the second terminal device. The first and second terminal devices are logged into the same account, and the WiFi and Bluetooth switches of both devices are enabled; the first terminal device receives the device configuration file of the second terminal device from the cloud server; if the first terminal device needs to invoke a first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes hardware capability information of the first hardware device; if so, the first terminal device uses the hardware capability information of the first hardware device to invoke the first hardware device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal, in particular to a hardware device calling method and device. BACKGROUND

[0002] For a terminal device supporting interconnection service, such as a mobile phone, a tablet computer, a notebook computer and the like, a user can interconnect a first terminal device and a second terminal device through the interconnection service, so as to use a hardware device of the second terminal device, such as a camera, a microphone, a loudspeaker and the like, on the first terminal device through an application program. When the first terminal device calls a first hardware device of the second terminal device, the first terminal device needs to first acquire hardware device capability information of the first hardware device of the second terminal device, for example, if the first hardware device is a camera, the hardware device capability information of the first hardware device can be information such as resolution and frame rate of the camera; and then the first terminal device calls the first hardware device by using the hardware device capability information. At present, the first terminal device usually acquires the hardware device capability information of the second terminal device from the second terminal device.

[0003] However, although the calling of the hardware device between the devices can be realized in the above scenario, how to improve the calling efficiency of the hardware device is an urgent problem to be solved. SUMMARY

[0004] The present application provides a hardware device calling method and device, which can shorten the time consumption of the first terminal device in acquiring the hardware device capability information of the second terminal device, so as to improve the calling efficiency of the hardware device.

[0005] In a first aspect, a hardware device calling method is provided, which is applied to a first terminal device, and the method comprises the following steps: when the first terminal device and a second terminal device initially establish device interconnection, the first terminal device sends a first request for requesting a device configuration file of the second terminal device to a cloud server, a same account is logged in the first terminal device and the second terminal device, and a wireless fidelity (WiFi) switch and a Bluetooth switch of the first terminal device and the second terminal device are both in an open state; the first terminal device receives the device configuration file of the second terminal device from the cloud server; in a case where the first terminal device needs to call a first hardware device of the second terminal device, the first terminal device judges whether hardware device capability information of the first hardware device is included in the device configuration file of the second terminal device; if yes, the first terminal device calls the first hardware device of the second terminal device by using the hardware device capability information of the first hardware device.

[0006] The hardware device calling method of the application, the first terminal device caches the device configuration file of the second terminal device; when the first terminal device needs to call the first hardware device of the second terminal device, if the hardware device capability information of the first hardware device is included in the device configuration file of the second terminal device, the first terminal device can obtain the hardware device capability information of the first hardware device from the device configuration file of the second terminal device, and then call the first hardware device based on the hardware device capability information of the first hardware device. Such a hardware device calling method, when the first terminal device obtains the device capability information of the first hardware device of the second terminal device, if the device capability information of the first hardware device is included in the device configuration file of the second terminal device cached by the first terminal device, the first terminal device does not need to interact with other devices and can directly obtain it from the cache of the first terminal device. Compared with obtaining the hardware device capability information of the first hardware device from the second terminal device, such a hardware device capability information obtaining method does not need to interact with multiple software modules of the application layer of the second terminal device, thereby making the first terminal device take less time to obtain the device capability information of the first hardware device of the second terminal device and improving the efficiency of the hardware device calling method.

[0007] It should be understood that initially establishing device interconnection means that each time the first terminal device is switched from the shutdown state to the startup state, or is switched from the unconnected state to the connected state, the first terminal device establishes device interconnection with the second terminal device for the first time. The device configuration file of the second terminal device can also be referred to as a profile, and the device configuration file of the second terminal device can include information such as the device model of the second terminal device. The cloud server can refer to one or more servers, and the cloud server can respectively perform data transmission with the first terminal device and the second terminal device. The cloud server can be used to store device configuration files of multiple terminal devices.

[0008] In some implementations of the first aspect, the method further includes: if the hardware device capability information of the first hardware device is not included in the device configuration file of the second terminal device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; and the first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

[0009] It should be understood that when the first terminal device does not include the hardware device capability information of the first hardware device in the device configuration file of the second terminal device stored by the first terminal device, the first terminal device needs to obtain the hardware device capability information of the first hardware device from the second terminal device.

[0010] In some implementations of the first aspect, the first terminal device sending a second request for requesting hardware device capability information of the first hardware device to the second terminal device comprises: a device virtualization module of the first terminal device establishing a signaling channel; the device virtualization module sending, by a first communication service manager of the first terminal device, the second request to a second communication service manager of the second terminal device via the signaling channel; and the first terminal device receiving the hardware device capability information from the second terminal device, comprising: the first communication service manager receiving the hardware device capability information from the second communication service manager; and the first communication service manager transmitting the hardware device capability information to the device virtualization module.

[0011] It should be understood that the first communication service manager and the second communication service manager can realize data transmission between the first terminal device and the second terminal device, and therefore the device virtualization module of the first terminal device can send the second request to the second communication service manager via the first communication service manager.

[0012] In some implementations of the first aspect, in a case where the first terminal device needs to invoke the first hardware device of the second terminal device, the first terminal device determining whether the hardware device capability information of the first hardware device is included in a device configuration file of the second terminal device comprises: a device virtualization module of the first terminal device sending a third request for requesting hardware device information of the first hardware device to a device management service module of the first terminal device, the third request comprising a device identifier of the second terminal device and an identifier of the first hardware device; the device management service module obtaining the device configuration file of the second terminal device corresponding to the device identifier of the second terminal device; and the device management service module determining whether information corresponding to the identifier of the first hardware device is included in the device configuration file of the second terminal device.

[0013] It should be understood that the device management service module of the first terminal device can be used to cache the device configuration file of the second terminal device. The device virtualization module of the first terminal device can determine whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device. In a case where the device virtualization module of the first terminal device determines whether the first terminal device needs to obtain the hardware device capability information of the hardware device of the second terminal device, the device virtualization module of the first terminal device can obtain the hardware device capability information of the hardware device of the second terminal device from the device management service module of the first terminal device, so that the first terminal device can efficiently determine the hardware device capability information of the first hardware device of the second terminal device.

[0014] In some implementations of the first aspect, the first request includes the device identifier of the second terminal device.

[0015] It should be understood that the device identifier of the second terminal device can refer to information such as the device model, device name, or device ID of the second terminal device. The cloud server can determine the device configuration file of the second terminal device that the first terminal device needs to obtain based on the device identifier in the first request.

[0016] In some implementations of the first aspect, the method further includes: the first terminal device receiving updated configuration information from the cloud server, the updated configuration information including a new device configuration file of the second terminal device or incremental information between the new device configuration file and the device configuration file; the first terminal device updating the device configuration file of the second terminal device based on the updated configuration information.

[0017] It should be understood that if the device configuration file of the second terminal device stored in the cloud server changes, the cloud server can instruct the first terminal device to update the device configuration file of the second terminal device cached by the first terminal device. In this way, when the first terminal device needs to obtain the device configuration information in the device configuration file of the second terminal device, it can obtain the updated device configuration information from the new device configuration file, which can improve the efficiency and accuracy of the first terminal device in obtaining the device configuration information of the second terminal device.

[0018] In some implementations of the first aspect, before the first terminal device receives updated configuration information from the cloud server, the method further includes: the first terminal device receiving an update message from the cloud server, the update message including a device identifier of the second terminal device; and in response to the update message, the first terminal device sending a fourth request to the cloud server to request a new device configuration file for the second terminal device.

[0019] It should be understood that the fourth request can be the same as or different from the first request. That is, the fourth request can be used to request a new device configuration file for the second terminal device stored on the cloud server, or it can be used to request incremental information between the new device configuration file and the existing device configuration file. When the device configuration information of the second terminal device stored on the cloud server changes, the cloud server sends an update message to the first terminal device. When the first terminal device needs to update the device configuration information of the second terminal device, the first terminal device sends a fourth request to the cloud server. In this way, the first terminal device can control when to update the device configuration file of the second terminal device, which helps to improve the success rate of updating the device configuration file of the second terminal device.

[0020] In some implementations of the first aspect, updating the device configuration file of the second terminal device includes: replacing the device configuration file of the second terminal device with the new device configuration file of the second terminal device when the updated configuration information includes the new device configuration file of the second terminal device; or, when the updated configuration information includes the incremental information, determining the new device configuration file of the second terminal device by combining the incremental information and the device configuration file of the second terminal device.

[0021] It should be understood that, in combination with the incremental information and the device configuration file of the second terminal device, determining the new device configuration file of the second terminal device can be as follows: if the incremental information is data information that needs to be added, the first terminal device adds the incremental information to the device configuration file of the second terminal device to obtain the new device configuration file; if the incremental information is data information that needs to be deleted, the first terminal device deletes the data information corresponding to the incremental information in the device configuration file of the second terminal device to obtain the new device configuration file; if the incremental information is newly added data information that needs to be changed, the first terminal device changes the data information that needs to be changed in the device configuration file of the second terminal device to obtain the new device configuration file.

[0022] In some implementations of the first aspect, after the first terminal device disconnects from the second terminal device while it is not powered off, it re-establishes a connection. The method further includes: when the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; if so, the first terminal device uses the hardware device capability of the second hardware device to call the second hardware device of the second terminal device.

[0023] It should be understood that, compared to the device configuration file, when the hardware device capability information of the second hardware device is added to the new device configuration file, this hardware device invocation method allows the first terminal device to obtain the hardware device capability information of the second hardware device from the new device configuration file of the second terminal device cached in the first terminal device. This helps to improve the efficiency of the first terminal device in obtaining the hardware device capability information of the second hardware device of the second terminal device, thereby improving the efficiency of the first terminal device invoking the second hardware device of the second terminal device.

[0024] In some implementations of the first aspect, the first hardware device is at least one of a camera, a microphone, or a speaker.

[0025] In some implementations of the first aspect, when the first hardware device is a camera, the hardware device capability information includes the resolution and / or frame rate of the camera.

[0026] It should be understood that when the first terminal device needs to access the camera of the second terminal device, it can determine the configuration supported by the camera based on the camera's resolution and / or frame rate, thereby allowing the user to select the corresponding camera configuration on the first terminal device.

[0027] Secondly, a hardware device invocation method is provided, applied to a system including a first terminal device, a second terminal device, and a cloud server. A user has logged into the same account on both the first and second terminal devices, and both the Wi-Fi and Bluetooth switches on the first and second terminal devices are enabled. The method includes: when the first and second terminal devices initially establish device interconnection, the first terminal device sends a first request to the cloud server to request a device configuration file for the second terminal device; the cloud server receives the first request from the first terminal device; based on the first request, the cloud server sends the device configuration file of the second terminal device to the first terminal device; the first terminal device receives the device configuration file of the second terminal device from the cloud server; when the first terminal device needs to invoke a first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes hardware capability information of the first hardware device; if so, the first terminal device uses the hardware capability information of the first hardware device to invoke the first hardware device of the second terminal device.

[0028] In some implementations of the second aspect, the method further includes: if it is determined that the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request to the second terminal device to request the hardware device capability information of the first hardware device; the second terminal device receives the second request from the first terminal device; and in response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device.

[0029] In some implementations of the second aspect, the method further includes: when the third terminal device obtains the hardware capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware capability information of the first hardware device to the cloud server; the cloud server receives the hardware capability information of the first hardware device and stores the hardware capability information of the first hardware device in the device configuration information of the second terminal device.

[0030] In some implementations of the second aspect, the method further includes: the second terminal device sending incremental information between the new device configuration file of the second terminal device and the device configuration file to the cloud server; the cloud server receiving the incremental information from the second terminal device and determining the new device configuration file of the second terminal device by combining the incremental information and the device configuration file of the second terminal device; the cloud server sending updated configuration information to the first terminal device, the updated configuration information including the new device configuration file of the second terminal device or the incremental information; and the first terminal device updating the device configuration file of the second terminal device based on the updated configuration information to obtain the new device configuration information of the second terminal device.

[0031] In some implementations of the second aspect, before the cloud server sends the updated configuration information to the first terminal device, the method further includes: the cloud server sending an update message to the first terminal device, the update message including a device identifier of the second terminal device; the first terminal device receiving the update message from the cloud server; in response to the update message, the first terminal device sending a fourth request to the cloud server for requesting a new device configuration file for the second terminal device; the cloud server sending the updated configuration information to the first terminal device includes: in response to the fourth request, the cloud server sending the updated configuration information to the first terminal device.

[0032] In some implementations of the second aspect, updating the device configuration file of the second terminal device includes: replacing the device configuration file of the second terminal device with the new device configuration file of the second terminal device when the updated configuration information includes the new device configuration file of the second terminal device; or, when the updated configuration information includes the incremental information, determining the new device configuration file of the second terminal device by combining the incremental information and the device configuration file of the second terminal device.

[0033] In some implementations of the second aspect, after the first terminal device disconnects from the second terminal device while it is not powered off, it re-establishes a connection. The method further includes: when the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; if so, the first terminal device uses the hardware device capability of the second hardware device to call the second hardware device of the second terminal device.

[0034] Thirdly, this application provides a hardware device invocation apparatus, including a processor coupled to a memory, which can be used to execute instructions in the memory to implement the method in any of the possible implementations of the first or second aspect described above. Optionally, the apparatus further includes a memory. Optionally, the apparatus further includes a communication interface, to which the processor is coupled.

[0035] In one implementation, the device is a first terminal device. When the device is a first terminal device, the aforementioned communication interface can be a transceiver, or an input / output interface.

[0036] In one implementation, the device is a second terminal device. When the device is a second terminal device, the aforementioned communication interface can be a transceiver, or an input / output interface.

[0037] In one implementation, the device is a cloud server. When the device is a cloud server, the aforementioned communication interface can be a transceiver, or an input / output interface.

[0038] In another implementation, the device is a chip configured in a terminal device. When the device is a chip configured in a terminal device, the aforementioned communication interface can be an input / output interface.

[0039] Fourthly, a processor is provided, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is configured to receive signals through the input circuit and transmit signals through the output circuit, causing the processor to execute the method in any possible implementation of the first or second aspect described above.

[0040] In the specific implementation process, the processor can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, gate circuit, flip-flop, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit can be output to, for example, but not limited to, a transmitter and transmitted by the transmitter. Furthermore, the input circuit and the output circuit can be the same circuit, which is used as the input circuit and the output circuit at different times. This application does not limit the specific implementation of the processor and various circuits.

[0041] Fifthly, a processing apparatus is provided, including a processor and a memory. The processor is used to read instructions stored in the memory and to receive signals via a receiver and transmit signals via a transmitter to execute the method in any possible implementation of the first or second aspect described above.

[0042] Optionally, the processor may be one or more, and the memory may be one or more.

[0043] Optionally, the memory may be integrated with the processor, or the memory may be separated from the processor.

[0044] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0045] It should be understood that the relevant data interaction process, such as sending instruction information, can be a process of outputting instruction information from the processor, and receiving capability information can be a process of the processor receiving input capability information. Specifically, the processed output data can be output to the transmitter, and the input data received by the processor can come from the receiver. Here, the transmitter and receiver can be collectively referred to as transceivers.

[0046] The processing device in the fifth aspect above can be a chip. The processor can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor that reads software code stored in memory. The memory can be integrated into the processor or located outside the processor and exist independently.

[0047] In a sixth aspect, a computer program product is provided, the computer program product comprising: a computer program (also referred to as code or instructions), which, when the computer program is run, causes a computer to perform the method in any possible implementation of the first or second aspect described above.

[0048] In a seventh aspect, a computer-readable storage medium is provided that stores a computer program (also referred to as code or instructions) that, when executed on a computer, causes the computer to perform the methods in any possible implementation of the first or second aspect described above. Attached Figure Description

[0049] Figure 1 A schematic diagram illustrating an application scenario provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0051] Figure 3 This is a software architecture block diagram of the Windows system of the terminal device in an embodiment of this application;

[0052] Figure 4 This is a software structure block diagram of the Android system of the terminal device in an embodiment of this application;

[0053] Figure 5 A flowchart illustrating a hardware device invocation method provided in an embodiment of this application;

[0054] Figure 6 A schematic diagram illustrating the process by which a first terminal device calls a first hardware device of a second terminal device, as provided in an embodiment of this application;

[0055] Figure 7 A flowchart illustrating a method for a first terminal device to access the camera of a second terminal device, provided in an embodiment of this application;

[0056] Figure 8 A schematic diagram of a laptop computer display device manager provided as an embodiment of this application;

[0057] Figure 9 This application provides a schematic diagram of an interface for a laptop computer to discover a mobile phone online.

[0058] Figure 10 A schematic diagram of another laptop computer display device manager interface provided in an embodiment of this application;

[0059] Figure 11 A schematic diagram of a camera selection interface in a conference application provided in this application embodiment;

[0060] Figure 12 This is a schematic block diagram of a hardware device calling device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0062] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect. For example, the first value and the second value are only used to distinguish different values ​​and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily imply that they are different.

[0063] It should be noted that, in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0064] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0065] Currently, for terminal devices that support Internet services, such as mobile phones, tablets, and laptops, users can connect the first terminal device and the second terminal device through Internet services, thereby enabling data transmission between the first terminal device and the second terminal device.

[0066] The following is combined Figure 1 Taking a laptop computer as the first terminal device and a mobile phone as the second terminal device as an example, the application scenarios of the embodiments of this application will be described in detail.

[0067] Figure 1 This is a schematic diagram illustrating an application scenario 100 provided in an embodiment of this application. For example... Figure 1 As shown, when a user conducts a video conference using a video conferencing application on laptop 101, and needs to photograph experimental equipment, the user can connect laptop 101 and mobile phone 102 to enable data transfer. Laptop 101 can then access the camera of mobile phone 102, and the video stream of the experimental equipment captured by the camera of mobile phone 102 can be transmitted to laptop 101, thus showcasing the experimental equipment to other participants in the video conference.

[0068] To better understand the terminal device in the embodiments of this application, the following is combined with... Figure 1 The hardware structure of the terminal device in the embodiments of this application will be described in detail.

[0069] Figure 2 This is a schematic diagram of the structure of the terminal device 200 provided in an embodiment of this application. Figure 2 As shown, the terminal device 200 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0070] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the terminal device 200. In other embodiments of this application, the terminal device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0071] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0072] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0073] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0074] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0075] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the terminal device 200.

[0076] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0077] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0078] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0079] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the shooting function of the terminal device 200. The processor 110 and the display screen 194 communicate via the DSI interface to enable the display function of the terminal device 200.

[0080] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 193, a display screen 194, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0081] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, or USB Type-C port. USB port 130 can be used to connect a charger to charge terminal device 200, and can also be used for data transfer between terminal device 200 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other terminal devices, such as AR devices.

[0082] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the terminal device 200. In other embodiments of this application, the terminal device 200 may also adopt different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0083] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the terminal device 200. While charging the battery 142, the charging management module 140 can also supply power to the terminal device via the power management module 141.

[0084] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0085] The wireless communication function of the terminal device 200 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0086] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 200 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0087] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the terminal device 200. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0088] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0089] The wireless communication module 160 can provide solutions for wireless communication applications on the terminal device 200, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0090] In some embodiments, antenna 1 of terminal device 200 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling terminal device 200 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0091] The terminal device 200 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0092] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, terminal device 200 may include one or N displays 194, where N is a positive integer greater than 1.

[0093] The terminal device 200 can perform shooting functions through an ISP, camera 193, video codec, GPU, display screen 194, and application processor.

[0094] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0095] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the terminal device 200 may include one or N cameras 193, where N is a positive integer greater than 1.

[0096] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when the terminal device 200 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0097] Video codecs are used to compress or decompress digital video. Terminal device 200 may support one or more video codecs. Thus, terminal device 200 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0098] NPU stands for Neural Network (NN) Computing Processor. By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in terminal devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0099] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the terminal device 200. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0100] Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the terminal device 200 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of the terminal device 200 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0101] Terminal device 200 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0102] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0103] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The terminal device 200 can listen to music or make hands-free calls through the speaker 170A.

[0104] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the terminal device 200 receives a telephone call or voice message, the receiver 170B can be brought close to the listener's ear to receive the voice message.

[0105] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Terminal device 200 may be equipped with at least one microphone 170C. In some embodiments, terminal device 200 may be equipped with two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, terminal device 200 may be equipped with three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0106] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0107] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Terminal device 200 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, terminal device 200 detects the intensity of the touch operation based on pressure sensor 180A. Terminal device 200 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example: when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS is executed.

[0108] The gyroscope sensor 180B can be used to determine the motion attitude of the terminal device 200. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the terminal device 200 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the terminal device 200's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the terminal device 200 through reverse movement, thus achieving image stabilization. The gyroscope sensor 180B can also be used in navigation and motion-sensing game scenarios.

[0109] The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the terminal device 200 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0110] The magnetic sensor 180D includes a Hall effect sensor. The terminal device 200 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the terminal device 200 is a flip phone, the terminal device 200 can detect the opening and closing of the flip cover using the magnetic sensor 180D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0111] The 180E accelerometer can detect the magnitude of acceleration in various directions (typically three axes) of the terminal device 200. When the terminal device 200 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the attitude of the terminal device, and can be applied to applications such as landscape / portrait switching and pedometers.

[0112] A distance sensor 180F is used to measure distance. The terminal device 200 can measure distance via infrared or laser. In some embodiments, during a shooting scene, the terminal device 200 can utilize the distance sensor 180F to measure distance for rapid focusing.

[0113] The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The terminal device 200 emits infrared light outward through the LED. The terminal device 200 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 200. When insufficient reflected light is detected, the terminal device 200 can determine that there is no object near the terminal device 200. The terminal device 200 may use the proximity sensor 180G to detect when a user holds the terminal device 200 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used in holster mode and pocket mode for automatic unlocking and screen locking.

[0114] The ambient light sensor 180L is used to sense ambient light intensity. The terminal device 200 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light intensity. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the terminal device 200 is in a pocket to prevent accidental touches.

[0115] The fingerprint sensor 180H is used to collect fingerprints. The terminal device 200 can use the collected fingerprint characteristics to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0116] Temperature sensor 180J is used to detect temperature. In some embodiments, terminal device 200 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, terminal device 200 reduces the performance of the processor located near temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is below another threshold, terminal device 200 heats battery 142 to prevent abnormal shutdown of terminal device 200 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, terminal device 200 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

[0117] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of terminal device 200, in a different position than display screen 194.

[0118] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

[0119] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Terminal device 200 can receive button input and generate key signal inputs related to user settings and function control of terminal device 200.

[0120] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0121] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0122] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the terminal device 200. The terminal device 200 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external storage cards. The terminal device 200 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the terminal device 200 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 200 and cannot be separated from the terminal device 200. The software system of the terminal device 200 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture of the Android system as an example to illustrate the software structure of the terminal device 200.

[0123] Currently, for terminal devices that support Internet services, such as mobile phones, tablets, and laptops, users can connect the first terminal device and the second terminal device through Internet services, thereby enabling the first terminal device to use the hardware devices of the second terminal device, such as cameras, microphones, and speakers, through an application.

[0124] The following is combined Figure 3 and Figure 4 The software structures of the first terminal device and the second terminal device involved in the information acquisition method provided in the embodiments of this application will be described in detail. Figure 3 Taking a personal computer (PC) as the first terminal device and a Windows system as the first terminal device as an example, its software structure will be explained. Figure 4 The software structure is explained using an Android phone as the second terminal device.

[0125] It should be understood that the first terminal device may also be a type of terminal device other than a personal computer, and the system of the first terminal device may also be an Android system; the second terminal device may also be a type of terminal device other than a mobile phone, and the system of the second terminal device may also be a Windows system. This application does not specifically limit the device type and operating system type of the first terminal device and the second terminal device.

[0126] Figure 3This is a software architecture diagram of the first terminal device. The layered architecture of the Windows system divides the software into several layers, which communicate with each other through software interfaces. For example, the application layer and the system layer can interact through an application programming interface (API), while the system layer and the kernel layer can interact through standard I / O interfaces. In some embodiments, the Windows system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system layer, and the kernel layer. The specific functional modules included in each layer are explained below.

[0127] 1. Application Layer

[0128] The application layer may include a series of applications, such as music, PC management, Bluetooth, WIFI, games, video, third-party applications, camera, and software development kits (SDKs). In this embodiment, the application layer also includes a device virtualization SDK, a device manager, etc. In one possible implementation, third-party applications can access the device virtualization SDK and call the system-level device virtualization module through the device virtualization SDK.

[0129] The aforementioned PC manager may include installation packages for various applications, allowing users to install multiple applications on their primary terminal device. It also enables system optimization, smart audio-visual features, and intelligent interconnection between devices, multi-screen collaboration, and hotspot sharing. The PC manager may also have other names, which are not specifically limited in this application.

[0130] The device manager is used to view and change device properties, modify device drivers, configure device settings, and uninstall devices. In this embodiment, the device manager is also used to implement secure authentication and account authorization for devices under the same account, self-discovery and self-organizing of devices, data transmission between devices, and access to communication services.

[0131] The aforementioned third-party applications may include video conferencing applications, social networking and chat applications, and so on.

[0132] 2. Application Framework Layer

[0133] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0134] like Figure 3As shown in this embodiment, the application framework layer includes at least: a device management service module, a communication service manager, and a device virtualization module.

[0135] The device management service module manages the device data profile of the first terminal device. This profile may include various attribute information of the first terminal device, such as device type. In this embodiment, the device data profile may also include hardware capability information of the first terminal device, such as camera resolution and frame rate. The device management service also monitors device online status; for example, after a second terminal device comes online, the first terminal device can detect its online status through the device management service. Furthermore, the device management service synchronizes the device data profile of the second terminal device from the cloud server.

[0136] The communication service manager enables data transfer between devices. For example, when a first terminal device accesses the camera of a second terminal device, the video or image data stream captured by that camera is transmitted through the communication service manager.

[0137] The communication service manager can also be called a smart interconnection module, MagicLink transmission management service module, etc., and this application does not make a specific limitation in this regard.

[0138] The device virtualization module manages the lifecycle of virtual device drivers, controlling their creation and deletion (also known as insertion and removal) via a virtual bus driver. It also processes requests from third-party applications to the virtual device drivers and returns the processing results to those applications. The device virtualization module can interact with other applications through the application-layer device virtualization SDK. In this embodiment, the device virtualization module also manages the data and signaling channels maintained by the communication service manager, enabling data transmission across devices.

[0139] The device virtualization module can also be referred to as a hardware service cross-device sharing module, a distributed mobile sensing development platform (DMSDP), a hardware virtualization module, etc., and this application does not specifically limit it in this way. Correspondingly, the device virtualization SDK can also be referred to as a hardware service cross-device sharing SDK, a DMSDP SDK, a hardware virtualization SDK, etc., and the embodiments of this application do not limit it in this way either.

[0140] 3. System layer

[0141] The system layer can include the Windows multimedia framework, etc., and can provide corresponding services for the execution of user application processes.

[0142] The Windows Multimedia Framework includes various Windows function calls that can be used by applications for application development. The Windows Multimedia Framework also connects the application layer and the kernel layer, enabling interaction between them. For example, a request sent by an application to the kernel layer can first be transmitted to the Windows Multimedia Framework via the Windows Application Framework (FWK) API, and then transmitted to the kernel layer via the standard I / O interface.

[0143] 4. Kernel layer

[0144] The kernel layer is the layer between hardware and software. It drives the hardware, enabling it to function. The kernel layer can include virtual bus drivers, virtual device drivers, Windows Driver Foundation (WDF) modules, and the operating system (OS) kernel.

[0145] Virtual device drivers can be recognized as Universal Serial Bus (USB) devices by terminal devices and used by them. Examples include virtual camera devices and virtual microphone devices. If the first terminal device recognizes an available USB device (also called a hardware device), a corresponding virtual device driver can be created for each USB device. The kernel layer of the first terminal device can include multiple virtual device drivers. If the first terminal device does not recognize any available USB devices, the kernel layer of the terminal device 200 includes zero virtual device drivers. When the first terminal device creates at least one virtual device driver, that virtual device driver can be displayed in its device manager.

[0146] The virtual bus driver is used to create or delete virtual device drivers, that is, it is responsible for managing the insertion, removal and communication process of virtual device drivers.

[0147] Optionally, PC Manager can call the installation script and installation package of the virtual bus driver to install the virtual bus driver.

[0148] The virtual bus driver can also be called a virtual bus controller, virtual drive bus, etc., and this application does not make a specific limitation on it.

[0149] The OS kernel is used to convert requests from the application layer into data processing instructions, which are then sent to the virtual device driver via the driver development framework.

[0150] The driver development framework is used to send data processing instructions from the OS kernel to the virtual device driver.

[0151] Figure 4 This is a software structure block diagram of a second terminal device provided in an embodiment of this application. The Android layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, the hardware abstraction layer, and the Linux kernel layer.

[0152] 1. Application Layer

[0153] The application layer can include a series of application packages. For example... Figure 4 As shown in the embodiments of this application, the application layer includes at least third-party applications, camera, Bluetooth, WIFI, and device virtualization SDK, etc.

[0154] Third-party applications can be video conferencing applications, social networking and chat applications, etc.

[0155] The device virtualization module SDK may also include public service interfaces, camera service interfaces, audio service interfaces, and speaker service interfaces. These service interfaces are exposed through the Android Interface Definition Language (AIDL) provided by the interface object abstraction layer. The interface object abstraction layer can be encapsulated by the service interface adaptation layer to enable cross-process access.

[0156] In this embodiment of the application, the device virtualization module SDK can call the device virtualization module in the system library through the public service interface.

[0157] 2. Application Framework Layer

[0158] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0159] like Figure 4 As shown in this embodiment, the application framework layer includes at least a communication service manager, a device management service module, and a device virtualization module. In this embodiment, when the camera of the second terminal device is invoked by the first terminal device, the image data stream captured by the camera is transmitted through the communication service manager.

[0160] In this embodiment, the device virtualization module can be responsible for interacting with hardware devices, such as opening and closing the camera, and acquiring camera video streams; it can also perform image processing and encoding / decoding, such as blurring images and encoding images acquired by the camera; the device virtualization module can also manage the data transmission channel of the communication service manager, such as pausing data transmission through the data transmission channel when the user chooses to pause camera access; in addition, the device virtualization module can also process user interface command messages through interaction with the device virtualization module SDK.

[0161] The device virtualization module may also include an interface security control module, a distribution control module, a channel management module, a device management module, and a service management module. In this embodiment, the service management module may include a camera service module, which may further include an image processing module and an encoding / decoding module. The image processing module can perform image blurring, and the encoding / decoding module can perform image encoding. Optionally, the service management module may also include an audio service module and a speaker service module. This application does not limit the hardware service modules and functions it includes.

[0162] 3. System layer

[0163] The system layer includes system libraries and the Android runtime. The Android runtime includes core libraries and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that Java needs to call, and the other part contains the core Android libraries. The application layer and application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and application framework layer into binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0164] A system library can include multiple functional modules. At a minimum, a system library includes: a graphics processing library, etc.

[0165] Graphics processing libraries can include 3D graphics processing libraries and 2D graphics processing libraries. 3D graphics processing libraries are used for 3D graphics drawing, image rendering, compositing, and layer processing; 2D graphics processing libraries are used for drawing and processing 2D graphics.

[0166] 4. Hardware Abstraction Layer

[0167] The Hardware Abstraction Layer (HAL) is an abstraction layer situated between the Linux kernel layer and the system runtime library layer. The HAL can be a wrapper around hardware drivers, providing a unified interface for upper-layer applications to call them. The HAL may include modules such as Bluetooth, Wi-Fi, and hardware configuration. For example, in this embodiment, the Bluetooth and Wi-Fi modules in the HAL can shield the specific hardware implementation details of the Bluetooth and Wi-Fi drivers in the kernel layer, allowing upper-layer applications to implement Bluetooth and Wi-Fi functions by calling interfaces without needing to know the specific implementation details of the kernel-level hardware drivers.

[0168] 5. Linux kernel layer

[0169] The Linux kernel layer is the layer between hardware and software. It includes at least display drivers, camera drivers, memory drivers, sensor drivers, and inter-process communication drivers.

[0170] When the first terminal device calls the first hardware device of the second terminal device, the first terminal device needs to first obtain the hardware device capability information of the first hardware device of the second terminal device. For example, if the first hardware device is a camera, the hardware device capability information of the first hardware device can be information such as the resolution and frame rate of the camera. Then, the first terminal device uses the hardware device capability information to call the first hardware device.

[0171] Currently, the first terminal device usually obtains hardware capability information of the second terminal device from the second terminal device.

[0172] However, this method of hardware device invocation requires the first terminal device to interact with multiple modules in the application framework layer of the first terminal device and multiple software modules in the application framework layer of the second terminal device each time it calls the hardware device of the second terminal device. This results in a long processing time for the first terminal device to obtain the hardware capability information of the second terminal device, thus reducing the efficiency of the first terminal device calling the hardware device of the second terminal device. Therefore, although hardware device invocation between devices can be achieved in the above scenario, improving the efficiency of hardware device invocation is an urgent problem to be solved.

[0173] To address the aforementioned technical problems, this application provides a hardware device invocation method and apparatus. A cloud server stores a device configuration file for a second terminal device. After the first and second terminal devices establish an interconnection, the first terminal device retrieves the device configuration file of the second terminal device from the cloud server. When the first terminal device needs to invoke a first hardware device of the second terminal device, it can obtain the hardware capability information of the first hardware device from the device configuration file of the second terminal device, and then invoke the first hardware device based on this hardware capability information. With this hardware device invocation method, when the first terminal device retrieves the device capability information of the first hardware device of the second terminal device, and the device configuration file of the second terminal device cached by the first terminal device includes the device capability information of the first hardware device, the first terminal device does not need to interact with other devices and can directly retrieve it from its cache. Compared to retrieving the hardware capability information of the first hardware device from the second terminal device, this method of obtaining hardware capability information does not require interaction with multiple software modules in the application layer of the second terminal device, thus reducing the time required for the first terminal device to retrieve the device capability information of the first hardware device of the second terminal device and improving the efficiency of the hardware device invocation method.

[0174] It should be understood that the aforementioned device configuration file may be uploaded by the second terminal device to the cloud server. If a terminal device requests the hardware capability information of the first hardware device of the second terminal device from the second terminal device, the second terminal device may send the hardware capability information of the first hardware device to the cloud server. The cloud server will then store the hardware capability information in the device configuration file of the second terminal device so that other terminal devices can directly obtain the hardware capability information of the first hardware device from the cloud server.

[0175] The following is combined Figures 5 to 11 This application provides a detailed description of the hardware invocation method. The embodiments shown in this application illustrate the hardware device invocation method from the perspective of device interaction. The specific forms and quantities of the devices shown are merely examples and should not be construed as limiting the implementation of the method provided in this application. Below, using a first terminal device, a second terminal device, and a cloud server as examples, the hardware device invocation method of this application's embodiments will be described in detail.

[0176] It should be understood that the terminal device can be the terminal device itself, or a chip, chip system or processor that supports the terminal device to implement cross-device hardware control methods, or a logic module or software that can implement all or part of the functions of the terminal device. This application does not make any specific limitations in this regard.

[0177] Figure 5This is a flowchart illustrating a hardware device invocation method 500 provided in an embodiment of this application. Method 500 is applied to a system including a first terminal device, a second terminal device, and a cloud server. The user has logged into the same account on both the first and second terminal devices, and both the Wi-Fi and Bluetooth switches on the first and second terminal devices are enabled. The hardware structures of the first and second terminal devices can be as follows: Figure 1 As shown, the software structure of the first terminal device can be as follows: Figure 3 As shown, the software structure of the second terminal device can be as follows: Figure 4 As shown.

[0178] It should be understood that an account can refer to a personally registered account provided by different terminal device brands to ensure the security of the user's device. After the first and second terminal devices log in to the same account, data sharing between the devices can be achieved. When the first and second terminal devices are logged in to the same account, and both the WiFi and Bluetooth switches on the first and second terminal devices are turned on, the first terminal device can detect the second terminal device.

[0179] Method 500 includes the following steps:

[0180] S501. When the first terminal device and the second terminal device establish device interconnection for the first time, the first terminal device sends a first request to the cloud server to request the device configuration file of the second terminal device. Correspondingly, the cloud server receives the first request from the first terminal device.

[0181] It should be understood that the initial device interconnection refers to the first time the first terminal device switches from a powered-off state to a powered-on state, or from a state not connected to the local area network (LAN) to a state connected to the LAN, that the first terminal device establishes a device interconnection with the second terminal device. The device configuration file of the second terminal device, also known as a profile, may include information such as the device model. The cloud server can refer to one or more servers, which can transmit data with both the first and second terminal devices. The cloud server can be used to store the device configuration files of multiple terminal devices.

[0182] S502, Based on the first request, the cloud server sends the device configuration file of the second terminal device to the first terminal device. Correspondingly, the first terminal device receives the device configuration file of the second terminal device from the cloud server.

[0183] S503. When the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device determines whether the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device.

[0184] It should be understood that hardware devices can also be called USB devices, such as cameras. The first hardware device can refer to any one or more hardware devices in the second terminal device. Hardware device capability information can refer to information used to indicate the attributes and configuration of the hardware device. The situation where the first terminal device needs to call the first hardware device of the second terminal device can also be understood as the situation where the first terminal device needs to create a virtual device driver corresponding to the first hardware device based on the hardware device capability information of the first hardware device; that is, the situation where the first terminal device needs to call the first hardware device of the second terminal device can refer to any situation where the first terminal device needs the hardware device capability information of the first hardware device.

[0185] S504. If the device configuration file of the second terminal device includes the hardware device capability information of the first hardware device, the first terminal device uses the hardware device capability information of the first hardware device to call the first hardware device of the second terminal device.

[0186] It should be understood that the first hardware device that calls the second terminal device can also refer to the creation of a virtual device driver corresponding to the first hardware device. That is, calling the first hardware device does not necessarily mean using the first hardware device. This application does not make any specific limitation in this regard.

[0187] The hardware device invocation method of this application involves a first terminal device caching a device configuration file of a second terminal device. When the first terminal device needs to invoke a first hardware device of the second terminal device, if the device configuration file of the second terminal device includes the hardware capability information of the first hardware device, the first terminal device can obtain the hardware capability information of the first hardware device from the device configuration file of the second terminal device, and then invoke the first hardware device based on the hardware capability information. With this hardware device invocation method, when the first terminal device obtains the device capability information of the first hardware device of the second terminal device, if the device configuration file of the second terminal device cached by the first terminal device includes the device capability information of the first hardware device, the first terminal device does not need to interact with other devices and can directly obtain it from its cache. Compared to obtaining the hardware capability information of the first hardware device from the second terminal device, this method of obtaining hardware capability information does not require interaction with multiple software modules in the application layer of the second terminal device, thus reducing the time required for the first terminal device to obtain the device capability information of the first hardware device of the second terminal device and improving the efficiency of the hardware device invocation method.

[0188] As an optional embodiment, method 500 further includes: S505, if the device configuration file of the second terminal device does not include the hardware device capability information of the first hardware device, the first terminal device sends a second request to the second terminal device to request the hardware device capability information of the first hardware device; correspondingly, the second terminal device receives the second request from the first terminal device. S506, in response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device; correspondingly, the first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

[0189] It should be understood that when the device configuration file of the second terminal device stored in the first terminal device does not include the hardware device capability information of the first hardware device, the first terminal device needs to obtain the hardware device capability information of the first hardware device from the second terminal device.

[0190] In one possible implementation, S505 is specifically implemented in the following manner: the device virtualization module of the first terminal device establishes a signaling channel; the device virtualization module sends a second request to the second communication service manager of the second terminal device through the first communication service manager of the first terminal device using the signaling channel. S506 is specifically implemented in the following manner: the first communication service manager receives hardware device capability information from the second communication service manager; the first communication service manager transmits the hardware device capability information to the device virtualization module.

[0191] It should be understood that the first communication service manager and the second communication service manager can realize data transmission between the first terminal device and the second terminal device. Therefore, the device virtualization module of the first terminal device can send a second request to the second communication service manager through the first communication service manager.

[0192] As an optional embodiment, method 500 further includes: when the third terminal device obtains the hardware capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware capability information of the first hardware device to the cloud server; the cloud server receives the hardware capability information of the first hardware device and stores the hardware capability information of the first hardware device in the device configuration information of the second terminal device.

[0193] As an optional embodiment, after S506, method 500 further includes: the second terminal device sending the hardware device capability information of the first hardware device to the cloud server; correspondingly, the cloud server receives the hardware device capability information of the first hardware device from the second terminal device and stores the hardware device capability information of the first hardware device in the device configuration file of the second terminal device.

[0194] It should be understood that the third terminal device can be the first terminal device or other terminal devices. When the third terminal device is the first terminal device, the step of the second terminal device sending the hardware capability information of the first hardware device to the cloud server can be executed after S505 or S506. After the second terminal device receives a request from other terminal devices or the first terminal device to obtain the hardware capability information of the second terminal device, the second terminal device sends the hardware capability information of the hardware device to the cloud server. In this way, at subsequent times, when other terminal devices need to obtain the hardware capability information of the hardware device, they can obtain it from the device configuration file of the second terminal device cached by the other terminal devices, which can improve the efficiency of other terminal devices in obtaining the hardware capability information of the hardware device.

[0195] In one possible implementation, the second terminal device sends the hardware capability information of the first hardware device to the cloud server, including: after the device virtualization module of the second terminal device detects a second request from the first terminal device, sending a first instruction to the device management service module of the second terminal device to instruct the hardware capability information of the first hardware device to be sent to the cloud server; in response to the first instruction, the device management service module of the second terminal device sends the hardware capability information of the first hardware device to the cloud server.

[0196] As an optional embodiment, method 500 further includes: S507, the second terminal device sends the device configuration information of the second terminal device to the cloud server; the cloud server receives the device configuration information from the second terminal device.

[0197] It should be understood that when a second terminal device first accesses the local area network, or establishes a device connection with a third terminal device for the first time, the second terminal device can send its device configuration file to the cloud server. The third terminal device can be any terminal device other than the second terminal device. This allows the cloud server to store the device configuration file of the second terminal device. When a fourth terminal device first establishes a device connection with the second terminal device, the fourth terminal device can obtain the device configuration file of the second terminal device from the cloud server. This helps improve the efficiency of other terminal devices obtaining the device configuration file of the second terminal device. The fourth terminal device can be any terminal device other than the third terminal device, or it can be the third terminal device itself. When the fourth terminal device is the third terminal device, the first establishment of a device connection between the fourth terminal device and the second terminal device means that after the fourth terminal device first establishes a device connection with the third terminal device, it switches from a powered-on state to a powered-off state; then switches back from a powered-off state to a powered-on state, and then establishes a device connection with the second terminal device for the first time.

[0198] In one possible implementation, S507 is implemented in the following way: the second terminal device sends the device configuration information of the second terminal device to the cloud server through the device management service.

[0199] As an optional embodiment, method 500 further includes: S508, the second terminal device sends incremental information between the new device configuration file of the second terminal device and the device configuration file to the cloud server; S509, the cloud server receives the incremental information from the second terminal device and determines the new device configuration file of the second terminal device by combining the incremental information and the device configuration file of the second terminal device; S510, the cloud server sends updated configuration information to the first terminal device, the updated configuration information including the new device configuration file of the second terminal device or the incremental information; S511, the first terminal device updates the device configuration file of the second terminal device based on the updated configuration information to obtain the new device configuration information of the second terminal device.

[0200] It should be understood that incremental information can refer to data information newly added, data information to be deleted, or data information that has been changed in the new device configuration file compared to the original device configuration file. Optionally, the incremental information may include information indicating how the data information in the new device configuration file has changed compared to the original device configuration file. For example, if the incremental information includes data information that needs to be added in the new device configuration file compared to the original device configuration file, then the incremental information may include 1, where 1 indicates the addition of data information; if the incremental information includes data information that needs to be deleted in the new device configuration file compared to the original device configuration file, then the incremental information may include 2, where 2 indicates the addition of data information; if the incremental information includes data information that has changed in the new device configuration file compared to the original device configuration file, then the incremental information may include 3, where 3 indicates the changed data information.

[0201] If the device configuration file of the second terminal device stored in the cloud server changes, the cloud server can instruct the first terminal device to update the device configuration file of the second terminal device cached by the first terminal device. In this way, when the first terminal device needs to obtain the device configuration information in the device configuration file of the second terminal device, it can obtain the updated device configuration information from the new device configuration file, which can improve the efficiency and accuracy of the first terminal device in obtaining the device configuration information of the second terminal device.

[0202] As an optional embodiment, prior to S510, method 500 further includes: S512, the cloud server sends an update message to the first terminal device, the update message including the device identifier of the second terminal device; correspondingly, the first terminal device receives the update message from the cloud server. S513, in response to the update message, the first terminal device sends a fourth request to the cloud server to request a new device configuration file for the second terminal device; correspondingly, the cloud server receives the fourth request from the first terminal device. S510 is specifically implemented as follows: in response to the fourth request, the cloud server sends updated configuration information to the first terminal device.

[0203] It should be understood that the fourth request can be the same as or different from the first request. That is, the fourth request can be used to request a new device configuration file for the second terminal device stored on the cloud server, or it can be used to request incremental information between the new device configuration file and the existing device configuration file. When the device configuration information of the second terminal device stored on the cloud server changes, the cloud server sends an update message to the first terminal device. When the first terminal device needs to update the device configuration information of the second terminal device, the first terminal device sends a fourth request to the cloud server. In this way, the first terminal device can control when to update the device configuration file of the second terminal device, which helps to improve the success rate of updating the device configuration file of the second terminal device.

[0204] In one possible implementation, S511 is specifically implemented in the following way: if the updated configuration information includes a new device configuration file of the second terminal device, the device configuration file of the second terminal device is replaced with the new device configuration file of the second terminal device; or, if the updated configuration information includes incremental information, the new device configuration file of the second terminal device is determined by combining the incremental information and the device configuration file of the second terminal device.

[0205] It should be understood that, in combination with the incremental information and the device configuration file of the second terminal device, determining the new device configuration file of the second terminal device can be as follows: if the incremental information is data information that needs to be added, the first terminal device adds the incremental information to the device configuration file of the second terminal device to obtain the new device configuration file; if the incremental information is data information that needs to be deleted, the first terminal device deletes the data information corresponding to the incremental information in the device configuration file of the second terminal device to obtain the new device configuration file; if the incremental information is newly added data information that needs to be changed, the first terminal device changes the data information that needs to be changed in the device configuration file of the second terminal device to obtain the new device configuration file.

[0206] As an optional embodiment, if the first terminal device is not powered off and disconnects from the second terminal device, and then re-establishes a connection, method 500 further includes: S514, if the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the new device configuration file of the second terminal device includes the hardware device capability information of the second hardware device; S515, if so, the first terminal device uses the hardware device capability of the second hardware device to call the second hardware device of the second terminal device.

[0207] It should be understood that after the first terminal device caches the device configuration information of the second terminal device, the device configuration information of the second terminal device will be cached in the first terminal device while it is not powered off. If the device configuration file of the second terminal device changes while the first terminal device is not powered off, the first terminal device will synchronously update the device configuration file of the second terminal device to obtain the new device configuration file. After the first terminal device caches the new device configuration file of the second terminal device, if the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device can determine whether the new device configuration file includes the hardware device capability information of the second hardware device, and the first terminal device can obtain the hardware device capability information of the second hardware device from the new device configuration file.

[0208] Compared to device configuration files, when the hardware capability information of the second hardware device is added to the new device configuration file, this hardware device invocation method allows the first terminal device to obtain the hardware capability information of the second hardware device from the new device configuration file of the second terminal device cached in the first terminal device. This helps to improve the efficiency of the first terminal device in obtaining the hardware capability information of the second hardware device of the second terminal device, thereby improving the efficiency of the first terminal device invoking the second hardware device of the second terminal device.

[0209] As an optional embodiment, S503 is specifically implemented in the following manner: the device virtualization module of the first terminal device sends a third request to the device management service module of the first terminal device for requesting hardware device information of the first hardware device. The third request includes the device identifier of the second terminal device and the identifier of the first hardware device; the device management service module obtains the device configuration file of the second terminal device corresponding to the device identifier of the second terminal device; the device management service module determines whether the device configuration file of the second terminal device includes information corresponding to the identifier of the first hardware device.

[0210] It should be understood that the device management service module of the first terminal device can be used to cache the device configuration file of the second terminal device. The device virtualization module of the first terminal device can determine whether the first terminal device needs to obtain the hardware capability information of the second terminal device. If the device virtualization module of the first terminal device determines whether the first terminal device needs to obtain the hardware capability information of the second terminal device, the device virtualization module of the first terminal device can obtain the hardware capability information of the second terminal device from the device management service module of the first terminal device. In this way, the first terminal device can efficiently determine the hardware capability information of the second terminal device.

[0211] As an optional embodiment, the first request includes the device identifier of the second terminal device.

[0212] It should be understood that the device identifier of the second terminal device can refer to information such as the device model, device name, or device ID of the second terminal device. The cloud server can determine the device configuration file of the second terminal device that the first terminal device needs to obtain based on the device identifier in the first request.

[0213] As an optional embodiment, the first hardware device is at least one of a camera, a microphone, or a speaker.

[0214] As an optional embodiment, when the first hardware device is a camera, the hardware device capability information includes the resolution and / or frame rate of the camera.

[0215] It should be understood that when the first terminal device needs to access the camera of the second terminal device, it can determine the configuration supported by the camera based on the camera's resolution and / or frame rate, thereby allowing the user to select the corresponding camera configuration on the first terminal device.

[0216] As an optional embodiment, the first terminal device calls the hardware device of the second terminal device, which can be implemented in the following way: the device virtualization module of the first terminal device creates a virtual device driver corresponding to the hardware device of the second terminal device based on the hardware device capability information of the hardware device of the second terminal device through the virtual bus driver of the first terminal device; then the first terminal device obtains the data stream of the hardware device of the second terminal device through the virtual device driver.

[0217] The following is combined Figure 6 Taking a camera as an example, this paper provides a detailed explanation of the process by which the first terminal device calls the first hardware device of the second terminal device.

[0218] The software structure of the first terminal device can be as follows: Figure 3As shown, the software structure of the second terminal device can be as follows: Figure 4 As shown. For ease of description, the device virtualization module of the first terminal device will be referred to as the first device virtualization module, and the device virtualization module of the second terminal device will be referred to as the second device virtualization module; the communication service manager of the first terminal device will be referred to as the first communication service manager, and the communication service manager of the second terminal device will be referred to as the second communication service manager; the device management service module of the first terminal device will be referred to as the first device management service module, and the device management service module of the second terminal device will be referred to as the second device management service module.

[0219] Figure 6 This is a schematic diagram illustrating the process by which a first terminal device calls a first hardware device of a second terminal device, as provided in an embodiment of this application. Figure 6 As shown, in the first process: after the second terminal device connects to the local area network, the second device management service module can send the device configuration file of the second terminal device to the cloud server. Assume that the device configuration file of the second terminal device does not include camera capability information.

[0220] The second process: After the first terminal device connects to the local area network, or after it switches from the power-off state to the power-on state, when the first device management service module discovers the second terminal device that is online for the first time, the first device management service module obtains the device configuration file of the second terminal device from the cloud server.

[0221] The third process: Hardware device invocation service refers to any first terminal device needing to access the camera service of a second terminal device. For example, a three-way conferencing application running on the first terminal device might need to access the camera of the second terminal device. When the first device management service module detects that the second terminal device is online, it can send a notification message to the hardware device invocation service, informing it that the second terminal device is online. The hardware device invocation service then issues an enable camera command to the first device virtualization module. The first device virtualization module then requests the camera capability information of the second terminal device from the first device management service module, such as the camera's frame rate and resolution. If the first device management service module determines that the second terminal device's device configuration file does not contain camera capability information, it notifies the first device virtualization module that the camera capability information retrieval failed.

[0222] Fourth process: The first device virtualization module sends a request for camera capability information to the second device virtualization module through the first communication service manager and the second communication service manager in sequence; in response to the request, the second device virtualization module collects camera capability information from the camera service; the second device virtualization module sends the camera capability information to the first device virtualization module through the second communication service manager and the first communication service manager in sequence.

[0223] Fifth process: After the second device virtualization module receives a request for camera capability information from another terminal device for the first time, it instructs the second device management service module to send the camera capability information to the cloud server. The cloud server then stores the camera capability information in the device configuration file of the second terminal device.

[0224] The sixth step: The first device virtualization module transmits the camera capability information to the virtual bus driver. Using this information, the virtual bus driver creates a virtual camera driver. In this way, the hardware device can access the second terminal device's camera through the virtual camera driver to obtain the video stream.

[0225] It should be understood that the fifth and sixth processes are not sequential and can be performed simultaneously; this application does not impose any specific restrictions on this.

[0226] The following is combined Figure 7 The process of the first terminal device accessing the camera of the second terminal device is explained in detail.

[0227] Figure 7 This is a flowchart illustrating a method 700 provided in an embodiment of this application for a first terminal device to access the camera of a second terminal device. Figure 7 As shown, method 700 includes the following steps:

[0228] S10, PC Manager instructs the first device virtualization module to enable camera service.

[0229] S11. The first device virtualization module sends a first message to the first device management service module, requesting capability information data of the second terminal device from the first device management service module. Correspondingly, the first device management service module receives the first message from the first device virtualization module.

[0230] S12. If the capability information data of the second terminal device is not included in the first device management service module, the first device management service module sends a second message to the first device virtualization module. The second message indicates that the first device virtualization module failed to acquire the capability information data of the second terminal device. Correspondingly, the first device virtualization module receives the second message from the first device management service module.

[0231] S13. The first device virtualization module starts up the second device virtualization module and creates a signaling channel.

[0232] S14. The second device virtualization module sends a third message to the camera frame. The third message is used to obtain camera capability information data of the second terminal device from the camera frame. Correspondingly, the camera frame receives the third message from the second device virtualization module.

[0233] S15, the camera frame sends a fourth message to the second device virtualization module. The fourth message includes camera capability information data of the second terminal device. Correspondingly, the second device virtualization module receives the fourth message from the camera frame.

[0234] S16. The second device virtualization module sequentially sends camera capability information data of the second terminal device to the first device virtualization module through the second communication service manager and the first communication service manager. Correspondingly, the first device virtualization module receives the camera capability information data of the second terminal device from the second device virtualization module.

[0235] S17. The first device virtualization module deletes the signaling channel and the second device virtualization module.

[0236] S18. The first communication service manager sends a fifth message to the first device virtualization module. The fifth message indicates that the first device virtualization module has successfully acquired the camera capability information data of the second terminal device. Correspondingly, the first device virtualization module receives the fifth message from the first communication service manager.

[0237] S19. The first device virtualization module instructs the virtual bus driver to create a virtual device driver based on the camera capability information data of the second terminal device. For example, the camera capability information data of the second terminal device includes that the camera supports 1080P, 720P and 480P resolution, and the camera supports 30FPS and 15FPS frame rate.

[0238] S20. Start the meeting using a third-party application. This third-party application can be a meeting application.

[0239] S21. The third-party application sends a command to the virtual device driver instructing the camera to be turned on. Optionally, the command may include capability information data of the camera to be turned on, such as 480P and 30FPS.

[0240] S22. The first device virtualization module reads the sixth message in the cached message queue of the virtual device driver through the virtual driver bus. The sixth message is used to instruct the first device virtualization module to open the camera and includes camera capability information data, such as 480P and 30FPS.

[0241] S23. The first device virtualization module starts up the second device virtualization module and creates a data channel.

[0242] S24. The first device virtualization module sequentially sends a camera-opening command to the second device virtualization module through the first communication service manager and the second communication service manager, instructing the camera to be turned on. The camera-opening command may include camera capability information data, such as 480P and 30FPS.

[0243] S25. The second device virtualization module sends a seventh message to the camera frame. The seventh message is used to instruct the camera frame to turn on the camera and to indicate the camera's capability information data, such as 480P and 30FPS.

[0244] S26. The camera frame sends the captured image data to the second device virtualization module. Correspondingly, the second device virtualization module receives the image data from the camera frame.

[0245] S27. The second device virtualization module processes the image data, such as encoding and decoding.

[0246] S28. The second device virtualization module sequentially sends the processed image data to the first device virtualization module through the second communication service manager and the first communication service manager. Correspondingly, the first device virtualization module receives the processed image data from the second device virtualization module.

[0247] S29. The second device virtualization module decodes the processed image data and transmits the decoded image data to the virtual bus driver. Correspondingly, the bus driver reads the decoded image data.

[0248] The S30 driver, which drives the bus, feeds back the decoded image data to the third-party application.

[0249] It should be understood that the specific implementation methods of S10 to S30 are similar to the steps in method 500, as can be seen above, and will not be repeated here.

[0250] It should be understood that the order of the methods listed above does not imply the order of execution. The execution order of each method should be determined by its function and internal logic.

[0251] The following is combined Figures 8 to 11 Taking a laptop computer as the first terminal device, a mobile phone with model number H90 as the second terminal device, and the front and rear cameras of the mobile phone as the first hardware devices, this paper illustrates the interface during the hardware device calling process.

[0252] Figure 8 This is a schematic diagram of a laptop computer display device manager provided as an embodiment of this application. Figure 8 As shown, after installing the virtual bus driver on a laptop, the system devices in the laptop's Device Manager include the virtual bus driver.

[0253] Figure 9 This is a schematic diagram of an interface for a laptop computer to discover a mobile phone online, provided as an embodiment of this application. Figure 9 As shown, after the laptop's device management service module detects the mobile phone's online status, the PC Manager application on the laptop can display the device model H90 of the second terminal device. Optionally, the device management service module can be a software module within the PC Manager application.

[0254] Figure 10 This is a schematic diagram of another laptop computer display device manager interface provided in an embodiment of this application. Figure 10 As shown, the available hardware devices of the mobile phone include a front-facing camera and a rear-facing camera. The phone's model number is H90. After creating virtual device drivers for the front-facing camera and the rear-facing camera in the laptop's virtual bus driver, the laptop's Device Manager displays the virtual devices corresponding to these two drivers: H90 rear-facing camera 1001 and H90 front-facing camera 1002.

[0255] Figure 11 This is a schematic diagram of a camera selection interface in a conference application provided as an embodiment of this application. Figure 11 As shown, if the virtual camera drivers for the H90 rear camera and the H90 front camera are already installed on the laptop, the camera selection interface of the meeting application on the laptop will include the cameras corresponding to these two virtual camera drivers: H90 rear camera 1101 and H90 front camera 1102.

[0256] The above text combined Figures 2 to 11 The hardware device invocation method of the embodiments of this application is described in detail below, in conjunction with Figure 12 This application describes in detail the hardware device invocation apparatus of its embodiments.

[0257] Figure 12This is a schematic block diagram of a hardware device calling apparatus 1200 provided in an embodiment of this application. The apparatus 1200 includes a processor 1201, a communication interface 1202, and a memory 1203. The processor 1201, communication interface 1202, and memory 1203 communicate with each other via internal connection paths. The memory 1203 is used to store instructions, and the processor 1201 is used to execute the instructions stored in the memory 1203. The communication interface 1202 can be used to send signals to other devices (e.g., the processor 1201 or the touchscreen of a terminal device) and to receive signals from other devices (e.g., the memory 1203). Exemplarily, the communication interface 1202 reads instructions stored in the memory 1203 and sends the instructions to the processor 1201.

[0258] It should be understood that the apparatus 1200 may specifically be the first terminal device, the second terminal device, or the cloud server in the above embodiments, and may be used to execute the various steps and / or processes corresponding to the first terminal device, the second terminal device, or the cloud server in the above method embodiments. Optionally, the memory 1203 may include read-only memory and random access memory, and provide instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information. The processor 1201 may be used to execute instructions stored in the memory, and when the processor 1201 executes instructions stored in the memory, the processor 1201 is used to execute the various steps and / or processes in the above method embodiments.

[0259] It should be understood that, in the embodiments of this application, the processor may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0260] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or as a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor executes the instructions in the memory, combining them with its hardware to complete the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0261] This application also provides a computer-readable storage medium for storing a computer program for implementing the methods shown in the above-described method embodiments.

[0262] This application also provides a computer program product, which includes a computer program (also referred to as code or instructions) that, when run on a computer, allows the computer to perform the methods shown in the above-described method embodiments.

[0263] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0264] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0265] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0266] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0267] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0268] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0269] The above description is merely a specific embodiment of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A hardware device invocation method, comprising: The method is applied to a first terminal device, and comprises the following steps: When the first terminal device and a second terminal device establish a device interconnection for the first time, the first terminal device sends a first request for requesting a device profile of the second terminal device to a cloud server, a same account is logged in the first terminal device and the second terminal device, and a wireless fidelity (WiFi) switch and a Bluetooth switch of the first terminal device and the second terminal device are both in an open state; The first terminal device receives the device profile of the second terminal device from the cloud server; In a case where the first terminal device needs to call a first hardware device of the second terminal device, the first terminal device judges whether hardware device capability information of the first hardware device is included in the device profile of the second terminal device; If yes, the first terminal device calls the first hardware device of the second terminal device by using the hardware device capability information of the first hardware device.

2. The method of claim 1, wherein, The method further comprises the following steps: If the hardware device capability information of the first hardware device is not included in the device profile of the second terminal device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; The first terminal device receives the hardware device capability information of the first hardware device from the second terminal device.

3. The method of claim 2, wherein, The first terminal device sending the second request for requesting the hardware device capability information of the first hardware device to the second terminal device comprises the following steps: A device virtualization module of the first terminal device establishes a signaling channel; The device virtualization module sends the second request to a second communication service manager of the second terminal device by using the signaling channel through a first communication service manager of the first terminal device; The first terminal device receiving the hardware device capability information from the second terminal device comprises the following steps: The first communication service manager receives the hardware device capability information from the second communication service manager; The first communication service manager transmits the hardware device capability information to the device virtualization module.

4. The method according to any one of claims 1 to 3, characterized in that, In the case where the first terminal device needs to call the first hardware device of the second terminal device, the first terminal device judging whether the hardware device capability information of the first hardware device is included in the device profile of the second terminal device comprises the following steps: A device virtualization module of the first terminal device sends a third request for requesting hardware device information of the first hardware device to a device management service module of the first terminal device, the third request comprising a device identifier of the second terminal device and an identifier of the first hardware device; The device management service module acquires a device profile of the second terminal device corresponding to the device identifier of the second terminal device; The device management service module judges whether information corresponding to the identifier of the first hardware device is included in the device profile of the second terminal device.

5. The method according to any one of claims 1 to 4, characterized in that, The first request comprises the device identifier of the second terminal device.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: The first terminal device receives the update configuration information from the cloud server, and the update configuration information comprises a new device profile of the second terminal device or incremental information between the new device profile and the device profile; The first terminal device updates the device profile of the second terminal device based on the update configuration information.

7. The method of claim 6, wherein, Before the first terminal device receives the update configuration information from the cloud server, the method further comprises: The first terminal device receives an update message from the cloud server, and the update message comprises a device identifier of the second terminal device; In response to the update message, the first terminal device sends a fourth request for requesting the new device profile of the second terminal device to the cloud server.

8. The method according to claim 6 or 7, characterized in that, The update of the device profile of the second terminal device comprises: In a case where the update configuration information comprises the new device profile of the second terminal device, the device profile of the second terminal device is replaced by the new device profile of the second terminal device; or In a case where the update configuration information comprises the incremental information, the new device profile of the second terminal device is determined by combining the incremental information and the device profile of the second terminal device.

9. The method according to any one of claims 6 to 8, characterized in that, The first terminal device is not powered off, and the connection with the second terminal device is disconnected, and the connection is established again, and the method further comprises: In a case where the first terminal device needs to call the second hardware device of the second terminal device, the first terminal device determines whether the hardware device capability information of the second hardware device is included in the new device profile of the second terminal device; If yes, the first terminal device calls the second hardware device of the second terminal device by using the hardware device capability of the second hardware device.

10. The method according to any one of claims 1 to 9, characterized in that, The first hardware device is at least one of a camera, a microphone or a loudspeaker.

11. The method according to any one of claims 1 to 10, characterized in that, In a case where the first hardware device is the camera, the hardware device capability information comprises a resolution and / or a frame rate of the camera.

12. A hardware device invocation method, comprising: The method is applied to a system comprising a first terminal device, a second terminal device and a cloud server, a user logs in a same account on the first terminal device and the second terminal device, and a wireless fidelity (WiFi) switch and a Bluetooth switch of the first terminal device and the second terminal device are both in an on state, and the method comprises: When the first terminal device and the second terminal device initially establish device interconnection, the first terminal device sends a first request for requesting a device profile of the second terminal device to the cloud server; The cloud server receives the first request from the first terminal device; The cloud server sends the device profile of the second terminal device to the first terminal device based on the first request; The first terminal device receives the device profile of the second terminal device from the cloud server; In a case that the first terminal device needs to invoke the first hardware device of the second terminal device, the first terminal device judges whether the hardware device capability information of the first hardware device is included in the device profile of the second terminal device; If yes, the first terminal device invokes the first hardware device of the second terminal device by using the hardware device capability information of the first hardware device.

13. The method of claim 12, wherein, The method further comprises: In a case that the first terminal device judges that the hardware device capability information of the first hardware device is not included in the device profile of the second terminal device, the first terminal device sends a second request for requesting the hardware device capability information of the first hardware device to the second terminal device; The second terminal device receives the second request from the first terminal device; In response to the second request, the second terminal device sends the hardware device capability information of the first hardware device to the first terminal device.

14. The method according to claim 12 or 13, characterized in that, The method further comprises: In a case that the third terminal device acquires the hardware device capability information of the first hardware device from the second terminal device, the second terminal device uploads the hardware device capability information of the first hardware device to the cloud server; The cloud server receives the hardware device capability information of the first hardware device, and stores the hardware device capability information of the first hardware device into the device configuration information of the second terminal device.

15. The method according to any one of claims 12 to 14, characterized in that, The method further comprises: The second terminal device sends the incremental information between the new device profile of the second terminal device and the device profile to the cloud server; The cloud server receives the incremental information from the second terminal device, and determines the new device profile of the second terminal device by combining the incremental information and the device profile of the second terminal device; The cloud server sends the update configuration information to the first terminal device, the update configuration information comprising the new device profile of the second terminal device or the incremental information; The first terminal device updates the device profile of the second terminal device based on the update configuration information, to obtain the new device profile of the second terminal device.

16. The method of claim 15, wherein, Before the cloud server sends the update configuration information to the first terminal device, the method further comprises: The cloud server sends an update message to the first terminal device, the update message comprising the device identifier of the second terminal device; The first terminal device receives the update message from the cloud server; In response to the update message, the first terminal device sends a fourth request for requesting the new device profile of the second terminal device to the cloud server; The cloud server sends the update configuration information to the first terminal device, comprising: In response to the fourth request, the cloud server sends the update configuration information to the first terminal device.

17. A hardware device invocation apparatus, comprising: Comprise: a processor coupled with the memory for storing a computer program which, when invoked by the processor, causes the apparatus to perform the method of any of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, a computer program product comprising computer program code to implement the method of any of claims 1 to 16.

19. A computer program product, characterised in that, a computer program product comprising computer program code to implement the method of any of claims 1 to 16.

Images