Hardware cross-device calling method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of computers, and particularly provides a hardware cross-device calling method and device, electronic equipment and a storage medium. The hardware cross-device calling method comprises the following steps: in response to a trigger operation of a user for a virtual hardware service corresponding to target hardware, obtaining hardware metadata of the target hardware through a hardware collaborative service interface; sending a hardware calling instruction to the target hardware based on the hardware metadata through the hardware cross-device calling interface; sending a data stream acquisition request for the target hardware to the called device through the data stream transmission interface; and receiving data stream information returned by the called device based on the data stream acquisition request. Thus, the target hardware of the called device is abstracted into the virtual hardware service of the calling device, a user can directly call the target hardware in a cross-device manner through the virtual hardware service without considering the difference between the device where the target hardware is located and the system, and the complex operation of hardware cross-device calling is simplified.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, electronic device and storage medium for hardware cross-device calling. Background Art

[0002] In a multi-device networking environment, users usually want to use the hardware resources of other devices in the network on one device. For example, since the camera device on a mobile phone has a relatively high configuration, while the camera device on a tablet has a relatively low configuration, the tablet can call the camera device of the mobile phone to shoot through intercommunication to obtain the captured pictures.

[0003] Under the prior art, when hardware is called across devices, users usually need to perform corresponding remote connection and hardware cross-device calling operations according to the device and system types of the calling device and the called device. However, in this way, the operation of hardware cross-device calling is relatively complicated. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, electronic device and storage medium for hardware cross-device calling, so as to simplify the tedious operations of hardware cross-device calling.

[0005] On the one hand, an embodiment of the present application provides a method for hardware cross-device calling, which is applied to calling a device, including:

[0006] In response to a user triggering operation on a virtual hardware service corresponding to target hardware, hardware metadata of the target hardware is obtained through a hardware coordination service interface; the target hardware is hardware set on the called device;

[0007] Through the hardware cross-device calling interface, based on the hardware metadata, a hardware calling instruction is sent to the target hardware, so that the calling device controls the target hardware to execute the hardware calling instruction;

[0008] Sending a data stream acquisition request for the target hardware to the called device through the data stream transmission interface;

[0009] Receive data stream information returned by the called device based on the data stream acquisition request.

[0010] In one implementation, obtaining hardware metadata of target hardware through a hardware collaboration service interface includes:

[0011] Through the hardware coordination service interface, call the distributed hardware coordination service to perform the following steps:

[0012] Sending a hardware information acquisition request to the called device for the target hardware;

[0013] Receive hardware metadata returned by the called device based on the hardware information acquisition request.

[0014] In one implementation, before sending a hardware call instruction to the target hardware through the hardware cross-device call interface based on the hardware metadata, the method further includes:

[0015] Call the distributed hardware collaboration service through the hardware collaboration service interface to obtain the cross-device permissions of the target hardware;

[0016] If the cross-device permission indicates that the call is prohibited, the hardware cross-device call is stopped.

[0017] In one implementation, sending a hardware call instruction to a target hardware based on hardware metadata through a hardware cross-device call interface includes:

[0018] Generate hardware call instructions based on hardware metadata through the hardware cross-device call interface;

[0019] Serialize the hardware call instruction to obtain the serialized hardware call instruction;

[0020] Transmit the serialized hardware call instructions to the target hardware.

[0021] In one implementation, sending a data stream acquisition request for target hardware to a called device through a data stream transmission interface includes:

[0022] Sending a data stream connection request to the called device through the data stream transmission interface;

[0023] If a connection response message returned by the called device based on the data stream connection request is received, it is determined that the data stream connection between the calling device and the called device is successfully established;

[0024] Send a data stream acquisition request to the called device through the data stream connection.

[0025] On the one hand, an embodiment of the present application provides a method for hardware cross-device calling, which is applied to a called device and includes:

[0026] When it is determined that a hardware call instruction sent by the calling device to the target hardware is received, the target hardware is controlled to execute the call instruction; the target hardware is the hardware set on the called device, and the hardware call instruction is sent by the calling device in response to the user's triggering operation of the virtual hardware service corresponding to the target hardware through the hardware cross-device call interface based on the hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through the hardware collaborative service interface;

[0027] When determining that a data stream acquisition request for target hardware is received from a calling device, data stream information of the target hardware is acquired based on the data stream request; the data stream acquisition request is sent by the calling device through a data stream transmission interface;

[0028] Returns data flow information to the calling device.

[0029] In one embodiment, the method further comprises:

[0030] When determining that a hardware information acquisition request sent by a calling device for target hardware is received, the hardware metadata is acquired; the hardware information acquisition request is sent by the calling device through a hardware collaborative service interface;

[0031] Returns hardware metadata to the calling device.

[0032] In one embodiment, the method further comprises:

[0033] Determine that upon receiving a permission request sent by a calling device for target hardware, obtain cross-device permissions of the target hardware; the permission request is sent by the calling device through the hardware collaborative service interface;

[0034] Returns cross-device permissions to the calling device.

[0035] In one embodiment, when it is determined that a data stream acquisition request for target hardware is received from a calling device, before acquiring data stream information of the target hardware based on the data stream request, the method further includes:

[0036] When determining that a data stream connection request sent by the calling device through the data transmission interface is received, a connection response message is sent to the calling device to establish a data stream connection between the calling device and the called device;

[0037] When it is determined that a data stream acquisition request for target hardware is received from a calling device, data stream information of the target hardware is acquired based on the data stream request, including:

[0038] It is determined that when a data stream acquisition request sent by a calling device through a data stream connection is received, data stream information is acquired.

[0039] On the one hand, an embodiment of the present application provides a device for hardware cross-device calling, which is applied to calling a device, including:

[0040] An acquisition unit, configured to acquire hardware metadata of the target hardware through a hardware collaboration service interface in response to a user triggering operation of a virtual hardware service corresponding to the target hardware; the target hardware is hardware set in the called device;

[0041] A calling unit, configured to send a hardware calling instruction to a target hardware through a hardware cross-device calling interface based on hardware metadata, so that the calling device controls the target hardware to execute the hardware calling instruction;

[0042] A sending unit, used to send a data stream acquisition request for target hardware to the called device through a data stream transmission interface;

[0043] The receiving unit is used to receive the data stream information returned by the called device based on the data stream acquisition request.

[0044] In one implementation, the acquisition unit is used to:

[0045] Through the hardware coordination service interface, call the distributed hardware coordination service to perform the following steps:

[0046] Sending a hardware information acquisition request to the called device for the target hardware;

[0047] Receive hardware metadata returned by the called device based on the hardware information acquisition request.

[0048] In one implementation, the calling unit is further configured to:

[0049] Call the distributed hardware collaboration service through the hardware collaboration service interface to obtain the cross-device permissions of the target hardware;

[0050] If the cross-device permission indicates that the call is prohibited, the hardware cross-device call is stopped.

[0051] In one implementation, the calling unit is used to:

[0052] Generate hardware call instructions based on hardware metadata through the hardware cross-device call interface;

[0053] Serialize the hardware call instruction to obtain the serialized hardware call instruction;

[0054] Transmit the serialized hardware call instructions to the target hardware.

[0055] In one implementation, the sending unit is used to:

[0056] Sending a data stream connection request to the called device through the data stream transmission interface;

[0057] If a connection response message returned by the called device based on the data stream connection request is received, it is determined that the data stream connection between the calling device and the called device is successfully established;

[0058] Send a data stream acquisition request to the called device through the data stream connection.

[0059] On the one hand, an embodiment of the present application provides a device for hardware cross-device calling, which is applied to a called device and includes:

[0060] The execution unit is used to determine that when receiving a hardware call instruction sent by the calling device to the target hardware, the target hardware is controlled to execute the call instruction; the target hardware is the hardware set on the called device, and the hardware call instruction is sent by the calling device in response to the user's triggering operation of the virtual hardware service corresponding to the target hardware through the hardware cross-device call interface based on the hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through the hardware collaborative service interface;

[0061] An acquisition unit, configured to acquire data stream information of the target hardware based on the data stream request when receiving a data stream acquisition request from the calling device for the target hardware; the data stream acquisition request is sent by the calling device through the data stream transmission interface;

[0062] The return unit is used to return data flow information to the calling device.

[0063] In one implementation, the execution unit is further configured to:

[0064] When determining that a hardware information acquisition request sent by a calling device for target hardware is received, the hardware metadata is acquired; the hardware information acquisition request is sent by the calling device through a hardware collaborative service interface;

[0065] Returns hardware metadata to the calling device.

[0066] In one implementation, the execution unit is further configured to:

[0067] Determine that upon receiving a permission request sent by a calling device for target hardware, obtain cross-device permissions of the target hardware; the permission request is sent by the calling device through the hardware collaborative service interface;

[0068] Returns cross-device permissions to the calling device.

[0069] In one implementation, the acquisition unit is further configured to:

[0070] When determining that a data stream connection request sent by the calling device through the data transmission interface is received, a connection response message is sent to the calling device to establish a data stream connection between the calling device and the called device;

[0071] The acquisition unit is used to:

[0072] It is determined that when a data stream acquisition request sent by a calling device through a data stream connection is received, data stream information is acquired.

[0073] On the one hand, an embodiment of the present application provides an electronic device, including:

[0074] Processor; and

[0075] A memory stores computer instructions, wherein the computer instructions are used to cause a processor to execute steps of a method provided in any of the above-mentioned various optional implementations of hardware cross-device calling.

[0076] On the one hand, an embodiment of the present application provides a storage medium storing computer instructions, which are used to enable a computer to execute the steps of the method provided in any of the various optional implementations of the hardware cross-device call as described above.

[0077] The method for hardware cross-device calling in the embodiment of the present application includes responding to the user's triggering operation on the virtual hardware service corresponding to the target hardware, obtaining the hardware metadata of the target hardware through the hardware collaborative service interface; the target hardware is the hardware set on the called device; through the hardware cross-device calling interface, based on the hardware metadata, sending a hardware call instruction to the target hardware, so that the calling device controls the target hardware to execute the hardware call instruction; through the data stream transmission interface, sending a data stream acquisition request for the target hardware to the called device; receiving the data stream information returned by the called device based on the data stream acquisition request. In this way, the target hardware of the called device is abstracted as the virtual hardware service of the calling device, and the user does not need to consider the differences in the device and system where the target hardware is located, and can directly call the target hardware across devices through the virtual hardware service, which simplifies the complex operation of hardware cross-device calling. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0079] Figure 1 This is an example diagram of the architecture of a system for hardware cross-device calling in an embodiment of the present application.

[0080] Figure 2 It is a flowchart of a method for hardware cross-device calling in an embodiment of the present application.

[0081] Figure 3 This is an example diagram of hardware metadata acquisition in an embodiment of the present application.

[0082] Figure 4 This is an example diagram of hardware management in an embodiment of the present application.

[0083] Figure 5This is an example diagram of a hardware cross-device calling interface in an embodiment of the present application.

[0084] Figure 6 This is an example diagram of data stream transmission in an embodiment of the present application.

[0085] Figure 7 It is a structural block diagram of a device for hardware cross-device calling in an embodiment of the present application.

[0086] Figure 8 It is a structural block diagram of another device for hardware cross-device calling in an embodiment of the present application.

[0087] Fig. 9 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0088] The technical solution of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described implementation methods are part of the implementation methods of the present application, rather than all of the implementation methods. Based on the implementation methods in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application. In addition, the technical features involved in the different implementation methods of the present application described below can be combined with each other as long as they do not conflict with each other.

[0089] First, some terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0090] Terminal device: can be a mobile terminal, a fixed terminal or a portable terminal, such as a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system device, a personal navigation device, a personal digital assistant, an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio receiver, an electronic book device, a gaming device or any combination thereof, including accessories and peripherals of these devices or any combination thereof. It is also foreseeable that the terminal device can support any type of interface for the user (such as a wearable device), etc.

[0091] Server: It can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms.

[0092] Based on the defects of the above-mentioned related technologies, a method, apparatus, electronic device and storage medium for hardware cross-device calling are provided in the embodiments of the present application, aiming to simplify the tedious operations of hardware cross-device calling.

[0093] A method for hardware cross-device calling is provided in an embodiment of the present application. The method can be applied to electronic devices. The present application does not limit the type of electronic device. It can be any type of device suitable for implementation, such as a terminal device and a server, etc. The present application will not go into details about this.

[0094] See also Figure 1 As shown, it is an example diagram of the architecture of a system for hardware cross-device calling. The system includes a calling device and a called device. Both the calling device and the called device can be any type of electronic device, such as a terminal device or a server.

[0095] In one implementation, the calling device and the called device interact via a distributed hardware service framework. The calling device also includes virtual hardware services and client application programming interfaces (Application Programming Interface, API). The called device also includes hardware and server APIs.

[0096] Among them, the distributed hardware service framework includes distributed hardware collaboration services and data stream transmission services.

[0097] The virtual hardware service is a program configured for the hardware of the called device. As an example, a virtual camera service, a virtual audio service and a virtual peripheral service may be configured for a camera device, an audio device and a peripheral device, respectively.

[0098] Among them, the hardware in the embodiments of the present application is computer hardware. It is composed of many modular components with different functions, and can complete operations such as input, processing, storage and output with the cooperation of software. Computer hardware usually includes the following types: output devices (such as displays, printers and speakers, etc.), input devices (such as mice, keyboards and cameras, etc.), central processing units, storage (such as memory, hard disks, optical disks, USB flash drives and storage cards, etc.), and motherboards.

[0099] Among them, the server-side API is set corresponding to the client API. API is a calling interface left by the operating system to the application program. The application program enables the operating system to execute the command (action) of the application program by calling the API. In the embodiment of the present application, the virtual hardware service of the calling device enables the distributed hardware service framework to execute commands such as data stream transmission of the virtual hardware service by calling the client API. The API of the calling device may include: hardware collaborative service interface, hardware cross-device calling interface and data stream transmission interface.

[0100] In this way, the virtual hardware service of the calling device can call the distributed hardware service framework through the client API to call the target hardware of the called device, thereby realizing cross-device hardware calling.

[0101] See also Figure 2 FIG. 1 is a flowchart of a method for hardware cross-device calling in an embodiment of the present application, which is applied to Figure 1 The hardware in the system calls the device across devices. Figure 1 and Figure 2 The method is described below. The specific implementation process of the method is as follows:

[0102] Step 200: In response to a user triggering operation on a virtual hardware service corresponding to target hardware, hardware metadata of the target hardware is obtained through a hardware coordination service interface.

[0103] The hardware metadata is basic attribute information of the hardware, such as the hardware identification, the functional parameters of the hardware (such as the resolution of the camera device), etc.

[0104] In one implementation, when executing step 200, the distributed hardware coordination service may be called through the hardware coordination service interface to perform the following steps:

[0105] S2001: In response to a user triggering operation on a virtual hardware service corresponding to a target hardware, a calling device sends a hardware information acquisition request to a called device for the target hardware.

[0106] S2002: When the called device determines that it has received a hardware information acquisition request sent by the calling device for the target hardware, it acquires hardware metadata.

[0107] The hardware information acquisition request is sent by the calling device through the hardware collaborative service interface.

[0108] S2003: The called device returns the hardware metadata to the calling device.

[0109] S2004: The calling device receives the hardware metadata returned by the called device based on the hardware information acquisition request.

[0110] In one implementation, when executing step 200, the following steps may also be included:

[0111] The calling device calls the distributed hardware collaborative service through the hardware collaborative service interface to obtain the cross-device permission of the target hardware; if the cross-device permission indicates that the call is prohibited, the hardware cross-device call is stopped.

[0112] The cross-device permission is used to indicate whether the calling device has the permission to use the target hardware in the called device across devices. Optionally, the cross-device permission can be obtained in real time or stored locally.

[0113] As an example, the calling device sends a permission request for the target hardware to the called device. The called device displays a permission confirmation interface to the user based on the permission request, and returns the cross-device permissions set by the user to the calling device based on the user's permission control operation.

[0114] See also Figure 3 The figure shows an example diagram of hardware metadata acquisition. The first distributed hardware collaboration service performs periodic detection, and when it determines that there is hardware to be synchronized, it obtains the hardware metadata of the hardware to be synchronized through the second distributed hardware collaboration service, and sends the hardware metadata to the virtual hardware service. Similarly, the first distributed hardware collaboration service can also obtain the cross-device permissions of the calling device for the hardware to be synchronized through the second distributed hardware collaboration service.

[0115] The virtual hardware service is a program for hardware settings of the called device. As an example, a virtual camera service, a virtual audio service, and a virtual peripheral service can be set for a camera device, an audio device, and a peripheral respectively. The virtual hardware service and the first distributed hardware collaboration service are located in the calling device, and the second distributed hardware collaboration service is located in the called device.

[0116] See also Figure 4 The figure shows an example diagram of hardware management. Figure 4 It includes virtual hardware services, hardware coordination service interfaces and distributed hardware coordination services. Virtual hardware services include virtual camera services, virtual audio services and virtual peripheral services.

[0117] Among them, the distributed hardware collaboration service can also include the following multiple modules: local hardware management module, remote hardware management module, persistent storage module, hardware online and offline management module, permission and security module and logical-physical mapping module.

[0118] Among them, the local hardware management module is used to store and manage the hardware metadata of the calling device; the remote hardware management module is used to store and manage the hardware metadata of the associated devices; the persistent storage module is used to store hardware-related information; the hardware online and offline management module is used to store and manage the online and offline status of the associated devices; the permission and security module is used to store the cross-device permissions of the hardware of the associated devices.

[0119] In an embodiment of the present application, the distributed hardware collaboration service manages the distributed hardware of each device using a unified global hardware resource pool abstract model, and can dynamically obtain hardware metadata of different devices, achieve cross-device hardware metadata synchronization, and achieve cross-device permission management of hardware of different devices. It also achieves online and offline monitoring of associated devices, providing a basis for subsequent cross-device calls of hardware.

[0120] Furthermore, the calling device establishes a collaborative relationship with the target hardware based on the hardware metadata of the target hardware, wherein the collaborative relationship includes a corresponding relationship between the calling device and the target hardware, and also includes the hardware metadata of the target hardware.

[0121] The following code example illustrates the collaborative relationship.

[0122] Future <List <associationinfo>>getAssociations(); / / Get collaborative relationships

[0123] Future <List <hardwareinfo>>getRemoteHardwares(); / / Get the hardware list

[0124] void associate(HardwareInfo hardwareInfo, AssociationListenerlistener, int m); / / Establish collaborative relationship

[0125] void disassociate(HardwareInfo hardwareInfo, AssociationListenerlistener); / / Disassociate the collaborative relationship

[0126] Future <boolean>checkPermission(AssociationInfo info); / / Permission check

[0127] In one implementation, the calling device obtains the established collaborative relationships and hardware metadata of each hardware of each associated device in the network. If there is hardware with no collaborative relationship established, a collaborative relationship with the hardware is established based on the hardware metadata of the hardware.

[0128] Among them, the associated device and the calling device are different devices located in the same network. The associated device and the calling device can be connected to the network via Bluetooth or wireless (wifi), and the associated device and the calling device can log in to the same account.

[0129] In an embodiment of the present application, the calling device can obtain the established collaborative relationships and the hardware of the associated devices in the network. It can also establish new collaborative information or terminate the established collaborative relationships, and obtain cross-device permissions.

[0130] Step 201: Send a hardware call instruction to the target hardware through a hardware cross-device call interface based on hardware metadata, so that the calling device controls the target hardware to execute the hardware call instruction.

[0131] In one implementation, when the called device determines that it has received a hardware calling instruction sent by the calling device to the target hardware, it controls the target hardware to execute the calling instruction.

[0132] Among them, the target hardware is the hardware set on the called device, and the hardware call instruction is sent by the calling device in response to the user's trigger operation of the virtual hardware service corresponding to the target hardware through the hardware cross-device call interface based on the hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through the hardware collaborative service interface.

[0133] See also Figure 5 As shown, it is an example diagram of a hardware cross-device calling interface, which includes a management module (RpcManager), a serialization module (Rpclnvoker), a deserialization module (RpcExcutor) and a transmission module (RpcTransport).

[0134] In one embodiment, when executing step 201, Figure 5 Use the following steps:

[0135] S2011: The calling device generates a hardware calling instruction based on the hardware metadata through the hardware cross-device calling interface.

[0136] In one implementation, RpcManager is called through a hardware cross-device call interface, and a hardware call instruction is generated based on hardware metadata.

[0137] S2012: The calling device serializes the hardware calling instruction to obtain the serialized hardware calling instruction.

[0138] In one implementation, Rpclnvoker is used to serialize the hardware call instructions into JSON.

[0139] S2013: The calling device transmits the serialized hardware calling instruction to the target hardware.

[0140] In one implementation, RpcTransport is used to send serialized hardware call instructions to the target hardware through a remote call connection.

[0141] The remote call connection can be a network connection based on the Remote Procedure Call (RPC) protocol. It can be configured according to user needs, can support the lightweight data exchange format (JavaScript Object Notation, JSON) RPC2.0 protocol, can also support synchronous and asynchronous calls, and can support multiple systems, such as Android system, Windows system and Vela system.

[0142] In one implementation, a remote call connection with the target hardware is established based on the cooperative relationship of the target hardware.

[0143] S2014: When the called device determines that the hardware calling instruction is received, it controls the target hardware to execute the calling instruction.

[0144] Furthermore, the call response information returned by the target hardware based on the hardware call instruction can be received through RpcTransport, and the call response information can be deserialized through RpcExcutor, and the deserialized call response information can be returned to the virtual hardware service through RpcManager.

[0145] As an example, the hardware call instruction may be:

[0146] {"jsonrpc":"2.0","method":"subtract","params":[42,23],"id":1};

[0147] Among them, "jsonrpc":"2.0" indicates that the RPC protocol version is version 2.0, "method":"subtract" indicates that the calling method is subtraction, "params":[42,23] indicates that the parameters are 42 and 23 respectively, and "id":1 indicates that the serial number of the hardware call instruction is 1.

[0148] As an example, the call response information can be: {"jsonrpc":"2.0","result":19,"id":1};

[0149] Among them, "jsonrpc":"2.0" means that the RPC protocol version is 2.0, "result":19 means the returned result is 19, and "id":1 means the serial number of the hardware call instruction is 1.

[0150] The following is an example of using the RPC protocol with a piece of RPC code:

[0151] public <t>void bindService(String deviceId,Class <t>serviceInterface,

[0152] ServiceConnection <t>serviceConnection) / / bind service;

[0153] public <t>void unbindService(String deviceId,

[0154] RpcCore.ServiceComponent <t>component) / / Unbind service;

[0155] public <t>void startService(T serviceInstance) / / Start the service;

[0156] public <t>void stopService(T serviceInstance) / / Stop the service.

[0157] In one implementation, the calling device and the called device can be bound by a binding service, and after binding, a hardware calling instruction for starting is sent to the target hardware by starting the service, and a calling stop instruction for stopping is sent to the target hardware by stopping the service. Furthermore, the calling device and the called device can be unbound by an unbinding service.

[0158] In the embodiment of the present application, through the hardware cross-device calling interface, the target hardware of the called device can be simply remotely called like interprocess communication (IPC), which simplifies the tedious operations of hardware remote calling and improves the efficiency and reliability of hardware remote calling.

[0159] Step 202: Send a data stream acquisition request for the target hardware to the called device through the data stream transmission interface.

[0160] In one implementation, when executing step 202, the following steps may be used:

[0161] S2021: The calling device sends a data stream connection request to the called device through the data stream transmission interface.

[0162] S2022: When the called device determines that it has received the data stream connection request sent by the calling device through the data transmission interface, it sends a connection response message to the calling device to establish a data stream connection between the calling device and the called device.

[0163] S2023: The calling device sends a data stream acquisition request to the called device through the data stream connection.

[0164] Step 203: Receive data stream information returned by the called device based on the data stream acquisition request.

[0165] In one implementation, when executing step 203, the following steps may be used:

[0166] S2031: When the called device determines that it has received the data stream acquisition request sent by the calling device through the data stream connection, it acquires the data stream information.

[0167] S2032: The called device returns data stream information to the calling device through the data stream connection.

[0168] S2033: The calling device receives the data stream information sent by the called device.

[0169] See also Figure 6 The figure shows an example diagram of data stream transmission. Figure 6 It includes the calling device and the called device. The calling device includes the client data stream transmission interface generated based on the distributed input output (IO) client software development kit (SDK) and the distributed IO runtime (Runtime). The called device includes the server data stream transmission interface and the distributed IO runtime generated based on the distributed IO server (Server) SDK. The distributed IO runtime is used to transmit data streams between the calling device and the called device through data stream connections.

[0170] The data stream transmission interface and the distributed IO Runtime can share memory and transmit data through IPC.

[0171] The following is a code example of a distributed IO Client calling a data stream transmission interface to illustrate calling the data stream transmission interface of the device.

[0172] public int init(String localIp,String clientName) / / Initialize the client information of the calling device;

[0173] public int start(String remoteIp,int remotePort) / / Initialize the server information of the called device;

[0174] public int stop();

[0175] public int attachSurface(Surface surface) / / Virtual hardware service mounting;

[0176] public int setMediaInfo(MiPlayCastMediaInfo info) / / Set the data stream format;

[0177] public int registerCallback(MiplayCastClientListener l) / / callback function registration;

[0178] public int unInit() / / Uninstall;

[0179] In this way, based on the client information of the calling device and the server information of the called device, a data stream connection can be established with the called device for the virtual hardware service corresponding to the target hardware.

[0180] The following is a code example of a distributed IO Server calling a data stream transmission interface to illustrate the data stream transmission interface of the called device.

[0181] public int init(String localIp,String serverName) / / Initialize the server information of the called device;

[0182] public int start();

[0183] public int stop();

[0184] public int write(MediaData mediaData) / / Write data stream;

[0185] public int setMediaInfo(MiPlayCastMediaInfo info) / / Set the data stream format;

[0186] public int unInit() / / Uninstall;

[0187] In one implementation, communication monitoring can be used to determine that when a data stream connection request sent by a calling device is received, a connection response message can be sent to the calling device, thereby establishing a data stream connection between the calling device and the called device.

[0188] As an example, the calling device is device A, the called device is device B, and the target hardware is a camera device. The virtual camera service of device A obtains the hardware metadata of the camera device through the hardware collaborative service interface, and sends a hardware call instruction for shooting to the camera device based on the hardware metadata through the hardware cross-device calling interface to control the camera device to perform the shooting operation, and receives the video stream sent by the camera device through the data stream transmission interface.

[0189] As another example, the calling device is device A, the called device is device B, and the target hardware is a telephone device. The virtual telephone service of device A obtains the hardware metadata of the telephone device through the hardware collaborative service interface, and sends a hardware call instruction for the call to the telephone device based on the hardware metadata through the hardware cross-device call interface to control the telephone device to perform the call operation, and receives the audio stream sent by the telephone device through the data stream transmission interface.

[0190] Furthermore, the calling device may also determine that when receiving the service stop request, it sends a call stop instruction to the target hardware through the hardware cross-device call interface.

[0191] In an embodiment of the present application, in response to a user triggering operation of a virtual hardware service corresponding to the target hardware of the called device in the calling device, the calling device obtains the hardware metadata of the target hardware of the called device through the hardware collaborative service interface, and sends a control command (i.e., a hardware call instruction) to the target hardware based on the hardware metadata through the hardware cross-device call interface to control the target hardware to perform the corresponding operation, and obtains the data stream (i.e., data stream information) of the target hardware through the data stream transmission interface, that is, based on the idea of ​​Device as a Service (DAAS), the device hardware is abstracted, serviced, and pooled, and the user does not need to consider the differences between different devices and systems, and can directly call the target hardware of any called device through the virtual hardware service, thereby simplifying the cumbersome operation of hardware cross-device calls. Hardware cross-device calls can be made between different devices and systems at the system hardware abstraction layer (HAL) or service layer, directly reaching the hardware capabilities themselves, and cross-device hardware service calls can be realized in Android, Windows, Linux and other systems, expanding the scope of application of hardware cross-device calls, improving the stability and performance of hardware cross-device calls, and providing a high user experience.

[0192] Based on the same inventive concept, an apparatus for hardware cross-device calling is also provided in the embodiment of the present application. Since the principle of solving the problem by the above apparatus and the device is similar to that of a method for hardware cross-device calling, the implementation of the above apparatus can refer to the implementation of the method, and the repeated parts will not be repeated. The apparatus can be applied to electronic devices. The present application does not limit the type of electronic devices. It can be any type of device suitable for implementation, such as terminal devices and servers, etc., and the present application will not repeat them.

[0193] See also Figure 7 , which is a structural block diagram of a device for hardware cross-device calling in an embodiment of the present application. In some implementations, the device for hardware cross-device calling in the example of the present application includes:

[0194] The acquisition unit 701 is used to obtain hardware metadata of the target hardware through the hardware coordination service interface in response to a user triggering operation of the virtual hardware service corresponding to the target hardware; the target hardware is the hardware set in the called device;

[0195] The calling unit 702 is used to send a hardware calling instruction to the target hardware based on the hardware metadata through the hardware cross-device calling interface, so that the calling device controls the target hardware to execute the hardware calling instruction;

[0196] The sending unit 703 is used to send a data stream acquisition request for the target hardware to the called device through the data stream transmission interface;

[0197] The receiving unit 704 is configured to receive data stream information returned by the called device based on the data stream acquisition request.

[0198] In one implementation, the acquisition unit 701 is used to:

[0199] Through the hardware coordination service interface, call the distributed hardware coordination service to perform the following steps:

[0200] Sending a hardware information acquisition request to the called device for the target hardware;

[0201] Receive hardware metadata returned by the called device based on the hardware information acquisition request.

[0202] In one implementation, the calling unit 702 is further configured to:

[0203] Call the distributed hardware collaboration service through the hardware collaboration service interface to obtain the cross-device permissions of the target hardware;

[0204] If the cross-device permission indicates that the call is prohibited, the hardware cross-device call is stopped.

[0205] In one implementation, the calling unit 702 is used to:

[0206] Generate hardware call instructions based on hardware metadata through the hardware cross-device call interface;

[0207] Serialize the hardware call instruction to obtain the serialized hardware call instruction;

[0208] Transmit the serialized hardware call instructions to the target hardware.

[0209] In one implementation, the sending unit 703 is configured to:

[0210] Sending a data stream connection request to the called device through the data stream transmission interface;

[0211] If a connection response message returned by the called device based on the data stream connection request is received, it is determined that the data stream connection between the calling device and the called device is successfully established;

[0212] Send a data stream acquisition request to the called device through the data stream connection.

[0213] See also Figure 8 , which is a structural block diagram of another device for hardware cross-device calling in an embodiment of the present application. In some implementations, the device for hardware cross-device calling in the example of the present application includes:

[0214] The execution unit 801 is used to determine that when receiving a hardware call instruction sent by a calling device to a target hardware, the target hardware is controlled to execute the call instruction; the target hardware is hardware set on the called device, and the hardware call instruction is sent by the calling device in response to a user's triggering operation of a virtual hardware service corresponding to the target hardware through a hardware cross-device call interface based on hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through a hardware collaborative service interface;

[0215] The acquisition unit 802 is used to determine that when a data stream acquisition request for target hardware is received from a calling device, based on the data stream request, the data stream information of the target hardware is acquired; the data stream acquisition request is sent by the calling device through a data stream transmission interface;

[0216] The returning unit 803 is used to return the data flow information to the calling device.

[0217] In one implementation, the execution unit 801 is further configured to:

[0218] When determining that a hardware information acquisition request sent by a calling device for target hardware is received, the hardware metadata is acquired; the hardware information acquisition request is sent by the calling device through a hardware collaborative service interface;

[0219] Returns hardware metadata to the calling device.

[0220] In one implementation, the execution unit 801 is further configured to:

[0221] Determine that upon receiving a permission request sent by a calling device for target hardware, obtain cross-device permissions of the target hardware; the permission request is sent by the calling device through the hardware collaborative service interface;

[0222] Returns cross-device permissions to the calling device.

[0223] In one implementation, the acquiring unit 802 is further configured to:

[0224] When determining that a data stream connection request sent by the calling device through the data transmission interface is received, a connection response message is sent to the calling device to establish a data stream connection between the calling device and the called device;

[0225] The acquisition unit 802 is used for:

[0226] It is determined that when a data stream acquisition request sent by a calling device through a data stream connection is received, data stream information is acquired.

[0227] The method for hardware cross-device calling in the embodiment of the present application includes responding to the user's triggering operation on the virtual hardware service corresponding to the target hardware, obtaining the hardware metadata of the target hardware through the hardware collaborative service interface; the target hardware is the hardware set on the called device; through the hardware cross-device calling interface, based on the hardware metadata, sending a hardware call instruction to the target hardware, so that the calling device controls the target hardware to execute the hardware call instruction; through the data stream transmission interface, sending a data stream acquisition request for the target hardware to the called device; receiving the data stream information returned by the called device based on the data stream acquisition request. In this way, the target hardware of the called device is abstracted as the virtual hardware service of the calling device, and the user does not need to consider the differences in the device and system where the target hardware is located, and can directly call the target hardware across devices through the virtual hardware service, which simplifies the complex operation of hardware cross-device calling.

[0228] In an embodiment of the present application, an electronic device is provided, including:

[0229] Processor; and

[0230] The memory stores computer instructions, where the computer instructions are used to enable the processor to execute the method of any of the above embodiments.

[0231] In an embodiment of the present application, a storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a computer to execute a method in any of the above-mentioned embodiments. Fig. 9 FIG. 9 is a schematic diagram showing the structure of an electronic device 9000. Fig. 9 As shown, the electronic device 9000 includes: a processor 9010 and a memory 9020 , and optionally, may also include a power supply 9030 , a display unit 9040 , and an input unit 9050 .

[0232] The processor 9010 is the control center of the electronic device 9000. It connects various components using various interfaces and lines, and performs various functions of the electronic device 9000 by running or executing software programs and / or data stored in the memory 9020, thereby monitoring the electronic device 9000 as a whole.

[0233] In the embodiment of the present application, the processor 9010 executes the various steps in the above embodiment when calling the computer program stored in the memory 9020.

[0234] Optionally, the processor 9010 may include one or more processing units; preferably, the processor 9010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 9010. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0235] The memory 9020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store data created according to the use of the electronic device 9000, etc. In addition, the memory 9020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices, etc.

[0236] The electronic device 9000 also includes a power source 9030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 9010 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0237] The display unit 9040 may be used to display information input by the user or information provided to the user and various menus of the electronic device 9000, and in the embodiment of the present application, is mainly used to display the display interface of each application in the electronic device 9000 and objects such as text and pictures displayed in the display interface. The display unit 9040 may include a display panel 9041. The display panel 9041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0238] The input unit 9050 may be used to receive information such as numbers or characters input by the user. The input unit 9050 may include a touch panel 9051 and other input devices 9052. The touch panel 9051, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a touch pen, or any other suitable object or accessory on or near the touch panel 9051).

[0239] Specifically, the touch panel 9051 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 9010, and receive and execute the command sent by the processor 9010. In addition, the touch panel 9051 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. Other input devices 9052 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0240] Of course, the touch panel 9051 can cover the display panel 9041. When the touch panel 9051 detects a touch operation on or near it, it transmits it to the processor 9010 to determine the type of touch event, and then the processor 9010 provides corresponding visual output on the display panel 9041 according to the type of touch event. Fig. 9 In the embodiment, the touch panel 9051 and the display panel 9041 are used as two independent components to realize the input and output functions of the electronic device 9000, but in some embodiments, the touch panel 9051 and the display panel 9041 can be integrated to realize the input and output functions of the electronic device 9000.

[0241] The electronic device 9000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the electronic device 9000 may also include other components such as a camera. Since these components are not the key components used in the embodiments of the present application, Fig. 9 It is not shown and will not be described in detail.

[0242] Those skilled in the art will understand that Fig. 9 The electronic device is merely an example and does not limit the electronic device, and may include more or less components than those shown in the figure, or may combine certain components, or may include different components.

[0243] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0244] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the embodiments. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. The obvious changes or modifications derived therefrom are still within the scope of protection created by this application.< / t> < / t> < / t> < / t> < / t> < / t> < / t> < / boolean> < / hardwareinfo> < / associationinfo>

Claims

1. A method for hardware cross-device calling, characterized in that: Applied to calling a device, the method comprises: In response to a user triggering operation on a virtual hardware service corresponding to a target hardware, obtaining hardware metadata of the target hardware through a hardware collaboration service interface; the target hardware is hardware set in a called device; Sending a hardware call instruction to the target hardware based on the hardware metadata through a hardware cross-device call interface, so that the calling device controls the target hardware to execute the hardware call instruction; Sending a data stream acquisition request for the target hardware to the called device through a data stream transmission interface; Receive data stream information returned by the called device based on the data stream acquisition request.

2. The method according to claim 1, characterized in that The obtaining of the hardware metadata of the target hardware through the hardware coordination service interface includes: Through the hardware coordination service interface, the distributed hardware coordination service is called to perform the following steps: Sending a hardware information acquisition request to the called device for the target hardware; Receive the hardware metadata returned by the called device based on the hardware information acquisition request.

3. The method according to claim 2, characterized in that Before sending a hardware call instruction to the target hardware based on the hardware metadata through the hardware cross-device call interface, the method further includes: Calling the distributed hardware collaborative service through the hardware collaborative service interface to obtain the cross-device permissions of the target hardware; If the cross-device permission indicates that the call is prohibited, the hardware cross-device call is stopped.

4. The method according to any one of claims 1 to 3, characterized in that: The sending of a hardware call instruction to the target hardware based on the hardware metadata through the hardware cross-device call interface includes: Generate a hardware call instruction based on the hardware metadata through the hardware cross-device call interface; Serializing the hardware call instruction to obtain a serialized hardware call instruction; The serialized hardware call instruction is transmitted to the target hardware.

5. The method according to any one of claims 1 to 3, characterized in that: The step of sending a data stream acquisition request for the target hardware to the called device through the data stream transmission interface includes: Sending a data stream connection request to the called device through a data stream transmission interface; If a connection response message returned by the called device based on the data stream connection request is received, it is determined that the data stream connection between the calling device and the called device is successfully established; The data stream acquisition request is sent to the called device through the data stream connection.

6. A method for hardware cross-device calling, characterized in that: Applied to the called device, the method includes: When it is determined that a hardware call instruction sent by a calling device to target hardware is received, the target hardware is controlled to execute the call instruction; the target hardware is hardware set on the called device, and the hardware call instruction is sent by the calling device in response to a user's trigger operation on a virtual hardware service corresponding to the target hardware through a hardware cross-device call interface based on hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through a hardware collaborative service interface; When determining that a data stream acquisition request for the target hardware is received from the calling device, acquiring data stream information of the target hardware based on the data stream request; the data stream acquisition request is sent by the calling device through a data stream transmission interface; The data flow information is returned to the calling device.

7. The method according to claim 6, characterized in that The method further comprises: When it is determined that a hardware information acquisition request sent by the calling device for the target hardware is received, the hardware metadata is acquired; the hardware information acquisition request is sent by the calling device through a hardware collaborative service interface; The hardware metadata is returned to the calling device.

8. The method according to claim 6, characterized in that The method further comprises: When it is determined that a permission request sent by the calling device for the target hardware is received, the cross-device permission of the target hardware is obtained; the permission request is sent by the calling device through the hardware collaborative service interface; The cross-device permission is returned to the calling device.

9. The method according to any one of claims 6 to 8, characterized in that: When it is determined that a data stream acquisition request for the target hardware is received by the calling device, before acquiring data stream information of the target hardware based on the data stream request, the method further includes: When determining that a data stream connection request sent by the calling device through the data transmission interface is received, sending a connection response message to the calling device to establish a data stream connection between the calling device and the called device; The determining that a data flow acquisition request for the target hardware is received by the calling device, and acquiring data flow information of the target hardware based on the data flow request, includes: When it is determined that a data stream acquisition request sent by the calling device through the data stream connection is received, the data stream information is acquired.

10. A device for hardware cross-device calling, characterized in that: Applied to calling a device, the device comprises: An acquisition unit, configured to acquire hardware metadata of the target hardware through a hardware collaboration service interface in response to a user triggering operation on a virtual hardware service corresponding to the target hardware; the target hardware is hardware set in a called device; A calling unit, configured to send a hardware calling instruction to the target hardware based on the hardware metadata through a hardware cross-device calling interface, so that the calling device controls the target hardware to execute the hardware calling instruction; A sending unit, configured to send a data stream acquisition request for the target hardware to the called device through a data stream transmission interface; The receiving unit is used to receive the data flow information returned by the called device based on the data flow acquisition request.

11. A device for hardware cross-device calling, characterized in that: Applied to a called device, the apparatus comprises: an execution unit, configured to control the target hardware to execute the hardware call instruction when receiving the hardware call instruction sent by the calling device to the target hardware; the target hardware is hardware set on the called device, the hardware call instruction is sent by the calling device in response to a user's triggering operation of the virtual hardware service corresponding to the target hardware through the hardware cross-device call interface based on the hardware metadata of the target hardware; the hardware metadata is obtained by the calling device through the hardware collaborative service interface; an acquisition unit, configured to acquire data stream information of the target hardware based on the data stream request when receiving a data stream acquisition request from the calling device for the target hardware; the data stream acquisition request is sent by the calling device through a data stream transmission interface; A returning unit is used to return the data flow information to the calling device.

12. An electronic device, characterized in that: include: processor; as well as A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any one of claims 1 to 9.

13. A storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 9.

Images