Method and apparatus for remote invocation of a device based on WebRTC

Abstract

The application relates to the technical field of remote control, in particular to a video assistance and device remote calling method and device based on WebRTC. The method comprises the following steps: establishing a WebRTC connection, a local terminal initiates a device information acquisition request to a remote terminal; after the remote terminal receives the device information acquisition request, the remote terminal collects device information through a hardware peripheral manager and returns the device information to the remote terminal; after the local terminal receives the device information, interface compatibility of hardware peripheral calling interface information contained in the device information is checked, if the interface is incompatible, a user is prompted to perform version matching upgrade, otherwise, the next step is performed; after the local terminal completes the device information checking and verification, whether the corresponding version of the remote hardware peripheral calling plug-in has been installed in the local terminal is checked; the device can support data types output by block devices and stream devices at the same time, and better performance and interactive experience are provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote control, in particular to a device remote calling method and device based on WebRTC. BACKGROUND

[0002] In fields such as education, government affairs, etc., there are often remote assistance and remote operation scenarios, which include scenarios that need to call the hardware peripherals of the remote terminal, collect relevant data, and then transmit and enter into the local terminal business system. The current technical solutions for realizing such remote calling scenarios mainly include remote desktop and proxy programs based on B / S architecture. The implementation of the remote desktop is to open and run a remote desktop client program in the remote terminal system, and the local terminal system is connected to the remote desktop client program through a remote desktop control program installed in the local terminal system and a server to realize remote control. Through remote control, the business system is run in the remote terminal, the hardware peripherals of the remote terminal are called, and the relevant data is collected and entered into the business system.

[0003] The above-mentioned traditional remote scenario calling technical solution needs to run the local business system in the remote terminal. If the local business system cannot be run in the remote terminal due to various conditions, an additional proxy system needs to be developed to call the hardware peripherals and collect the corresponding data, which is sent to the local business system through the relay.

[0004] Based on the above problems, there is an urgent need for a WebRTC video assistance and device remote calling system, which does not need to develop an additional proxy system even if the local business system cannot be run in the remote terminal. SUMMARY

[0005] In view of the current remote scenario calling technical solution, the local business system cannot be run in the remote terminal, and an additional proxy system needs to be developed, the present application provides a device remote calling method and device based on WebRTC to solve the above problems.

[0006] To achieve the above-mentioned purpose, the present application is realized by the following technical solutions:

[0007] The present application discloses a device remote calling method based on WebRTC, characterized in that it comprises:

[0008] The WebRTC connection is ready, the local terminal initiates a request for obtaining device information to the remote terminal; after the remote terminal receives the request for obtaining device information, the device information is collected through the hardware peripheral manager module and returned to the local terminal;

[0009] After the local terminal receives the device information, the interface compatibility of the hardware peripheral calling interface information contained therein is checked, if the interface is not compatible, the user is prompted to perform version matching upgrade, otherwise the next step is performed;

[0010] After the local terminal completes the device information checking and verification, whether the corresponding version of the remote hardware peripheral calling plug-in has been installed locally is checked, if the corresponding version of the remote hardware peripheral calling plug-in has been installed, the step of creating a remote hardware peripheral object is entered, otherwise the next step is entered;

[0011] The local terminal sends a request for obtaining a remote hardware peripheral calling plug-in to the remote terminal; after the remote terminal receives the request for obtaining the remote hardware peripheral calling plug-in, the plug-in based on HTML+CSS+JavaScript is packaged and sent to the local terminal through the data channel of WebRTC;

[0012] After the local terminal receives the plug-in, the plug-in is checked, unpacked, installed, loaded and registered, then the corresponding remote hardware peripheral object is created and handed over to the abstract hardware peripheral manager module for management and scheduling;

[0013] When the business system initiates a device calling request, and the user directs the device calling request to the remote hardware peripheral object by selection or configuration, the remote hardware peripheral object is responsible for encapsulating the device calling request and the corresponding related parameters, and sending them to the remote terminal;

[0014] After the remote terminal receives the device calling request, the device calling request is transferred to the remote calling processor module for analysis, then distributed to the hardware peripheral manager module, and the corresponding hardware peripheral is scheduled and driven by the hardware peripheral manager module for data acquisition, finally the collected data is transmitted to the local terminal by the remote calling processor module;

[0015] After the local terminal receives the collected data, the collected data is processed and finally submitted to the business system.

[0016] The application also discloses a device remote calling device based on WebRTC, which is applied to the device remote calling method based on WebRTC and characterized by comprising a browser container device end and a WebRTC technology end.

[0017] The hardware peripheral calling module can drive the hardware peripherals on the local terminal and the remote terminal, initiate calling and collect corresponding data;

[0018] Hardware peripheral manager module: capable of managing various hardware peripherals integrated in the local terminal and the remote terminal, and uniformly allocating and scheduling the hardware peripherals;

[0019] Abstract hardware peripheral manager module: the browser container device end has the capability of abstractly managing hardware peripherals, and can abstract and manage hardware peripherals on the remote terminal in addition to managing hardware peripherals on the local terminal, and uniformly allocate and schedule the hardware peripherals;

[0020] Hardware peripheral calling interface object module: the browser container device end provides corresponding uniform hardware peripheral interface objects according to the types of hardware peripherals, facing the business system, and the business system can drive hardware peripherals and collect corresponding data by calling the hardware peripheral interface objects;

[0021] Remote calling interface module: abstracts the remote calling mode and parameters to form a unified calling interface, so that different network communication modes can be called in a unified manner;

[0022] Remote calling processor module: mainly responsible for parsing, allocating and responding to remote calling requests;

[0023] Plug-in extension module: the browser container provides a plug-in extension mechanism based on HTML+CSS+JavaScript technology, supporting plug-in verification, installation, loading, registration, management and uninstallation functions.

[0024] Preferably, after the hardware peripheral calling interface object module receives a device calling request, the device calling request and corresponding related parameters are encapsulated and then sent to the abstract hardware peripheral manager module, which schedules remote hardware peripheral objects according to user selection or configuration.

[0025] Further preferably, the remote calling processor module receives and parses the hardware peripheral remote calling request of the opposite end, then allocates the hardware peripheral remote calling request to the abstract hardware peripheral manager module, which is responsible for driving and calling local hardware peripherals and collecting corresponding data, and finally the remote calling processor module encapsulates the result data and responds to return to the opposite end.

[0026] Further preferably, the plug-in extension module dynamically installs and registers remote hardware peripheral calling plug-ins, creates remote hardware peripheral objects, and hands them over to the abstract hardware peripheral manager module for management, depending on the plug-in extension mechanism.

[0027] Technical effects:

[0028] 1. The technical scheme of the present application is based on WebRTC technology, and through the data channel and multimedia stream channel of WebRTC, the data types output by block devices and stream devices can be well supported at the same time, and better performance and interactive experience are provided.

[0029] 2. A more flexible calling mode and interactive mode are provided. The technical scheme of the present application is based on the plug-in extension mechanism of the browser container, and the plug-in is implemented by the HTML+CSS+JavaScript technology stack. The calling mode of the background interface and the interactive mode of the user interface can be provided, and the characteristics of the hardware peripheral can be flexibly adapted, and the needs of the user and the business system can be met.

[0030] 3. In the technical scheme of the present application, the remote hardware peripheral object is implemented by inheriting and implementing the hardware peripheral interface, and the network communication is abstracted and encapsulated, the decoupling of the hardware peripheral remote calling and the network communication is realized, so that the technical scheme of the present application can not only support WebRTC, but also support network communication modes such as WebSocket, HTTP, etc. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical schemes in the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 The flow chart of the video assistance and device remote calling method based on WebRTC of the present application;

[0033] Figure 2 The connection block diagram of the video assistance and device remote calling method based on WebRTC of the present application.

[0034] In the figure:

[0035] 1. Hardware peripheral calling module; 2. Hardware peripheral manager module; 3. Abstract hardware peripheral manager module; 4. Hardware peripheral calling interface object module; 5. Remote calling interface module; 6. Remote calling processor module; 7. Plug-in extension module. DETAILED DESCRIPTION

[0036] In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0037] It is to be understood that the terminology "includes", "has", "holds", "contains" and / or other similar forms used herein are used in the sense of providing support for, allowing support for, or capable of supporting the described feature, element, step, operation, component and / or group thereof, but do not in themselves constitute a limitation as to what can or might be present otherwise.

[0038] Please refer to Figure 1 and Figure 2 , the existing remote device remote call needs to develop a proxy system to call the hardware peripheral and collect the corresponding data if the local business system cannot run in the remote terminal due to various conditions. The device remote call method based on WebRTC disclosed in the embodiments of the present application is as follows:

[0039] The WebRTC connection is ready, and the local terminal initiates a request for obtaining device information to the remote terminal. After the remote terminal receives the request for obtaining device information, the device information is collected through the hardware peripheral manager module, and is returned to the local terminal.

[0040] After the local terminal receives the device information, the interface compatibility of the hardware peripheral call interface information contained in the device information is checked. If the interface is not compatible, the user is prompted to perform version matching and upgrading, otherwise the next step is performed.

[0041] After the local terminal completes the device information checking and verification, it is checked whether the corresponding version of the remote hardware peripheral call plug-in has been installed in the local terminal. If the corresponding version of the remote hardware peripheral call plug-in has been installed, the step of creating a remote hardware peripheral object is entered, otherwise the next step is entered.

[0042] The local terminal sends a request for obtaining a remote hardware peripheral call plug-in to the remote terminal. After the remote terminal receives the request for obtaining a remote hardware peripheral call plug-in, the plug-in based on HTML+CSS+JavaScript is packaged and sent to the local terminal through the data channel of WebRTC.

[0043] After the local terminal receives the plug-in, the plug-in is verified, unpacked, installed, loaded and registered, and then the corresponding remote hardware peripheral object is created and handed over to the abstract hardware peripheral manager module for management and scheduling.

[0044] When the business system initiates a device call request, and the user points the device call request to the remote hardware peripheral object by selection or configuration, the remote hardware peripheral object is responsible for encapsulating the device call request and corresponding related parameters, and sending them to the remote terminal;

[0045] After the remote terminal receives the device call request, it is transferred to the remote call processor module for analysis, and then assigned to the hardware peripheral manager module, which schedules and drives the corresponding hardware peripheral to collect data, and finally the remote call processor module transmits the collected data to the local terminal;

[0046] After the local terminal receives the collected data, it processes the collected data and finally submits it to the business system.

[0047] Compared with the remote desktop method, the technical solution provided by the present application can directly call the hardware peripherals of the remote terminal in the local terminal in the form of remote call, collect the corresponding data and directly enter the business system in the local terminal, without opening the business system in the remote terminal.

[0048] Compared with the remote terminal agent service program, the technical solution provided by the present application can more flexibly call the hardware peripherals of the remote terminal, and can well support the device type whose output is multimedia stream data.

[0049] Hardware peripherals can be divided into two categories according to the output data mode - one category is block devices whose output is text, image, file, etc.; the other category is stream devices whose output is multimedia audio and video, etc. The technical solution of the present application mainly connects and communicates based on WebRTC technology, wherein the multimedia channel of the WebRTC technology end is used to transmit the output data of the stream device, and the data channel is used to transmit the output data of the block device.

[0050] In the technical solution of the present application, the remote hardware peripheral object of the local terminal is derived from the remote hardware peripheral call plug-in provided by the remote terminal. According to the specification of the browser container plug-in extension mechanism, the remote hardware peripheral call plug-in based on HTML+CSS+JavaScript is packaged and transmitted to the local terminal through the data channel of WebRTC, and unpacked, verified, installed and registered by the browser container of the local terminal, and finally managed by the abstract hardware peripheral manager module. Among them, the remote hardware peripheral object is responsible for encapsulating the remote hardware peripheral call request of the local terminal business system and related parameters, and sending them to the remote terminal through the data channel of WebRTC, and the remote call processor module in the remote terminal browser container is responsible for analyzing the request, and assigning it to the hardware peripheral manager module of the remote terminal, and finally transmitting the collected data back to the local terminal.

[0051] The application further discloses a device remote calling apparatus based on WebRTC, which is applied to the device remote calling method based on WebRTC and comprises a browser container apparatus end and a WebRTC technology end.

[0052] A hardware peripheral calling module 1 is capable of driving hardware peripherals on the local terminal and the remote terminal, initiating calling and collecting corresponding data.

[0053] A hardware peripheral manager module 2 is capable of managing various hardware peripherals integrated on the local terminal and the remote terminal, and uniformly distributing and scheduling.

[0054] An abstract hardware peripheral manager module 3 is capable of abstractly managing hardware peripherals, and is capable of abstracting and managing hardware peripherals on the remote terminal in addition to managing hardware peripherals on the local terminal, and uniformly distributing and scheduling.

[0055] A hardware peripheral calling interface object module 4 provides corresponding uniform hardware peripheral interface objects according to the types of hardware peripherals, and a business system is capable of driving hardware peripherals and collecting corresponding data by calling the hardware peripheral interface objects.

[0056] A remote calling interface module 5 is capable of abstracting remote calling modes and parameters, forming a uniform calling interface, and enabling different network communication modes to be called in a uniform manner.

[0057] A remote calling processor module 6 is mainly responsible for analyzing, distributing and responding to remote calling requests.

[0058] A plug-in extension module 7 provides a plug-in extension mechanism based on HTML+CSS+JavaScript technology, and supports the functions of checking, installing, loading, registering, managing and uninstalling plug-ins.

[0059] After the hardware peripheral calling interface object module 4 receives a device calling request, the device calling request and corresponding related parameters are encapsulated and then sent to the abstract hardware peripheral manager module 3, which schedules remote hardware peripheral objects according to user selection or configuration.

[0060] The remote call processor module 6 receives and analyzes the remote call request of the hardware peripheral of the opposite end, then distributes the hardware peripheral remote call request to the abstract hardware peripheral manager module 3, which is responsible for driving the call of the local hardware peripheral and the collection of the corresponding data, and finally the result data is packaged by the remote call processor module 6 and returned to the opposite end.

[0061] The plug-in extension module 7 depends on the plug-in extension mechanism, dynamically installs and registers the remote hardware peripheral call plug-in, creates the remote hardware peripheral object, and hands it over to the abstract hardware peripheral manager module 3 for management.

[0062] It is worth mentioning that the above scheme enables to support the remote call of the block device and the stream device based on the WebRTC technology, and the overall architecture design scheme based on the browser container can realize the dynamic installation and creation of the plug-in and the remote hardware peripheral object.

[0063] In the above embodiments, the device elements involved are all conventional device elements, and the connection mode and the control mode involved are all conventional connection mode and control mode, unless otherwise specified.

[0064] In the above embodiments, the device elements involved are all conventional device elements, and the connection mode and the control mode involved are all conventional connection mode and control mode, unless otherwise specified.

[0065] The above embodiments are described in detail in combination with the embodiments, but those skilled in the art can understand that the various specific parameters in the above embodiments can be changed to form multiple specific embodiments without departing from the purpose of the present application, which are all within the common variation range of the present application, and will not be described one by one.

Claims

1. A method for remote device invocation based on WebRTC, characterized in that, include: Once the WebRTC connection is established and ready, the local terminal initiates a request to the remote terminal to obtain device information. After receiving a request to obtain device information, the remote terminal collects the device information through the hardware peripherals manager module and returns it to the local terminal. After receiving the device information, the local terminal performs an interface compatibility check on the hardware peripheral call interface information contained therein. If the interface is incompatible, the user is prompted to upgrade the version; otherwise, the user proceeds to the next step. After the local terminal completes the device information check and verification, it checks whether the corresponding version of the remote hardware peripheral calling plugin has been installed locally. If the corresponding version of the remote hardware peripheral invocation plugin has been installed, proceed to the step of creating a remote hardware peripheral object; otherwise, proceed to the next step. The local terminal sends a plugin retrieval request to the remote terminal; after receiving the plugin retrieval request, the remote terminal packages the HTML+CSS+JavaScript-based plugin and sends it to the local terminal through the WebRTC data channel. After receiving the plugin, the local terminal verifies, unpacks, installs, loads, and registers the plugin, then creates the corresponding remote hardware peripheral object, and hands it over to the abstract hardware peripheral manager module for management and scheduling. When a business system initiates a call request, and the user selects or configures the call request to be directed to a remote hardware peripheral object, the remote hardware peripheral object is responsible for encapsulating the call request and the corresponding related parameters and sending them to the remote terminal. After receiving the call request, the remote terminal forwards the call request to the remote call processor module for parsing, and then distributes it to the hardware peripheral manager module. The hardware peripheral manager module schedules and drives the corresponding hardware peripherals to collect data, and finally the remote call processor module transmits the collected data to the local terminal. After receiving the collected data, the local terminal processes the data and finally submits it to the business system. WebRTC's multimedia channel is used to transmit output data from streaming devices, while the data channel is used to transmit output data from block devices.

2. A WebRTC-based device remote invocation apparatus, applied to the method as described in claim 1, characterized in that, include: The browser container device and the WebRTC technology side, wherein the browser container device includes: Hardware peripheral invocation module: used to drive hardware peripherals on local and remote terminals, initiate invocations and collect corresponding data; Hardware Peripheral Manager Module: Used to manage, allocate and schedule various hardware peripherals integrated into local and remote terminals. Abstract Hardware Peripheral Manager module: Used to abstract and manage hardware peripherals on local and remote terminals, and to uniformly allocate and schedule them; Hardware peripheral call interface object module: used to provide a unified hardware peripheral interface object to the business system. The business system drives the hardware peripheral and collects data by calling the hardware peripheral interface object. Remote Invocation Interface Module: Used to abstract remote invocation methods and parameters to form a unified invocation interface; Remote Invocation Processor Module: Used to parse, allocate, and respond to remote invocation requests; Plugin extension module: Provides a plugin extension mechanism based on HTML+CSS+JavaScript, supporting plugin verification, installation, loading, registration, management and uninstallation; The multimedia channel on the WebRTC technology side is used to transmit output data from streaming devices, and the data channel is used to transmit output data from block devices. After receiving a call request, the hardware peripheral invocation interface object module encapsulates the call request and corresponding parameters and sends them to the abstract hardware peripheral manager module. The abstract hardware peripheral manager module then schedules remote hardware peripheral objects based on user selection or configuration. The remote invocation processor module receives and parses the remote invocation request from the peer, then allocates the request to the abstract hardware peripheral manager module. The abstract hardware peripheral manager module drives the local hardware peripheral and collects data. Finally, the remote invocation processor module encapsulates the result data and returns it to the peer. The plugin extension module relies on the browser container's plugin extension mechanism to dynamically install and register remote hardware peripheral invocation plugins, create remote hardware peripheral objects, and hand them over to the abstract hardware peripheral manager module for management.

Images