A remote screen mirroring method based on message queue telemetry transfer

Abstract

The application discloses a remote screen mirroring method based on message queue telemetry transfer, and belongs to the technical field of engineering machinery software, which comprises the following steps: developing a set of application software code, performing cross-platform compilation, and deploying the code to a remote end and a vehicle-mounted end; the remote end and the vehicle-mounted end establish a remote screen mirroring connection through an MQTT server; the remote end and the vehicle-mounted end operate the screen, and realize screen interface and data unification through the MQTT server. The method can enable the remote end and the vehicle-mounted end to use the same set of application software code, without the need of re-development, and can realize bidirectional or multi-directional interaction.

Description

TECHNICAL FIELD

[0001] The application relates to a remote screen mirroring method based on message queue telemetry transmission and belongs to the technical field of engineering machinery software. BACKGROUND

[0002] At present, remote screen mirroring patents mainly focus on remote education, home video, robot positioning and the like.

[0003] One reason why the remote screen mirroring function cannot be widely applied in the engineering machinery industry is that two sets of software need to be developed, and the software developed by the equipment end cannot be directly applied to the remote PC end or mobile end. The vehicle-mounted end and the remote end need to develop software respectively in the traditional remote screen mirroring function, which is very laborious, and the software of the engineering machinery industry changes very frequently, so the software of the vehicle-mounted end and the remote end both need to be changed, which is very troublesome.

[0004] In addition, the previous remote screen mirroring is limited to display screen mirroring, which is similar to the function of copying the screen. Interaction at the operation level is not realized, that is, the vehicle-mounted end is operated, and the remote end can only view the interface and cannot operate. SUMMARY

[0005] The application aims to provide a remote screen mirroring method based on message queue telemetry transmission, which uses the same set of application software code for the remote end and the vehicle-mounted end, does not need to be redeveloped, and can realize two-way or multi-way interaction.

[0006] To achieve the above-mentioned purpose, the application provides the following technical scheme:

[0007] A remote screen mirroring method based on message queue telemetry transmission comprises the following steps:

[0008] Developing a set of application software code, performing cross-platform compilation, and deploying it to the remote end and the vehicle-mounted end;

[0009] The remote end and the vehicle-mounted end establish a remote screen mirroring connection through an MQTT server;

[0010] The remote end and the vehicle-mounted end operate the screen and realize screen interface and data unification through the MQTT server.

[0011] Further, developing a set of application software code comprises the following steps:

[0012] Developing a set of application software code using Qt or other cross-platform compilation supported software;

[0013] The application software code supports local processing and external processing for the operation interface in the screen interface;

[0014] For data processing of the vehicle terminal, the application software code reserves a local data interface;

[0015] For data processing of the remote terminal, the application software code reserves a network data interface;

[0016] For different operating systems, the application software code configures different compilation environments and cross-tool chains.

[0017] Further, the operation interface in the screen interface includes buttons, sliders, lists, and pop-up windows.

[0018] Further, the data processing of the vehicle terminal includes CAN data transceiving, ferroelectric data, and database file processing.

[0019] Further, the operating system includes Linux systems, Windows systems, and Android systems.

[0020] Further, the remote terminal and the vehicle terminal establish a remote screen mirroring connection through an MQTT server, including:

[0021] The remote terminal logs in through an Internet of Things platform, selects a corresponding vehicle model, and sends a screen mirroring request to the vehicle terminal through the MQTT server;

[0022] In response to the screen mirroring request, the interface of the vehicle terminal displays a screen mirroring request confirmation reminder;

[0023] After the user confirms the screen mirroring request, the remote terminal and the vehicle terminal establish a remote screen mirroring connection.

[0024] Further, after the remote terminal and the vehicle terminal establish a remote screen mirroring connection, the vehicle terminal operates the screen to realize screen interface and data unification of the vehicle terminal and the remote terminal, including:

[0025] The vehicle terminal sends local data to the MQTT server and distributes corresponding response data to the remote terminal that has established a remote screen mirroring connection through the MQTT server;

[0026] The remote terminal receives the response data and processes interface jumping and interface data refreshing according to a preset software logic to realize screen interface and data unification of the vehicle terminal and the remote terminal;

[0027] The response data includes interface information and interface internal data.

[0028] Further, after the remote terminal and the vehicle terminal establish a remote screen mirroring connection, the remote terminal operates the screen to realize screen interface and data unification of the vehicle terminal and the remote terminal, including:

[0029] The remote end responds to the operation on the screen, and responds to the screen interface, and sends the operation signal to the vehicle-mounted end through the MQTT server;

[0030] The vehicle-mounted end receives the operation signal, responds to the screen interface according to the operation signal, and sends the corresponding response data to the remote end through the MQTT server, so as to realize the screen interface and data unification of the vehicle-mounted end and the remote end.

[0031] Compared with the prior art, the beneficial effects of the present application are:

[0032] On the one hand, the present application uses the same set of application software code for the vehicle-mounted end and the remote end, reduces the workload of software development, and can automatically backup the latest software code to the server, so that the user can automatically push the latest version of the software when using the remote screen mirroring function, which not only reduces the workload of application and maintenance, but also creates conditions for batch application.

[0033] On the other hand, the present application can realize two-way or multi-way interaction, and each device can interact with the screen interface after the device is connected, and other screen interfaces will respond to the operation, which provides great convenience for remote service, remote diagnosis and remote control in combination with permission management. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a remote screen mirroring method flowchart based on message queue telemetry transmission provided by the embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions of the present application will be further described in detail below in combination with specific implementation manners.

[0036] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the drawings, wherein the same or similar reference signs represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation on the present application. The technical features in the embodiments of the present application and the embodiments can be combined with each other without conflict.

[0037] Figure 1 is a remote screen mirroring method flowchart based on message queue telemetry transmission provided by the embodiment one of the present application, and the flowchart only shows the logical order of the method of the present embodiment, and the steps shown or described can be completed in an order different from that shown in the present application and other possible embodiments of the present application without conflict. Figure 1

[0038] ​The remote screen sharing method based on message queue telemetry transport provided by the embodiment can be applied to a terminal and can be executed by a corresponding remote screen sharing device. The device can be implemented in software and / or hardware, and can be integrated in the terminal, such as any tablet computer or computer device with a communication function. Figure 1 The method of the embodiment specifically includes the following steps.

[0039] Step 1: Develop a set of application software code, perform cross-platform compilation, and deploy it to the remote end and the vehicle-mounted end.

[0040] Developing a set of application software code includes the following steps.

[0041] Step A: Develop a set of application software code using Qt or other software supporting cross-platform compilation.

[0042] Step B: For the operation interface in the screen interface, the application software code supports local processing and external processing at the same time.

[0043] Step C: For data processing of the vehicle-mounted end, the application software code reserves a local data interface.

[0044] Step D: For data processing of the remote end, the application software code reserves a network data interface.

[0045] Step E: For different operating systems, the application software code configures different compilation environments and cross-tool chains.

[0046] The operation interface in the screen interface includes buttons, sliders, lists, pop-up windows, and the like. When developing these operation interfaces, the same event processing function is called, and the local processing and external processing are supported at the same time, so that a set of application software can respond to multiple input modes.

[0047] The data processing of the vehicle-mounted end includes CAN data transceiving, ferroelectric data, and database file processing.

[0048] The operating system includes a Linux system, a Windows system, an Android system, and the like.

[0049] In this way, the code related to the UI and the function part during cross-platform compilation does not need to be changed, and a set of software can run on multiple platforms.

[0050] Step 2: The remote end and the vehicle-mounted end establish a remote screen sharing connection through an MQTT server.

[0051] The remote end and the vehicle-mounted end establish a remote screen sharing connection through an MQTT server, including the following steps.

[0052] Step a: the remote end logs in through the IoT platform, selects the corresponding vehicle model, and sends a screen mirroring request to the vehicle end through the MQTT server;

[0053] Step b: in response to the screen mirroring request, the interface of the vehicle end displays a screen mirroring request confirmation reminder;

[0054] Step c: after the user confirms the screen mirroring request, the remote end and the vehicle end establish a remote screen mirroring connection.

[0055] Step three: the remote end and the vehicle end operate the screen, and realize screen interface and data unification through the MQTT server;

[0056] After the remote end and the vehicle end establish a remote screen mirroring connection, the vehicle end operates the screen to realize screen interface and data unification between the vehicle end and the remote end, including the following steps:

[0057] Step I: the vehicle end sends local data to the MQTT server and distributes corresponding response data to the remote end that has established a remote screen mirroring connection through the MQTT server;

[0058] Step II: the remote end receives the response data and processes interface jump and interface data refresh according to the preset software logic to realize screen interface and data unification between the vehicle end and the remote end;

[0059] Among them, the response data includes interface information and interface internal data.

[0060] After the remote end and the vehicle end establish a remote screen mirroring connection, the remote end operates the screen to realize screen interface and data unification between the vehicle end and the remote end, including the following steps:

[0061] Step i: the remote end responds to the operation of the screen and processes the screen interface, and sends the operation signal to the vehicle end through the MQTT server;

[0062] Step ii: the vehicle end receives the operation signal, processes the screen interface according to the operation signal, and sends the corresponding response data to the remote end through the MQTT server to realize screen interface and data unification between the vehicle end and the remote end.

[0063] The remote same screen system in the current engineering machinery industry has a delay of about 1.5s, and the system delay of the application is within 100ms without considering network delay. With the wide application of 4G and 5G, the communication speed of the current mobile network has been significantly improved, and based on 4G communication, the total delay of the system of the application can be controlled within 1s, and if a 5G module is used, the total delay can be controlled within 500ms, so that the remote same screen function can be practically applied, such as a remote service scene, and the service personnel do not need to rush to a place thousands of miles away, but only need to connect the vehicle-mounted display screen in the office to check the system fault and remotely debug, troubleshoot and solve customer problems, which not only reduces the service cost, but also improves the service efficiency and user satisfaction. In addition, MQTT is lightweight and based on a client-server protocol, and on this basis, the application optimizes the communication protocol part, reduces the content that needs to be communicated for each interface, and improves the communication efficiency.

[0064] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) having computer-usable program code embodied in the medium.

[0065] The application is described with reference to flowcharts and / or block diagrams that illustrate the methods, apparatus (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions that are executed by the processor of the computer or other programmable data processing apparatus generate an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0066] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0067] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide a process for implementing the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the flow Figure 1 one flow or multiple flows and / or the functions specified in the flow

[0068] The above is only the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the technical principles of the present application, can also make a number of improvements and variations, these improvements and variations should also be considered as the protection scope of the present application.

Claims

1. A method for remote co-screening based on message queue telemetry transport, characterized in that, The application relates to a remote screen mirroring method and system. Developing an application software code, cross-platform compiling, and deploying to remote and vehicle ends; Remote and vehicle ends establish remote screen mirroring connection through MQTT server; Remote and vehicle ends operate the screen and realize screen interface and data unification through MQTT server; Developing an application software code includes: Developing an application software code using Qt or other cross-platform compiling software; For operation interfaces in the screen interface, the application software code supports local processing and external processing; For data processing of the vehicle end, the application software code reserves a local data interface; For data processing of the remote end, the application software code reserves a network data interface; For different operating systems, the application software code configures different compiling environments and cross-tool chains; After the remote and vehicle ends establish remote screen mirroring connection, the vehicle end operates the screen, and realizes screen interface and data unification of the vehicle end and the remote end includes: The vehicle end sends local data to the MQTT server and distributes corresponding response data to the remote end which has established remote screen mirroring connection through the MQTT server; The remote end receives the response data and processes interface jump and interface data refresh according to the preset software logic, realizing screen interface and data unification of the vehicle end and the remote end; After the remote and vehicle ends establish remote screen mirroring connection, the remote end operates the screen, and realizes screen interface and data unification of the vehicle end and the remote end includes: The remote end processes the screen interface in response to the operation on the screen, and sends operation signals to the vehicle end through the MQTT server; The vehicle end receives the operation signals, processes the screen interface according to the operation signals, and sends corresponding response data to the remote end through the MQTT server, realizing screen interface and data unification of the vehicle end and the remote end.

2. The remote co-screen method based on message queue telemetry transport according to claim 1, wherein, The operation interfaces in the screen interface include buttons, sliders, lists and pop-up windows.

3. The remote co-screen method based on message queue telemetry transport according to claim 1, wherein, The data processing of the vehicle end includes CAN data transceiving, ferroelectric data and database file processing.

4. The remote co-screen method based on message queue telemetry transport according to claim 1, wherein, The operating system includes Linux system, Windows system and Android system.

5. The remote co-screen method based on message queue telemetry transport according to claim 1, wherein, Remote and vehicle ends establish remote screen mirroring connection through MQTT server includes: The remote end logs in through an Internet of Things platform, selects a corresponding vehicle model, and sends a screen mirroring request to the vehicle end through the MQTT server; In response to the screen mirroring request, the interface of the vehicle end displays a screen mirroring request confirmation reminder; After the user confirms the screen mirroring request, the remote end and the vehicle end establish remote screen mirroring connection.

6. The remote co-screen method based on message queue telemetry transport according to claim 1, wherein, The response data includes interface information and interface internal data.

Images