Android handheld device point-to-point screen projection control method

Abstract

The invention discloses a point-to-point screen projection control method for an Android handheld device. The point-to-point screen projection control method comprises an Android device end and a receiving end, the Android equipment end captures equipment screen data, compresses and packages the equipment screen data into general data and sends the general data to the receiving end, and meanwhile obtains an operation instruction of the receiving end, analyzes the operation instruction and executes the operation instruction; the receiving end obtains equipment screen data through a browser and then analyzes and displays a corresponding picture, and meanwhile the browser receives instruction data, classifies and packages the instruction data into universal instruction characters and then sends the universal instruction characters to the Android equipment end; according to the method, cross-device projection and control can be quickly realized through general packaging, receiving and sending of data by the Android device end and the receiving end, and the practicability is good.

Description

Technical Field

[0001] The present invention relates to the field of information processing, and in particular to a point-to-point screen projection control method for an Android handheld device. Background Art

[0002] With the development of mobile Internet and the popularity of Android devices, more and more users are starting to use Android devices for various operations, such as application development, office work, entertainment, etc. How to use Android devices efficiently and improve work efficiency has become an important issue. Screen projection is an important function that can project the screen of an Android device to external devices, such as TVs, projectors, etc., to achieve remote demonstration, remote assistance and other functions. The mainstream systems on the market provide some screen projection functions, most of which support the TV side and require the lower computer to support complex customized streaming media protocols and drivers, such as Apple, Xiaomi and other screen projection services, which have great limitations; another way is to use third-party software, such as AirServer, Vysor, etc., but these software only support display on the receiving end and do not support command operations. Summary of the invention

[0003] The purpose of the present invention is to provide a point-to-point screen projection control method for an Android handheld device. The present invention can quickly realize cross-device projection and control through the universal encapsulation, reception and transmission of data by the Android device end and the receiving end, and has good practicality.

[0004] The technical solution of the present invention is: a point-to-point screen projection control method for an Android handheld device, comprising an Android device end and a receiving end; the Android device end captures device screen data, compresses and encapsulates it into general data and sends it to the receiving end, and simultaneously obtains the receiving end's operation instructions, parses them and executes them; the receiving end obtains the device screen data through a browser, parses them and displays the corresponding pictures, and simultaneously the browser receives the instruction data and classifies and encapsulates them into general instruction characters and sends them to the Android device end.

[0005] In the above-mentioned point-to-point screen projection control method of an Android handheld device, the screen data sending, acquiring and displaying process is performed according to the following steps:

[0006] Step A1: The Android device starts the projection foreground service to capture the screen data of the Android device;

[0007] Step A2: Start the WebSocket service to maintain the WebSocket connection and push screen data to the receiving end;

[0008] Step A3: acquiring frame data from the screen data, and converting and compressing the frame data;

[0009] Step A4: Cache the compressed frame data into the sending queue, and the WebSocket service encapsulates and sends it;

[0010] Step A5: The receiving end creates a WebSocket link to receive the frame data, parses it, renders it into a picture, and outputs it.

[0011] In the aforementioned point-to-point screen projection control method for an Android handheld device, in step S3, frame data including screen width, height, color format and pixel data are obtained through ImageReader, and frame data planes are further obtained, and then the frame data planes are converted into a cache Bitmap in the format of ARGB565, the cache Bitmap is processed with respect to the image width and height, and then it is converted into JPEG format and compressed using the JPEG compression algorithm.

[0012] In the aforementioned point-to-point screen projection control method for an Android handheld device, in step S4, the WebSocket service encapsulates the compressed frame data through run-length encoding and outputs it in binary.

[0013] In the aforementioned point-to-point screen projection control method for an Android handheld device, in step S5, the receiving end creates and parses a WebSocket link through an H5 page.

[0014] In the aforementioned point-to-point screen projection control method for an Android handheld device, the process of receiving input command data, encapsulating and sending it, and receiving it by the Android device end is performed according to the following steps:

[0015] Step B1: The browser at the receiving end receives the operation instruction data, encapsulates it accordingly according to the instruction data category, and sends it to the Android device end through the WebSocket service;

[0016] Step B2: The Android device receives the command data via WebSocket;

[0017] Step B3: The Android device parses the instruction data and calls the corresponding system interface to execute according to the category.

[0018] In the aforementioned point-to-point screen projection control method for an Android handheld device, the browser at the receiving end in step B1 intercepts and obtains operation instruction data including action events and input events through Javascript, the action events are packaged after coordinate calculation based on the image scaling ratio, and the input events are packaged for input content.

[0019] In the aforementioned point-to-point screen projection control method for an Android handheld device, in step B1, the action event is encapsulated as an adb command string, and the input event is encapsulated as a qts command string.

[0020] In the aforementioned point-to-point screen projection control method for an Android handheld device, in step B3, when the command data is an adb command string, the Android device side converts it into an adb command and sends the command to the local adb daemon through the embedded abdlib and executes it; when the command data is a qts command string, the Android device side converts it into a qts command and calls the system's API interface to execute it.

[0021] Compared with the prior art, the present invention captures device screen data through the Android device end, compresses and encapsulates it into general data and sends it to the receiving end. The receiving end obtains the device screen data through the browser, parses and displays the corresponding picture, and at the same time, the receiving end can also receive instruction data and encapsulate it into general instruction characters and send it to the Android device end. The Android device end obtains the receiving end operation instructions, parses and executes them, thereby realizing cross-device projection and control; the browser existing in each system is used as the receiving end, and the general data is adapted and parsed at the same time, without the need for complex protocols and drivers, which effectively improves the adaptation range; after receiving the classified and packaged general instruction characters, the Android device end can quickly parse and use them, which has the characteristics of low performance occupation and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the structure of the present invention;

[0023] Figure 2 It is a schematic diagram of the processing flow of the present invention;

[0024] Figure 3 It is a schematic diagram of the process of frame data compression of the present invention;

[0025] Figure 4 It is a schematic diagram of instruction data transmission of the present invention. DETAILED DESCRIPTION

[0026] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.

[0027] Embodiment: A point-to-point screen projection control method for an Android handheld device, as shown in the attached Figure 1 and attached Figure 2As shown, it includes an Android device end and a receiving end; the Android device end captures the device screen data, compresses and encapsulates it into general data and sends it to the receiving end, and at the same time obtains the receiving end operation instruction, parses it and executes it; the receiving end obtains the device screen data through the browser, parses it and displays the corresponding picture, and at the same time the browser receives the instruction data and classifies and encapsulates it into general instruction characters and sends it to the Android device end;

[0028] The screen data sending, acquisition and display process is carried out according to the following steps:

[0029] Step A1: The Android device starts the projection foreground service, which is based on the MediaProjection API provided by Android FrameWork and is used to capture the screen data of the Android device.

[0030] Step A2: Create a WebSocket service on the Android device to maintain the WebSocket connection and push screen data to the receiving end. Compared with Socket, WebSocket has richer interfaces and supports heartbeat mechanism and automatic reconnection, making the entire network transmission more stable.

[0031] Step A3: As attached Figure 3 As shown, frame data including screen width, height, color format and pixel data are obtained through ImageReader, and frame data planes in the format of ARGB_8888 are further obtained through ImageReader, in order to facilitate rendering of screen data captured by MediaProjection; frame data planes are converted into cached Bitmap in the format of ARGB565 through ProjectionUtils to reduce memory size; MediaProjectActionMananger processes the image width and height of the cached Bitmap according to the target receiving end, converts it into JPEG format and compresses it using the JPEG compression algorithm;

[0032] Step A4: Cache the compressed frame data into the sending queue, and the WebSocket service encapsulates and sends it. Before sending, the WebSocket service encapsulates it through run-length encoding and outputs it in binary, where the first 8 bytes contain the type and image file length information, and the rest of the data is the binary data of the image file, which effectively improves the data processing performance of both ends.

[0033] Step A5: The receiving end creates a WebSocket link to receive the compressed frame data, which is rendered as a picture after parsing and output; the receiving end of this embodiment is an H5 page, which can be run in the browser of each system. During the startup of the H5 page, a WebSocket link will be created to receive the binary data transmitted from the Android device, and the header and picture files will be parsed in turn. Javascript directly writes the binary data of the picture file into the canvas for rendering to obtain the picture, thereby realizing real-time screen projection.

[0034] As attached Figure 4 As shown, the process of receiving input instruction data, encapsulating and sending it, and receiving it by the Android device is carried out in the following steps:

[0035] Step B1: The browser at the receiving end receives the operation instruction data, encapsulates it according to the instruction data category, and sends it to the Android device end through the WebSocket service; the operation instruction data including action events and input events are intercepted and obtained through Javascript, the action event is encapsulated after the coordinate calculation is combined with the image scaling ratio, and the input event encapsulates the input content; the configuration protocol stipulates that the action event is encapsulated as an adb instruction string, and the input event is encapsulated as a qts instruction string, both of which can be run in the underlying driver of the Android device end, which is safe, stable and efficient;

[0036] Step B2: The Android device receives the command data via WebSocket;

[0037] Step B3: The Android device parses the command data and calls the corresponding system interface to execute according to the category; when the command data is an adb command string, the Android device converts it into an adb command and sends the command to the local adb daemon through the embedded abdlib and executes it; when the command data is a qts command string, the Android device converts it into a qts command and calls the system API interface to execute it. For example, for Chinese input, it will be copied to the clipboard first, and then the paste operation will be performed.

[0038] To summarize, the present invention captures device screen data through the Android device end, compresses and encapsulates it into general data and sends it to the receiving end. The receiving end obtains the device screen data through the browser, parses and displays the corresponding picture, and at the same time, the receiving end can also receive instruction data and encapsulate it into general instruction characters and send it to the Android device end. The Android device end obtains the operation instructions of the receiving end, parses and executes them, thereby realizing cross-device projection and control; wireless data transmission relies on the WebSocket service, which can be achieved only by WIFI without additional hardware modules; the browser existing in each system is used as the receiving end, and the general data is adapted and parsed at the same time, without the need for complex protocols and drivers, which effectively improves the adaptation range; after receiving the classified and packaged general instruction characters, the Android device end can quickly parse and use them, which has the characteristics of low performance occupation and high efficiency.

Claims

1. A point-to-point screen projection control method for an Android handheld device, characterized in that: It includes an Android device end and a receiving end; the Android device end captures the device screen data, compresses and encapsulates it into general data and sends it to the receiving end, and at the same time obtains the receiving end operation instructions for parsing and execution; The receiving end obtains the device screen data through the browser and parses and displays the corresponding picture. At the same time, the browser receives the instruction data and classifies and encapsulates it into general instruction characters and then sends it to the Android device end.

2. The point-to-point screen projection control method of an Android handheld device according to claim 1, characterized in that: The screen data sending, acquisition and display process is carried out according to the following steps: Step A1: The Android device starts the projection foreground service to capture the screen data of the Android device; Step A2: Start the WebSocket service to maintain the WebSocket connection and push screen data to the receiving end; Step A3: acquiring frame data from the screen data, and converting and compressing the frame data; Step A4: Cache the compressed frame data into the sending queue, and the WebSocket service encapsulates and sends it; Step A5: The receiving end creates a WebSocket link to receive the frame data, parses it, renders it into a picture, and outputs it.

3. The point-to-point screen projection control method of an Android handheld device according to claim 2, characterized in that: In step S3, frame data including screen width, height, color format and pixel data are obtained through ImageReader, and frame data planes are further obtained, and then the frame data planes are converted into a cache Bitmap in the format of ARGB565, the cache Bitmap is processed for image width and height, and then converted into JPEG format and compressed using the JPEG compression algorithm.

4. The point-to-point screen projection control method of an Android handheld device according to claim 2, characterized in that: In step S4, the WebSocket service encapsulates the compressed frame data through run-length encoding and outputs it in binary.

5. The point-to-point screen projection control method of an Android handheld device according to claim 2, characterized in that: In step S5, the receiving end creates and parses the WebSocket link through the H5 page.

6. The point-to-point screen projection control method of an Android handheld device according to claim 1, characterized in that: The process of receiving input instruction data, encapsulating and sending it, and receiving it on the Android device side is performed in the following steps: Step B1: The browser at the receiving end receives the operation instruction data, encapsulates it accordingly according to the instruction data category, and sends it to the Android device end through the WebSocket service; Step B2: The Android device receives the command data via WebSocket; Step B3: The Android device parses the instruction data and calls the corresponding system interface to execute according to the category.

7. The point-to-point screen projection control method of an Android handheld device according to claim 6, characterized in that: In step B1, the browser at the receiving end intercepts and obtains operation instruction data including action events and input events through Javascript. The action events are packaged after coordinate calculation based on the image scaling ratio, and the input events are packaged for input content.

8. The point-to-point screen projection control method of an Android handheld device according to claim 7, characterized in that: In the step B1, the action event is encapsulated as an adb command string, and the input event is encapsulated as a qts command string.

9. The point-to-point screen projection control method of an Android handheld device according to claim 7, characterized in that: In step B3, when the instruction data is an adb instruction string, the Android device converts it into an adb instruction and sends the instruction to the local adb daemon through the embedded abdlib and executes it; when the instruction data is a qts instruction string, the Android device converts it into a qts instruction and calls the system API interface to execute it.

Images