Android terminal mirror image projection screen control method, device and system
Patent Information
- Application Number
- CN202411022761.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing Android terminal mirror screen projection method is not fully functional in remote command scenarios, and it is impossible to achieve reverse screen projection, voice command and remote operation.
Through the collaborative work of the Android terminal, the server and the second terminal, the two-way transmission of screen content, prompt information, audio data and operation data is realized. The prompt information is displayed using a suspended window, and remote operation is performed using the RTSP protocol and the ADB actuator.
Reverse screen projection, voice command and remote operation functions are realized, enhancing the functional integrity of remote command scenarios.
Smart Images

Figure CN118939217A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of mirror screen projection technology, and in particular to a mirror screen projection control method, device and system for an Android terminal. Background Art
[0002] Mirroring screen projection refers to the technology of projecting the image displayed on the screen of one device to the screen of another device. Mirroring screen projection technology has been widely used in life and work. For example, in life, the movie screen played on the mobile phone screen is projected on the TV screen to get a better viewing experience; in a meeting, the slides played on the computer screen are projected on the projector screen so that other participants can also see the content of the slides and improve the reporting effect. These scenarios are all common applications of mirroring screen projection technology.
[0003] The existing Android terminal mirroring screen projection method includes: collecting the screen image of the Android terminal (such as a mobile phone) and sending the screen image to a server via the network; the server then forwards the screen image to a second terminal (such as a TV, projector, computer), and displays the screen image of the Android terminal on the screen of the second terminal.
[0004] This existing Android terminal mirroring screen projection method can also be used for remote command, but the function is not perfect. For example, the elderly in their hometown cannot operate Android devices (mobile phones), and their children in other places can use the existing mirroring screen projection technology to see the screen of the elderly’s Android device through their second terminal (computer). However, if the children want to instruct the elderly to operate a certain button on the Android device, they need to project the text prompt information or pattern mark on the Android device. The existing Android terminal mirroring screen projection method can only realize the one-way projection of the screen of the Android terminal to the second terminal, and cannot reversely project the mark of the second terminal to the Android terminal, nor can it transmit audio for voice command, and it cannot remotely operate the Android terminal through the second terminal. The function in the remote command scenario is not perfect enough. Summary of the invention
[0005] Based on this, the purpose of the present invention is to provide an Android terminal mirror screen projection control method, device and system, which has the functions of reverse screen projection, voice command and remote operation, and overcomes the defect that the functions of the remote command scenario in the prior art are not perfect.
[0006] The present invention provides a method for controlling screen mirroring of an Android terminal, which is executed by an Android terminal, a server and a second terminal, and comprises the following steps: SA, the Android terminal obtains its screen content, and encodes the screen content into video stream data, which is forwarded to a second terminal through a server, and the second terminal displays the screen content of the Android terminal; SB, the second terminal obtains prompt information, and forwards the prompt information to the Android terminal through a server; the Android terminal displays the prompt information on the top layer of an application through a floating window.
[0007] The present invention overcomes the defect that the existing Android terminal mirroring screen projection method can only realize unidirectional projection of the screen content of the Android terminal to the second terminal, and the function in the remote command scene is not perfect, and realizes the reverse projection of the marking information of the second terminal to the Android terminal, which facilitates the user of the second terminal to remotely command the user of the Android terminal.
[0008] Furthermore, the prompt information includes text information; step SB specifically includes: SB0, the second terminal obtains text information data, and forwards the text information data to the Android terminal through the server; the Android terminal calls the Toast component to display the text information at the top level of the application according to the text information data.
[0009] Furthermore, the prompt information includes marking information; step SB specifically includes: SB1, the second terminal obtains the marking information data, and forwards the marking information data to the Android terminal through the server; SB2, after receiving the marking information data, the Android terminal creates a drawing board with the same size as the screen, and sets the background color of the drawing board to transparent, and then places the marking content corresponding to the marking information data at the corresponding position of the drawing board; SB3, the Android terminal creates a system-level floating window that floats on the top layer of the application, and displays the marking content in the drawing board through the floating window.
[0010] Furthermore, after step SA, the method further includes step SC, in which the second terminal collects audio data and forwards the audio data to the Android terminal through the server; the Android terminal plays the audio content corresponding to the audio data through a player according to the audio data.
[0011] Furthermore, step SC specifically includes: SC1, the second terminal calls the microphone to collect an audio data segment at a preset time interval, and encodes the continuous audio data segments into PCM audio stream data, and sends the PCM audio stream data to the server; SC2, the server forwards the PCM audio stream data to the Android terminal; SC3, after the Android terminal receives the PCM audio stream data, it calls the Oboe PCM stream player to decode the PCM audio stream data to obtain audio content, and then plays the audio content through the audio output device.
[0012] Furthermore, after step SA, step SD is also included: the second terminal monitors the user's operation data, and forwards the operation data to the Android terminal through the server; the Android terminal calls the ADB executor to execute the operation corresponding to the operation data.
[0013] Furthermore, step SD is specifically as follows: the second terminal monitors the user's operation data in the mirror window, and converts the operation data into ADB operation instructions; the second terminal encapsulates the ADB operation instructions and transmits them to the server; the server forwards the ADB operation instructions to the Android terminal; the Android terminal calls the ADB executor, parses and executes the ADB operation instructions.
[0014] Further, step SA specifically includes: SA1, the second terminal obtains the remote matching code input by the user, and sends the remote matching code to the server; SA2, the server queries the subscription information corresponding to the remote matching code in the database according to the remote matching code; and then transmits the subscription information to the second terminal; SA3, after receiving the subscription information, the second terminal requests the server to create a live streaming address; after receiving the request of the second terminal, the server creates a live streaming address and pushes the live streaming address to the Android device; SA4, after receiving the live streaming address, the Android device connects to the server through the RTSP transmission protocol; SA5, the Android device calls its screen recording function to obtain screen image data; renders and encodes the screen image data, and converts the screen image data into video stream data; sends the video stream data to the server through the live streaming address and the RTSP transmission protocol; SA6, the server forwards the video stream data to the second terminal; after receiving the video stream data, the second terminal parses the video stream data, creates a mirror window according to the specifications of the video stream data, and displays the screen content of the Android device in the mirror window.
[0015] Based on the same inventive concept, the present invention also provides an Android terminal mirroring screen control device, including: a screen content transmission module, used to enable the Android terminal to obtain its screen content, and encode the screen content into video stream data, forward it to a second terminal through a server, and the second terminal displays the screen content of the Android terminal through a mirroring window; a prompt information display module, used to enable the second terminal to obtain prompt information, and forward the prompt information to the Android terminal through a server; the Android terminal displays the prompt information on the top level of the application through a floating window.
[0016] Based on the same inventive concept, the present invention also provides an Android terminal mirroring screen projection control system, including an Android terminal, a server and a second terminal; the Android terminal obtains its screen content, and encodes the screen content into video stream data, and forwards it to the second terminal through the server, and the second terminal displays the screen content of the Android terminal through a mirroring window; the second terminal obtains prompt information, and forwards the prompt information to the Android terminal through the server; the Android terminal displays the prompt information on the top level of the application through a floating window.
[0017] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 This is a module schematic diagram of the Android terminal mirroring screen projection control system of the present invention;
[0019] Figure 2 This is a module schematic diagram of an Android terminal mirroring screen projection device according to Embodiment 1 of the present invention;
[0020] Figure 3 This is a flow chart of the method for mirroring screen of an Android terminal according to Embodiment 1 of the present invention;
[0021] Figure 4 This is a module schematic diagram of an Android terminal mirroring screen projection device according to Embodiment 2 of the present invention;
[0022] Figure 5 This is a flow chart of the method for mirroring screen of an Android terminal according to Embodiment 2 of the present invention;
[0023] Figure 6 This is a module schematic diagram of an Android terminal mirroring screen projection device according to Embodiment 3 of the present invention;
[0024] Figure 7 This is a flow chart of the method for mirroring screen of an Android terminal according to Embodiment 3 of the present invention;
[0025] Figure 8 The present invention is a schematic diagram of a module of an Android terminal mirroring screen projection device in one embodiment of the present invention. DETAILED DESCRIPTION
[0026] It should be clear that the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.
[0027] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0028] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.
[0029] It should be understood that the embodiments of the present application are not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present application is limited only by the appended claims.
[0030] See also Figure 1 , Figure 1 The module schematic diagram of the Android terminal mirror screen projection control system of the present invention. The Android terminal mirror screen projection control system of the present invention comprises an Android terminal 1, a server 2 and a second terminal 3. The Android terminal 1 and the second terminal 3 communicate with each other through the server 2. In this embodiment, the Android terminal 1 and the second terminal 3 are not in the same local area network, so the communication between the Android terminal 1 and the second terminal 3 needs to be transferred through the server 2.
[0031] The Android terminal 1 of the present invention includes but is not limited to smart devices equipped with Android system such as mobile phones, smart watches, smart glasses, etc. The second terminal 3 includes but is not limited to smart devices such as mobile phones, televisions, projectors, computers, etc., and its screen area is not necessarily larger than that of the Android terminal 1.
[0032] Example 1
[0033] As described in the background technology, the existing Android terminal mirroring screen projection method can only display the screen image of the Android terminal 1 in the second terminal 3, and cannot reversely project the text or mark in the second terminal to the Android device. Therefore, the Android terminal mirroring screen projection control method of embodiment 1 of the present invention is used to realize reverse screen projection based on the existing mirroring screen projection method.
[0034] See also Figure 2 and Figure 3 , Figure 2 This is a module diagram of an Android terminal mirroring screen projection device according to Embodiment 1 of the present invention. Figure 3 The present invention is a flowchart of an Android terminal mirroring method according to Embodiment 1 of the present invention.
[0035] The Android terminal mirroring screen projection device of embodiment 1 of the present invention comprises: a screen content transmission module A and a prompt information display module B. The screen content transmission module A is used to execute step SA: enable the Android terminal to obtain its screen content, and encode the screen content into video stream data, forward it to the second terminal through the server, and the second terminal displays the screen content of the Android terminal through the mirroring window.
[0036] Specifically, step SA includes:
[0037] SA1, the second terminal obtains the remote matching code input by the user, and sends the remote matching code to the server. The user inputs the remote matching code by operating the control software in the second terminal. Each remote matching code corresponds to an Android device to be connected.
[0038] SA2, the server queries the subscription information corresponding to the remote matching code in the database according to the remote matching code; and then transmits the subscription information to the second terminal. In this embodiment, the subscription information is MQTT (Message Queuing Telemetry Transport) subscription information. The server can query the online / offline status of the corresponding Android device through MQTT subscription information; when the corresponding Android device is found to be online, the server transmits the subscription information to the second terminal.
[0039] SA3, after receiving the subscription information, the second terminal requests the server to create a live streaming address. After receiving the request from the second terminal, the server creates a live streaming address and pushes the live streaming address to the Android device. In this embodiment, the server pushes the live streaming address to the Android device via the MQTT (Message Queuing Telemetry Transport) protocol.
[0040] SA4, after receiving the live streaming address, the Android device connects to the server via the RTSP transmission protocol (Real Time Streaming Protocol). In this embodiment, after the Android device connects to the server via the RTSP transmission protocol, it enters the standby state and locks the APP operation interface to prevent the user from performing other operations on the Android device that cause the screen projection to fail.
[0041] SA5, the Android device calls its built-in screen recording function to obtain screen image data; renders and encodes the screen image data, and converts the screen image data into video stream data; and sends the video stream data to the server via the live streaming address and RTSP transmission protocol.
[0042] SA6, the server forwards the video stream data to the second terminal. After receiving the video stream data, the second terminal parses the video stream data, creates a mirror window according to the specification of the video stream data, and displays the screen content of the Android device in the mirror window.
[0043] Furthermore, when the Android device calls its built-in screen recording function, it also needs to obtain the screen recording function permission (MediaProjection). After the user authorizes the screen recording function permission, the Android device calls the createVirtualDisplay module to create a virtual display for displaying the screen image, and calls the GLSurfaceView module to further process and render the screen image; calls the MediaCodec module to encode the screen image data rendered by the GLSurfaceView module to obtain video stream data in H264 / H265 format; caches the video stream data in a preset cache array, and executes streaming through the thread pool, thereby transmitting the video stream data to the server through the live streaming address and RTSP transmission protocol. Converting screen image data into video stream data in H264 / H265 format can reduce the amount of data transmission and increase the transmission speed.
[0044] Furthermore, the second terminal parses the video stream data and creates a mirror window according to the specifications of the video stream data, specifically including: after receiving the first frame key frame data of the video stream data, the second terminal parses SPS data (Sequence Paramater Set) and PPS data (Picture Paramater Set), obtains the video encoding format and the video width and height, and then creates a mirror window according to the video encoding format and the video width and height.
[0045] The prompt information display module B is used to execute step SB: enable the second terminal to obtain prompt information, and forward the prompt information to the Android terminal through the server; the Android terminal displays the prompt information on the top layer of the application through a floating window.
[0046] The prompt information includes text information and mark information. The text information refers to the text string content input by the user. The mark information refers to the picture, the mouse track drawn by the user, and the pattern content composed of lines, graphics, arrows or text. The user can operate the brush control provided by the second terminal to draw the mouse track, or draw a pattern composed of lines, graphics, arrows or text by operating the line, graphic, arrow and text controls.
[0047] In an embodiment where the prompt information is text information, step SB is specifically as follows: SB0, the second terminal obtains text information data, and forwards the text information data to the Android terminal through the server; the Android terminal calls the Toast component to display the text information at the top level of the application according to the text information data. The Toast component is an existing message pop-up component in the Android system, which belongs to a type of floating window. In this embodiment, the second terminal transmits the text information data to the server through the MQTT protocol; the server also forwards the text information data to the Android terminal through the MQTT protocol.
[0048] In an embodiment where the prompt information is tag information, step SB specifically includes: SB1, the second terminal obtains the tag information data, and forwards the tag information data to the Android terminal through the server. SB2, after receiving the tag information data, the Android terminal creates a drawing board with the same size as the screen, sets the background color of the drawing board to transparent, and then places the tag content corresponding to the tag information data at the corresponding position of the drawing board. SB3, the Android terminal creates a system-level floating window suspended at the top level of the application, and displays the tag content in the drawing board through the floating window. In this embodiment, the second terminal transmits the tag information data to the server through the MQTT protocol; the server also forwards the tag information data to the Android terminal through the MQTT protocol.
[0049] In an embodiment in which the prompt information includes both text data and tag data, the Android device displays the text data through step SB0 and displays the tag data through steps SB1-SB3, thereby realizing multi-dimensional prompt information display.
[0050] Example 2
[0051] As described in the background technology, the existing Android terminal mirroring screen projection method can only display the screen image of the Android terminal 1 in the second terminal 3, but cannot perform real-time voice transmission. Therefore, the Android terminal mirroring screen projection control method of embodiment 2 of the present invention is used to further realize the voice command function based on embodiment 1.
[0052] See also Figure 4 and Figure 5 , Figure 4 This is a module diagram of an Android terminal mirroring screen projection device according to Embodiment 2 of the present invention. Figure 5 The present invention is a flowchart of an Android terminal mirroring screen projection method according to Embodiment 2 of the present invention.
[0053] The Android terminal mirroring screen projection device of Embodiment 2 of the present invention further includes: an audio transmission module C. The audio transmission module C is used to execute step SC after step SA: instruct the second terminal to collect audio data, and forward the audio data to the Android terminal through the server; the Android terminal plays the audio content corresponding to the audio data through the player according to the audio data.
[0054] Step SC specifically includes: SC1, the second terminal calls the microphone to collect an audio data segment every preset time, and encodes the continuous audio data segments into PCM (Pulse Code Modulation) audio stream data, and sends the PCM audio stream data to the server through the MQTT protocol. SC2, the server forwards the PCM audio stream data to the Android terminal through the MQTT protocol. SC3, after receiving the PCM audio stream data, the Android terminal calls the Oboe PCM stream player to decode the PCM audio stream data to obtain the audio content, and then plays the audio content through an audio output device (such as a speaker, earphones). The Oboe PCM stream player is a PCM stream audio player in the existing Android system Oboe development library.
[0055] Furthermore, before step SC1, the method further includes step SC0, where the second terminal obtains microphone permission. Step SC1 is performed only after the microphone permission is successfully obtained. If the microphone does not exist or the microphone permission acquisition fails, a prompt message is popped up, and step SC1 is not performed.
[0056] In this embodiment, the preset time length is 10 ms, that is, the second terminal collects an audio data segment every 10 ms.
[0057] Example 3
[0058] As described in the background technology, the existing Android terminal mirroring screen projection method can only display the screen image of the Android terminal 1 in the second terminal 3, but cannot directly operate the Android terminal through the second terminal. Therefore, the Android terminal mirroring screen projection control method of embodiment 3 of the present invention is used to further realize the remote operation function on the basis of embodiment 1.
[0059] See also Figure 6 and Figure 7 , Figure 6 This is a module diagram of an Android terminal mirroring screen projection device according to Embodiment 3 of the present invention. Figure 7 The present invention is a flowchart of an Android terminal mirroring screen projection method according to Embodiment 3 of the present invention.
[0060] The Android terminal mirroring screen projection device of Embodiment 3 of the present invention further includes a remote operation module D. The remote operation module D is used to execute step SD after step SA: instruct the second terminal to monitor the user's operation data, and forward the operation data to the Android terminal through the server; the Android terminal calls the ADB executor to execute the operation corresponding to the operation data.
[0061] Step SD specifically includes: the second terminal monitors the user's operation data in the mirror window, and converts the operation data into ADB (Android Debug Bridge) operation instructions required by the Android terminal; the second terminal encapsulates the ADB operation instructions and transmits them to the server through the MQTT protocol; the server forwards the ADB operation instructions to the Android terminal; the Android terminal calls the ADB executor to parse and execute the ADB operation instructions. The operation data includes operation path data such as mouse clicks / slides. ADB is a command line tool in the Android system that can be used for cross-platform debugging.
[0062] Furthermore, before step SD1, step SD0 is also included: the second terminal requests the Android terminal to enable the developer mode through the server. Step SD1 is performed only after the Android terminal enables the developer mode; otherwise, a prompt message is popped up and step SD1 is not performed.
[0063] Embodiment 2 and Embodiment 3 of the present invention can be combined technically, that is, after step SA, steps SB, SC and SD are performed simultaneously, so as to realize the functions of reverse screen projection, voice command and remote operation at the same time, and obtain a more comprehensive Android terminal mirror screen projection control method for remote command scenarios. The module diagram of the Android terminal mirror screen projection control device obtained by combining embodiment 2 and embodiment 3 is shown in FIG. Figure 8 shown.
[0064] The present invention has the following technical effects: the present invention overcomes the defect that the existing Android terminal mirroring screen projection method can only realize the one-way projection of the screen content of the Android terminal to the second terminal, and the function in the remote command scene is not perfect, and realizes the reverse projection of the marking information of the second terminal to the Android terminal, which facilitates the user of the second terminal to remotely command the user of the Android terminal. The present invention also provides the functions of voice command and remote operation, which further adapts to the application scenario of remote command.
[0065] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the Android terminal mirroring method of the embodiment of the present invention; and the memory is used to store a computer program executable by the processor.
[0066] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the embodiment of the aforementioned Android terminal mirroring screen projection method, and the computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the Android terminal mirroring screen projection method recorded in any of the above embodiments.
[0067] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
[0068] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.
Claims
1. A method for controlling screen mirroring of an Android terminal, executed by an Android terminal, a server and a second terminal, characterized in that: Includes steps: SA, the Android terminal obtains its screen content, encodes the screen content into video stream data, and forwards it to the second terminal through the server, and the second terminal displays the screen content of the Android terminal; SB, the second terminal obtains the prompt information, and forwards the prompt information to the Android terminal through the server; The Android terminal displays prompt information on the top layer of the application through a floating window.
2. The Android terminal mirroring screen control method according to claim 1, characterized in that: The prompt information includes text information; step SB specifically includes: SB0, the second terminal obtains text information data, and forwards the text information data to the Android terminal through the server; the Android terminal calls the Toast component to display the text information at the top level of the application according to the text information data.
3. The Android terminal mirroring screen control method according to claim 1, characterized in that: The prompt information includes marking information; step SB specifically includes: SB1, the second terminal obtains the marking information data, and forwards the marking information data to the Android terminal through the server; SB2, after receiving the marking information data, the Android terminal creates a drawing board with the same size as the screen, sets the background color of the drawing board to transparent, and then places the marking content corresponding to the marking information data at the corresponding position of the drawing board; SB3, the Android terminal creates a system-level floating window floating on the top layer of the application, and displays the marked content in the drawing board through the floating window.
4. The Android terminal mirroring screen control method according to claim 1, characterized in that: After step SA, the method further includes step SC, in which the second terminal collects audio data and forwards the audio data to the Android terminal through the server; the Android terminal plays the audio content corresponding to the audio data through a player according to the audio data.
5. The Android terminal mirroring screen control method according to claim 4 is characterized in that: Step SC specifically includes: SC1, the second terminal uses the microphone to collect an audio data segment every preset time, encodes the continuous audio data segments into PCM audio stream data, and sends the PCM audio stream data to the server; SC2, the server forwards the PCM audio stream data to the Android terminal; SC3, after receiving the PCM audio stream data, the Android terminal calls the Oboe PCM stream player to decode the PCM audio stream data to obtain audio content, and then plays the audio content through the audio output device.
6. The Android terminal mirroring screen control method according to claim 1, characterized in that: After step SA, step SD is also included: the second terminal monitors the user's operation data, and forwards the operation data to the Android terminal through the server; the Android terminal calls the ADB executor to execute the operation corresponding to the operation data.
7. The method for controlling screen mirroring of an Android terminal according to claim 6, characterized in that: Step SD is specifically as follows: the second terminal monitors the operation data of the user in the mirror window and converts the operation data into an ADB operation instruction; the second terminal encapsulates the ADB operation instruction and transmits it to the server; The server forwards the ADB operation instruction to the Android terminal; The Android terminal calls the ADB executor to parse and execute the ADB operation instruction.
8. The Android terminal mirroring screen control method according to claim 1, characterized in that: Step SA specifically includes: SA1, the second terminal obtains a remote matching code input by the user, and sends the remote matching code to the server; SA2, the server searches the database for subscription information corresponding to the remote matching code according to the remote matching code; and then transmits the subscription information to the second terminal; SA3, after receiving the subscription information, the second terminal requests the server to create a live streaming address; after receiving the request from the second terminal, the server creates the live streaming address and pushes the live streaming address to the Android device; SA4, after receiving the live streaming address, the Android device connects to the server via the RTSP transmission protocol; SA5, the Android device calls its screen recording function to obtain screen image data; renders and encodes the screen image data, and converts the screen image data into video stream data; and sends the video stream data to the server through the live streaming address and RTSP transmission protocol; SA6, the server forwards the video stream data to the second terminal; after receiving the video stream data, the second terminal parses the video stream data, creates a mirror window according to the specifications of the video stream data, and displays the screen content of the Android device in the mirror window.
9. An Android terminal mirror screen control device, characterized in that: include: A screen content transmission module, used to enable the Android terminal to obtain its screen content, and encode the screen content into video stream data, and forward it to the second terminal through the server, and the second terminal displays the screen content of the Android terminal through a mirror window; A prompt information display module, used to enable the second terminal to obtain prompt information, and forward the prompt information to the Android terminal through the server; The Android terminal displays prompt information on the top layer of the application through a floating window.
10. An Android terminal mirror screen projection control system, characterized in that: Including Android terminal, server and second terminal; The Android terminal obtains its screen content, encodes the screen content into video stream data, and forwards it to the second terminal through the server, and the second terminal displays the screen content of the Android terminal through a mirror window; The second terminal obtains the prompt information, and forwards the prompt information to the Android terminal through the server; the Android terminal displays the prompt information on the top layer of the application through a floating window.
Citation Information
Patent Citations
Remote control method based on mobile terminal
CN106161619A
Remote assistance method, device and system for smart terminal
CN107070980A
Method and device for transmitting conference content
CN109586929A
Screen projection method, device, system and computer equipment
CN110515573A
Information interaction method, interaction system and application
CN110727361A
