Screen projection link establishment method and system

Abstract

The embodiment of the invention provides a screen projection link establishment method which is applied to a windows computer, the windows computer is connected with an IOS device through a USB cable, the screen projection link establishment method comprises the steps that an Apple service program is installed on the windows computer, and target service is established on a wi nDows computer through the Apple service program; a target software library is installed on the windows computer, and the target software library is used for carrying out local communication and management with the IOS device; screen projection data transmitted by the IOS equipment through the USB cable are received through the target service; and transmitting the screen projection data to a first application program of the windows computer through the target software library for screen projection playing. According to the technical scheme, the screen projection link can be established in a wired mode, the quality of the screen projection link is improved, and the problems existing in wireless screen projection are avoided.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to a method, system, computer device, computer-readable storage medium, and computer program product for establishing a screen projection link. Background Art

[0002] When using a mobile device for screen casting and live streaming, the mobile device first pushes the media stream to a local computer with more powerful performance (such as a Windows computer), and then uses dedicated live streaming software on the computer to receive the streaming data from the mobile device. The live streaming software on the computer integrates the images and sounds in the streaming data into a live streaming scene with more flexible configuration and richer materials, and then pushes the stream to the live streaming room.

[0003] Before streaming from a mobile device to a computer, you need to build a data link between the two. The traditional method is to use wireless screen projection: connect the computer and mobile device (via WiFi) to the same network segment of the same local area network, then create a network socket on the computer application, and the mobile device obtains the address and port number required to connect to the socket by scanning the QR code, and thereby establishes a data link between the two. After the data link is established, the mobile device uses the link to send the encoded and encapsulated audio and video data.

[0004] However, although wireless screen projection is convenient, the link quality is directly related to the LAN network equipment and WiFi signal quality, and the configuration of the LAN network equipment will also affect the device connection. For example, if the computer and mobile device are not in the same network segment, or the router is configured with device isolation, the computer and mobile device will not be able to communicate; for example, if the WiFi signal quality used by the link is low, such as through a wall, it may cause screen projection interruption or freeze or delay.

[0005] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0006] The embodiments of the present application provide a screen projection link method, system, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0007] One aspect of an embodiment of the present application provides a method for establishing a screen projection link, which is applied to a Windows computer, wherein the Windows computer is connected to an iOS device via a USB cable, and the method includes:

[0008] Installing an Apple service program on the Windows computer, and creating a target service on the Windows computer through the Apple service program;

[0009] Installing a target software library on the Windows computer, wherein the target software library is used to perform local communication and management with the IOS device;

[0010] Receiving, through the target service, the screen projection data transmitted by the iOS device via the USB cable;

[0011] The projection data is transmitted to the first application of the Windows computer through the target software library for projection and playback.

[0012] Optionally, the method further comprises:

[0013] When the IOS device is connected to the Windows computer for the first time, installing a driver for the IOS device on the Windows computer;

[0014] Enumerate the IOS device list connected to the Windows computer, and select the IOS device from the IOS device list;

[0015] The connection function of the target software library is called to connect to the IOS device, and a target port number is specified so that the IOS device can start a TCP server according to the target port number to transfer the screen projection data.

[0016] Another aspect of an embodiment of the present application provides a method for establishing a screen projection link, which is applied to an iOS device, wherein the iOS device is connected to a Windows computer via a USB cable, and the method includes:

[0017] When a second application of the IOS device is started, monitoring a target port number of a connection between the Windows computer and the IOS device, wherein the second application is used to obtain screen projection data;

[0018] When the target port number is monitored, starting a TCP server in the IOS device according to the target port number;

[0019] Acquire the screen projection data of the IOS device through the second application;

[0020] The projection data is transferred through the TCP server and sent to the Windows computer through the USB cable based on the usbmux protocol.

[0021] Optionally, the projection data includes video frame data, microphone audio data, and playback audio data collected by the IOS device, and the projection data is transferred through the TCP server and sent to the Windows computer through the USB cable based on the usbmux protocol, including:

[0022] Rendering the video frame data, and performing predetermined processing on the video frame data during the rendering process;

[0023] Encoding the processed video frame data to obtain a video data packet;

[0024] Mixing the microphone audio data and the playback audio data to obtain mixed audio data;

[0025] Encoding the mixed audio data to obtain an audio data packet;

[0026] Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream;

[0027] The encapsulated data stream is transferred through the TCP server and sent to the Windows computer through the USB cable based on the USBmux protocol.

[0028] Another aspect of the embodiment of the present application provides a screen projection link establishment system, comprising: a Window computer and an IOS device, wherein the Window computer and the IOS device are connected via a USB cable;

[0029] The windows computer is used to create a target service through an installed Apple service program, receive the screen projection data transmitted by the iOS device via the USB cable through the target service, and transmit the screen projection data to the first application of the windows computer through an installed target software library for screen projection playback;

[0030] The IOS device is used to listen to the target port number of the connection between the Windows computer and the IOS device when the second application in the IOS device is started. When the target port number is listened to, the TCP server is started on the IOS device according to the target port number, the screen projection data of the IOS device is obtained through the second application, the screen projection data is transferred through the TCP server, and sent to the Windows computer through the USB cable based on the usbmux protocol.

[0031] Another aspect of an embodiment of the present application provides a method for establishing a screen projection link, which is applied to a Windows computer, wherein the Windows computer is connected to an Android device via a USB cable, and the method includes:

[0032] Installing a first USB library and a second USB library for the Android device on the Windows computer;

[0033] Sending a target instruction to the Android device to switch the Android device to an accessory mode;

[0034] When the Android device is switched to the accessory mode, reinstalling the first USB library for the Android device;

[0035] Receiving, through the first USB library, the screen projection data transmitted by the Android device via the USB cable;

[0036] The screen projection data is transmitted to the first application of the Windows computer through the second USB library for screen projection and playback.

[0037] Optionally, installing the first USB library and the second USB library for the Android device on the Windows computer includes:

[0038] Get device information through the device management API;

[0039] Enumerating devices according to the device information to obtain a list of portable devices connected to the Windows computer;

[0040] The Android device is selected according to the portable device list, and a first USB library and a second USB library are installed for the Android device.

[0041] Another aspect of an embodiment of the present application provides a method for establishing a screen projection link, which is applied to an Android device, wherein the Android device is connected to a Windows computer via a USB cable, and the method includes:

[0042] receiving a target instruction sent by the Windows computer, and switching the Android device to an accessory mode according to the target instruction;

[0043] Enumerate devices to obtain an accessory list of the Android device, select the Windows computer from the accessory list, and obtain a file descriptor for communicating with the Windows computer;

[0044] The projection data of the Android device is obtained by using a second application, and the projection data is sent to the Windows computer according to the file descriptor.

[0045] Optionally, the obtaining the screen projection data of the Android device by using the second application, and sending the screen projection data to the Windows computer according to the file descriptor includes:

[0046] Acquire screen acquisition data of the Android device, and acquire audio acquisition data of the Android device;

[0047] Encoding the screen acquisition data to obtain a video data packet;

[0048] Encoding the audio acquisition data to obtain an audio data packet;

[0049] Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream;

[0050] The encapsulated data stream is sent to the windows computer according to the file descriptor.

[0051] Another aspect of an embodiment of the present application provides a screen projection link establishment system, comprising a Windows computer and an Android device, wherein the Windows computer and the Android device are connected via a USB cable;

[0052] The windows computer is used to install a first USB library and a second USB library for the Android device, send a target instruction to the Android device to switch the Android device to an accessory mode, and reinstall the first USB library for the Android device when the Android device switches to the accessory mode, receive the screen projection data transmitted by the Android device through the USB cable through the first USB library, and transmit the screen projection data to the first application of the windows computer through the second USB library for screen projection playback;

[0053] The Android device is used to receive the target instruction, switch to the accessory mode according to the target instruction, enumerate the accessory list of the Android device, select the window computer from the accessory list, and obtain a file descriptor for communicating with the windows computer, use a second application to obtain the projection data of the Android device, and send the projection data to the window computer according to the file descriptor.

[0054] Another aspect of an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0055] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0056] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0057] The above technical solution adopted in the embodiment of the present application may have the following advantages:

[0058] By installing the Apple service program on the Windows computer, creating a target service on the Windows computer through the Apple service program, installing the target software library on the Windows computer, receiving the projection data transmitted by the IOS device via the USB cable through the target service, and transmitting the projection data to the first application of the Windows computer through the target software library for screen projection and playback, the connection between the Windows computer and the IOS device and the reception and playback of the projection data of the IOS device can be achieved in a wired manner, thereby improving the quality of the projection link and avoiding the problems existing in wireless projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0060] Figure 1 The operating environment diagram of the screen projection link establishment method according to the first embodiment of the present application is schematically shown;

[0061] Figure 2 The flowchart of the method for establishing a screen projection link according to the first embodiment of the present application is schematically shown;

[0062] Figure 3 The schematic diagram shows a method for establishing a screen projection link according to the first embodiment of the present application;

[0063] Figure 4 The newly added steps of the method for establishing a screen projection link according to the first embodiment of the present application are schematically shown;

[0064] Figure 5 The flowchart of the method for establishing a screen projection link according to the second embodiment of the present application is schematically shown;

[0065] Figure 6 Schematically shows Figure 5 Flow chart of sub-steps of step S406;

[0066] Figure 7 A block diagram of a screen projection link establishment system according to Embodiment 3 of the present application is schematically shown;

[0067] Figure 8 The flowchart of the method for establishing a screen projection link according to the fourth embodiment of the present application is schematically shown;

[0068] Fig. 9 The timing diagram of switching the Android device to the accessory mode in the screen projection link establishment method according to the fourth embodiment of the present application is schematically shown;

[0069] Fig.10 An example diagram of the process of establishing a screen projection link between a Windows computer and an Android device;

[0070] Fig.11 The schematic diagram shows a method for establishing a screen projection link according to the fourth embodiment of the present application;

[0071] Fig.12 for Figure 8 Flow chart of sub-steps of step S600;

[0072] Fig.13 The flowchart of the method for establishing a screen projection link according to the fifth embodiment of the present application is schematically shown;

[0073] Fig.14 A flowchart schematically shows the response of an Android device to an accessory mode in a method for establishing a screen projection link according to Embodiment 5 of the present application;

[0074] Fig.15 for Fig.13 Flow chart of sub-steps of step S804;

[0075] Fig.16 A block diagram of a screen projection link establishment system according to Embodiment 7 of the present application is schematically shown; and

[0076] Fig.17 The hardware architecture diagram of the computer device according to the eighth embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0078] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0079] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0080] First, the following terms are explained:

[0081] Screen projection: also known as same screen, flying screen, screen sharing, screen mirroring, is a special term for a technology and function in the digital field. It usually refers to the phenomenon or behavior of two identical or different devices displaying the same picture, that is, the picture on one device (such as a mobile phone, tablet, computer, etc.) is transmitted to another device (such as a TV, projector, computer monitor, etc.) for display in some way (wireless or wired).

[0082] libimobiledevice: is a powerful open source library that allows developers to communicate with iOS devices through native protocols without relying on any Apple proprietary libraries or requiring device jailbreaking. This means that developers can write software on multiple operating system platforms (such as Linux, macOS, Windows, and Android) to manage and interact with iOS devices (such as iPhone, iPad, and iPod Touch).

[0083] AppleMobileDeviceSupport: is a driver program for Apple mobile devices (such as iPhone, iPad and iPod). This service is part of the automatic installation process of iTunes software, and is designed to solve the problem of Apple mobile devices not being able to be recognized and accessed properly after connecting to Windows computers.

[0084] TCP (Transmission Control Protocol) is a computer communication protocol used to reliably transmit data on the Internet. It is part of the Internet Protocol (IP) and works at the fourth layer of the OSI seven-layer model, the transport layer. TCP provides a connection-oriented, reliable data transmission service for applications.

[0085] usbmux (USB Multiplexer): is a service provided by Apple, mainly used to implement multiple TCP connections on the USB protocol and abstract USB communication into TCP communication.

[0086] libusb: is a cross-platform open source USB library designed to provide low-level access to USB devices for various operating systems. It provides a unified API that allows developers to easily write cross-platform USB applications without having to deeply understand the details of USB drivers under different operating systems.

[0087] libusbK: is an open source USB library, which is designed to provide an interface similar to WinUSB, but with wider compatibility and richer functions. It is mainly used in Windows operating system and aims to simplify the development of USB devices under Windows platform.

[0088] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below:

[0089] In related technologies, wireless screen projection is usually used for screen projection, but the link quality of screen projection is easily affected by factors such as local area network equipment and WIFI signal quality, which can easily lead to screen projection interruption, freeze, or delay.

[0090] To this end, the embodiment of the present application provides a technical solution for establishing a screen projection link. In this technical solution, the screen projection link can be established in a wired manner to improve the quality of the screen projection link and avoid the problems existing in wireless screen projection. See below for details.

[0091] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0092] like Figure 1 As shown, the operating environment diagram includes: a mobile device 2 and a windows computer 4, and the mobile device 2 is connected to the windows computer 4 via a USB cable 6. Among them, the mobile device 2 can be an iOS device running an iOS operating system, or an Android device running an Android system. The mobile device 2 is used to collect screen projection data and send the screen projection data to the windows computer 4 via the USB cable 6 for screen projection playback.

[0093] The following uses Windows computers, iOS devices, and Android devices as the execution subjects to introduce the technical solutions of the present application through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be interpreted as being limited to the embodiments described here.

[0094] Embodiment 1

[0095] Figure 2 The flowchart of the method for establishing a screen projection link according to the first embodiment of the present application is schematically shown.

[0096] like Figure 2 As shown, the screen projection link establishment method is applied to a Windows computer, and the Windows computer is connected to an IOS device via a USB cable, wherein the receiving application on the Windows computer serves as the initiator of the connection, and the application on the IOS device serves as the recipient. The screen projection link establishment method may include steps S200 to S206, wherein:

[0097] Step S200, installing the Apple service program on the Windows computer, and creating a target service on the Windows computer through the Apple service program.

[0098] Step S202, installing a target software library on a Windows computer, wherein the target software library is used for local communication and management with an IOS device.

[0099] Step S204, receiving the screen projection data transmitted by the IOS device via the USB cable through the target service.

[0100] Step S206, transmitting the projection data to the first application of the Windows computer through the target software library for projection and playback.

[0101] The method for establishing a screen projection link provided in this embodiment installs an Apple service program on a Windows computer, creates a target service on the Windows computer through the Apple service program, installs a target software library on the Windows computer, receives the projection data transmitted by the IOS device via a USB cable through the target service, and transmits the projection data to the first application of the Windows computer through the target software library for screen projection playback. The connection between the Windows computer and the IOS device and the reception and playback of the projection data of the IOS device can be achieved in a wired manner, thereby improving the quality of the screen projection link and avoiding the problems existing in wireless screen projection.

[0102] The following combination Figure 2 , each step in steps S200 to S206 and other optional steps are described in detail.

[0103] Step S200 , install the Apple service program on the Windows computer, and create the target service on the Windows computer through the Apple service program.

[0104] The Apple service program can be AppleMobileDeviceSupport, which can create a Windows service named Apple Mobile Device Service that runs in the background as the target service. Since iTunes comes with AppleMobileDeviceSupport, it will be installed when iTunes is installed, so Windows computers can also use AppleMobileDeviceSupport by installing iTunes.

[0105] Step S202 , install the target software library on the Windows computer, where the target software library is used to communicate and manage the IOS device locally.

[0106] The target software library may be libimobiledevice, and the AppleMobileDeviceSupport is a bridge for libimobiledevice to communicate with IOS devices.

[0107] Step S204 , receive the screen projection data transmitted by the iOS device via the USB cable through the target service.

[0108] Step S206 , the screen projection data is transmitted to the first application of the Windows computer through the target software library for screen projection and playback.

[0109] Please refer to Figure 3 , which is a schematic diagram of the principle of establishing a screen projection link between a Windows computer and an IOS device. As shown in the figure, the Windows computer receives the projection data transmitted by the IOS mobile terminal (IOS device) through the USB cable through the Apple service (target service), and then transmits the projection data to the receiving end software (first application) of the Windows computer through libimobiledevice (target software library), and the receiving end software decodes or decapsulates the projection data and plays it.

[0110] In an optional embodiment, if Figure 4 As shown, the screen projection link establishment method may further include steps S300 to S304:

[0111] Step S300: When the iOS device is connected to the Windows computer for the first time, a driver is installed for the iOS device on the Windows computer.

[0112] Step S302, enumerate the list of IOS devices connected to the Windows computer, and select an IOS device from the IOS device list.

[0113] Step S304, call the connection function of the target software library to connect to the IOS device, and specify a target port number so that the IOS device can start the TCP server according to the target port number to transfer the screen projection data.

[0114] Specifically, when the IOS device is connected to the Windows computer for the first time, the Windows computer can install a suitable driver for the IOS device on the Windows computer. After the driver is installed, the devices can be enumerated on the Windows computer to obtain the Windows computer IOS device list, the IOS device can be selected from the IOS device list, and the connection function in libimobiledevice (which can be idevic e_connect() API) is called to connect to the IOS device, and a target port number is specified for the IOS device to start the TCP server in the IOS device according to the target port number, which is used to transfer the screen projection data so that the IOS device can serve as the server of the Windows computer. When installing the driver for the IOS device, the driver may be replaced by the driver with the same Vendor ID and Product ID, and the link cannot be established. At this time, the corresponding driver can be manually unloaded, the IOS device can be reconnected, and the driver can be reinstalled. Among them, the Vendor ID is the identification of the manufacturer that produces the device, and the Product ID is the identification of a certain model of the device. Devices of the same model have the same Product ID.

[0115] When connecting a Windows computer and an IOS device, the device connection logic can be:

[0116] 1. Check whether the Apple service has been installed: Use the idevice_get_device_list() API provided by libimobiledevice to check its return value. If the return value is an error, it means that the Apple service is not installed. If the return value is successful, it means that the service has been installed.

[0117] 2. Get the list of IOS devices currently connected to the Windows computer;

[0118] 3. Select the iOS device from the iOS device list and use the idevice_connect() API to connect. When calling, you need to specify a pre-negotiated port number so that the application on the iOS device can use the port number to create a local socket as a server.

[0119] After the link is established, the receiving program (first application) on the Windows computer can call the idevice_connection_send() and idevice_connection_receive_timeout() APIs in libimobiledevice to send and receive arbitrary data. The calls for sending and receiving data are synchronous, and the reception can set a timeout to avoid being stuck on the reception call when reconnecting the USB cable.

[0120] In this optional embodiment, when the IOS device is connected to the Windows computer for the first time, a driver is installed for the IOS device on the Windows computer, a list of IOS devices connected to the Windows computer is enumerated, and the IOS device is selected from the IOS device list. The connection function of the target software library is called to connect to the IOS device, and a target port number is specified for the IOS device to start a TCP server according to the target port number to transfer the screen projection data. This can effectively install the driver for the IOS device and effectively establish a wired screen projection link between the Windows computer and the IOS device.

[0121] Embodiment 2

[0122] Figure 5 The flowchart of the method for establishing a screen projection link according to the second embodiment of the present application is schematically shown.

[0123] like Figure 5 As shown, the screen projection link establishment method is applied to an iOS device, and the iOS device is connected to a Windows computer via a USB cable. The screen projection link establishment method may include steps S400 to S406:

[0124] Step S400, when the second application of the IOS device is started, monitor the target port number of the connection between the Windows computer and the IOS device, wherein the second application is used to obtain the projection data.

[0125] Step S402: when the target port number is monitored, a TCP server is started in the IOS device according to the target port number.

[0126] Step S404, obtaining the screen projection data of the IOS device through the second application.

[0127] Step S406, transfer the screen projection data through the TCP server and send it to the Windows computer through the USB cable based on the USBmux protocol.

[0128] Specifically, after the second application is started, the IOS device can monitor the target port number of the connection between the Windows computer and the IOS device. When the target port number is monitored, a local socket is created according to the target port number, and a TCP server is started in the IOS device. The Windows computer connected to the IOS device acts as a client and calls the API provided by libimobiledevice to connect to the IOS device according to the target port number. The IOS device obtains the screen projection data of the IOS device through the second application, transfers the screen projection data through the TCP server, and sends it to the Windows computer through the USB cable based on the usbmux protocol of the IOS device.

[0129] Please continue to refer to Figure 3 ,like Figure 3 As shown, the screen projection data of the iOS device is transferred through the TCP server and sent to the Windows computer through the USB cable based on the usbmux. In addition, the recording of the screen projection data can actually be recorded by the RelayExtension shown in the figure, and the Relay Extension and the host APP (second application) are independent processes. The data interaction between the Relay Extension and the host APP can also be based on the TCP server-client mechanism for data transmission.

[0130] The screen projection link establishment method provided in this embodiment monitors the target port number of the connection between the Windows computer and the IOS device when the second application of the IOS device is started. When the target port number is monitored, the TCP server is started in the IOS device according to the target port number, the screen projection data of the IOS device is obtained through the second application, the screen projection data is transferred through the TCP server, and sent to the Windows computer through the USB cable based on the USB mux protocol. The screen projection link between the IOS device and the Windows computer can be effectively established, the quality of the screen projection link can be improved, and the problems existing in wireless screen projection can be avoided.

[0131] In an optional embodiment, the screen projection data includes video frame data collected by the iOS device, microphone audio data, and playback audio data, such as Figure 6 As shown, in step S406, the screen projection data is transferred through the TCP server and sent to the Windows computer through the USB cable based on the usbmux protocol, which may further include steps S500 to S510:

[0132] Step S500, rendering the video frame data, and performing predetermined processing on the video frame data during the rendering process.

[0133] Step S502: Encode the processed video frame data to obtain a video data packet.

[0134] Step S504: Mix the microphone audio data and the playback audio data to obtain mixed audio data.

[0135] Step S506: Encode the mixed audio data to obtain an audio data packet.

[0136] Step S508: encapsulate the video data packet and the audio data packet to obtain an encapsulated data stream.

[0137] Step S510, the encapsulated data stream is transferred through the TCP server and sent to the Windows computer through the USB cable based on the USBmux protocol.

[0138] Specifically, the video frame data collected by the IOS device has a resolution of the video frame image that is proportional to the device screen size of the IOS device. The IOS device can render the video frame data and perform predetermined processing on the video frame data during the rendering process, such as cropping, scaling, color correction, adding filters, etc., and then encode the processed video frame data, such as using VideoToolBox to perform H.264 encoding to obtain an encoded video data packet; for the microphone audio data and the playback audio data of the IOS device, they can be mixed to synthesize a single-channel audio stream to obtain mixed data; the mixed data is encoded, such as using AudioToolBox to perform AAC encoding to obtain an encoded audio data packet; the video data packet and the audio data packet are then encapsulated, such as encapsulated into FLV Packet to obtain an encapsulated data stream, and the encapsulated data stream is then transferred through a TCP server and sent to a Windows computer via a USB cable based on the usbmux protocol.

[0139] In this optional embodiment, the video frame data is rendered, and the video frame data is predetermined processed during the rendering process, the processed video frame data is encoded to obtain a video data packet, the microphone audio data and the playback audio data are mixed to obtain mixed data, the mixed data is encoded to obtain an audio data packet, the video data packet and the audio data packet are encapsulated to obtain an encapsulated data stream, the encapsulated data stream is transferred through a TCP server, and sent to a Windows computer via a USB cable based on the USBmux protocol. The projection data can be encapsulated and sent to the Windows computer, thereby facilitating the transmission of the projection data.

[0140] Embodiment 3

[0141] Figure 7 The block diagram of the screen projection link establishment system 600 according to the third embodiment of the present application is schematically shown. Figure 7 As shown, the system 600 may include a windows computer 610 and an IOS device 620, and the windows computer 610 and the IOS device 620 are connected via a USB cable, wherein:

[0142] The windows computer 610 is used to create a target service through an installed Apple service program, receive the projection data transmitted by the iOS device 620 via the USB cable through the target service, and transmit the projection data to the first application of the windows computer 610 through an installed target software library for projection and playback;

[0143] The IOS device 620 is used to monitor the target port number of the connection between the Windows computer 610 and the IOS device 620 when the second application in the IOS device 620 is started, and when the target port number is monitored, start the TCP server in the IOS device 620 according to the target port number, obtain the screen projection data of the IOS device 620 through the second application, transfer the screen projection data through the TCP server, and send it to the Windows computer 610 through the USB cable based on the usbmux protocol.

[0144] In an optional embodiment, the windows computer 610 is also used to:

[0145] When the IOS device 620 is connected to the Windows computer 610 for the first time, installing a driver for the IOS device on the Windows computer;

[0146] Enumerate the IOS device list connected to the Windows computer 610, and select the IOS device 620 from the IOS device list;

[0147] The connection function of the target software library is called to connect to the IOS device 620, and the target port number is specified so that the IOS device 620 can start the TCP server according to the target port number to transfer the screen projection data.

[0148] In an optional embodiment, the IOS device 620 is further configured to:

[0149] Rendering the video frame data, and performing predetermined processing on the video frame data during the rendering process;

[0150] Encoding the processed video frame data to obtain a video data packet;

[0151] Mixing the microphone audio data and the playback audio data to obtain mixed audio data;

[0152] Encoding the mixed audio data to obtain an audio data packet;

[0153] Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream;

[0154] The encapsulated data stream is transferred through the TCP server and sent to the Windows computer 610 through the USB cable based on the usbmux protocol.

[0155] Embodiment 4

[0156] Figure 8 The flowchart of the method for establishing a screen projection link according to the fourth embodiment of the present application is schematically shown.

[0157] like Figure 8 As shown, the screen projection link establishment method is applied to a Windows computer, and the Windows computer is connected to an Android device via a USB cable, wherein the receiving end application of the Windows computer serves as the initiator of the connection, and the application on the Android device serves as the recipient. The screen projection link establishment method may include steps S600 to S608, wherein:

[0158] Step S600: Install a first USB library and a second USB library for the Android device on the Windows computer.

[0159] The first USB library may be, for example, libusbK, and the second USB library may be, for example, libusb. When the Android device is connected via a USB cable, the Windows computer may install the first USB library and the second USB library for the Android device.

[0160] Step S602: Send a target instruction to the Android device to switch the Android device to the accessory mode.

[0161] The target instruction may be, for example, a device control command sent using libusb to switch the Android device to accessory mode. The instruction sent may include fields such as accessory manufacturer name, accessory model name, accessory version, accessory description, and accessory-related website address, wherein the accessory manufacturer name, accessory model name, and accessory version are necessary fields for matching Android-side applications; the accessory description and accessory-related website address are optional fields and may be displayed in a system prompt box that pops up on the Android device.

[0162] Please refer to Fig. 9, which is a timing diagram of a Windows computer switching an Android device to accessory mode. As shown in the figure, the Windows computer requests the protocol version of the Android device, and the Android device returns the protocol version to the Windows computer. The Windows computer sends a request including accessory information and matching applications according to the protocol version, and then sends a request to make the device enter accessory mode, so that the Android device enters accessory mode.

[0163] Step S604: when the Android device is switched to the accessory mode, reinstall the first USB library for the Android device.

[0164] When the Android device is switched to accessory mode, the first USB library needs to be reinstalled because the Vendor ID and Product ID will change.

[0165] Please refer to Fig.10 , which is a flow chart of establishing a screen projection link between a Windows computer and an Android device. As shown in the figure, after the Android device is connected to the Windows computer, it is determined whether the driver has been installed; if not, the driver is installed; if the driver has been installed, libusb is used to open the device, and then it is determined whether the Android device is in accessory mode; if it is in accessory mode, the connection is successful; if it is not, it is converted to accessory mode and the driver is reinstalled; then libusb is used to open the device, and then the communication point is found, and finally the connection is successful.

[0166] Step S606, receiving the screen projection data transmitted by the Android device via the USB cable through the first USB library.

[0167] Step S608, transferring the projection data to the first application of the Windows computer through the second USB library for projection and playback.

[0168] After the screen projection link is established, the receiving application on the Windows computer can call libusb to communicate with the device. libusb provides interfaces similar to send and recv to send and receive data. There are synchronous and asynchronous versions, and timeouts can be set.

[0169] Please refer to Fig.11 , which is a schematic diagram of the principle of establishing a screen projection link between a Windows computer and an Android device. As shown in the figure, the Windows computer receives the projection data transmitted by the Android mobile terminal (Android device) through the USB cable through libusbK (the first USB library), and then transmits the projection data to the receiving end software (the first application) of the Windows computer through libusb (the second USB library), and the receiving end software decodes or decapsulates the projection data and plays it.

[0170] The screen projection link establishment method provided in this embodiment installs the first USB library and the second USB library for the Android device on the Windows computer, sends a target instruction to the Android device to switch the Android device to the accessory module, and reinstalls the first USB library for the Android device when the Android device switches to the accessory mode, receives the screen projection data transmitted by the Android device via the USB cable through the first USB library, and transmits the screen projection data to the first application of the Windows computer through the second USB library for screen projection playback. The connection between the Windows computer and the Android device and the reception and playback of the screen projection data of the Android device can be achieved in a wired manner, thereby improving the quality of the screen projection link and avoiding the problems existing in wireless screen projection.

[0171] In an optional embodiment, if Fig.12 As shown, in step 600, installing the first USB library and the second USB library for the Android device on the Windows computer may further include steps S700 to S704:

[0172] Step S700: Obtain device information through the device management API.

[0173] Step S702: enumerate devices according to device information to obtain a list of portable devices connected to the Windows computer.

[0174] Step S704: select an Android device according to the portable device list, and install the first USB library and the second USB library for the Android device.

[0175] The device management API may be, for example, SetupAPI, through which device information such as device description, device instance ID, device class, parent class, and child class may be obtained. In order to facilitate device enumeration, the class GUID in the INF file (Information File) may be changed to a specific GUID. After obtaining the device information, the Windows computer may enumerate the device according to the device information, obtain a list of portable devices connected to the Windows computer, and then select the Android device from the list of portable devices to install the first USB library and the second USB library for the Android device.

[0176] In this optional embodiment, device information is obtained through the device management API, devices are enumerated based on the device information to obtain a list of portable devices connected to the Windows computer, an Android device is selected based on the portable device list, and the first USB library and the second USB library are installed for the Android device. This can effectively install a driver for the Android device to facilitate establishment of a wired screen projection link between Windows and the Android device.

[0177] Embodiment 5

[0178] Fig.13 The flowchart of the method for establishing a screen projection link according to the fifth embodiment of the present application is schematically shown.

[0179] like Fig.13 As shown, the screen projection link establishment method is applied to an Android device, and the Android device is connected to a Windows computer via a USB cable. The screen projection link establishment method may include steps S800 to S804, wherein:

[0180] Step S800, receiving a target instruction sent by a Windows computer, and switching the Android device to an accessory mode according to the target instruction.

[0181] Step S802: enumerate the devices to obtain an accessory list of the Android device, select a Windows computer from the accessory list, and obtain a file descriptor for communicating with the Windows computer.

[0182] Step S804, use the second application to obtain the projection data of the Android device, and send the projection data to the Windows computer according to the file descriptor.

[0183] When the Android device enters accessory mode, if the second application wants to communicate in accessory mode, the information it declares must correspond to the information provided by the accessory. Specifically, the second application needs to: 1. Declare in the manifest file <intent-filter>and <meta-data>android.hardware.usb.action.USB_ACCESSORY_ATTACHED, when the accessory is connected, the second application will receive a notification; 2. Declare the list of accessories of interest in res / xml / accessory_filter.xml in the resource directory of the second application, including: model, manufacturer and version, and these three fields need to be consistent with the fields sent by the receiving program on the Windows computer. In the wired screen projection scenario, the Windows computer can be regarded as an "accessory".

[0184] Please continue to refer to Fig.11 As shown in the figure, the Android device can obtain the file descriptor through UsbManager. The application layer corresponding to the second application can obtain the UsbManager proxy object through the Android Framework layer, and obtain the file descriptor after authorization. At this time, the Framework layer will open the " / dev / usb_accessory" based on the Linux kernel through UsbDev iceManager and obtain the necessary description information for accessory matching. After confirming the match and authorization, the application layer corresponding to the second application can perform read and write operations.

[0185] Please refer to Fig.14 , which is a flow diagram of an Android device responding to an accessory mode. As shown in the figure, the flow is a flow corresponding to when an accessory is connected to the Android device and the Android device enters the accessory mode, and the Android device has an Android application matching the accessory information installed. It can be divided into two situations: the second application is not started and is started:

[0186] 1. The second application is not started:

[0187] When the USB is connected, the system recognizes the accessory information declared in the manifest file and wakes up the second application after the user agrees. The Android device can obtain the USBManager through the Android service to enumerate the connected accessories. After the verification is passed, a file descriptor will be obtained when the accessory is opened. The file descriptor can be read and written through the IO stream to complete the transmission.

[0188] 2. The second application has been started:

[0189] Android devices can monitor USB connections through Android BroadcastReceiver. When a new device is found to be connected, it will actively initiate an authorization request to the system service USBManager. After obtaining the user's authorization, the system will notify the application in the form of a broadcast, and then after verifying the accessory permissions, open the file descriptor for reading and writing.

[0190] When the device is connected, the Android device can apply for UsbAccessory operation permissions through UsbManager and obtain ParcelFileDescriptor to implement IO output.

[0191] The screen projection link establishment method provided in this embodiment receives a target instruction sent by a Windows computer, switches the Android device to accessory mode according to the target instruction, enumerates the devices to obtain a list of accessories for the Android device, selects the Windows computer from the accessory list, obtains a file descriptor for communicating with the Windows computer, uses a second application to obtain the screen projection data of the Android device, and sends the screen projection data to the Windows computer according to the file descriptor. This can effectively establish a screen projection link between the Android device and the Windows computer, improve the quality of the screen projection link, and avoid problems with wireless screen projection.

[0192] In an optional embodiment, if Fig.15 As shown, in step 804, the screen projection data of the Android device is obtained by using the second application, and the screen projection data is sent to the Windows computer according to the file descriptor, which may further include steps S900 to S908:

[0193] Step S900, obtaining screen acquisition data of the Android device, and obtaining audio acquisition data of the Android device.

[0194] Step S902: Encode the screen acquisition data to obtain a video data packet.

[0195] Step S904: Encode the audio acquisition data to obtain an audio data packet.

[0196] Step S906: encapsulate the video data packet and the audio data packet to obtain an encapsulated data stream.

[0197] Step S908: Send the encapsulated data stream to the Windows computer according to the file descriptor.

[0198] The output of screen projection data can include three stages: acquisition, encoding, and encapsulation. It can finally be encapsulated into a data stream in FLV format, and then the encapsulated data stream is sent to the Windows computer according to the file descriptor. In screen projection, MediaProjection can be used to capture the screen to obtain screen acquisition data, and AudioRecord can be used to capture the microphone to obtain audio acquisition data. After obtaining the acquisition data, the video and audio can be encoded separately through MediaCodec to generate AVC video stream and AAC audio stream.

[0199] When the device is connected, the Android device can apply for UsbAccessory operation permissions through UsbManager and obtain ParcelFileDescriptor to achieve IO output. Subsequently, the AVC video stream and AAC audio stream can be encapsulated into FLV Packets in a time sequence to form a continuous data stream, and then transmitted to the Windows computer through this IO output.

[0200] In this optional embodiment, by obtaining the screen acquisition data of the Android device and the audio acquisition data of the Android device, encoding the screen acquisition data to obtain a video data packet, encoding the audio acquisition data to obtain an audio data packet, encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream, and sending the encapsulated data stream to the Windows computer according to the file descriptor, the screen projection data of the Android device can be effectively encapsulated and sent to the Windows computer, thereby facilitating the transmission of the screen projection data.

[0201] Embodiment 7

[0202] Fig.16 The block diagram of the screen projection link establishment system 1000 according to the seventh embodiment of the present application is schematically shown. Fig.16 As shown, the system 1000 may include a Windows computer 1010 and an Android device 1020, wherein the Windows computer 1010 and the Android device 1020 are connected via a USB cable, wherein:

[0203] The windows computer 1010 is used to install a first USB library and a second USB library for the Android device 1020 on the windows computer 1010, send a target instruction to the Android device 1020 to switch the Android device 1020 to the accessory mode, and reinstall the first USB library for the Android device 1020 when the Android device 1020 switches to the accessory mode, receive the screen projection data transmitted by the Android device 1020 through the USB cable through the first USB library, and transmit the screen projection data to the first application of the windows computer 1010 through the second USB library for screen projection playback;

[0204] The Android device 1020 is used to receive the target instruction, switch to the accessory mode according to the target instruction, enumerate the accessory list of the Android device 1020, select the Windows computer 1010 from the accessory list, and obtain the file descriptor for communicating with the Windows computer 1010, use the second application to obtain the projection data of the Android device 1020, and send the projection data to the Windows computer 1010 according to the file descriptor.

[0205] In an optional embodiment, the windows computer 1010 is also used for:

[0206] Get device information through the device management API;

[0207] Enumerating devices according to the device information to obtain a list of portable devices connected to the Windows computer;

[0208] The Android device 1020 is selected according to the portable device list, and the first USB library and the second USB library are installed for the Android device 1020 .

[0209] In an optional embodiment, the Android device 1020 is further configured to:

[0210] Acquire screen acquisition data of the Android device 1020, and acquire audio acquisition data of the Android device 1020;

[0211] Encoding the screen acquisition data to obtain a video data packet;

[0212] Encoding the audio acquisition data to obtain an audio data packet;

[0213] Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream;

[0214] The encapsulated data stream is sent to the Windows computer 1010 according to the file descriptor.

[0215] Embodiment 8

[0216] Fig.17 The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing the method for establishing a screen projection link according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.17 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them:

[0217] The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes a flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10010 may be an internal storage module of the computer device 10000, such as a hard disk or a memory of the computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 may also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store an operating system and various application software installed on the computer device 10000, such as program code of a method for establishing a screen projection link, etc. In addition, the memory 10010 can also be used to temporarily store various types of data that have been output or are to be output.

[0218] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0219] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0220] It should be pointed out that Fig.17 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0221] In this embodiment, the screen projection link establishment method stored in the memory 10010 can also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0222] Embodiment 9

[0223] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the screen projection link establishment method in the embodiment are implemented.

[0224] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEP ROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the screen projection link establishment method in the embodiment. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0225] Embodiment 10

[0226] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0227] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0228] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for establishing a screen projection link, characterized in that: Applied to a Windows computer, the Windows computer is connected to an IOS device via a USB cable, and the method comprises: Installing an Apple service program on the Windows computer, and creating a target service on the Windows computer through the Apple service program; Installing a target software library on the Windows computer, wherein the target software library is used to perform local communication and management with the IOS device; Receiving, through the target service, the screen projection data transmitted by the iOS device via the USB cable; The projection data is transmitted to the first application of the Windows computer through the target software library for projection and playback.

2. The method according to claim 1, characterized in that The method further comprises: When the IOS device is connected to the Windows computer for the first time, installing a driver for the IOS device on the Windows computer; Enumerate the IOS device list connected to the Windows computer, and select the IOS device from the IOS device list; The connection function of the target software library is called to connect to the IOS device, and a target port number is specified so that the IOS device can start a TCP server according to the target port number to transfer the screen projection data.

3. A method for establishing a screen projection link, characterized in that: Applied to an IOS device, the IOS device is connected to a Windows computer via a USB cable, and the method comprises: When a second application of the IOS device is started, monitoring a target port number of a connection between the Windows computer and the IOS device, wherein the second application is used to obtain screen projection data; When the target port number is monitored, starting a TCP server in the IOS device according to the target port number; Acquire the screen projection data of the IOS device through the second application; The projection data is transferred through the TCP server and sent to the Windows computer through the USB cable based on the usbmux protocol.

4. The method according to claim 3, characterized in that The screen projection data includes video frame data, microphone audio data and playback audio data collected by the IOS device, and the screen projection data is transferred through the TCP server and sent to the Windows computer through the USB cable based on the USBmux protocol, including: Rendering the video frame data, and performing predetermined processing on the video frame data during the rendering process; Encoding the processed video frame data to obtain a video data packet; Mixing the microphone audio data and the playback audio data to obtain mixed audio data; Encoding the mixed audio data to obtain an audio data packet; Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream; The encapsulated data stream is transferred through the TCP server and sent to the Windows computer through the USB cable based on the USBmux protocol.

5. A screen projection link establishment system, characterized in that: It includes a Windows computer and an IOS device, wherein the Windows computer and the IOS device are connected via a USB cable; The windows computer is used to create a target service through an installed Apple service program, receive the screen projection data transmitted by the iOS device via the USB cable through the target service, and transmit the screen projection data to the first application of the windows computer through an installed target software library for screen projection playback; The IOS device is used to listen to the target port number of the connection between the Windows computer and the IOS device when the second application in the IOS device is started. When the target port number is listened to, the TCP server is started on the IOS device according to the target port number, the screen projection data of the IOS device is obtained through the second application, the screen projection data is transferred through the TCP server, and sent to the Windows computer through the USB cable based on the usbmux protocol.

6. A method for establishing a screen projection link, characterized in that: Applied to a Windows computer, the Windows computer is connected to an Android device via a USB cable, and the method comprises: Installing a first USB library and a second USB library for the Android device on the Windows computer; Sending a target instruction to the Android device to switch the Android device to an accessory mode; When the Android device is switched to the accessory mode, reinstalling the first USB library for the Android device; Receiving, through the first USB library, the screen projection data transmitted by the Android device via the USB cable; The screen projection data is transmitted to the first application of the Windows computer through the second USB library for screen projection and playback.

7. The method according to claim 6, characterized in that The step of installing the first USB library and the second USB library for the Android device on the Windows computer includes: Get device information through the device management API; Enumerating devices according to the device information to obtain a list of portable devices connected to the Windows computer; The Android device is selected according to the portable device list, and a first USB library and a second USB library are installed for the Android device.

8. A method for establishing a screen projection link, characterized in that: Applied to an Android device, the Android device is connected to a Windows computer via a USB cable, and the method comprises: receiving a target instruction sent by the Windows computer, and switching the Android device to an accessory mode according to the target instruction; Enumerate devices to obtain an accessory list of the Android device, select the Windows computer from the accessory list, and obtain a file descriptor for communicating with the Windows computer; The projection data of the Android device is obtained by using a second application, and the projection data is sent to the Windows computer according to the file descriptor.

9. The method according to claim 8, characterized in that The method of obtaining the projection data of the Android device by using the second application program and sending the projection data to the Windows computer according to the file descriptor comprises: Acquire screen acquisition data of the Android device, and acquire audio acquisition data of the Android device; Encoding the screen acquisition data to obtain a video data packet; Encoding the audio acquisition data to obtain an audio data packet; Encapsulating the video data packet and the audio data packet to obtain an encapsulated data stream; The encapsulated data stream is sent to the windows computer according to the file descriptor.

10. A screen projection link establishment system, characterized in that: Comprising a Windows computer and an Android device, wherein the Windows computer and the Android device are connected via a USB cable; The windows computer is used to install a first USB library and a second USB library for the Android device on the windows computer, send a target instruction to the Android device to switch the Android device to an accessory mode, and when the Android device switches to the accessory mode, reinstall the first USB library for the Android device, receive the screen projection data transmitted by the Android device through the USB cable through the first USB library, and transmit the screen projection data to the first application of the windows computer through the second USB library for screen projection playback; The Android device is used to receive the target instruction, switch to the accessory mode according to the target instruction, enumerate the accessory list of the Android device, select the window computer from the accessory list, and obtain a file descriptor for communicating with the windows computer, use a second application to obtain the projection data of the Android device, and send the projection data to the window computer according to the file descriptor.

11. A computer device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4 or 6-9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1-4 or 6-9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 4 or 6 to 9 are implemented.

Images