Data processing method and related device

Abstract

The present application provides a data processing method and related device. This method can change the default data encapsulation format of the tablet for reverse control data by leveraging the data encapsulation module that supports UIBC HID encapsulation capabilities carried by the upper wireless display module of the tablet and the expansion function provided by the underlying chip. This allows the tablet to encapsulate the reverse control data in UIBC HID format using the encapsulation capability carried by the upper wireless display module, so that the tablet can encapsulate the reverse control data in UIBC HID format through the upper wireless display module and then send the reverse control data to the computer. In this way, the computer can successfully parse and identify the reverse control data encapsulated in UIBC HID format and respond accordingly to the reverse control data, thereby achieving the user's purpose of reverse controlling the computer through the tablet.

Description

Technical Field

[0001] The present application relates to the field of terminal screen projection technology, and in particular to data processing methods and related devices. Background Art

[0002] With the development of conference terminals, standard screen projection systems such as Miracast and AirPlay can realize screen projection and reverse control systems. Currently, many terminals can also support private wireless screen projection and reverse control functions, so that conference terminals can solve the problem of users being constrained by cable length during use, and the reverse control function can free users from their personal computers.

[0003] In some scenarios, after projecting the computer screen onto a tablet, users may want to control the computer through the tablet. However, currently, tablet devices, acting as a secondary screen for the computer, cannot control the computer during the projection process, which undoubtedly brings inconvenience to users.

[0004] Therefore, how to enable tablet devices to control computers during the screen projection process is an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to provide a data processing method and related devices, which can change the default data encapsulation form of the tablet for reverse control data with the help of the data encapsulation module that supports the UIBC HID form encapsulation capability carried by the upper wireless display module of the tablet and the expansion function provided by the underlying chip, so that the tablet can encapsulate the reverse control data in the UIBC HID form with the help of the encapsulation capability carried by the upper wireless display module, so that the tablet can encapsulate the reverse control data in the UIBC HID form through the upper wireless display module, and then send the reverse control data to the computer. In this way, the computer can successfully parse and identify the reverse control data encapsulated in the UIBC HID form, and respond accordingly to these reverse control data, so as to achieve the purpose of the user reverse controlling the computer through the tablet.

[0006] In the first aspect, the present application provides a data processing method, which is applied to a first terminal, wherein the first terminal includes a data encapsulation module and a chip, and the chip includes a wireless screen projection management module and a port multiplexing module. The method includes: the port multiplexing module obtains a handle of a reverse control path corresponding to a first reverse control port, and the first reverse control port is a port for transmitting reverse control data negotiated by the wireless screen projection management module and the second terminal during the screen projection process between the first terminal and the second terminal. During the screen projection process, the first terminal is the receiving end and the second terminal is the sending end; the data encapsulation module encapsulates the reverse control data obtained by the first terminal in UIBC HID format, and then sends the reverse control data to the port multiplexing module; the port multiplexing module sends the reverse control data to the second terminal through the first reverse control port.

[0007] In this method, the data encapsulation module can be used to encapsulate the counter-control data in UIBC HID format. The chip does not have the ability to encapsulate the counter-control data in UIBC HID format, and the second terminal is a terminal that cannot parse the counter-control data encapsulated in UIBC generic format.

[0008] In addition, the data encapsulation module (including the module to which the data encapsulation module belongs) and the chip can be produced by different electronic manufacturers. In the prior art, according to the logic circuit designed in the chip, after the chip and the second terminal establish a reverse control path, the chip will encapsulate the reverse control data in a generic form by default, and send the encapsulated reverse control data to the second terminal through the first reverse control port to reverse control the second terminal. However, for the second terminal that cannot parse the reverse control data encapsulated in the UIBC generic form, since it cannot correctly identify and parse the reverse control data encapsulated in the UIBC generic form, the second terminal cannot respond according to the reverse control operation of the user on the first terminal, and the purpose of reverse controlling the first terminal through the first terminal cannot be achieved.

[0009] In the present application, in order to change the default encapsulation method of the reverse control data by the first terminal, the first terminal can obtain the reverse control path occupied by the chip through the port multiplexing module in the chip, that is, the reverse control path corresponding to the first reverse control port. After obtaining the reverse control path, it is opened to the port multiplexing module in the same process as the wireless screen projection management module. Correspondingly, in the subsequent process, the unencapsulated reverse control data will also be transmitted to the data encapsulation module, and the data encapsulation module will encapsulate the reverse control data in the form of UIBC HID. Afterwards, the wireless display module can call the interface of the port multiplexing module to send the encapsulated reverse control data to the port multiplexing module, and the port multiplexing module will call the corresponding interface to send the encapsulated reverse control data to the second terminal via the above-mentioned reverse control path. The second terminal can parse the received reverse control data and successfully respond to it, and the purpose of the user's reverse control of the second terminal through the first terminal can be achieved.

[0010] In combination with the first aspect, in a possible implementation, the data encapsulation module includes a wireless display module, and before the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port, the method further includes: the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module; the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port, including: when the wireless screen projection management module does not close the first reverse control port, the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port.

[0011] In an embodiment of the present application, the wireless display module included in the first terminal has the ability to independently establish a reverse control path with the second terminal, and the wireless display module also includes a data encapsulation module capable of encapsulating reverse control data in UIBC HID format. In other words, in addition to using the reverse control path occupied by the chip to send reverse control data encapsulated in UIBC HID format to the second terminal through port multiplexing technology, if the first terminal can release the occupation of the reverse control port by the underlying chip, the first terminal can renegotiate the port used to transmit reverse control data with the second terminal through the wireless display module, and can directly use the data encapsulation module to encapsulate the reverse control data in UIBC HID format, and then directly send it to the second terminal through the reverse control path occupied by the wireless display module.

[0012] Therefore, in an embodiment of the present application, in this implementation mode, before the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port, the wireless display module can first send an instruction to the wireless screen projection management module for closing the first reverse control port. If the wireless screen projection management module can successfully close the first reverse control port, the wireless display module can transmit the encapsulated data to the second terminal through the self-built reverse control path, which can effectively reduce the link length and save system resources. Of course, if the wireless screen projection management module cannot successfully close the first reverse control port, the port multiplexing module can obtain the handle of the reverse control path corresponding to the first reverse control port, and transmit the encapsulated data through the reverse control path occupied by the multiplexing chip, which can also achieve the purpose of successfully reverse controlling the second terminal through the first terminal.

[0013] In combination with the first aspect, in a possible implementation, before the port multiplexing module obtains the handle of the path corresponding to the first reverse-control port, the method also includes: when the wireless screen projection management module does not close the first reverse-control port, the wireless screen projection management module sends a first instruction to the wireless display module, and the first instruction is used to notify the wireless display module that the first reverse-control port is not closed, and to control the wireless display module to send a multiplexing instruction to the port multiplexing module, and the multiplexing instruction is used to control the port multiplexing module to obtain the handle of the reverse-control path corresponding to the first reverse-control port.

[0014] In this embodiment, when the above-mentioned reverse control port is not successfully closed, the wireless screen projection management module can send the first instruction to the wireless display module. The first instruction can be used to notify the wireless display module that the reverse control port has failed to be closed (or the wireless screen projection management module contained in the underlying chip currently still occupies the reverse control path between the first terminal and the second terminal), and is used to control the wireless display module to send an instruction for obtaining the reverse control path to the port multiplexing module. After the port multiplexing module obtains the reverse control path, it means that the port multiplexing module has obtained the right to use the above-mentioned reverse control path. In the subsequent process, the port multiplexing module can send the reverse control data encapsulated in the form of UIBC HID to the computer by multiplexing the reverse control path occupied by the wireless screen projection management module.

[0015] In combination with the first aspect, in a possible implementation, the port multiplexing module and the wireless screen projection management module are modules in the same process, and the port multiplexing module obtains the handle of the path corresponding to the first reverse control port, including: the port multiplexing module queries the Socket handle in the open state according to the process number of the process to which the port multiplexing module belongs; the port multiplexing module determines the handle of the Socket handle in the open state whose corresponding port number is a TCP port number and a reverse control port number as the handle of the path corresponding to the first reverse control port.

[0016] In this embodiment, the port multiplexing module and the screen projection management module belong to the same process.

[0017] In order to obtain the handle of the path corresponding to the above-mentioned first reverse control port, in this embodiment, the port multiplexing module can obtain all the open socket handles of the above-mentioned process in the kernel of the first terminal according to the process ID of the process to which the port multiplexing module belongs, and send these open socket handles to the port multiplexing module.

[0018] Since the tablet and the computer are in a normal screen projection state at this time, and a reverse control channel has been created between the computer and the tablet, all the aforementioned open handles include handles for the screen projection path and handles for the reverse control path, and may also include other handles. It can be understood that Scoket is a combination of an IP address and a port number. For the service port in the TCP / IP protocol, the port number can be used to identify the service provided by a specific process in a computer. Therefore, after obtaining the socket handle sent by the kernel, the port multiplexing module can determine whether the port number contained in each socket handle is a TCP port, and further determine whether the port number is a reverse control port if the port number is a TCP port. If there is a socket handle, the port multiplexing module determines that the port corresponding to the handle is a TCP port and a reverse control port, then the port multiplexing module can save the handle. After saving the handle, it means that the port multiplexing module has obtained the reverse control path occupied by the wireless screen projection management module, that is, the port multiplexing module has obtained the use right of the path corresponding to the first reverse control port.

[0019] In combination with the first aspect, in a possible implementation, before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method also includes: the wireless screen projection management module determines whether the chip has the ability to encapsulate the reverse control data in UIBC HID format; before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method includes: when the wireless screen projection management module determines whether the chip does not have the ability to encapsulate the reverse control data in UIBC HID format, the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module.

[0020] In some embodiments, the chip contained in the first terminal may include a submodule capable of encapsulating the reverse control data in the UIBC HID format. In this case, after the reverse control port number is negotiated between the wireless screen projection management module in the chip and the computer and a reverse control path is established, the first terminal can directly use the chip (the submodule contained therein) to encapsulate the reverse control data in the UIBC HID format and send it to the computer through the reverse control path occupied by the chip. Then, the wireless screen projection management module does not need to close the reverse control port number that has been negotiated with the computer. Therefore, in this embodiment, after the wireless screen projection manager module receives the instruction sent by the wireless display module to close the reverse control port, the wireless screen projection management module can first determine whether the chip has the ability to encapsulate the data in the UIBC HID format. If the chip does not have the ability to encapsulate the reverse control data in the UIBC HID format, the wireless screen projection management module will respond to the instruction for closing the first reverse control port and try to close the reverse control port it occupies. In this way, the first terminal can be prevented from performing meaningless operations, further saving the system resources of the first terminal.

[0021] In combination with the first aspect, in a possible implementation manner, the first terminal is a tablet, and the second terminal is a computer.

[0022] In the second aspect, the present application provides a data processing method, which is applied to a first terminal, wherein the first terminal includes a wireless display module and a chip, the wireless display module includes a data encapsulation module, and the chip includes a wireless screen projection management module, the method including: the wireless display module sends an instruction to the wireless screen projection management module for closing a first reverse control port, the first reverse control port being a port for transmitting reverse control data negotiated by the wireless screen projection management module and the second terminal during the screen projection process between the first and second terminals, and the first terminal is a receiving end and the second terminal is a sending end during the screen projection process; when the wireless screen projection management module closes the first reverse control port, the wireless display module sends a reconnection request instruction to the second terminal, and the reconnection request instruction is used to request the second terminal to renegotiate the second reverse control port with the wireless display module; after the wireless display module encapsulates the reverse control data obtained by the first terminal in UIBC HID format, the wireless display module sends the reverse control data to the second terminal through the second reverse control port.

[0023] In this method, the data encapsulation module can be used to encapsulate the counter-control data in UIBC HID format. The chip does not have the ability to encapsulate the counter-control data in UIBC HID format, and the second terminal is a terminal that cannot parse the counter-control data encapsulated in UIBC generic format.

[0024] In this method, the wireless display module contained in the first terminal has the ability to independently establish a reverse control path with the second terminal, and the wireless display module also includes a data encapsulation module that can encapsulate the reverse control data in the form of UIBC HID. That is to say, if the first terminal can release the occupation of the reverse control port by the underlying chip, the first terminal can renegotiate the port used to transmit the reverse control data with the second terminal through the wireless display module, and can directly use the data encapsulation module to encapsulate the reverse control data in the form of UIBC HID, and then directly send it to the second terminal through the reverse control path occupied by the wireless display module. It can be understood that in this method, the reverse control data received by the second terminal is also encapsulated in the form of UIBC HID, so the second terminal can parse the received reverse control data and successfully respond to it, and the purpose of the user's reverse control of the second terminal through the first terminal can be achieved.

[0025] In combination with the second aspect, in a possible implementation, before the wireless display module sends a reconnection request instruction to the second terminal, the method also includes: when the wireless screen projection management module closes the first reverse control port, the wireless screen projection management module sends a second instruction to the wireless display module, and the second instruction is used to notify the wireless display module that the first reverse control port has been closed, and is used to control the wireless display module and send the reconnection request instruction to the second terminal.

[0026] In this embodiment, when the first reverse control port is successfully closed, the wireless screen projection management module sends the second instruction to the wireless display module. The second instruction is used to notify the wireless display module that the reverse control port has been successfully closed (or the wireless screen projection management module contained in the underlying chip has currently released the occupation of the reverse control path between the first terminal and the second terminal), and is also used to control the wireless display module to subsequently autonomously negotiate the second reverse control port number with the computer and establish a reverse control path.

[0027] After the wireless display module receives the above-mentioned second instruction and determines based on the second instruction that the current chip has closed the reverse control port negotiated with the computer before the wireless screen projection management module, the wireless display module can try to independently establish a reverse control path with the computer. Therefore, in this embodiment, the wireless display module can send the above-mentioned reconnection request instruction to the computer, requesting the computer to re-conduct an RTSP session with it, and the session content includes the second reverse control port number. It can be understood that the reverse control path corresponding to the second port number is directly occupied by the wireless display module, so the wireless display module can encapsulate the reverse control data in the form of UIBC HID, and send the encapsulated reverse control data directly to the computer through the reverse control path it occupies.

[0028] In conjunction with the second aspect, in a possible implementation manner, the first terminal is a tablet, and the second terminal is a computer.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code comprises computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, and the method in the second aspect or any possible implementation of the second aspect.

[0030] In a fourth aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, and the method in the second aspect or any possible implementation of the second aspect.

[0031] In a fifth aspect, a computer program product comprising instructions, when the computer program product is run on an electronic device, enables the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, and the method in the second aspect or any possible implementation of the second aspect.

[0032] In the sixth aspect, a computer-readable storage medium is provided, comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method in the first aspect or any possible implementation of the first aspect, and the method in the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A software architecture diagram of a screen projection device provided in an embodiment of the present application;

[0034] Figure 2 A software architecture diagram of a screen projection device provided in an embodiment of the present application;

[0035] Figure 3 A flowchart of data interaction between a tablet and a computer during screen projection provided in an embodiment of the present application;

[0036] Figure 4 A flowchart of data interaction between a tablet and a computer during screen projection provided in an embodiment of the present application;

[0037] Figure 5 This is an architectural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The terms used in the following examples of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and encompasses any or all possible combinations of one or more of the listed items.

[0039] To facilitate understanding, the following first introduces the relevant terms involved in the embodiments of this application.

[0040] (1) Miracast

[0041] Miracast is a wireless display standard developed by the Wi-Fi Alliance in 2012, based on Wi-Fi Direct. Based on Miracast, the Wi-Fi Alliance launched another important technical specification, Wi-Fi P2P (Peer-to-Peer). P2P is a device discovery method specified by the Miracast standard that supports multiple Wi-Fi devices connecting to each other without a wireless access point (AP). Two or more devices that support the Miracast standard and have P2P capabilities can establish a wireless communication connection, after which they can send and receive audio and video data.

[0042] During wireless screen projection, devices can be divided into two categories: a transmitter (also known as the Source) and a receiver (also known as the Sink). The transmitter is the device that streams multimedia content to the receiver via a WiFi link, while the receiver receives and renders the multimedia content sent from the transmitter via the WiFi link.

[0043] After establishing a communication link, data can be transmitted between the sender and receiver by establishing a socket. A socket is data describing an IP address and port number, and serves as the handle for the communication link between the sender and receiver. A socket is the basic operating unit for network communications that supports the TCP / IP protocol. It is an abstract representation of an endpoint in the network communication process and contains five pieces of information necessary for network communication: the protocol used for the connection; the IP address of the local host; the local and remote protocol ports; the IP address of the remote host; and the protocol port number of the remote process.

[0044] (2) Reverse control

[0045] In this application, the display screen of the sending end can be called the "main screen" and the display screen of the receiving end can be called the "secondary screen". During the wireless screen projection process, the user can control the secondary screen by operating the user on the main screen, such as clicking, dragging, etc.; in addition, in this application, the user can also reversely control the main screen by operating the secondary screen. This operation of controlling the sending end by the receiving end during the screen projection process is called "reverse control" (hereinafter referred to as "reverse control").

[0046] For example, in this application, when the user projects the computer screen onto the tablet, the computer is the transmitter and the tablet is the receiver. After establishing a reverse control channel between the computer and the tablet through Miracast, the user can reversely control the computer through the tablet. For example, the user can write and annotate on the tablet screen by touching or clicking, and the screen displayed on the computer will also change with the user's operation of the tablet. Ignoring a certain delay, the screen displayed on the computer will remain synchronized with the screen displayed on the tablet.

[0047] (3) Anti-control data encapsulation

[0048] Data encapsulation is to map data into the payload of a certain encapsulation protocol, then fill in the packet header of the corresponding protocol to form a data packet of the encapsulation protocol and complete rate adaptation.

[0049] To protect data and device security, once a reverse control path is established between the sender and receiver, the receiver must encapsulate the reverse control data. Once the reverse control data is encapsulated, the receiver sends it to the sender via the reverse control path, completing the reverse control of the sender.

[0050] Currently, there are two types of anti-control data encapsulation: UIBC generic and UIBC HID. UIBC stands for User Input Back Channel (UIBC), which transmits user operations to the sender and receiver via TCP. "Generic" refers to hardware-independent operations, such as mouse clicks, button clicks, touch clicks, and zooming. "HID" stands for Human Interface Device (HID), which includes infrared, USB, Bluetooth, Wi-Fi, joysticks, remote controls, and more.

[0051] Computers running the Windows operating system generally only support UIBC HID data encapsulation. This means that when a computer is used as a transmitter to project its screen onto another receiver, such as an Android tablet, if a user wishes to control the computer from the tablet, the tablet must encapsulate the control data in UIBC HID format. This ensures that upon receiving the encapsulated control data, the computer can correctly identify the receiving device through the HID report descriptor and parse the subsequent control data in the format needed to successfully respond to the user's control request. However, the communication channel between the tablet and the computer is established by the tablet's chip, which does not support UIBC HID encapsulation of control data, only UIBC generic encapsulation. Although the data encapsulation module within the tablet's upper-layer wireless display module can encapsulate control data in UIBC HID format, the communication channel between the tablet and the computer is already occupied by the chip. Therefore, the tablet cannot encapsulate the data in UIBC HID format through the wireless display module, preventing the user from controlling the computer from the tablet. The main purpose of this application is to adaptively process the reverse control path of the Pad so that when it acts as a receiving end, it can normally encapsulate the reverse control data in the form of UIBC HID through the data encapsulation module contained in the upper wireless display module, thereby successfully reverse controlling the computer.

[0052] With the development of conference terminals, standard screen projection systems such as Miracast and AirPlay can implement screen projection and reverse control systems. Many current terminals also support proprietary wireless screen projection and reverse control functions, allowing conference terminals to solve the problem of users being constrained by cable length during use. For example, when users are holding a meeting in a conference room or conducting a knowledge competition, the presenter can project the screen of their laptop onto one or more electronic devices so that participants can view the content displayed on the presenter's computer through these electronic devices. After the user projects the screen of their computer onto another electronic device, they may need to reverse control the computer through this electronic device. For example, in a meeting, after projecting the screen of a tablet onto a mobile phone, if the user wants to control the computer from a distance (for example, switching the page of the currently displayed PowerPoint presentation on the computer or annotating on a page in the PowerPoint presentation), the user can use the mobile phone to achieve reverse control of the computer (for example, by clicking or drawing on the mobile phone's display).

[0053] However, in some screen projection scenarios, when the sending end is a computer and the receiving end is a pad, when the computer projects the screen to the tablet, due to the computer's requirements for the reverse control data packaging format, the tablet may not be able to package the reverse control data according to the format specified by the computer, resulting in the user being unable to reverse control the computer through the tablet, causing inconvenience to the user.

[0054] like Figure 1 As shown, the tablet 10 operates under the Android system, and the computer 11 operates under the Windows system. Both the tablet 10 and the computer 11 are electronic devices that support the Miracast standard and P2P functions. Specifically, the tablet 10 may include a wireless display module 101, an application framework layer, a chip 102, and an input device. Among them:

[0055] The wireless display module 101 is a module responsible for upper-layer services during the screen projection process. Specifically, during the screen projection process, the wireless display module 101 is responsible for providing the user with a corresponding operation interface. After the user triggers the screen projection function through operation on the operation interface, the wireless display module 101 is also used to establish a Wi-Fi P2P channel, and further call the corresponding interface provided by the underlying chip 102, and send a request connection signal based on the established Wi-Fi P2P channel, so that other devices can receive the request connection signal and discover the tablet 10. After that, the two parties can transmit data based on the TCP protocol. In addition, if an abnormality occurs during the data transmission between the two parties, the wireless display module 101 will also be responsible for outputting abnormal prompts and other operations to the user. In this application, the wireless display module 101 may include a data encapsulation module 101a. When the tablet 10 is used as the receiving end, the data encapsulation module 101a supports the encapsulation of the reverse control data obtained by the tablet 10 in the form of UIBC HID.

[0056] The application framework layer may include an input manager and an event manager. During the reverse control process, when the user interacts with the tablet through an input device (for example, the user operates on a touch screen or the keyboard on the pad), the input manager will collect these events, and further transmit them to the event manager, and encapsulate them into InputEvent (that is, the object created by the event manager for these events, that is, reverse control data). During the reverse control process, the above-mentioned InputEvent will be transmitted to the corresponding encapsulation module for encapsulation. Finally, these encapsulated reverse control data will be transmitted to the computer 11 through the reverse control path, and the computer 11 can parse these reverse control data, and the computer 11 can know the specific operation content of the user on the tablet 10 and respond based on the content.

[0057] Chip 102 may include a wireless screen projection management module and a reverse control module 102a. Generally speaking, during the screen projection process, the wireless screen projection management module is responsible for negotiating with computer 11. The negotiation content may include the encoding formats supported by computer 11 and tablet 10, as well as the port number for user reverse control data transmission. When tablet 10 is the receiving end, reverse control module 102a can be used to encapsulate the reverse control data in UIBC generic format.

[0058] However, most computers currently need to use HID report descriptors to identify reverse control data. Specifically, HID report descriptors are generally used to describe to the sender what type of device the receiver is, and for the sender to determine the parsing format of subsequent control data. For example, the reverse control data sent by the tablet 10 as the receiver to the computer 11 contains coordinate information of four screens a1, a2, a3, and a4. However, after receiving this information, the computer 11 as the sender does not recognize this information and needs to use HID report descriptors to determine that a1 and a2 represent the horizontal coordinates, and a3 and a4 represent the vertical coordinates in these four screen coordinate information. Because the working mode of the chip 102 in the tablet 10 (the working mode is generally determined by the logic circuit in the chip 102, and the logic circuit is designed and specified by the manufacturer of the chip 102) is set before it leaves the factory. Therefore, in the absence of intervention, the tablet 10 will, by default, encapsulate the reverse control data in generic form through the reverse control module 102a in the chip 102, and send the encapsulated reverse control data to the computer 11 through the port previously negotiated between the wireless screen projection management module and the computer 11, so as to reverse control the computer 11. Furthermore, the reverse control data transmitted through the above-mentioned negotiated port is generally encapsulated by the wireless screen projection management module in UIBC generic form rather than UIBC HID form by default. Therefore, these encapsulated reverse control data will lack the necessary HID report descriptor symbols. Even if the computer 11 successfully receives the reverse control data sent by the tablet 10, the computer 11 will not be able to correctly identify and parse it. The computer 11 will not be able to respond to the reverse control operation of the user on the tablet 10, and the purpose of reverse controlling the computer 11 through the tablet 10 will not be achieved.

[0059] In response to the above problems, the present application provides a data processing method. This method can change the default data encapsulation format of the tablet for reverse control data by means of a data encapsulation module that supports UIBC HID encapsulation capabilities carried by the upper wireless display module of the tablet and the expansion function provided by the underlying chip, so that the tablet can encapsulate the reverse control data in UIBC HID format with the help of the encapsulation capability carried by the upper wireless display module, so that the tablet can encapsulate the reverse control data in UIBC HID format through the upper wireless display module and then send the reverse control data to the computer. In this way, the computer can successfully parse and identify the reverse control data encapsulated in UIBC HID format and respond accordingly to the reverse control data, so as to achieve the purpose of the user controlling the computer through the tablet.

[0060] Next, we will first introduce two different forms of data processing methods provided by the embodiment of the present application in combination with the specific architecture of the tablet provided by the embodiment of the present application and the above related descriptions. Figure 2 .

[0061] like Figure 2 As shown, the tablet 20 works under the Android system, the computer 21 works under the Windows system, and both the tablet 20 and the computer 21 are electronic devices that support the Miracast standard and P2P function. Specifically, the tablet 20 may include a wireless display module 201, an application framework layer, a chip 202, and an input device. The specific functions of these modules in the screen projection process can be referred to the aforementioned Figure 1 The relevant descriptions are not repeated here. It should be noted that in the embodiments of this application:

[0062] The wireless display module 201 may include a data encapsulation module 201a and a reverse control transmission management module 201b. When the tablet 20 is used as a receiving end, the data encapsulation module 201a supports encapsulating the reverse control data obtained by the tablet 20 in the UIBC HID format.

[0063] In addition to the wireless screen projection management module 202a and the reverse control module 202b, the chip 202 may also include a port multiplexing module 202c and a reverse control port opening module 202d. The port multiplexing module 202c and the wireless screen projection management module 202a are in the same process. It is understandable that when a module creates and occupies a Socket path, another module in the same process as the module can also reuse this path, and the module that reuses the path can provide a corresponding interface, and other modules on the upper layer of the interface perform data transmission and send the data to other devices through the above-mentioned Socket path. That is to say, during the screen projection process, if the wireless screen projection management module 202a in the tablet 20 creates and occupies the reverse control socket path (hereinafter referred to as the "reverse control path"), if the reverse control port opening module 202d opens the use permission of the reverse control port, then the port multiplexing module 202c in the same process as the wireless screen projection management module 202a can also reuse the reverse control path, and the port multiplexing module 202c can further provide an interface for data transmission with the upper-level wireless display module (or the sub-modules it contains, such as the data encapsulation module 201a), and obtain reverse control data from the wireless display module through the interface, and then transmit it to the computer 21 through the above-mentioned reverse control path.

[0064] It should be noted in advance that in the embodiment of the present application, when the tablet 20 acts as the receiving end, after a Wi-Fi channel p2p is established between the wireless display module 201 and the computer 21, the tablet 20 will still default to negotiating with the computer 11 through the wireless screen projection management module in the chip 202. The negotiation content includes the port number used for transmitting reverse control data between the tablet 20 and the computer 21. After the negotiation is completed, the wireless display module 201 will control the reverse control port to be open to facilitate the subsequent output of the reverse control data.

[0065] In combination with the above description, it can be seen that without any intervention, the tablet 20 will default to encapsulating the reverse control data in generic form through the reverse control module 202b in the chip 202, and send the encapsulated reverse control data to the computer 21 through the wireless projection management module 202a and the port previously negotiated by the computer 21. Since the reverse control data lacks the necessary HID report descriptor, the computer 21 cannot respond according to the reverse control operation of the user on the tablet 20, and the process of the user reverse controlling the computer 21 through the tablet 20 will fail.

[0066] However, in the present application, after the wireless screen projection management module 202a and the computer 21 have negotiated the port number for transmitting the reverse control data, in order to successfully use the data encapsulation module 201a in the wireless display module 201 to encapsulate the reverse control data, and after encapsulation, the tablet 20 can successfully send the reverse control data to the computer 21 through the negotiated reverse control port, the wireless display module can send an instruction for closing the above-mentioned reverse control port to the wireless screen projection management module 202a in the chip. In the application, the wireless screen projection management module 202a may have two different processing methods for this instruction: 1. If the chip 202 does not implement the logic circuit for closing the reverse control port, then after receiving the above-mentioned instruction, the wireless screen projection management module 202a will not respond to the instruction (or the wireless screen projection management module 202a cannot close the above-mentioned reverse control port as required by the instruction), and the above-mentioned reverse control port remains open. In this case, the reverse control path between the tablet 20 and the computer 21 is still occupied by the chip. At this time, the reverse control port opening module 202d opens the use rights of the reverse control path to other modules in the same process as the wireless screen projection management module 202a, such as the port multiplexing module 202c. The tablet 20 can then search for the socket handle of the reverse control path (the "reverse control path" mentioned here is a TCP connection dedicated to the tablet 20 to send reverse control data to the computer 21, that is, the reverse control socket communication path) through the port multiplexing module 202c in the chip 202, and on this basis, add a send interface and open the interface to the upper-level wireless display module 201. In this way, the data encapsulation module 201a in the upper-level wireless display module 201 can send the encapsulated reverse control data to the chip through the interface after encapsulating the reverse control data in the form of UIBC HID, and send it to the computer 21 through the above-mentioned reverse control path. ②. If there is a logic circuit in the chip 202 that realizes closing the reverse control port, the wireless screen projection management module 202a can successfully respond to the instruction and close the reverse control port after receiving the above instruction. At this time, the chip 202 no longer occupies the reverse control path. Therefore, in this case, the upper wireless display module 201 can renegotiate with the computer 21 the port number for transmitting reverse control data, that is, re-establish the reverse control path with the computer 21 through the reverse control transmission management module 201b in the upper wireless display module 201. Correspondingly, since the reverse control path between the reverse control transmission management module 201b and the computer 21 is not occupied by the bottom layer of the chip, the data encapsulation module 201a in the wireless display module 201 encapsulates the unencapsulated reverse control data in the form of UIBC HID, and then transmits it directly to the computer 21 through the reverse control path between the reverse control transmission management module 201b and the computer 21.

[0067] It can be understood that in the above two data transmission methods, the counter-control data ultimately sent by the tablet 20 to the computer 21 is encapsulated in the form of UIBC HID. Therefore, the computer 21 can successfully identify and parse the counter-control data sent by the tablet 20 and respond accordingly based on the counter-control data. The user's purpose of counter-controlling the computer through the tablet 20 is also achieved.

[0068] In addition, in the tablet 20, the functions of the above modules can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. Figure 2 The structure shown in the figure does not constitute a specific limitation on the electronic device provided in this application. In other embodiments of this application, the tablet 20 may include Figure 2 The present application does not limit the number of modules shown, or the combination or separation of certain modules.

[0069] Next, combine Figure 3 and Figure 4 The interactive flow chart shown illustrates the data processing methods in the above description respectively.

[0070] It should be noted in advance that due to space limitations, Figure 3 and Figure 4 In the figure, some modules include submodules (such as the data encapsulation module, the reverse control transmission management module and the reverse control port opening module included in the wireless display module) are not shown in the figure. Figure 3 and Figure 4 The tablet in the system works under Android. Figure 3 and Figure 4 The computer is running under Windows system, and Figure 3 and Figure 4 The tablet and computer 11 are both electronic devices that support the Miracast standard and P2P function. Specifically, Figure 3 and Figure 4 The tablet and computer can be Figure 2 The tablet 20 and the computer 21, Figure 3 and Figure 4 The wireless display module and chip can be respectively Figure 2 The wireless display module 201 and the chip 202; accordingly, Figure 3 and Figure 4 The wireless screen projection management module and port multiplexing module contained in the chip can be respectively Figure 2 The chip 202 includes a wireless screen projection management module 202a and a port multiplexing module 202c.

[0071] Figure 3 The figure shows the data interaction process between the tablet and the computer when the reverse control path occupied by the chip is not successfully closed. Figure 3 As shown, the data interaction process includes but is not limited to the following steps:

[0072] S101: The wireless display module opens a screen receiving interface.

[0073] S102: Receive the user's screen projection operation in the screen projection sending interface.

[0074] When Wi-Fi is turned on on both the computer and the tablet, the user can also perform corresponding user operations on the tablet (for example, turning on the screen projection function in the "Settings" application) to enable the tablet's wireless display module to display the screen projection receiving interface; after the above-mentioned screen projection receiving interface is turned on, the tablet as the receiving end will periodically send request signals to the outside.

[0075] At the same time, the user can perform corresponding user operations on the computer (such as pressing the "Win+K" keys) to make the computer display the screen projection sending interface; correspondingly, the computer as the sending end will continuously monitor the request signal in the environment.

[0076] When the computer successfully detects the request signal sent by the tablet, it means that the computer has discovered the tablet. At this time, the computer will display a corresponding prompt message to the user to remind the user that the computer has successfully discovered the presence of a device nearby that supports computer screen projection (that is, the above-mentioned tablet), and the computer can display the corresponding connection options for the user to choose whether to project the computer screen to the tablet.

[0077] After the user confirms through the computer that the computer screen will be projected onto the tablet, a P2P connection can be established between the computer and the tablet. Next, the computer and the tablet will negotiate the projection port number for transmitting the projection data, which generally defaults to 7236. After the two have negotiated the port number for transmitting the projection data, it is equivalent to the projection path between the computer and the tablet (the "projection path" and the "reverse control path" mentioned here are TCP connections used for the computer to send projection data to the tablet, that is, the projection socket communication path) being established. Subsequently, the computer can send the projection data it obtains to the tablet through the projection path, and the tablet can render the projection data and output it to the tablet's display screen, thereby displaying the screen synchronously with the computer.

[0078] S103: If the tablet supports reverse control, the computer sends the reverse control port number to the wireless screen projection management module.

[0079] It should be understood that the above-mentioned screen projection port number will also be used to manage and control the real-time streaming protocol (RTSP) session between the computer and the tablet. The specific session content includes some encoding formats supported by both parties in the communication between the computer and the tablet, as well as the parameters used subsequently, such as some RTSP methods supported by the computer and the tablet respectively, and the above-mentioned reverse control port number.

[0080] Specifically, during the conversation between the two parties, the computer will first ask the tablet whether it supports reverse control. If the tablet replies to the computer with a message indicating that it supports reverse control, the computer will send a port number to the tablet (wireless screen projection management module), namely the reverse control port number mentioned above. In some embodiments of the present application, the port corresponding to this port number can be called the "first reverse control port."

[0081] Specifically, the reverse control port number may be 50000.

[0082] S104: The wireless screen projection management module sends a first confirmation message to the wireless display module.

[0083] It is understandable that after receiving the reverse control port number, the chip will assume that the port number has been occupied by the reverse control process (i.e., for the tablet, the reverse control port has been opened). In this case, the wireless display module can send a first confirmation message to the wireless display module, which is used to prompt the wireless display module that the reverse control port has been negotiated between the tablet and the computer, that is, the underlying chip has pre-occupied the reverse control path between the tablet and the computer.

[0084] S105: The wireless display module determines whether the current port is a reverse control port.

[0085] S106: The wireless display module sends a control instruction for closing the reverse control port to the wireless screen projection management module.

[0086] After the wireless display module receives the first confirmation information, it can determine whether the communication path corresponding to the reverse control port number is a reverse control path based on the first confirmation information. If the communication path is a reverse control path, then combined with the above description, it can be seen that the wireless display module can send an instruction to the wireless screen projection management module to close the reverse control port, that is, the wireless display module attempts to release the underlying chip from occupying the reverse control path.

[0087] S107: The wireless screen projection management module fails to close the reverse control port.

[0088] In an embodiment of the present application, the wireless screen projection management module fails to successfully close the reverse control port after receiving the control instruction. For example, the tablet chip does not implement the logic circuit to close the reverse control port, or under some program anomalies, the wireless screen projection display management module may not respond to the control instruction to close the occupied reverse control port.

[0089] If the above-mentioned reverse control port is not successfully closed, the wireless screen projection management module will establish a reverse control path with the computer according to the original working logic, that is, the underlying chip still occupies the reverse control path. Combined with the above description, it can be seen that without intervention, the underlying chip will still use the reverse control module it contains ( Figure 3 (not shown) the reverse control data is encapsulated in UIBC generic format and the encapsulated data is transmitted to the computer through the reverse control path. However, the computer cannot correctly parse and identify the received reverse control data.

[0090] Therefore, in an embodiment of the present application, in order to change the tablet's default data encapsulation format for reverse control data (i.e., not using UIBC generic encapsulation but using UIBC HID encapsulation), so that the computer can successfully identify and parse the reverse control data after receiving it, the tablet can obtain the reverse control path through the port multiplexing module in the chip and open it to the port multiplexing module in the same process as the wireless screen projection management module. Accordingly, in the subsequent process, the unencapsulated reverse control data will also be transmitted to the upper-level wireless display module. After the upper-level wireless display module encapsulates the reverse control data in the UIBC HID format, the encapsulated reverse control data will be sent to the port multiplexing module, and the port multiplexing module will send the encapsulated reverse control data to the computer via the above-mentioned reverse control path.

[0091] Specifically, if the reverse control port is not successfully closed, the tablet and the computer may execute subsequent steps S108 to S118.

[0092] S108: The wireless screen projection management module sends a first instruction to the wireless display module.

[0093] When the above-mentioned reverse control port is not successfully closed, the wireless screen projection management module sends a first instruction to the wireless display module. The first instruction is used to notify the wireless display module that the reverse control port has failed to be closed (or the wireless screen projection management module contained in the underlying chip currently still occupies the reverse control path between the tablet and the computer), and is also used to control the wireless display module to turn on the reverse control function, that is, to control the wireless display module to send an instruction for obtaining the reverse control path to the port multiplexing module (in some embodiments of the present application, the instruction can be referred to as a "multiplexing instruction") to control the port multiplexing module to send the reverse control data encapsulated in UIBC HID format to the computer by multiplexing the reverse control path occupied by the wireless screen projection management module in the subsequent process.

[0094] S109: The wireless display module sends an instruction for obtaining a reverse control path to the port multiplexing module.

[0095] For network devices like tablets and computers, only one application can be bound to an open port at a time. If other applications attempt to use the port, a conflict will occur in the device's system scheduling. Port multiplexing technology allows multiple processes on the same host to share the same port, enabling concurrent listening and processing of connection requests.

[0096] That is to say, in this application, although the wireless screen projection management module of the underlying chip occupies the reverse control port, the upper wireless display module can send instructions for obtaining the reverse control path to the port multiplexing module (the port multiplexing module must be in the same process as the wireless screen projection management module). After the port multiplexing module obtains the reverse control path, it means that the port multiplexing module has obtained the right to use the above-mentioned reverse control path. Therefore, after receiving the reverse control data encapsulated in the form of UIBC HID by the upper wireless display module, the encapsulated reverse control data can be transmitted to the computer through the reverse control path.

[0097] Specifically, the port multiplexing module can obtain the reverse control path occupied by the wireless screen projection management module according to subsequent steps S110 to S115.

[0098] S110: The port multiplexing module sends a query instruction to the kernel.

[0099] S111: The kernel sends the queried handle to the port multiplexing module.

[0100] The kernel is the core of the operating system, responsible for managing the system's processes, memory, device drivers, files, and network systems. For example, the data files involved in the tablet's screen projection process, including handles to the projection path and the reverse control path, etc. Therefore, after receiving the instruction sent by the upper-level wireless display module to obtain the reverse control path, the port multiplexing module can respond to the instruction and send a query instruction to the kernel to query the corresponding handle of the reverse control path.

[0101] Specifically, the kernel can first query the process ID of the process to which the port multiplexing module belongs according to the above query instruction. Combined with the above description, it can be seen that the port multiplexing module and the screen projection management module belong to the same process. Therefore, the kernel can obtain all open socket handles of the process based on the process ID of the port multiplexing module obtained, and send these open socket handles to the port multiplexing module.

[0102] S112: The port multiplexing module determines whether the port corresponding to the Scoket handle is a TCP port.

[0103] S113: The port multiplexing module determines whether the port corresponding to the Scoket handle is a reverse control port.

[0104] S114: When the port corresponding to the Scoket handle is a TCP port and is a reverse control port, save the Socket handle.

[0105] Since the tablet and computer are in a normal screen projection state at this time, and a reverse control channel has been established between the computer and the tablet, all the aforementioned open handles include handles to the screen projection channel and handles to the reverse control channel, and may also include other handles.

[0106] As you can understand, Scoket is a combination of an IP address and a port number. The ports of an IP address are numbered using 16 bits, and there can be a maximum of 65536 ports, which can be marked by port numbers ranging from 0 to 65535.

[0107] For the service port in the TCP / IP protocol, the port number can be used to identify the service provided by a specific process in a computer. For example, the common port number "21" represents the file transfer service, the port number "23" represents the remote login service, and the port number 25 refers to the mail transfer service. Therefore, after obtaining the socket handle sent by the kernel, the port multiplexing module can determine whether the port number contained in each socket handle is a TCP port, and further determine whether the port number is a reverse control port when the port number is a TCP port. If there is a socket handle, the port multiplexing module determines that the port corresponding to the handle is a TCP port and is a reverse control port, then the port multiplexing module can save the handle. After saving the handle, it means that the port multiplexing module has obtained the reverse control path occupied by the wireless screen projection management module, that is, the port multiplexing module has obtained the right to use the reverse control path.

[0108] S115: The port multiplexing module sends second confirmation information to the wireless display module.

[0109] After obtaining the above-mentioned reverse control path, the port multiplexing module can send a second confirmation message to the above-mentioned wireless display module to notify the above-mentioned wireless display module that the port multiplexing module has obtained the reverse control path occupied by the underlying wireless projection management module.

[0110] S116: The wireless display module performs UIBC HID encapsulation on the reverse control data.

[0111] When the user subsequently controls the computer through the tablet, the control data generated by the user operation will not be transmitted to the control module in the chip for UIBC generic packaging, but will be transmitted to the upper-layer wireless display module, and the data packaging module in the wireless display module will encapsulate it in UIBC HID format.

[0112] S117: The wireless display module sends the reverse control data encapsulated in the UIBC HID format to the port multiplexing module.

[0113] S118: The port multiplexing module sends the reverse control data encapsulated in UIBC HID format to the computer.

[0114] After the data encapsulation module in the wireless display module encapsulates the reverse control data in the UIBC HID format, the wireless display module will further send the encapsulated reverse control data to the port multiplexing module. Specifically, after obtaining the reverse control path, the port multiplexing module can add a send interface for data transmission with the upper-layer wireless display module. After the data encapsulation module in the upper-layer wireless display module encapsulates the reverse control data in the UIBC HID format, the wireless display module can call the send interface to send the encapsulated reverse control data to the port multiplexing module.

[0115] Because the port multiplexing module has already obtained the reverse control path occupied by the underlying chip, it can use this reverse control path to send reverse control data encapsulated in UIBC HID format to the computer. The computer can then interpret the received reverse control data and successfully respond to it, achieving the user's goal of reverse control of the computer through the tablet.

[0116] In an embodiment of the present application, when the chip occupies the reverse control path, the tablet can obtain the reverse control path occupied by the bottom layer of the chip through port multiplexing technology. After the reverse control data is encapsulated in the UIBC HID format using the upper-layer infinite display module, the encapsulated reverse control data can still be transmitted to the computer through the reverse control path obtained by the port multiplexing module, so that the computer can successfully parse the reverse control data and respond, and the user can successfully reverse control the computer through the tablet.

[0117] Figure 4 It shows the data interaction process between the tablet and the computer when the reverse control path occupied by the chip is successfully closed.

[0118] like Figure 4 As shown, the data interaction process includes but is not limited to the following steps:

[0119] S201: The wireless display module opens the screen projection receiving interface.

[0120] S202: Receive the user's screen projection operation in the screen projection sending interface.

[0121] S203: If the tablet supports reverse control, the computer sends the reverse control port number to the wireless screen projection management module.

[0122] S204: The wireless screen projection management module sends a first confirmation message to the wireless display module.

[0123] S205: The wireless display module determines whether the current port is a reverse control port.

[0124] S206: The wireless display module sends a control instruction for closing the reverse control port to the wireless screen projection management module.

[0125] For details of steps S201 to S206 , please refer to the aforementioned description of steps S101 to S106 , which will not be repeated here.

[0126] S207: The wireless screen projection management module successfully closes the reverse control port.

[0127] Combined with the above Figure 3 From the relevant instructions, it can be seen that by default, the wireless screen projection management module of the chip and the computer will negotiate the port number for transmitting reverse control data between the two through the RTSP session, that is, the reverse control port number mentioned above. At this time, it can be understood that the reverse control path between the tablet and the computer is occupied by the underlying chip. In the absence of intervention, the chip will work according to its original working method, that is, a reverse control path is established between the wireless screen projection management module and the computer, and the reverse control module in the chip is encapsulated in the form of UIBC generic. Afterwards, even if the computer cannot successfully parse the received reverse control data encapsulated in the form of UIBCgeneric, the tablet will still send the encapsulated data to the computer through the reverse control path occupied by the chip.

[0128] In order to change the default data encapsulation method for reverse control data, in an embodiment of the present application, the wireless display module can also send an instruction to the wireless projection management module to close the above-mentioned reverse control port, that is, the wireless display module can try to release the occupation of the reverse control path by the underlying chip.

[0129] In an embodiment of the present application, the wireless screen projection management module successfully closes the reverse control port upon receiving the control instruction. For example, the tablet chip already includes logic circuitry for closing the reverse control port. In this case, the wireless screen projection display management module can respond to the control instruction and close the occupied reverse control port.

[0130] In the case where the above-mentioned reverse control port is successfully closed, the wireless screen projection management module will not establish a reverse control path with the computer according to the original working logic, that is, the underlying chip will no longer occupy the reverse control path. Therefore, in an embodiment of the present application, in order to change the default data encapsulation form of the tablet for reverse control data (that is, not using UIBC generic form encapsulation but using UIBC HID form encapsulation), so that the computer can successfully identify and parse the reverse control data after receiving the reverse control data, the tablet can renegotiate the reverse control port with the computer through the upper wireless display module and establish a reverse control path. Correspondingly, in the subsequent process, the unencapsulated reverse control data will also be transmitted to the upper wireless display module. After the upper wireless display module encapsulates the reverse control data in the UIBC HID form, the encapsulated reverse control data will be sent to the computer directly by the wireless display module through the reverse control path established by it.

[0131] Specifically, when the reverse control port is successfully closed, the tablet and the computer may execute subsequent steps S208 to S212.

[0132] S208: The wireless screen projection management module sends a second instruction to the wireless display module.

[0133] When the above-mentioned reverse control port is successfully closed, the wireless screen projection management module sends a second instruction to the wireless display module. The second instruction is used to notify the wireless display module that the reverse control port has been successfully closed (or the wireless screen projection management module contained in the underlying chip has currently released the occupation of the reverse control path between the tablet and the computer). It is also used to control the wireless display module to turn on the reverse control function, that is, to control the wireless display module to subsequently negotiate with the computer and establish a reverse control path. It can be understood that the reverse control path is directly occupied by the wireless display module, so the wireless display module can encapsulate the reverse control data in the form of UIBC HID and send the encapsulated reverse control data directly to the computer through the reverse control path it occupies.

[0134] S209: The wireless display module sends a reconnection request instruction to the computer.

[0135] After the wireless display module receives the second instruction and determines based on the second instruction that the current chip has closed the reverse control port previously negotiated between the wireless projection management module and the computer, the wireless display module can attempt to independently establish a reverse control path with the computer. Therefore, in this embodiment of the application, the wireless display module can send the reconnection request instruction to the computer, requesting the computer to re-establish an RTSP session with it, and the session content includes the reverse control port number.

[0136] S210: The computer sends the reverse control port number to the wireless display module.

[0137] The above-mentioned reverse control port number can be the same as the port number negotiated between the aforementioned wireless screen projection management module and the computer, or it can be different from the port number negotiated between the aforementioned wireless screen projection management module and the computer, and this application does not limit this. In some embodiments of the present application, the port corresponding to the reverse control port number sent by the computer to the wireless display module can be called the "second reverse control port."

[0138] Specifically, the reverse control port sent by the computer to the wireless display module may be 50000.

[0139] S211: The wireless display module performs UIBC HID encapsulation on the reverse control data.

[0140] S212: The wireless display module sends the reverse control data encapsulated in the UIBC HID format to the computer.

[0141] When the user subsequently controls the computer through the tablet, the control data generated according to the user's operation will not be sent to the control module in the chip for UIBC generic packaging, but will be sent to the upper-layer wireless display module, and the data packaging module in the wireless display module will encapsulate it in UIBC HID format.

[0142] After the data encapsulation module in the wireless display module encapsulates the reverse control data in the UIBC HID format, since the wireless display module has directly established a reverse control path with the computer, the wireless display module will directly send the data to the computer through the reverse control path it has established.

[0143] Afterwards, the computer can analyze the received counter-control data and successfully respond to it, and the user's purpose of counter-controlling the computer through the tablet is achieved.

[0144] In an embodiment of the present application, when the chip has successfully closed the reverse control port and is no longer occupying the reverse control path, the tablet can directly establish a path between the upper wireless display module and the computer. After the upper wireless display module encapsulates the reverse control data in the UIBC HID format, the wireless display module can directly transmit the encapsulated reverse control data to the computer through the reverse control path that it independently establishes and occupies, so that the computer can successfully parse the reverse control data and respond, and the user can successfully reverse control the computer through the tablet.

[0145] In some embodiments, the chip contained in the tablet provided by this application may include a submodule capable of encapsulating the reverse control data in the form of UIBC HID. In this case, after the reverse control port number is negotiated between the wireless screen projection management module in the chip and the computer and a reverse control path is established, the tablet can directly use the chip (the submodules contained therein) to encapsulate the reverse control data in the form of UIBC HID and send it to the computer through the reverse control path occupied by the chip. Then, the wireless screen projection management module does not need to close the reverse control port number that has been negotiated with the computer. Therefore, in some embodiments, Figure 3 or Figure 4 After the wireless screen projection manager module receives the instruction from the wireless display module to close the reverse control port, the wireless screen projection management module can first determine whether the chip has the ability to encapsulate the data in UIBC HID format. If the chip has the ability to encapsulate the data in UIBC HID format, the wireless screen projection management module does not need to respond to the instruction and does not need to close the reverse control port number that has been negotiated with the computer. If the chip does not have the ability to encapsulate the data in UIBC HID format, the wireless screen projection management module in the tablet will respond to the instruction and attempt to close the reverse control port it occupies and execute subsequent steps.

[0146] In addition, in some embodiments, regardless of whether the chip contained in the tablet provided by the present application has the ability to respond to instructions to close the reverse control port, it can be adaptively selected based on whether the port occupied by the chip is successfully closed. Figure 3 The data processing method shown or Figure 4 The data processing method shown processes the reverse control data. For example, even if the chip of the tablet provided by the present application includes a logic circuit for closing the reverse control port, if the reverse control port occupied by the chip is not successfully closed due to a program exception, the tablet will still send the reverse control data encapsulated in the form of UIBC HID to the computer by reusing the reverse control path occupied by the wireless projection management module; for example, even if the chip of the tablet provided by the present application does not include a logic circuit for closing the reverse control port, if the reverse control port occupied by the chip is closed due to a program exception, the tablet will send the reverse control data encapsulated in the form of UIBC HID to the computer by building a reverse control path through the wireless display module.

[0147] Next, the electronic device provided by the embodiment of the present application is introduced. Figure 5 The structure of the electronic device is exemplarily shown.

[0148] Specifically, the electronic device 100 may be the aforementioned Figure 2-Figure 4 Tablet in.

[0149] The electronic device 100 may include a processor 110, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a wireless screen projection management module 180, a reverse control module 181, a port multiplexing module 182, a reverse control port opening module 183, a display screen 194 and a wireless display module 195, etc.

[0150] It is understood that the structure shown in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include Figure 5 More or fewer components may be shown, or some components may be combined or separated, or the components may be arranged differently. Figure 5 The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0151] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0152] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0153] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0154] The USB interface 130 is an interface that complies with USB standards and may be a MiniUSB interface, a MicroUSB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 and to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0155] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0156] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0157] The power management module 141 is used to connect the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the display 194, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.

[0158] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0159] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0160] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0161] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0162] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0163] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0164] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0165] The wireless display module 195 is a module responsible for upper-layer services during the screen projection process. During the screen projection process, the wireless display module 195 is responsible for providing the user with a corresponding operation interface. After the user triggers the screen projection function through an operation on the operation interface, the wireless display module 195 is also used to establish a Wi-Fi P2P channel with other devices together with the wireless communication module 160, and further call the corresponding interface provided by the processor 110 to send a connection request signal based on the established Wi-Fi P2P channel, so that other devices can receive the connection request signal and discover the electronic device 100. After that, the two parties can perform data transmission based on the TCP protocol. In addition, if an abnormality occurs during the data transmission between the two parties, the wireless display module 195 will also be responsible for outputting abnormal prompts to the user and other operations. In this application, the wireless display module 195 may include a data encapsulation module 195a and a reverse control transmission management module 195b. When the electronic device 100 is the receiving end, the data encapsulation module 195a supports the encapsulation of the reverse control data obtained by the electronic device 100 in the form of UIBC HID. When the wireless screen projection management module 180 does not occupy the reverse control port, the reverse control transmission management module 195b can renegotiate with the computer 21 the port number for transmitting reverse control data and establish a reverse control path with other devices.

[0166] During the screen projection process, the wireless screen projection management module 180 is responsible for negotiating with other devices. The negotiation content may include some encoding formats supported by the electronic device 100 and the port number for the user to transmit reverse control data. When the electronic device 100 is the receiving end, the reverse control module 181 can be used to encapsulate the reverse control data in the UIBC generic format.

[0167] The port multiplexing module 182 is in the same process as the wireless screen projection management module 180. During the screen projection process, if the wireless screen projection management module 180 creates and occupies a reverse control path, and the reverse control port opening module 183 opens the use rights of the reverse control port, the port multiplexing module 182 can also reuse the reverse control path, and the port multiplexing module 182 can further transmit data with the wireless display module 195, for example, obtain encapsulated reverse control data from the wireless display module 195, and then transmit it to other devices through the above-mentioned reverse control path to reverse control other devices.

[0168] In the present application, if the processor 110 does not have the ability to close the reverse control port occupied by the wireless screen projection management module 180, the wireless screen projection management module 180 can start the port multiplexing module 182 and the reverse control port opening module 183 under the control of the wireless display module 195. The port multiplexing module 182 can search for the socket handle of the reverse control path, and on this basis, add a send interface and open the interface to the wireless display module 195. In this way, the data encapsulation module 195a in the wireless display module 195 can encapsulate the reverse control data in the form of UIBC HID, and then send the encapsulated reverse control data to the port multiplexing module 182 through the interface, and send it to other devices (such as computers) as the source end through the above-mentioned reverse control path.

[0169] If the processor 110 has the ability to close the reverse control port occupied by the wireless screen projection management module 180, the wireless screen projection management module 180 can, under the control of the wireless display module 195, renegotiate the port number used to transmit reverse control data with other devices serving as the source end through the reverse control transmission management module 195b, that is, re-establish the reverse control path with the source end through the reverse control transmission management module 195b. Therefore, the data encapsulation module 195a in the wireless display module 195 can encapsulate the unencapsulated reverse control data in UIBC HID format and transmit it directly to the source end (e.g., a computer) through the reverse control path between the reverse control transmission management module 195b and the source end.

[0170] An embodiment of the present application also provides an electronic device, which includes: one or more processors and a memory; wherein the memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the steps performed by the tablet in the method shown in the aforementioned embodiment.

[0171] As used in the above embodiments, the term “when…” may be interpreted to mean “if…” or “after…” or “in response to determining…” or “in response to detecting…”, depending on the context. Similarly, the phrases “upon determining…” or “if (stated condition or event) is detected” may be interpreted to mean “if determining…” or “in response to determining…” or “upon detecting (stated condition or event)” or “in response to detecting (stated condition or event)”, depending on the context.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0173] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium, and when executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data processing method, characterized in that: The method is applied to a first terminal, the first terminal including a data encapsulation module and a chip for encapsulating reverse control data in UIBC HID format, the chip including a wireless screen projection management module, a port multiplexing module, and a reverse control module for encapsulating reverse control data in UIBC generic format, the method comprising: The port multiplexing module obtains a handle of a reverse control path corresponding to a first reverse control port, where the first reverse control port is a port for transmitting reverse control data negotiated between the first terminal and the second terminal by the wireless screen projection management module and the second terminal during the screen projection process, and generates a first interface based on the handle, where the first interface is used to communicate with the data encapsulation module. During the screen projection process, the first terminal is a receiving end and the second terminal is a sending end. The data encapsulation module encapsulates the reverse control data acquired by the first terminal in the UIBC HID format and sends the reverse control data to the port multiplexing module; The port multiplexing module sends the reverse control data to the second terminal through the first reverse control port, the first terminal is a tablet, and the second terminal is a computer.

2. The method according to claim 1, characterized in that The data encapsulation module includes a wireless display module, and before the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port, the method further includes: The wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module; The port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port, including: When the wireless screen projection management module does not close the first reverse control port, the port multiplexing module obtains the handle of the reverse control path corresponding to the first reverse control port.

3. The method according to claim 2, characterized in that Before the port multiplexing module obtains the handle of the path corresponding to the first reverse control port, the method further includes: When the wireless screen projection management module does not close the first reverse control port, the wireless screen projection management module sends a first instruction to the wireless display module, where the first instruction is used to notify the wireless display module that the first reverse control port is not closed, and to control the wireless display module to send a multiplexing instruction to the port multiplexing module, where the multiplexing instruction is used to control the port multiplexing module to obtain the handle of the reverse control path corresponding to the first reverse control port.

4. The method according to any one of claims 1 to 3, characterized in that The port multiplexing module and the wireless screen projection management module are modules in the same process. The port multiplexing module obtains the handle of the path corresponding to the first reverse control port, including: The port multiplexing module queries the Socket handle in the open state in the process according to the process number of the process to which the port multiplexing module belongs; The port multiplexing module determines the handle whose corresponding port number is a TCP port number and a reverse control port number among the open Socket handles as the handle of the path corresponding to the first reverse control port.

5. The method according to claim 2 or 3, characterized in that Before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method further includes: The wireless screen projection management module determines whether the chip has the ability to encapsulate the reverse control data in UIBC HID format; Before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method includes: When the wireless screen projection management module determines whether the chip does not have the ability to encapsulate the reverse control data in UIBC HID format, the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module.

6. The method according to claim 4, characterized in that The data encapsulation module includes a wireless display module. Before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method further includes: The wireless screen projection management module determines whether the chip has the ability to encapsulate the reverse control data in UIBC HID format; Before the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module, the method includes: When the wireless screen projection management module determines whether the chip does not have the ability to encapsulate the reverse control data in UIBC HID format, the wireless display module sends an instruction for closing the first reverse control port to the wireless screen projection management module.

7. A data processing method, characterized in that: The method is applied to a first terminal, the first terminal including a wireless display module and a chip, the wireless display module including a data encapsulation module, the chip including a wireless screen projection management module and a reverse control module for encapsulating reverse control data in a UIBC generic format, and the method includes: The wireless display module sends an instruction to the wireless screen projection management module for closing a first reverse control port, where the first reverse control port is a port for transmitting reverse control data negotiated between the first terminal and the second terminal during the screen projection process, wherein the first terminal is a receiving end and the second terminal is a sending end; When the wireless screen projection management module closes the first reverse control port, the wireless display module sends a reconnection request instruction to the second terminal, where the reconnection request instruction is used to request the second terminal to renegotiate the second reverse control port with the wireless display module; After the wireless display module encapsulates the reverse control data obtained by the first terminal in UIBC HID format, the wireless display module sends the reverse control data to the second terminal through the second reverse control port. The first terminal is a tablet and the second terminal is a computer.

8. The method according to claim 7, characterized in that Before the wireless display module sends a reconnection request instruction to the second terminal, the method further includes: When the wireless screen projection management module closes the first reverse control port, the wireless screen projection management module sends a second instruction to the wireless display module, and the second instruction is used to notify the wireless display module that the first reverse control port has been closed, and to control the wireless display module and send the reconnection request instruction to the second terminal.

9. An electronic device, characterized in that: The electronic device includes: one or more processors, a memory and a display screen; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 8.

10. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 8.

11. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 6 or the method according to any one of claims 7 to 8.

Images