Data processing method and related device

Abstract

The data processing method and related device provided by the embodiment of the present application relate to the field of terminal technology. The method includes: a first electronic device can exchange data with a second electronic device with stronger drawing capabilities, so that a game screen can be drawn in the second electronic device, and the relevant data of the drawn game screen is sent to the first electronic device, and the first electronic device displays the game screen. In this way, based on the drawing capability of the second electronic device, the image quality of the game interface in the first electronic device can be improved, the fluency of the game can be improved, and a better gaming experience can be brought to the user.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a data processing method and related devices. Background Art

[0002] With the development of technology, more and more applications are installed in electronic devices. For example, electronic devices can support the installation of game applications.

[0003] However, when users use electronic devices to play games, the game screen may not be smooth, and the electronic devices may become hot or stuck. Summary of the invention

[0004] The data processing method and related device provided in the embodiment of the present application can enable a first electronic device to exchange data with a second electronic device with stronger drawing capabilities, so that a game screen can be drawn in the second electronic device, and the relevant data of the drawn game screen can be sent to the first electronic device, and the first electronic device displays the game screen. In this way, based on the drawing capabilities of the second electronic device, the image quality of the game interface in the first electronic device can be improved, the fluency of the game can be improved, and a better gaming experience can be brought to the user.

[0005] In a first aspect, an embodiment of the present application provides a data processing method, which is applied to a first electronic device, and includes:

[0006] Establish a communication connection with the second electronic device; if the first electronic device determines that the network meets the network quality requirements, send a drawing instruction for drawing the first screen to the second electronic device; display the first screen, wherein the first screen is drawn based on the GPU device of the second electronic device, and the drawing capability of the GPU device of the first electronic device is less than the drawing capability of the GPU device of the second electronic device; if the first electronic device determines that the network does not meet the network quality requirements, draw the second screen; display the second screen. In this way, based on the drawing capability of the second electronic device, the image quality of the interface in the first electronic device can be improved, the smoothness of application operation can be improved, the power consumption of the first electronic device can be reduced, and the heating of the first electronic device can be reduced, thereby improving the user experience.

[0007] In a possible implementation, if the first electronic device determines that the network meets the network quality requirements, a drawing instruction for drawing the first screen is sent to the second electronic device, including: if the number of instructions in the target instruction queue is less than a preset value, a drawing instruction for drawing the first screen is sent to the second electronic device, wherein the target instruction queue is used to store the drawing instructions, and the drawing instructions for the first screen are obtained from the target instruction queue; if the first electronic device determines that the network does not meet the network quality requirements, a second screen is drawn, including: if the number of instructions in the target instruction queue is greater than or equal to the preset value, a second screen is drawn, wherein the drawing instructions for the second screen are obtained from the target instruction queue. In this way, the quality of the network can be determined according to the number of instructions in the target instruction queue, and the status of the network signal can be quickly judged. Through the number of instructions in the target instruction queue, the change of the network signal can be detected in real time, so as to switch the local screen drawing and the remote screen drawing in time.

[0008] In a possible implementation, if the first electronic device determines that the network meets the network quality requirements, a drawing instruction for drawing the first screen is sent to the second electronic device, including: if the target flag is a first value, a drawing instruction for drawing the first screen is sent to the second electronic device, wherein, when the number of instructions in the target instruction queue is less than a preset value, the target flag is a first value, and the first value is used to instruct the GPU device based on the second electronic device to draw the screen; if the first electronic device determines that the network does not meet the network quality requirements, a second screen is drawn, including: if the target flag is a second value, a second screen is drawn, wherein, when the number of instructions in the target instruction queue is greater than or equal to the preset value, the target flag is a second value, and the second value is used to instruct the GPU device based on the first electronic device to draw the screen. In this way, the quality of the network can be determined according to the target flag, and the status of the network signal can be quickly judged. The target flag is used to conveniently record the status of the network signal, simplify the control process of code execution in the first electronic device, and improve the readability and flexibility of the code, so that the changes in the network signal can be concisely identified, and the local screen drawing and remote screen drawing can be switched in time.

[0009] In a possible implementation, the method further includes: if the first electronic device determines that the network switches from not meeting the network quality requirements to meeting the network quality requirements, the electronic device executes the following process: sending a drawing instruction to the second electronic device; within a preset time period, the first electronic device draws the screen, and the first electronic device does not use the screen sent by the second electronic device; after the preset time period, the first electronic device does not draw the screen, updates the target flag to the first value, and uses the screen sent by the second electronic device; if the first electronic device determines that the network switches from meeting the network quality requirements to not meeting the network quality requirements, the first electronic device executes the following process: drawing the screen and updating the target flag to the second value. In this way, the first electronic device can first send the instructions accumulated in the target instruction queue, give the second electronic device some processing time, and wait for the second electronic device to resume normal processing speed, so that the screen can be connected between the first electronic device and the second electronic device to improve the smoothness of the screen.

[0010] In a possible implementation, the method further includes: if the first electronic device determines that the network does not meet the network quality requirements, establish a communication connection with a third electronic device; after establishing a communication connection with the third electronic device, if the first electronic device determines that the network meets the network quality requirements, send a drawing instruction for drawing a third screen to the third electronic device; display the third screen, wherein the third screen is drawn based on the GPU device of the third electronic device, and the drawing capability of the GPU device of the first electronic device is less than the drawing capability of the GPU device of the third electronic device. In this way, the first electronic device switches with multiple electronic devices to find an electronic device with a better network status, such as the third electronic device, to improve the efficiency of screen drawing, thereby bringing a better game screen experience to the user.

[0011] In one possible implementation, before sending a drawing instruction for drawing a third screen to a third electronic device, it also includes: obtaining target data from a second electronic device, the target data being used to identify the drawing progress of the second electronic device; and sending the target data to the third electronic device. In this way, the first electronic device can increase the speed of screen drawing by establishing a connection with a third electronic device with a better network status, thereby reducing the impact of network signals on screen display. In addition, the target data can make the context of the GPU consistent, and the screen drawing can be seamlessly connected. In this way, when the first electronic device switches to the third electronic device, the screen can be continuous, thereby improving the user experience.

[0012] In a possible implementation, establishing a communication connection with a second electronic device includes: establishing a first Socket communication and a second Socket communication with the second electronic device; before sending a drawing instruction for drawing a first screen to the second electronic device, it also includes: numbering the drawing instruction; sending a drawing instruction for drawing the first screen to the second electronic device includes: sending a drawing instruction whose number satisfies a first condition to the second electronic device through the first Socket communication, and sending a drawing instruction whose number satisfies a second condition to the second electronic device through the second Socket communication; wherein the first condition is used to determine whether the first Socket communication is applicable, and the second condition is used to determine whether the second Socket communication is applicable. In this way, establishing multiple socket communications between the first electronic device and the second electronic device can increase the speed of sending drawing instructions, thereby reducing problems such as the screen not following the hand and the screen freezing.

[0013] In a possible implementation, before sending the drawing instruction for drawing the first screen to the second electronic device, the method further includes: performing differential calculation on the drawing instruction and compressing the drawing instruction after the differential calculation; sending the drawing instruction for drawing the first screen to the second electronic device includes: sending the compressed drawing instruction for drawing the first screen to the second electronic device. In this way, the modified part is extracted through the differential algorithm, thereby improving the compression rate, reducing the amount of data transmitted, and speeding up the data transmission.

[0014] In a second aspect, an embodiment of the present application provides a data processing method, which is applied to a second electronic device, and includes:

[0015] Establish a communication connection with one or more electronic devices; receive drawing instructions from one or more electronic devices; based on the GPU device of the second electronic device, draw a screen for one or more electronic devices according to the drawing instructions of one or more electronic devices, wherein the drawing capabilities of the GPU devices of the one or more electronic devices are less than the drawing capabilities of the GPU device of the second electronic device; send the screen corresponding to each electronic device to the one or more electronic devices respectively. In this way, based on the drawing capabilities of the second electronic device, the image quality of the interface in one or more electronic devices can be improved, the smoothness of application operation can be improved, the power consumption of one or more electronic devices can be reduced, and the heating of one or more electronic devices can be reduced, thereby improving the user experience.

[0016] In a possible implementation, one or more electronic devices include a first electronic device, and a GPU device based on a second electronic device, before drawing a screen for one or more electronic devices according to drawing instructions of one or more electronic devices, further includes: receiving a target flag from the first electronic device; based on the GPU device of the second electronic device, drawing a screen for one or more electronic devices according to drawing instructions of one or more electronic devices, including: if the target flag is a first value, the GPU device based on the second electronic device, according to the drawing instructions of the first electronic device, draws a screen for the first electronic device, wherein the first value is used to instruct the GPU device based on the second electronic device to draw the screen. In this way, the quality of the network can be determined according to the target flag, and the status of the network signal can be quickly judged. The use of the target flag facilitates recording the status of the network signal, simplifies the control process of code execution in the first electronic device, and improves the readability and flexibility of the code, so that changes in the network signal can be concisely identified, and local screen drawing and remote screen drawing can be switched in a timely manner.

[0017] In a possible implementation, establishing a communication connection with one or more electronic devices includes: establishing a first Socket communication and a second Socket communication with a first electronic device; receiving drawing instructions from one or more electronic devices, including: receiving drawing instructions from the first electronic device through the first Socket communication whose number satisfies a first condition, and storing them in a first queue, wherein the first condition is used to indicate that the drawing instructions are transmitted through the first Socket communication; receiving drawing instructions from the first electronic device through the second Socket communication whose number satisfies a second condition, and storing them in a second queue, wherein the second condition is used to indicate that the drawing instructions are transmitted through the second Socket communication; before drawing a screen for one or more electronic devices based on the GPU device of the second electronic device according to the drawing instructions of one or more electronic devices, it also includes: obtaining drawing instructions for drawing the screen corresponding to the first electronic device from the first queue or the second queue in the order of numbers. In this way, establishing multiple socket communications between the first electronic device and the second electronic device can increase the speed of sending drawing instructions, thereby reducing problems such as the screen not following the hand and the screen freezing.

[0018] In a possible implementation, after receiving the drawing instructions from one or more electronic devices, it also includes: decompressing the drawing instructions of the one or more electronic devices and performing differential calculation on the decompressed drawing instructions; based on the GPU device of the second electronic device, drawing a screen for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, including: based on the GPU device of the second electronic device, using the drawing instructions of the one or more electronic devices after differential calculation, drawing a screen for the one or more electronic devices. In this way, the modified part is extracted through the differential algorithm, thereby improving the compression rate, reducing the amount of data transmission, and speeding up the data transmission speed.

[0019] In a third aspect, an embodiment of the present application provides a data processing device, which may be an electronic device, or a chip or chip system in an electronic device. The device may include a processing unit and a display unit. The processing unit is used to implement any method related to processing performed by the first electronic device in the first aspect or any possible implementation of the first aspect, or performed by the second electronic device in the first aspect or any possible implementation of the first aspect, or performed by the first electronic device in the second aspect or any possible implementation of the second aspect, or performed by the second electronic device in the second aspect or any possible implementation of the second aspect. The display unit is used to implement any method related to display performed by the first electronic device in the first aspect or any possible implementation of the first aspect, or performed by the second electronic device in the first aspect or any possible implementation of the first aspect, or performed by the first electronic device in the second aspect or any possible implementation of the second aspect, or performed by the second electronic device in the second aspect or any possible implementation of the second aspect.

[0020] When the device is an electronic device, the processing unit may be a processor. The device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to enable the electronic device to implement the method performed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the second electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the first electronic device described in the second aspect or any possible implementation of the second aspect, or the method performed by the second electronic device described in the second aspect or any possible implementation of the second aspect.

[0021] When the device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit to enable the electronic device to implement the method performed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the second electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the first electronic device described in the second aspect or any possible implementation of the second aspect, or the method performed by the second electronic device described in the second aspect or any possible implementation of the second aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).

[0022] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute a method executed by a first electronic device described in the first aspect or any possible implementation of the first aspect, or a method executed by a second electronic device in the method described in the first aspect or any possible implementation of the first aspect, or a method executed by a first electronic device in the method described in the second aspect or any possible implementation of the second aspect, or a method executed by a second electronic device in the method described in the second aspect or any possible implementation of the second aspect.

[0023] In a fifth aspect, an embodiment of the present application provides an interactive system, including a first electronic device and a second electronic device, the first electronic device being used to execute a method executed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or a method executed by the first electronic device described in the second aspect or any possible implementation of the second aspect, and the second electronic device being used to execute a method executed by the second electronic device described in the first aspect or any possible implementation of the first aspect, or a method executed by the second electronic device in a method described in the second aspect or any possible implementation of the second aspect.

[0024] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the method executed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method executed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method executed by the second electronic device described in the second aspect or any possible implementation of the second aspect, or the method executed by the second electronic device described in the second aspect or any possible implementation of the second aspect.

[0025] In the seventh aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, it enables the computer to execute the method executed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method executed by the second electronic device described in the first aspect or any possible implementation of the first aspect, or the method executed by the first electronic device described in the second aspect or any possible implementation of the second aspect, or the method executed by the second electronic device described in the second aspect or any possible implementation of the second aspect.

[0026] In an eighth aspect, the present application provides a chip or a chip system, the chip or chip system includes at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method performed by the first electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the second electronic device described in the first aspect or any possible implementation of the first aspect, or the method performed by the first electronic device described in the second aspect or any possible implementation of the second aspect, or the method performed by the second electronic device described in the second aspect or any possible implementation of the second aspect. Among them, the communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0027] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0028] It should be understood that the third to eighth aspects of the present application correspond to the technical solutions of the first or second aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of a usage scenario provided for an embodiment of the present application;

[0032] Figure 4 A schematic diagram of interaction between a mobile phone and a tablet computer provided in an embodiment of the present application;

[0033] Figure 5 A schematic diagram of command transmission blocking between a mobile phone and a tablet computer provided in an embodiment of the present application;

[0034] Figure 6 A schematic diagram of switching between local drawing and remote drawing provided in an embodiment of the present application;

[0035] Figure 7 A schematic diagram of an interaction process between a mobile phone and a tablet computer provided in an embodiment of the present application;

[0036] Figure 8 A logic diagram of flag bit judgment provided in an embodiment of the present application;

[0037] Fig. 9 A schematic diagram of switching between a mobile phone and different tablet computers provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of context transmission between a mobile phone and different tablet computers provided in an embodiment of the present application;

[0039] Fig.11 A schematic diagram of a single socket communication provided in an embodiment of the present application;

[0040] Fig.12 A schematic diagram of a dual-threaded socket communication provided in an embodiment of the present application;

[0041] Fig.13 A schematic diagram of a dual-threaded socket communication process provided in an embodiment of the present application;

[0042] Fig.14 A schematic diagram of a scenario in which a tablet computer is connected to multiple mobile phones provided in an embodiment of the present application;

[0043] Fig.15A schematic diagram of independent communication between multiple mobile phones in a tablet computer provided in an embodiment of the present application;

[0044] Fig.16 A schematic diagram of creating multiple socket communications for a tablet computer provided in an embodiment of the present application;

[0045] Fig.17 A schematic diagram of module interaction between a mobile phone and a tablet computer provided in an embodiment of the present application;

[0046] Fig.18 A schematic diagram of a layer effect display provided in an embodiment of the present application;

[0047] Fig.19 A schematic diagram of another layer effect display provided in an embodiment of the present application;

[0048] Fig. 20 A schematic diagram of the display effect of an application layer provided in an embodiment of the present application;

[0049] Fig.21 A flowchart of a mobile phone screen display provided in an embodiment of the present application;

[0050] Fig. 22 A schematic diagram of a differential mechanism and a compression algorithm provided in an embodiment of the present application;

[0051] Fig.23 A schematic diagram of a mobile phone side differential mechanism provided in an embodiment of the present application;

[0052] Fig.24 A schematic diagram of a tablet computer side differential mechanism provided in an embodiment of the present application;

[0053] Fig.25 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0054] Fig.26 A schematic diagram of another data processing method provided in an embodiment of the present application;

[0055] Fig. 27 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0057] 1. Terminology

[0058] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0059] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0060] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0061] With the development of technology, more and more applications are installed in electronic devices. For example, electronic devices can support the installation of game applications.

[0062] However, when users use electronic devices to play games, the electronic devices may have a weak ability to draw game images, resulting in an unsmooth game image and the electronic devices may become hot or stuck.

[0063] In view of this, the data processing method provided in the embodiment of the present application allows the first electronic device to exchange data with the second electronic device with stronger drawing capabilities, so that the game screen can be drawn in the second electronic device, and the relevant data of the drawn game screen can be sent to the first electronic device, and the first electronic device displays the game screen. In this way, based on the drawing capability of the second electronic device, the image quality of the game interface in the first electronic device can be improved, the fluency of the game can be improved, the power consumption of the first electronic device can be reduced, and the heating of the first electronic device can be reduced, thereby bringing a better gaming experience to the user. Among them, the screen in the embodiment of the present application can also be understood as an image.

[0064] The first electronic device may be a terminal device in any form, for example, the terminal device may include: a mobile phone, a tablet computer, a PDA, a laptop computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network or a future evolved public land mobile communication network (public land mobile communication network); The terminal equipment in the land mobile network (PLMN), etc., is not limited to this in the embodiments of the present application.

[0065] As an example but not limitation, in the embodiments of the present application, the terminal device may also be a wearable device. Wearable devices may also be referred to as wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also powerful functions achieved through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include full-featured, large-sized, and fully or partially independent of smartphones, such as smart watches or smart glasses, as well as devices that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various types of smart bracelets and smart jewelry for vital sign monitoring.

[0066] In addition, in the embodiments of the present application, the terminal device may also be a terminal device in an Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0067] The terminal device in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0068] In an embodiment of the present application, a terminal device or each network device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software.

[0069] For example, Figure 1 A schematic structural diagram of a first electronic device is shown.

[0070] The first electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0071] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the first electronic device. In other embodiments of the present application, the first electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may include hardware, software, or a combination of software and hardware.

[0072] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0073] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it may be directly called from the above-mentioned memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. For example, in an embodiment of the present application, the processor 110 may be used to process the serialization of transmission instructions, the decoding of data, etc.

[0074] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc.

[0075] It is understandable that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the first electronic device. In other embodiments of the present application, the first electronic device may also adopt a different interface connection method in the above embodiments, or a combination of multiple interface connection methods.

[0076] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, at least one application required for a function, etc. The data storage area may store data created during the use of the first electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the first electronic device by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor. For example, in an embodiment of the present application, the internal memory 121 may be used to store an open graphics library application program interface, an application management service, and related codes required for the operation of an application, etc.

[0077] The wireless communication module 160 can provide wireless communication solutions for application on the first electronic device, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0078] The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be sent from the processor 110, modulate the frequency of the signal, amplify the signal, and convert it into an electromagnetic wave for radiation via the antenna.

[0079] Figure 2: is a software structure block diagram of the first electronic device of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware adaptation layer, HAL), and the kernel layer.

[0080] The application layer can also be called the application layer, which can include a series of application packages. Figure 2 As shown, the application package may include applications such as games, mobile game assistants, etc. Applications may include system applications and third-party applications.

[0081] The application framework layer can also be called the Framework layer. The Framework layer can provide application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The Framework layer can include some predefined functions.

[0082] like Figure 2 As shown, the Framework layer may include an activity manager service (AMS), a codec module (MediaCodec module) and SurfaceFlinger, a device connection service, an open graphics library application programming interface (OpenGL API), an OpenGL HOOK mechanism, and the like.

[0083] AMS can also be called an activity management service or an application manager. AMS can be responsible for managing the life cycle of an application, task stacks, switching between processes and activities, etc. For example, in an embodiment of the present application, AMS can be used to create a buffer surface, which can be used to store screen data to be displayed. AMS can also be used to pass a surface pointer to an application.

[0084] The MediaCodec module can be used to convert data from one format to another. For example, the MediaCodec module can convert raw data into a compressed format, and can also convert the compressed format back to raw data. For example, in an embodiment of the present application, the MediaCodec module can be used to decode the relevant data of the screen returned by the tablet computer, and can also write the decoded screen to the buffer area surface, etc. In some scenarios, the MediaCodec module can also be referred to as a codec module.

[0085] SurfaceFlinger can be responsible for managing and synthesizing display content. For example, SurfaceFlinger can manage the display of application windows, including the creation, destruction and hierarchical relationship of windows, so that the windows of each application can be correctly displayed on the screen. SurfaceFlinger can also be responsible for synthesizing multiple layers (Layers) to generate a picture displayed on the screen. In an embodiment of the present application, SurfaceFlinger can display the picture from the tablet computer on the screen of the mobile phone.

[0086] The device connection service can be used to establish a connection between electronic devices, and the device connection service can also be used to transmit OpenGL instructions, etc. between electronic devices.

[0087] The Open Graphics Library Application Programming Interface (OpenGL API) is a cross-language, cross-platform application programming interface for rendering 2D and 3D vector graphics.

[0088] The OpenGL HOOK mechanism can intercept the OpenGL instructions issued by the game application and determine whether local drawing is required.

[0089] The Framework layer may also include an activity manager, a window manager, a resource manager, a notification manager, a content provider, and a view system, etc. For a specific description, please refer to the relevant technology and will not be repeated here.

[0090] Android runtime includes core libraries and virtual machines. Android runtime is responsible for the control and management of the Android system.

[0091] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0092] The application layer and the framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0093] The system library can also be called the Native layer, which can include multiple functional modules, such as media libraries, function libraries, and graphics processing libraries (such as OpenGL ES).

[0094] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0095] The function library provides developers with API interfaces for multiple services, making it easy for developers to quickly integrate and implement various functions.

[0096] The graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis and layer processing.

[0097] The hardware abstraction layer is an abstract structure between the kernel layer and the Android runtime. The hardware abstraction layer can be a package of hardware drivers, providing a unified interface for upper-layer applications to call.

[0098] The kernel layer is the layer between hardware and software. The kernel layer can include OpenGL drivers, GPU drivers, and display drivers, etc.

[0099] It should be noted that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0100] It is understandable that the second electronic device can also be any form of terminal device mentioned above. In addition, the second electronic device can also include a personal computer (PC). PC can be understood as a computer suitable for personal use. For example, PC can include desktops, laptops, small laptops, tablets, large screens, and ultrabooks, etc., which are not limited in the embodiments of the present application. The structural diagram and software structure block diagram of the second electronic device can refer to the first electronic device. It is understandable that the structural diagram and software structure block diagram of the first electronic device do not constitute a specific limitation on the second electronic device. In other embodiments of the present application, the second electronic device can be compared to Figure 1 or Figure 2 The components shown may include more or fewer components, or some components may be combined, or some components may be separated, or different component arrangements may be provided. The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0101] It should be noted that the drawing capability of the second electronic device, such as GPU capability, is stronger than that of the first electronic device.

[0102] For ease of description, the following description will be given by taking the first electronic device as a mobile phone and the second electronic device as a tablet computer as an example.

[0103] like Figure 3 As shown, the target application can be installed on the mobile phone 301 and the tablet computer 302 respectively. When the target application 303 in the mobile phone 301 and the target application 304 in the tablet computer 302 are both running, a communication connection can be established on the mobile phone 301 and the tablet computer 302 by scanning the interconnection, for example, a socket communication connection is established, so as to transmit data between the mobile phone 301 and the tablet computer 302. Among them, the target application 303 and the target application 304 can be a mobile game assistant application, or other applications that can realize functions such as data transmission, etc., which are not limited in the embodiments of the present application.

[0104] It is understandable that the mobile phone 301 and the tablet computer 302 can establish a wireless or wired communication connection, such as a socket connection, a USB connection, a Bluetooth connection, a near-field communication connection, a hypertext transfer protocol (HTTP) connection, a hypertext transfer protocol secure (HTTPS) connection, etc., which is not limited in the embodiments of the present application.

[0105] For ease of description, the following explanation will be given by taking the example that target application 303 and target application 304 are both mobile game assistant applications.

[0106] When a user plays a game on the mobile phone 301, the game screen displayed on the interface of the mobile phone 301 can be drawn through the open graphics library (openGL) instructions. Since the computing power of the GPU in the tablet computer 302 is stronger than that of the GPU in the mobile phone 301, when the network signal is strong, the mobile phone 301 can send the openGL instructions to the tablet computer 302 to draw the screen based on the GPU capability of the tablet computer 302. The tablet computer 302 can also return the relevant data of the drawn screen to the mobile phone 301, so that the mobile phone 301 can display the game interface based on the returned screen-related data. In some implementations, the openGL instructions can also be called GPU instructions.

[0107] Optionally, the tablet computer 302 can draw pictures in the foreground or in the background. When the tablet computer 302 draws pictures in the background, the user can still handle other things in the foreground of the tablet computer 302, or the user can close the tablet computer 302 when not using the tablet computer 302, and the tablet computer 302 can still draw pictures when the tablet computer 302 is not in sleep mode.

[0108] Optionally, a tablet computer can also be connected to multiple mobile phones, and the tablet computer can process the data of multiple mobile phones in parallel and provide screen drawing capabilities for multiple mobile phones. In this way, the GPU capability of the tablet computer 302 can be used to bring better game experience to multiple users.

[0109] The following takes a mobile phone and a tablet computer as an example to explain the data transmission process after the mobile phone and tablet computer establish a communication connection.

[0110] like Figure 4 As shown in the figure, on the mobile phone side, the application layer can include game applications and mobile game assistant applications. The Framework layer can include OpenGL interface, AMS, MediaCodec module, SurfaceFlinger and OpenGL Hook mechanism, and the driver layer can include OpenGL driver, GPU driver and display driver. For the specific functions of each module in the above layers, please refer to Figure 2 The relevant descriptions in the corresponding embodiments are not repeated here.

[0111] For example, when the game application is running, it can call the OpenGL interface and pass the OpenGL instructions to the OpenGLHook mechanism. Then, the OpenGL Hook mechanism can intercept the OpenGL instructions and determine whether local drawing is required. The process of intercepting the OpenGL instructions by the OpenGL Hook mechanism can also be understood as the process of intercepting the system's OpenGL instructions based on the Graphics Layer (GL Layer) mechanism of the Android system. Among them, the GL Layer mechanism can be used to manage and render graphic elements so that the image elements can be correctly displayed on the screen.

[0112] In a possible implementation, the OpenGL Hook mechanism can determine whether local drawing is required through the local drawing flag. The data type of the local drawing flag may include string, integer, Boolean, enumeration and other types. For example, taking the Boolean type as an example, when local drawing is required, the local drawing flag may be true, and when local drawing is not required, the local drawing flag may be false. The specific representation method of the local drawing flag is not limited in the embodiments of the present application.

[0113] Optionally, the OpenGL Hook mechanism can also determine whether local drawing is required through the remote drawing flag. The data type of the remote drawing flag may include string, integer, Boolean, enumeration and other types. For example, taking the Boolean type as an example, when local drawing is required, the remote drawing flag may be false, and when local drawing is not required, the remote drawing flag may be true. The specific representation method of the remote drawing flag is not limited in the embodiments of the present application.

[0114] Optionally, the OpenGL Hook mechanism can also set other flags to indicate whether local drawing is required. This is not limited in the embodiments of the present application. The need for local drawing can also be understood as the need for remote drawing, the need for a tablet computer to draw the screen, the need for sending OpenGL instructions to the tablet computer, or the need for displaying the screen sent by the tablet computer. The need for local drawing can also be understood as the need for remote drawing, the need for a tablet computer to draw the screen, the need for sending OpenGL instructions to the tablet computer, or the need for displaying the screen sent by the tablet computer, which is not limited in the embodiments of the present application.

[0115] For the convenience of description, the following description will be made by taking the case where the local drawing flag is true, indicating that local drawing is required, and the case where the local drawing flag is false, indicating that local drawing is not required.

[0116] If the OpenGL Hook mechanism determines that local drawing is required, on the one hand, the OpenGL Hook mechanism can place the intercepted OpenGL instructions in the OpenGL instruction queue to determine the network situation and set the local drawing flag. On the other hand, the OpenGL Hook mechanism can pass the OpenGL instructions to the driver layer to draw the screen based on the OpenGL driver and GPU driver. It can be understood that the OpenGL Hook mechanism can be executed in parallel in these two aspects.

[0117] If the OpenGL Hook mechanism determines that local drawing is not needed, on the one hand, the OpenGL Hook mechanism can also place the intercepted OpenGL instructions in the OpenGL instruction queue to determine the network situation and set the local drawing flag. On the other hand, the OpenGL Hook mechanism can perform operations such as serialization and compression on the OpenGL instructions in the OpenGL instruction queue and send them to the tablet computer, so that the screen can be drawn based on the drawing capabilities of the tablet computer. It can be understood that the OpenGL Hook mechanism can be executed in parallel in these two aspects.

[0118] Serialization may include encapsulating the OpenGL instruction in a certain format, for example, the encapsulation format includes the length of the OpenGL instruction and the content of the OpenGL instruction, so as to facilitate data transmission through socket communication. In addition, the OpenGL Hook mechanism may use the LZ4 compression algorithm to compress the OpenGL instruction.

[0119] In a possible implementation, the mobile phone can send serialized OpenGL instructions to the tablet computer via the transmission control protocol (TCP). The mobile phone can send the instructions in a wired or wireless manner, which is not limited in the embodiment of the present application.

[0120] After the tablet receives the OpenGL instruction, the mobile game assistant application of the tablet can decompress and deserialize the OpenGL instruction, and pass the decompressed and deserialized OpenGL instruction to the driver layer, and draw the screen based on the OpenGL driver and GPU driver. Among them, deserialization can include parsing the OpenGL instruction encapsulated in a certain format.

[0121] After the tablet draws the screen, the relevant data of the rendered screen can be encoded and sent back to the mobile game assistant application on the mobile phone through socket communication. The mobile game assistant application can call the MediaCodec module to decode the data, write the screen to the cache surface, and call SurfaceFlinger to display the screen. SurfaceFlinger can obtain the screen from the cache surface and synthesize and display the screen based on the display driver.

[0122] It is understandable that before the mobile game assistant application calls the cache surface, AMS has already created the surface and passed the surface pointer to the mobile game assistant application. For example, when the game application is started, AMS can create the surface. When the mobile game assistant application calls the MediaCodec module to decode the data, it can carry the surface pointer.

[0123] However, if Figure 5 As shown in the figure, when the mobile phone and the tablet are transmitting data, if the network signal is weak, the mobile phone may be congested when sending OpenGL instructions to the tablet, so that the instructions cannot be sent out. At this time, the screen drawing based on the tablet will be slower than the local drawing of the mobile phone, so the game screen may be out of sync and the operation may not keep up with the user's hand when playing games.

[0124] Based on this, the embodiment of the present application can detect the strength of the network signal and switch between local drawing and remote drawing.

[0125] like Figure 6 As shown, when the mobile phone detects that the network signal is weak, it may not send OpenGL instructions to the tablet computer, and may send OpenGL instructions again after the network signal is restored.

[0126] In a possible implementation, the mobile phone can detect the strength of the network signal by the number of OpenGL instructions remaining in the OpenGL instruction queue. It is understandable that the OpenGL instructions remaining in the OpenGL instruction queue can be understood as instructions that have not been sent out in the queue. The mobile phone can make a judgment when the OpenGL instruction is added to the OpenGL instruction queue, or when the OpenGL instruction queue is updated, or periodically. The timing of specifically judging the number of OpenGL instructions remaining in the OpenGL instruction queue is not limited in the embodiments of the present application.

[0127] If the number of remaining OpenGL instructions is less than the preset number, it means that the OpenGL instructions in the current OpenGL instruction queue can be sent out in time and the network signal is strong, then the mobile phone can draw the screen based on the tablet computer. In this case, the OpenGL Hook mechanism can set the local drawing flag to false to indicate that local drawing is not required. Among them, the embodiment of the present application does not limit the value of the preset number, for example, the preset number can be set to 3, or it can be set to other values.

[0128] It should be understood that if the local drawing flag is already false before the local drawing flag is set, it may not be set, which can reduce the execution of the code and improve the execution efficiency of the electronic device.

[0129] If the number of remaining OpenGL instructions is greater than or equal to the preset number, it means that the OpenGL instructions in the current OpenGL instruction queue have not been sent out in time and the network signal is weak, then the mobile phone can perform local screen drawing. In this case, the OpenGL Hook mechanism can set the local drawing flag to true to indicate that local drawing is required.

[0130] It should be understood that if the local drawing flag is already true before the local drawing flag is set, it may not be set, which can reduce the execution of the code and improve the execution efficiency of the electronic device.

[0131] When the mobile phone detects that the number of remaining OpenGL instructions in the OpenGL instruction sending queue is greater than or equal to 3, that is, the network signal is weak, the mobile phone can perform local drawing and display the picture drawn locally by the mobile phone; the mobile phone can also notify the tablet computer to stop sending picture-related data.

[0132] When the mobile phone detects that the number of remaining OpenGL instructions in the OpenGL instruction sending queue is less than 3, that is, the network signal is good or the network signal is restored, the mobile phone can notify the tablet computer to resume the transmission of screen-related data, that is, the tablet computer can send the screen-related data back to the mobile phone. When the mobile phone receives the screen-related data sent from the tablet computer, it can display the screen drawn by the tablet computer and close the local drawing of the mobile phone.

[0133] It is understandable that when the mobile phone does not receive the picture data of the preset frame number within a certain period of time, it can also switch to local rendering, so that the display of the game screen will not be affected.

[0134] Figure 7 A schematic diagram showing the process of data interaction between a mobile phone and a tablet computer is shown.

[0135] The OpenGL Hook mechanism of the mobile phone can intercept OpenGL instructions and store the OpenGL instructions in the OpenGL instruction queue. The mobile phone can transmit OpenGL instructions to the tablet computer based on TCP. For example, when the mobile phone is based on TCP transmission, it can obtain OpenGL instructions from the OpenGL instruction queue, and the OpenGL instruction queue can return OpenGL instructions. Based on TCP transmission, the mobile phone can send the OpenGL instructions returned by the OpenGL instruction queue to the tablet computer. After the tablet computer executes the OpenGL instructions, it can send the drawn screen related data back to the mobile phone, and then the mobile phone can display the screen.

[0136] Among them, the specific method for determining the strength of the network signal can be referred to Figure 6 The relevant descriptions in the corresponding embodiments are not repeated here.

[0137] For example, when the number of remaining OpenGL instructions in the OpenGL instruction queue is greater than or equal to the preset number, it means that the network signal is weak, and the mobile phone can stop sending OpenGL instructions, switch to local screen drawing, and display the locally drawn screen. When the number of remaining OpenGL instructions in the OpenGL instruction queue is less than the preset number, it means that the network signal is strong, and the mobile phone can send OpenGL instructions to the tablet computer, and can also receive and display the screen drawn by the tablet computer, and turn off local drawing.

[0138] Figure 8 A logical schematic diagram of flag bit judgment is shown.

[0139] In a possible implementation, after intercepting the OpenGL instruction, the mobile phone can save it in the OpenGL instruction queue. The thread in the mobile phone can continuously obtain the OpenGL instruction from the queue, send it to the tablet computer, and determine the number of remaining OpenGL instructions in the queue.

[0140] (1) The number of remaining OpenGL instructions is less than 3

[0141] When the number of remaining OpenGL instructions is less than 3, it means that the network signal is strong and the network status is good, then the mobile phone can update the New flag to false. The New flag can be understood as the latest local drawing flag bit assigned, which is assigned to false, indicating that the current network status is good and no local drawing is required.

[0142] Furthermore, the mobile phone can also determine the Last flag bit, which can be understood as the local drawing flag bit assigned last time.

[0143] If the Last flag is true, it means that the previous network status was poor and local drawing was performed, but the current network status is good, that is, the network signal has changed from weak to strong. Therefore, the mobile phone can notify the tablet to start sending screen-related data.

[0144] In addition, the mobile phone can also turn off local drawing and display the picture sent by the tablet computer after a delay of about 100 milliseconds (ms). It is understandable that when the network signal changes from weak to strong, the mobile phone can display the picture based on the picture drawing capability of the tablet computer. In this case, the mobile phone can first send the OpenGL instructions accumulated in the OpenGL instruction queue and give the tablet computer some processing time. Therefore, the mobile phone can first delay for about 100ms and wait for the tablet computer to resume normal processing speed, which can improve the smoothness of the picture. Among them, about 100ms is an empirical value of laboratory testing. Whether the specific mobile phone is delayed and the length of the delay of the mobile phone can be pre-set in the electronic device, and the embodiments of this application are not limited.

[0145] Optionally, if the tablet sends screen data to the mobile phone within 100ms, the mobile phone can determine whether it is currently local rendering based on the flag bit. If it is local rendering, the mobile phone may not perform subsequent decoding and other processes, and may not display the screen data transmitted by the tablet. If it is not local rendering, the mobile phone may perform subsequent decoding and other processes, and display the screen data transmitted by the tablet.

[0146] The mobile phone can also assign the Last flag bit to the value of the New flag bit. It is understandable that the execution order of the mobile phone sending a notification to the tablet computer, the mobile phone delaying closing the local drawing, and updating the Last flag bit can be regardless of the execution order.

[0147] If the Last flag is false, it means that the network status was good before and no local drawing was performed, and the current network status is also good, that is, the network signal has not changed significantly and has not become weaker. Therefore, there is no need to set the Last flag, which can reduce the execution of code and improve the execution efficiency of electronic devices.

[0148] (2) The number of remaining OpenGL instructions is greater than or equal to 3

[0149] When the number of remaining OpenGL instructions is greater than or equal to 3, it means that the network signal is weak and the network status is poor. The mobile phone can update the New flag to true to indicate that the current network status is poor and local drawing is required.

[0150] Furthermore, if the Last flag is false, it means that the network status was good before and no local rendering was performed, but the current network status is poor, that is, the network signal becomes weaker. Therefore, the mobile phone can notify the tablet to stop sending screen-related data and perform local rendering.

[0151] In addition, the mobile phone can also assign the Last flag bit to the value of the New flag bit. It is understandable that the execution order of the mobile phone sending a notification to the tablet computer, the mobile phone performing local drawing, and updating the Last flag bit can be performed in any order.

[0152] If the Last flag is true, it means that the previous network status was poor and local drawing was performed, and the current network status is also poor, that is, the network signal has not changed significantly and has not become stronger. Therefore, there is no need to set the Last flag, which can reduce the execution of code and improve the execution efficiency of electronic devices.

[0153] It should be noted that when the Last flag in the mobile phone is updated, the tablet computer needs to be notified. In this way, the tablet computer can update the flag recorded in the tablet computer so that the flag recorded in the tablet computer can be consistent with the meaning indicated by the Last flag in the mobile phone.

[0154] Among them, the meaning indicated by the mark bit recorded in the tablet computer includes but is not limited to whether to perform local drawing, whether to perform remote drawing, whether the tablet computer is required to draw the screen, whether the network signal is good, or whether it is necessary to send screen-related data to the mobile phone, etc., and the embodiments of the present application do not limit this.

[0155] For example, taking the flag bit recorded in the tablet computer as an example to indicate whether the network signal is good, when the Last flag bit in the mobile phone indicates that the network signal is strong, the flag bit recorded in the tablet computer should also indicate that the network signal is strong; when the Last flag bit in the mobile phone indicates that the network signal is weak, the flag bit recorded in the tablet computer should also indicate that the network signal is weak. In this way, the mobile phone and the tablet computer can perform related processing of the screen based on the same network status, so that the mobile phone and the tablet computer can coordinate the processing of services.

[0156] The tablet computer can determine the strength of the network signal based on the mark bit recorded in the tablet computer. When the network signal is strong, the tablet computer returns the drawn picture to the mobile phone, so that the mobile phone can display the game interface based on the picture drawn by the tablet computer. When the network signal is weak, the tablet computer will not return the drawn picture to the mobile phone, and the mobile phone can display the game interface based on the picture drawn locally.

[0157] In some scenarios, the tablet computer may have a poor network signal. For example, the network card performance or hardware performance of some tablet computers may be relatively poor, which may affect the picture drawing or data transmission. In a possible implementation, the mobile phone can also switch with multiple second electronic devices to find a second electronic device with a better network status, improve the efficiency of picture drawing, and thus bring a better game picture experience to the user.

[0158] like Fig. 9 As shown, switching between a mobile phone and two tablet computers is taken as an example.

[0159] When the mobile phone detects that the network status for interacting with tablet computer A is poor, the mobile phone can notify tablet computer A and send the relevant data of tablet computer A to the mobile phone. For example, the relevant data may include context data, etc. The specific method for the mobile phone to detect the network signal can be referred to Figure 6 The relevant descriptions in the corresponding embodiments are not repeated here.

[0160] It is understandable that when the network status of the interaction between the mobile phone and the tablet computer deteriorates, on the one hand, the mobile phone can perform local screen drawing, and on the other hand, the mobile phone can also try to discover other second electronic devices at the same time, so as not to affect the display of the game screen.

[0161] After the mobile phone discovers tablet computer B, it can establish a connection with tablet computer B. If the mobile phone detects that the network status for interacting with tablet computer B is good, the mobile phone can send the relevant data of tablet computer A, such as context data, to tablet computer B, and transmit OpenGL instructions to tablet computer B. When tablet computer B returns screen-related data, the mobile phone can display the screen drawn by tablet computer B, and the mobile phone can also turn off local drawing.

[0162] For example, Fig.10 As shown, when the mobile phone switches from tablet A to tablet B, the GPU context needs to be consistent for seamless connection. In this way, the mobile phone does not need to restart the game application when switching between different devices, so that the game screen can be continuous and the user can continue to play the game, improving the user experience.

[0163] When the mobile phone finds that the network status is poor, it can notify tablet A, which will then serialize GPU context-related information, such as Context data, and send it to the mobile phone through socket communication. After the mobile phone connects to tablet B, it can send tablet A's GPU context information to tablet B through socket communication, and the mobile phone can also send OpenGL instructions to tablet B.

[0164] After receiving the GPU context information, tablet computer B can deserialize the context data and initialize the OpenGL environment. After the GPU environment of tablet computer B is initialized, tablet computer B can draw the screen based on the OpenGL instructions sent by the mobile phone.

[0165] like Fig.11 As shown, when the mobile phone draws the screen based on the GPU computing power of the tablet computer, for some large or complex game applications, a large number of OpenGL instructions will be generated when drawing the screen, and the large amount of data may cause the accumulation of OpenGL instructions in the mobile phone. If a single socket communication is established between the mobile phone and the tablet computer to transmit OpenGL instructions, since the data that can be transmitted by a single socket communication is fixed, it may not be possible to send the OpenGL instructions to the tablet computer in time, which may cause the game to not follow the hand, the game screen to freeze, and the mobile phone memory to occupy too much.

[0166] In the embodiment of the present application, multiple socket communications can also be established between the tablet computer and the mobile phone to increase the speed of sending OpenGL instructions, thereby reducing problems such as the game not following the hands and the game screen being stuck.

[0167] like Fig.12 As shown, a dual-thread socket communication between a tablet computer and a mobile phone is taken as an example for explanation.

[0168] For example, a dual-thread communication mechanism can be added to the mobile phone and the tablet computer respectively. The dual-thread communication mechanism can create two socket communications. Before the mobile phone sends the OpenGL instructions, the OpenGL instructions can be sorted and the sequence number can also be sent to the tablet computer. When the tablet computer receives the OpenGL instructions, it can parse the OpenGL instructions and execute the OpenGL instructions according to the sequence number. In this way, the mobile phone can send the OpenGL instructions to the tablet computer in an orderly and fast manner, and the tablet computer can also process the OpenGL instructions in an orderly manner based on the sequence number.

[0169] Fig.13 A schematic diagram of the dual-thread communication process is shown.

[0170] For example, the mobile phone can discover the tablet computer through the device discovery connection service, which may also be referred to as a device discovery service, a device connection service, a connection service, or an automated device discovery connection service, etc. For ease of description, the device connection service is used as an example for description.

[0171] When the mobile phone discovers the tablet computer and establishes a socket communication connection with the tablet computer, two socket processes can be created to send data. For ease of description, the two socket communications established between the mobile phone and the tablet computer are referred to as socket communication 1 and socket communication 2 respectively.

[0172] When the mobile phone sends an OpenGL instruction to the tablet computer, a serial number may be added to the OpenGL instruction to be sent. In a possible implementation, the serial number may be placed in the header of the sent data, and the serial number may be incremented from 0.

[0173] Optionally, the sequence number may also be placed in other parts of the sent data, the sequence number may be incremented starting from other numbers, or may be numbered in other ways, which is not limited in the embodiments of the present application.

[0174] Exemplarily, when the sequence number of the OpenGL instruction is an odd number, data can be sent by socket communication 1; when the sequence number of the OpenGL instruction is an even number, data can be sent by socket communication 2. Optionally, when the sequence number of the OpenGL instruction is an odd number, data can also be sent by socket communication 2; when the sequence number of the OpenGL instruction is an even number, data can also be sent by socket communication 1, which is not limited in the embodiment of the present application.

[0175] When the tablet computer receives OpenGL instruction data, two queues may be used to store the OpenGL instruction data transmitted by socket communication 1 and socket communication 2 respectively.

[0176] Exemplarily, the two queues may include queue 1 and queue 2, wherein queue 1 may store OpenGL command data transmitted by socket communication 1, and queue 2 may store OpenGL command data transmitted by socket communication 2. Optionally, queue 1 may store OpenGL command data transmitted by socket communication 2, and queue 2 may store OpenGL command data transmitted by socket communication 1, which is not limited in the embodiments of the present application.

[0177] When the thread of the tablet computer executes an OpenGL instruction, the tablet computer can first save a sequence number, for example, sequence number 0, and record the sequence number of the next OpenGL instruction to be executed, and then the sequence number is incremented. In this way, the tablet computer can process the OpenGL instructions in order based on the sequence number.

[0178] When the tablet computer takes data from the queue, it can first take it from queue 1. If an OpenGL instruction that satisfies the next instruction number is found in queue 1, the OpenGL instruction can be executed; if an OpenGL instruction that satisfies the next instruction number is not found in queue 1, it can be found in queue 2.

[0179] If an OpenGL instruction that satisfies the next instruction number is found in queue 2, the OpenGL instruction can be executed; if an OpenGL instruction that satisfies the next instruction number is not found in queue 2, the OpenGL instruction can be searched from queue 1, and so on, until an OpenGL instruction that satisfies the instruction number is found. In this way, the dual-thread communication mechanism can be used to reduce problems such as game lag and game screen freeze caused by slow OpenGL instruction sending.

[0180] It is understandable that the solution of streaming OpenGL instructions after connecting a mobile phone and a tablet computer can also be applied to the situation of multiple mobile phones and a tablet computer.

[0181] like Fig.14 As shown, the tablet computer can establish connections with N mobile phones, where N is a positive integer greater than 1. For example, N is 3, that is, the tablet computer establishes connections with 3 mobile phones, and the 3 mobile phones may include mobile phone A, mobile phone B, and mobile phone C. The tablet computer can perform device discovery, connection management, OpenGL instruction parsing and rendering, and screen echo management with the 3 mobile phones respectively.

[0182] like Fig.15 As shown, a tablet computer is connected to three mobile phones as an example. In the tablet computer, each mobile phone can correspond to one or more socket connections, corresponding to the OpenGL instruction stream module and the echo module. The tablet computer can also initialize one or more network management modules for network connection with the mobile phone. In addition, the tablet computer can also initialize the corresponding OpenGL instruction stream module, echo module, etc. for different mobile phones.

[0183] It is understandable that when a tablet computer is connected to multiple mobile phones at the same time, the interaction between the tablet computer and each mobile phone can be performed independently and in parallel. For example, the tablet computer will establish an independent socket communication connection with each mobile phone, and the OpenGL instruction stream executed by the tablet computer and one of the mobile phones will not affect the OpenGL instruction stream of the tablet computer and other mobile phones, and the screen transmission executed by the tablet computer and one of the mobile phones will not affect the screen transmission of the tablet computer and other mobile phones.

[0184] like Fig.16As shown, the tablet computer can initialize the mobile game assistant application, wherein initializing the mobile game assistant application can also be understood as initializing the streaming main program or initializing the streaming application. The tablet computer can also create one or more socket servers for connecting to the mobile phone. If the mobile phone is successfully connected, the tablet computer can stream and parse the OpenGL command stream sent by the mobile phone, and after drawing the picture, send it back to the mobile phone for screen display. If the connection to the mobile phone fails, the tablet computer can continue to wait for connection with the mobile phone or other mobile phones. It is understandable that the tablet computer can wait for other devices to connect in parallel.

[0185] The phone can start the mobile game assistant application and start the streaming service to connect to the tablet. If the tablet is successfully connected, the phone can create streaming communication, send OpenGL command streams to the tablet, and receive the screen data returned by the tablet, so as to render and display the screen. If the tablet fails to be connected, the phone can continue to wait for the tablet to connect.

[0186] Optionally, the number of device connection services initialized by the tablet computer may be greater than the number of currently connected mobile phones, so that the tablet computer may have one or more idle device connection services. In addition, when the tablet computer is not connected to a mobile phone, the tablet computer may also initialize one or more device connection services first, so that when the tablet computer is connected to a new mobile phone, it can connect and interact with the mobile phone in a timely manner, thereby improving the connection efficiency and processing related services quickly.

[0187] Fig.17 The schematic diagram of the interaction process between a tablet computer and a mobile phone is shown. The interaction process may include (1) establishing a connection between the tablet computer and the mobile phone, (2) initializing the module, and (3) drawing and displaying the screen.

[0188] (1) Establish a connection between the tablet and the mobile phone.

[0189] The tablet computer can run and initialize the main program of the mobile game assistant to wait for connection with other devices. The main program of the mobile game assistant can also create a device connection service.

[0190] The mobile phone can run and initialize the main program of the mobile game assistant to wait for connection with the tablet computer. The main program of the mobile game assistant can also start applications that need streaming, such as game applications, and the main program of the mobile game assistant can also create a device connection service.

[0191] This way, the tablet and phone can establish a connection based on the device connection service.

[0192] (2) Initialization of the module.

[0193] In the tablet computer, the main program of the mobile game assistant can instruct the OpenGL instruction stream parsing module to create an OpenGL instruction parsing thread, and can also instruct the echo module to create an echo thread, which can be used to send screen-related data. It is understandable that the creation process of the echo thread and the creation process of the OpenGL instruction stream parsing module can be executed in any order.

[0194] After creating the echo thread and the OpenGL command stream parsing module, the main program of the mobile game assistant can send a receipt message to the mobile phone, which can be used to indicate that the tablet has completed the module initialization and can also be used to indicate that the mobile phone can send OpenGL commands. Therefore, the mobile phone can start the OpenGL command interception module based on the receipt message to intercept the OpenGL commands and send the intercepted OpenGL commands to the tablet.

[0195] (3) Screen drawing and display.

[0196] In a tablet computer, the OpenGL instruction stream parsing module can parse the OpenGL instructions, the echo module can render the screen for the parsed OpenGL instructions, and transmit the relevant data of the screen to the mobile phone based on the device connection service, so that the echo module of the mobile phone can display the interface based on the relevant data of the screen.

[0197] It is understandable that after the main program service of the tablet computer is started, multiple socket servers can be created, for example, supporting 5 socket service connections, and creating 5 thread pools to support the connection of 5 application devices. Among them, the main program service can also be called the main program thread.

[0198] After the tablet computer waits for mobile phone A to connect, it can start the socket service thread A1 of mobile phone A, start the OpenGL instruction parsing module thread A2 of mobile phone A, and start the echo module thread A3 of mobile phone A, so as to perform the streaming echo processing of mobile phone B.

[0199] When mobile phone B requests to access the tablet computer, the tablet computer can connect to mobile phone B, start the socket service thread B1 of mobile phone B, start the OpenGL instruction parsing module thread B2 of mobile phone B, and start the echo module thread B3 of mobile phone B, so as to perform streaming echo processing of mobile phone B.

[0200] It is understandable that the connection between the tablet and mobile phone A, the connection between the tablet and mobile phone B, the streaming and the echo all belong to different context handles, that is, different context data. In other words, the data domain of mobile phone A and the data domain of mobile phone B are isolated and will not affect each other.

[0201] The tablet can also wait for the next device to connect, no further details are given.

[0202] When mobile phone A disconnects, the tablet device can release the resources of various modules of mobile phone B, such as streaming echo, etc. After the main program service of the tablet detects that all devices are offline, it can release all resources and enter a dormant state.

[0203] After the tablet computer (device B) uses streaming to parse the OpenGL command stream, it needs to echo the currently parsed image to the mobile phone (device A).

[0204] like Fig.18 As shown, the current mobile phone's media layer and window layer use the default display logic, and cannot render the echoed image directly onto the application layer. In addition, the echoed image does not intercept user touch events and interacts with the window layer UI. The echoed layer lifecycle needs to be consistent with the current application, so as not to affect the creation and display logic of the current application layer.

[0205] The application layer may include a media layer and a window layer. The media layer is responsible for the carrying and display of multimedia images and is at the lowest end of the application layer. The window layer is responsible for UI interactive display and is at the top of the current application layer.

[0206] In the present application embodiment, Fig.19 As shown, when the game assistant application on the mobile phone (device A) is started, it can be determined whether it supports screen drawing based on the tablet computer (device B), which can also be understood as whether it supports the OpenGL instruction streaming function. Optionally, the user can manually trigger the function to be turned on or off in the game assistant application. If the function is turned on in the game assistant application, it means that the game assistant application supports screen drawing on the tablet computer. When the game assistant application is started, the system side of the mobile phone will create a new echo layer based on the current application layer information, such as Fig. 20 The following figure shows the display effect of the application layer. The echo layer is responsible for streaming the echo screen and is located in the middle of the application layer. The echo layer is located above the media layer.

[0207] At the same time, the mobile phone can bind the life cycle of the echo layer to the application layer, and the echo layer does not process touch events. The touch events of the entire screen will be transmitted to the application layer for processing.

[0208] For example, when a game application in a mobile phone requests the GPU to synthesize a frame, the mobile phone can intercept the current OpenGL command and transmit the OpenGL command to the tablet. After receiving the OpenGL command, the tablet can call the tablet's GPU drawing capabilities to draw and synthesize the picture. After synthesizing a frame, the tablet can encode and compress the picture and transmit the picture data back to the mobile phone through a network connection.

[0209] After receiving a frame of picture data, the mobile phone can decode the picture data and render it to the echo layer for display.

[0210] Taking Android devices as an example, Fig.21 The flowchart of the mobile phone screen echo is shown.

[0211] S2101. When the mobile phone is started, a layer management class (RemoteSurfaceManager) object may be created. The layer management class object may be used to create an echo layer.

[0212] S2102. When the game application is started, the game application can call the interface provided by the system, initialize the application layer (SurfaceView), and create a media layer. It is understandable that the layer management class object can listen to the creation event of the media layer. When the layer management class object listens to the game application creating a media layer, the layer management class object can request the activity task management service (ActivityTaskManagerService) to create an echo layer, and the activity task management service can request the layer management class to create an echo layer. Among them, the echo layer is above the media layer of the mobile phone screen and below the window layer.

[0213] S2103. The activity task management service (ActivityTaskManagerService) may start a streaming connection to perform OpenGL command streaming transmission.

[0214] It can be understood that the process of transmitting the OpenGL instruction stream in step S2103 and the process of creating the echo layer in step S2101 can be executed in parallel.

[0215] S2104. The mobile phone may start a picture analysis class (RemoteMediaCodec), which may wait for decoding and echoing of the picture video. When the picture analysis class is started, the created echo layer may be obtained from the layer management class, thereby waiting for the echo.

[0216] It is understandable that the process of transmitting the OpenGL instruction stream in step S2103 and the process of obtaining the echo layer in step S2104 can be executed in parallel.

[0217] S2105. Through the device connection service (RemoteConnect) on the mobile phone side and the device connection service (PCRemoteConnect) on the tablet computer side, the OpenGL streaming data can be transmitted to the tablet computer, and then the management module (PCRemoteManager) of the tablet computer can perform command parsing and screen synthesis based on the OpenGL streaming data.

[0218] S2106. After the image synthesis is completed, the image analysis class (PCRemoteMediaCodec) in the tablet computer can encode and compress the image, and transmit the encoded and compressed image data back to the mobile phone through the device connection service.

[0219] S2107. After receiving a frame of picture data, the mobile phone may request the picture analysis class to first decode the picture.

[0220] S2108. After the picture is decoded, the picture parsing class can render the decoded frame of data to the echo layer through surfaceFlinger.

[0221] It is understandable that the amount of original data after game rendering is large. For example, when the resolution of some game applications is 60FPS, the data volume may reach about 2800Mbit / s, while the actual speed of mobile phone Wi-Fi may be up to about 720Mbit / s, so it cannot meet the original data transmission of the game.

[0222] When the mobile phone Wi-Fi speed cannot meet the original data transmission of the game, the data of the mobile phone may not be sent to the tablet in time, which may cause the game screen to not follow the hand, the game to freeze, etc.

[0223] It is understandable that, when intercepting the data after game rendering, although each piece of data is almost different, there are only slight changes between the data of most consecutive pictures, which makes the compression effect of these data poor and the amount of data transmitted does not decrease significantly.

[0224] like Fig. 22 As shown, in the embodiment of the present application, the OpenGL Hook mechanism of the mobile phone can obtain OpenGL instruction data. When the amount of data is too large, the mobile phone can reduce the amount of data through the differential mechanism and compression algorithm, thereby sending the OpenGL instruction data processed by the differential mechanism and compression algorithm to the tablet computer.

[0225] After receiving the OpenGL instruction data, the tablet computer may process the OpenGL instruction data through a decompression algorithm and a differential mechanism, thereby obtaining processed OpenGL instruction data.

[0226] During the game, some continuous images have small changes, but the complete data needs to be transmitted. Therefore, the mobile phone can transmit the changed part to the tablet computer. Based on this, the differential mechanism can be used for data transmission. In a possible implementation, the changed part can be extracted through the differential algorithm, thereby improving the compression rate and reducing the amount of data transmission.

[0227] Fig.23 The calculation method of the difference mechanism on the mobile phone side is shown. Since the OpenGL instruction data can be transmitted in binary form, a small piece of binary data is used as an example.

[0228] In a possible implementation, the differential mechanism can compare the new data with the old data to find out the changed parts. It can also be understood as subtracting the new data from the old data to finally obtain the differential data, where the new data can be understood as the current frame data and the old data can be understood as the previous frame data.

[0229] If the difference between the two frames of data is small, there will be more continuous and repeated 0s in the differential data. At this time, most of the 0s can be removed through the compression algorithm, thereby improving the compression rate, reducing the amount of transmitted data, and speeding up data transmission.

[0230] Fig.24 The calculation method of the differential mechanism on the tablet computer side is shown. After receiving the OpenGL instruction data, the tablet computer can decompress the data and compare the new data with the old data, which can also be understood as adding the new data to the old data to obtain the new data. Further, by parsing the OpenGL instruction data, the tablet computer can execute the OpenGL instruction and draw the screen.

[0231] The following specific embodiments are used to describe the method of the present application in detail. The following embodiments may be combined with each other or implemented independently, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0232] Fig.25 The data processing method of an embodiment of the present application is shown. The method is applied to a first electronic device, and the method includes:

[0233] S2501. Establish a communication connection with a second electronic device.

[0234] In the embodiment of the present application, the first electronic device may include any electronic device with a GPU device, and the second electronic device may include any electronic device with a GPU device. It should be understood that the graphics capability of the GPU device of the first electronic device is less than the graphics capability of the GPU device of the second electronic device. For example, the first electronic device may include the mobile phone in the above embodiment, and the second electronic device may include the tablet computer in the above embodiment.

[0235] Among them, the drawing capability includes one or more of the following: the computing power of the GPU device, the running speed of the GPU device, the hardware configuration of the GPU device, the network card performance of the GPU device, the hardware performance of the GPU device, the quality of the GPU device and / or the memory capacity of the GPU device, etc.

[0236] A wireless or wired communication connection can be established between the first electronic device and the second electronic device, including, for example, a socket connection, a USB connection, a Bluetooth connection, a near-field communication connection, an HTTP connection, an HTTPS connection, etc., which is not limited in the embodiments of the present application.

[0237] S2502: If the first electronic device determines that the network meets the network quality requirement, it sends a drawing instruction for drawing a first picture to the second electronic device.

[0238] In the embodiment of the present application, the network meeting the network quality requirements can be understood as a good network signal. According to the above embodiment, whether the network meets the network quality requirements can be judged according to the number of OpenGL instructions remaining in the OpenGL instruction queue. Exemplarily, when the number of OpenGL instructions remaining in the OpenGL instruction queue is greater than or equal to 3, it can be considered that the network meets the network quality requirements.

[0239] The first screen may be understood as a screen that the first electronic device needs to display, for example, the first screen includes a game screen. The drawing instruction may include an OpenGL instruction. The drawing instruction may include a function name and function parameters of a function for drawing a screen, and the function parameters may include information required for drawing a screen, for example, a picture, a string, etc., required for drawing a screen, a screen resolution, etc.

[0240] S2503: Display a first picture, wherein the first picture is drawn based on a GPU device of the second electronic device, and the drawing capability of the GPU device of the first electronic device is smaller than the drawing capability of the GPU device of the second electronic device.

[0241] In the embodiment of the present application, the drawing capability can refer to the relevant description in step S2501 and will not be repeated here.

[0242] S2504: If the first electronic device determines that the network does not meet the network quality requirement, draw a second screen.

[0243] In the embodiment of the present application, the network not meeting the network quality requirement can be understood as a poor network signal. According to the above embodiment, whether the network meets the network quality requirement can be judged according to the number of OpenGL instructions remaining in the OpenGL instruction queue. Exemplarily, when the number of OpenGL instructions remaining in the OpenGL instruction queue is less than 3, it can be considered that the network does not meet the network quality requirement.

[0244] When the network does not meet the network quality requirements, the first electronic device can perform local rendering, that is, the first electronic device can render the second screen based on the GPU device of the first electronic device. The second screen can be understood as the screen that the first electronic device needs to display, for example, the second screen includes a game screen.

[0245] It is understandable that when the network does not meet the network quality requirements, the first electronic device may not send a drawing instruction for drawing the second screen to the second electronic device; or, the first electronic device may also send a drawing instruction for drawing the second screen to the second electronic device, but does not receive the second screen from the second electronic device; or, the first electronic device may also send a drawing instruction for drawing the second screen to the second electronic device, but does not receive the second screen from the second electronic device; or, the first electronic device may also send a drawing instruction for drawing the second screen to the second electronic device and receive the second screen from the second electronic device, but does not display the second screen from the second electronic device. This is not limited to the embodiments of the present application.

[0246] S2505: Display the second screen.

[0247] In the embodiment of the present application, the first electronic device can exchange data with the second electronic device with stronger drawing capability, so that the second electronic device can draw a screen, and send the drawn screen to the first electronic device, and then the first electronic device displays the screen. In this way, based on the drawing capability of the second electronic device, the image quality of the interface in the first electronic device can be improved, the smoothness of application operation can be improved, the power consumption of the first electronic device can be reduced, and the heating of the first electronic device can be reduced, thereby improving the user experience.

[0248] Optional, in Fig.25On the basis of the corresponding embodiments, if the first electronic device determines that the network meets the network quality requirements, sending a drawing instruction for drawing the first screen to the second electronic device may include: if the number of instructions in the target instruction queue is less than a preset value, sending a drawing instruction for drawing the first screen to the second electronic device, wherein the target instruction queue is used to store the drawing instructions, and the drawing instructions for the first screen are obtained from the target instruction queue; if the first electronic device determines that the network does not meet the network quality requirements, drawing the second screen may include: if the number of instructions in the target instruction queue is greater than or equal to the preset value, drawing the second screen, wherein the drawing instructions for the second screen are obtained from the target instruction queue.

[0249] In the embodiment of the present application, the target instruction queue can be understood as Figure 4 The OpenGL command queue in the corresponding embodiment.

[0250] The preset value may be pre-set by the electronic device, for example, the preset value may include 3, and the specific value of the preset value is not limited in the embodiment of the present application.

[0251] The specific implementation process of determining whether to send a drawing instruction to the second electronic device according to the number of instructions in the target instruction queue can refer to Figure 4 The relevant descriptions in the corresponding embodiments are not repeated here.

[0252] It is understandable that by determining the quality of the network based on the number of instructions in the target instruction queue, the network signal status can be quickly determined. The number of instructions in the target instruction queue can be used to detect changes in the network signal in real time, thereby switching between local screen drawing and remote screen drawing in a timely manner.

[0253] Optional, in Fig.25 On the basis of the corresponding embodiments, if the first electronic device determines that the network meets the network quality requirements, sending a drawing instruction for drawing the first screen to the second electronic device may include: if the target flag is a first value, sending a drawing instruction for drawing the first screen to the second electronic device, wherein, when the number of instructions in the target instruction queue is less than a preset value, the target flag is the first value, and the first value is used to instruct the GPU device based on the second electronic device to draw the screen; if the first electronic device determines that the network does not meet the network quality requirements, drawing the second screen may include: if the target flag is a second value, drawing the second screen, wherein, when the number of instructions in the target instruction queue is greater than or equal to the preset value, the target flag is the second value, and the second value is used to instruct the GPU device based on the first electronic device to draw the screen.

[0254] In the embodiment of the present application, the target flag bit can be understood as Figure 4The local drawing flag bit or the remote drawing flag bit in the corresponding embodiment is not limited in the embodiment of the present application. The target flag bit being the first value may indicate that the network signal is good and the screen drawing can be performed through the second electronic device. The target flag bit being the second value may indicate that the network signal is poor and the screen drawing can be performed locally.

[0255] For example, when the target flag is a local drawing flag, the first value may include false, and the second value may include true. When the target flag is a remote drawing flag, the first value may include true, and the second value may include false. For a specific description of the local drawing flag or the remote drawing flag, please refer to Figure 4 Corresponding to the relevant description in the embodiment, the target flag bit can also be understood as Figure 8 The Last flag bit in the corresponding embodiment will not be described in detail.

[0256] It is understandable that by determining the quality of the network based on the target flag, the network signal status can be quickly determined. The target flag is used to conveniently record the status of the network signal, simplify the control process of code execution in the first electronic device, and improve the readability and flexibility of the code, so that changes in the network signal can be concisely identified, and local screen drawing and remote screen drawing can be switched in a timely manner.

[0257] Optional, in Fig.25 On the basis of the corresponding embodiments, the method may further include: if the first electronic device determines that the network switches from not meeting the network quality requirements to meeting the network quality requirements, the electronic device executes the following process: sending a drawing instruction to the second electronic device; within a preset time period, the first electronic device draws the screen, and the first electronic device does not use the screen sent by the second electronic device; after the preset time period, the first electronic device does not draw the screen, updates the target flag to the first value, and uses the screen sent by the second electronic device; if the first electronic device determines that the network switches from meeting the network quality requirements to not meeting the network quality requirements, the first electronic device executes the following process: drawing the screen and updating the target flag to the second value.

[0258] In the embodiment of the present application, when the network switches from not meeting the network quality requirement to meeting the network quality requirement, it can be understood that the network signal changes from weak to strong, and the first electronic device can draw the screen through the second electronic device. In this way, the image quality of the interface of the first electronic device can be improved based on the stronger drawing ability of the second electronic device.

[0259] When the network switches from satisfying the network quality requirement to not satisfying the network quality requirement, it can be understood that the network signal changes from strong to weak, and the first electronic device can perform local screen drawing. In this way, the situation where the drawing instruction cannot be sent can be reduced, thereby reducing the situation where the game screen is out of sync and the operation is not in sync with the hand when the user plays the game.

[0260] The process of updating the target flag can refer to Figure 8 The relevant descriptions in the corresponding embodiments are not repeated here.

[0261] The preset time period may be preset by the electronic device, for example, the preset time period may include 100 ms. The specific value of the preset time period is not limited in the embodiment of the present application.

[0262] It is understandable that after delaying the preset time period, the first electronic device does not draw the screen and displays the screen sent by the second electronic device. In this way, the first electronic device can first send the instructions accumulated in the target instruction queue, and give the second electronic device some processing time, waiting for the second electronic device to resume normal processing speed, so that the screen can be connected between the first electronic device and the second electronic device, thereby improving the smoothness of the screen.

[0263] Optional, in Fig.25 Based on the corresponding embodiments, the method may further include: if the first electronic device determines that the network does not meet the network quality requirements, establishing a communication connection with a third electronic device; after establishing the communication connection with the third electronic device, if the first electronic device determines that the network meets the network quality requirements, sending a drawing instruction for drawing a third screen to the third electronic device; displaying the third screen, wherein the third screen is drawn based on the GPU device of the third electronic device, and the drawing capability of the GPU device of the first electronic device is less than the drawing capability of the GPU device of the third electronic device.

[0264] In the embodiment of the present application, the third electronic device may include any electronic device with a GPU device. It should be understood that the graphics capability of the GPU device of the first electronic device is less than the graphics capability of the GPU device of the third electronic device. For example, the third electronic device may include the tablet computer in the above embodiment. The graphics capability can refer to the relevant description in step S2501 and will not be repeated here.

[0265] The third screen may be understood as a screen that the first electronic device needs to display, for example, the third screen includes a game screen.

[0266] It is understandable that the second electronic device may have a poor network signal, for example, the network card performance or hardware performance of the second electronic device may be relatively poor, which may affect screen drawing or data transmission. In a possible implementation, the first electronic device can also switch with multiple electronic devices to find an electronic device with a better network status, such as the third electronic device, to improve the efficiency of screen drawing, thereby bringing a better game screen experience to the user.

[0267] Optional, in Fig.25 Based on the corresponding embodiment, before sending a drawing instruction for drawing a third screen to the third electronic device, the method may also include: acquiring target data from the second electronic device, where the target data is used to identify the drawing progress of the second electronic device; and sending the target data to the third electronic device.

[0268] In the embodiment of the present application, the target data may include Fig. 9 The relevant data in the corresponding embodiment, for example, the relevant data may include context data. The specific process of obtaining the target data from the second electronic device and sending the target data to the third electronic device can refer to Fig. 9 Corresponding embodiments and Fig.10 The relevant description of the corresponding embodiment will not be repeated here.

[0269] It is understandable that the first electronic device can increase the speed of screen drawing by establishing a connection with the third electronic device with better network status, thereby reducing the impact of network signals on screen display. In addition, the target data can make the GPU context consistent, and the screen drawing can be seamlessly connected. In this way, when the first electronic device switches to the third electronic device, the screen can be continuous, improving the user experience.

[0270] Optional, in Fig.25 On the basis of the corresponding embodiments, establishing a communication connection with the second electronic device may include: establishing a first Socket communication and a second Socket communication with the second electronic device; before sending a drawing instruction for drawing a first screen to the second electronic device, it may also include: numbering the drawing instruction; sending a drawing instruction for drawing the first screen to the second electronic device may include: sending a drawing instruction whose number meets a first condition to the second electronic device through the first Socket communication, and sending a drawing instruction whose number meets a second condition to the second electronic device through the second Socket communication; wherein the first condition is used to determine whether the first Socket communication is applicable, and the second condition is used to determine whether the second Socket communication is applicable.

[0271] In the embodiment of the present application, the first Socket communication can be understood as Fig.13Corresponding to the socket communication 1 in the embodiment, the second socket communication can be understood as Fig.13 Corresponding to the socket communication 2 in the embodiment. Multiple socket communications can be established between the first electronic device and the second electronic device, which is not limited in the embodiment of the present application.

[0272] The first condition may include that the number of the drawing instruction is an odd number, and the second condition may include that the number of the drawing instruction is an even number, or the first condition may include that the number of the drawing instruction is an even number, and the second condition may include that the number of the drawing instruction is an odd number. Of course, other conditions may also be used, as long as it can determine whether the first Socket communication or the second Socket communication is applicable, and the embodiment of the present application does not limit this.

[0273] The specific process of numbering drawing instructions can be referred to Fig.13 The relevant description of the corresponding embodiment will not be repeated here.

[0274] It is understandable that establishing multiple socket communications between the first electronic device and the second electronic device can increase the speed of sending drawing instructions, thereby reducing problems such as the screen not following the hand and the screen freezing.

[0275] Optional, in Fig.25 On the basis of the corresponding embodiment, before sending the drawing instruction for drawing the first screen to the second electronic device, it may also include: performing differential calculation on the drawing instruction, and compressing the drawing instruction after the differential calculation; sending the drawing instruction for drawing the first screen to the second electronic device may include: sending the compressed drawing instruction for drawing the first screen to the second electronic device.

[0276] In the embodiment of the present application, the specific process of performing differential calculation and compression on the drawing instruction can be referred to Fig. 22 The process of the first electronic device performing the differential calculation can refer to the relevant description of the corresponding embodiment. Fig.23 The relevant description of the corresponding embodiment will not be repeated here.

[0277] It is understandable that during the display process, since some consecutive images have small changes, the first electronic device can transmit the modified part to the second electronic device, that is, the differential mechanism can be used for data transmission. In this way, the modified part is extracted through the differential algorithm, thereby improving the compression rate, reducing the amount of data transmission, and speeding up the data transmission.

[0278] Fig.26 Another data processing method according to an embodiment of the present application is shown. The method is applied to a second electronic device, and the method includes:

[0279] S2601. Establish a communication connection with one or more electronic devices.

[0280] In the embodiment of the present application, the second electronic device can refer to Fig.25 The relevant description in step S2501 of the corresponding embodiment is not repeated here.

[0281] S2602: Receive drawing instructions from one or more electronic devices.

[0282] In the embodiment of the present application, one or more electronic devices may include any electronic device with a GPU device. It should be understood that the graphics capabilities of the GPU devices of the one or more electronic devices are less than the graphics capabilities of the GPU device of the second electronic device. For example, the one or more electronic devices may include the mobile phone in the above embodiment.

[0283] S2603. Based on the GPU device of the second electronic device, draw a picture for one or more electronic devices according to drawing instructions of one or more electronic devices, wherein the drawing capabilities of the GPU devices of the one or more electronic devices are smaller than the drawing capabilities of the GPU device of the second electronic device.

[0284] In the embodiment of the present application, the process of the second electronic device drawing a screen for one or more electronic devices can refer to Fig.14 Corresponding embodiments to Fig.15 The relevant descriptions in the corresponding embodiments are not repeated here.

[0285] It is understandable that the process of the second electronic device drawing a screen for each electronic device can be isolated and not affect each other. For example, the second electronic device can establish a socket communication connection with each electronic device respectively, and the data interaction between the second electronic device and one of the electronic devices will not affect the data interaction between the second electronic device and other electronic devices, and the screen transmission performed by the second electronic device and one of the electronic devices will not affect the screen transmission of the tablet computer and other mobile phones.

[0286] S2604. Send the screen corresponding to each electronic device to one or more electronic devices respectively.

[0287] In the embodiment of the present application, the second electronic device with stronger drawing ability draws a screen for one or more electronic devices with weaker drawing ability, and sends the drawn screen to each electronic device. In this way, based on the drawing ability of the second electronic device, the image quality of the interface in one or more electronic devices can be improved, the smoothness of application operation can be improved, the power consumption of one or more electronic devices can be reduced, and the heating of one or more electronic devices can be reduced, thereby improving the user experience.

[0288] Optional, in Fig.26On the basis of the corresponding embodiments, one or more electronic devices include a first electronic device, and a GPU device based on a second electronic device, before drawing a screen for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, it may also include: receiving a target flag from the first electronic device; based on the GPU device of the second electronic device, drawing a screen for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, it may include: if the target flag is a first value, based on the GPU device of the second electronic device, drawing a screen for the first electronic device according to the drawing instructions of the first electronic device, wherein the first value is used to instruct the GPU device based on the second electronic device to draw the screen.

[0289] In the embodiment of the present application, the first electronic device can refer to Fig.25 The relevant description in step S2501 of the corresponding embodiment is not repeated here.

[0290] The target flag can be understood as Figure 4 The local drawing flag bit or the remote drawing flag bit in the corresponding embodiment is not limited in the embodiment of the present application. The target flag bit being the first value may indicate that the network signal is good and the screen can be drawn through the second electronic device. For the specific description of the target flag bit and the first value, please refer to Figure 4 Corresponding embodiments and Fig.25 Corresponding to the relevant description in the embodiment, the target flag bit can also be understood as Figure 8 The Last flag bit in the corresponding embodiment will not be described in detail.

[0291] It is understandable that by determining the quality of the network based on the target flag, the network signal status can be quickly determined. The target flag is used to conveniently record the status of the network signal, simplify the control process of code execution in the first electronic device, and improve the readability and flexibility of the code, so that changes in the network signal can be concisely identified, and local screen drawing and remote screen drawing can be switched in a timely manner.

[0292] Optional, in Fig.26On the basis of the corresponding embodiments, establishing a communication connection with one or more electronic devices may include: establishing a first Socket communication and a second Socket communication with a first electronic device; receiving drawing instructions from one or more electronic devices may include: receiving, through the first Socket communication, a drawing instruction whose number satisfies a first condition from the first electronic device, and storing it in a first queue, wherein the first condition is used to indicate that the drawing instruction is transmitted through the first Socket communication; receiving, through the second Socket communication, a drawing instruction whose number satisfies a second condition from the first electronic device, and storing it in a second queue, wherein the second condition is used to indicate that the drawing instruction is transmitted through the second Socket communication; based on the GPU device of the second electronic device, according to the drawing instructions of one or more electronic devices, before drawing a picture for one or more electronic devices, it may also include: obtaining the drawing instructions for drawing the picture corresponding to the first electronic device from the first queue or the second queue in the order of numbers.

[0293] In the embodiment of the present application, the first Socket communication can be understood as Fig.13 Corresponding to the socket communication 1 in the embodiment, the second socket communication can be understood as Fig.13 Corresponding to the socket communication 2 in the embodiment. Multiple socket communications can be established between the first electronic device and the second electronic device, which is not limited in the embodiment of the present application.

[0294] The first queue can be understood as Fig.13 Corresponding to the queue 1 in the embodiment, the second queue can be understood as Fig.13 Corresponding to queue 2 in the embodiment,

[0295] For the specific process of obtaining drawing instructions from the first queue or the second queue in the order of numbers, please refer to Fig.13 The relevant description of the corresponding embodiment will not be repeated here.

[0296] It is understandable that establishing multiple socket communications between the first electronic device and the second electronic device can increase the speed of sending drawing instructions, thereby reducing problems such as the screen not following the hand and the screen freezing.

[0297] Optional, in Fig.26On the basis of the corresponding embodiment, after receiving the drawing instructions from one or more electronic devices, it can also include: decompressing the drawing instructions of the one or more electronic devices and performing differential calculation on the decompressed drawing instructions; based on the GPU device of the second electronic device, drawing a picture for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, it can include: based on the GPU device of the second electronic device, using the drawing instructions of the one or more electronic devices after differential calculation to draw a picture for the one or more electronic devices.

[0298] In the embodiment of the present application, the specific process of performing differential calculation and compression on the drawing instruction can be referred to Fig. 22 The process of the second electronic device performing the differential calculation can refer to the relevant description of the corresponding embodiment. Fig.24 The relevant description of the corresponding embodiment will not be repeated here.

[0299] It is understandable that during the display process, since some consecutive images have small changes, the first electronic device can transmit the modified part to the second electronic device, that is, the differential mechanism can be used for data transmission. In this way, the modified part is extracted through the differential algorithm, thereby improving the compression rate, reducing the amount of data transmission, and speeding up the data transmission.

[0300] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0301] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0302] The embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0303] like Fig. 27 FIG. 2 is a schematic diagram of a chip structure provided by an embodiment of the present application. The chip 2700 includes one or more (including two) processors 2701 , a communication line 2702 , a communication interface 2703 and a memory 2704 .

[0304] In some embodiments, the memory 2704 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0305] The method described in the above embodiment of the present application can be applied to the processor 2701, or implemented by the processor 2701. The processor 2701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 2701 or an instruction in the form of software. The above processor 2701 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 2701 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0306] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 2704, and the processor 2701 reads the information in the memory 2704 and completes the steps of the above method in combination with its hardware.

[0307] The processor 2701 , the memory 2704 , and the communication interface 2703 may communicate with each other via the communication line 2702 .

[0308] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0309] The embodiment of the present application also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium may include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0310] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0311] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically using lasers.

[0312] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. A data processing method, characterized in that: The method is applied to a first electronic device, and the method includes: establishing a communication connection with a second electronic device; If the first electronic device determines that the network meets the network quality requirement, sending a drawing instruction for drawing a first picture to the second electronic device; Displaying the first picture, wherein the first picture is drawn based on a GPU device of the second electronic device, and a drawing capability of the GPU device of the first electronic device is smaller than a drawing capability of the GPU device of the second electronic device; If the first electronic device determines that the network does not meet the network quality requirement, drawing a second screen; displaying the second picture; Before sending the drawing instruction for drawing the first picture to the second electronic device, the method further includes: Performing differential calculation on the drawing instructions, and compressing the drawing instructions after the differential calculation; The sending a drawing instruction for drawing the first picture to the second electronic device includes: The compressed drawing instruction for drawing the first picture is sent to the second electronic device.

2. The method according to claim 1, characterized in that If the first electronic device determines that the network meets the network quality requirement, sending a drawing instruction for drawing a first picture to the second electronic device includes: If the number of instructions in the target instruction queue is less than a preset value, sending a drawing instruction for drawing the first screen to the second electronic device, wherein the target instruction queue is used to store drawing instructions, and the drawing instruction for the first screen is obtained from the target instruction queue; If the first electronic device determines that the network does not meet the network quality requirement, drawing a second screen includes: If the number of instructions in the target instruction queue is greater than or equal to the preset value, the second screen drawing is performed, wherein the drawing instructions of the second screen are obtained from the target instruction queue.

3. The method according to claim 2, characterized in that If the first electronic device determines that the network meets the network quality requirement, sending a drawing instruction for drawing a first picture to the second electronic device includes: If the target flag is a first value, a drawing instruction for drawing the first screen is sent to the second electronic device, wherein when the number of instructions in the target instruction queue is less than the preset value, the target flag is the first value, and the first value is used to instruct the GPU device based on the second electronic device to draw the screen; If the first electronic device determines that the network does not meet the network quality requirement, drawing a second screen includes: If the target flag is a second value, the second screen drawing is performed, wherein, when the number of instructions in the target instruction queue is greater than or equal to the preset value, the target flag is the second value, and the second value is used to instruct the GPU device based on the first electronic device to perform screen drawing.

4. The method according to claim 3, characterized in that The method further comprises: If the first electronic device determines that the network switches from not meeting the network quality requirement to meeting the network quality requirement, the electronic device performs the following process: Sending a drawing instruction to the second electronic device; During a preset time period, the first electronic device draws a picture, and the first electronic device does not use the picture sent by the second electronic device; After the preset time period, the first electronic device does not draw the screen, updates the target flag to the first value, and uses the screen sent by the second electronic device; If the first electronic device determines that the network switches from satisfying the network quality requirement to not satisfying the network quality requirement, the first electronic device executes the following process: drawing a screen and updating the target flag to the second value.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: If the first electronic device determines that the network does not meet the network quality requirement, establishing a communication connection with a third electronic device; After establishing a communication connection with the third electronic device, if the first electronic device determines that the network meets the network quality requirement, sending a drawing instruction for drawing a third picture to the third electronic device; The third picture is displayed, wherein the third picture is drawn based on the GPU device of the third electronic device, and the drawing capability of the GPU device of the first electronic device is smaller than the drawing capability of the GPU device of the third electronic device.

6. The method according to claim 5, characterized in that Before sending the drawing instruction for drawing the third screen to the third electronic device, the method further includes: Acquire target data from the second electronic device, where the target data is used to identify a drawing progress of the second electronic device; The target data is sent to the third electronic device.

7. The method according to any one of claims 1 to 4, characterized in that: The establishing a communication connection with the second electronic device includes: Establishing a first Socket communication and a second Socket communication with the second electronic device; Before sending the drawing instruction for drawing the first picture to the second electronic device, the method further includes: Numbering the drawing instructions; The sending a drawing instruction for drawing the first picture to the second electronic device includes: A drawing instruction whose number satisfies a first condition is sent to the second electronic device through the first Socket communication, and a drawing instruction whose number satisfies a second condition is sent to the second electronic device through the second Socket communication, wherein the first condition is used to determine whether the first Socket communication is applicable, and the second condition is used to determine whether the second Socket communication is applicable.

8. A data processing method, characterized in that: The method is applied to a second electronic device, and the method includes: establishing a communication connection with one or more electronic devices; receiving a drawing instruction from the one or more electronic devices; Based on the GPU device of the second electronic device, according to the drawing instructions of the one or more electronic devices, drawing pictures for the one or more electronic devices, wherein the drawing capabilities of the GPU devices of the one or more electronic devices are all smaller than the drawing capabilities of the GPU device of the second electronic device; Sending the screen corresponding to each electronic device to the one or more electronic devices respectively; After receiving the drawing instruction from the one or more electronic devices, the method further includes: Decompressing the drawing instructions of the one or more electronic devices, and performing differential calculation on the decompressed drawing instructions; The GPU device based on the second electronic device draws a picture for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, including: Based on the GPU device of the second electronic device, the drawing instructions of the one or more electronic devices after the difference calculation are used to draw pictures for the one or more electronic devices.

9. The method according to claim 8, characterized in that The one or more electronic devices include a first electronic device, and the GPU device based on the second electronic device, before drawing a picture for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, further includes: Receiving a target flag bit from the first electronic device; The GPU device based on the second electronic device draws a picture for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, including: If the target flag is a first value, a GPU device of the second electronic device draws a picture for the first electronic device according to a drawing instruction of the first electronic device, wherein the first value is used to instruct the GPU device of the second electronic device to draw the picture.

10. The method according to claim 9, characterized in that The establishing of a communication connection with one or more electronic devices comprises: Establishing a first Socket communication and a second Socket communication with the first electronic device; The receiving of the drawing instruction from the one or more electronic devices comprises: Receiving, through the first Socket communication, a drawing instruction whose number satisfies a first condition from the first electronic device and storing the drawing instruction in a first queue, wherein the first condition is used to indicate that the drawing instruction is transmitted through the first Socket communication; receiving, via the second Socket communication, a drawing instruction whose number satisfies a second condition from the first electronic device and storing the instruction in a second queue, wherein the second condition is used to indicate that the drawing instruction is transmitted via the second Socket communication; Before the GPU device based on the second electronic device draws a picture for the one or more electronic devices according to the drawing instructions of the one or more electronic devices, the GPU device further includes: A drawing instruction for drawing a screen corresponding to the first electronic device is obtained from the first queue or the second queue in a sequence of numbers.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, wherein the computer program code includes computer instructions, and the one or more processors call the computer instructions to execute the method executed by the first electronic device in the method described in any one of claims 1-10, or the method executed by the second electronic device in the method described in any one of claims 1-10.

12. An interactive system, characterized in that: include: A first electronic device and a second electronic device, wherein the first electronic device is used for the method performed by the first electronic device in the method described in any one of claims 1-10, and the second electronic device is used for the method performed by the second electronic device in the method described in any one of claims 1-10.

13. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method executed by the first electronic device in the method described in any one of claims 1-10, or the method executed by the second electronic device in the method described in any one of claims 1-10.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method executed by the first electronic device in the method described in any one of claims 1 to 10, or the method executed by the second electronic device in the method described in any one of claims 1 to 10.

15. A computer program product, characterized in that The computer program product includes a computer program code, which, when executed on an electronic device, enables the electronic device to execute the method executed by the first electronic device in the method described in any one of claims 1 to 10, or the method executed by the second electronic device in the method described in any one of claims 1 to 10.

Images