Display data processing method, device, electronic device and computer storage medium

Abstract

Embodiments of the present invention provide a display data processing method, apparatus, electronic device, and computer storage medium. The display data processing method includes: generating an application request based on a user's operation on a terminal application on a terminal device, and sending the application request to a data processing end; obtaining second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on first display data with preset data filtered out, and the first display data is data obtained by a cloud application on the data processing end in response to the application request; and displaying the second display data. Through embodiments of the present invention, application requests can be processed by a cloud application without the user being aware of the cloud application.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a display data processing method, device, electronic device, and computer storage medium. Background Art

[0002] As cloud computing matures, an increasing number of applications, particularly those requiring high device performance, are being deployed and run on data processing platforms (e.g., in the cloud, on servers, or other IoT devices with sufficient computing power). These applications are also known as cloud applications. A characteristic of cloud applications is that, although the corresponding terminal application is installed on the terminal device, the primary data processing for the terminal application is performed by the cloud application rather than the terminal application itself, thus reducing the performance requirements placed on the terminal device.

[0003] Existing cloud applications run in containers or virtual machines, which in turn run on data processing terminals. The data processing terminals send the cloud application's execution results to the user's terminal device, which then outputs the results through its local terminal application, such as displaying and / or playing data.

[0004] However, there are usually some problems when terminal applications and cloud applications are used together. For example, some image data of the operating system where the cloud application is located will be displayed in the terminal application interface, causing users to be able to perceive the cloud application and its operating system, affecting the user experience. In addition, these image data also increase the data processing burden of the terminal device and waste resources. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a display data processing solution to solve some or all of the above problems.

[0006] According to a first aspect of an embodiment of the present invention, a display data processing method is provided, comprising: generating an application request based on a user's operation on a terminal application on a terminal device, and sending the application request to a data processing end; obtaining second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data with preset data filtered out, and the first display data is data obtained by a cloud application of the data processing end in response to the application request; and displaying the second display data.

[0007] According to a second aspect of an embodiment of the present invention, a display data processing method is provided, comprising: obtaining first display data generated by a cloud application when processing an application request from a terminal application; filtering out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; and sending second display data determined based on the filtered first display data to the terminal application.

[0008] According to a third aspect of an embodiment of the present invention, there is provided a data processing method, comprising: receiving a user's exit operation on a terminal application on a terminal device, generating an application exit request according to the exit operation, and sending the application exit request to a data processing end; receiving an exit message returned by the data processing end in response to the application exit request, wherein the exit message carries information for indicating that the cloud application corresponding to the terminal application has exited; exiting the terminal application according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device. According to a fourth aspect of an embodiment of the present invention, there is provided a data processing method, comprising: exiting the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application; sending an exit message to the terminal application indicating that the cloud application has exited, so as to instruct the terminal application to perform an exit operation through the exit message, and cause the terminal device where the terminal application is located to display the local desktop.

[0009] According to a fifth aspect of an embodiment of the present invention, a display data processing device is provided, including: a sending module, used to generate an application request based on a user's operation on a terminal application on a terminal device, and send the application request to a data processing end; a first acquisition module, used to obtain second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data with preset data filtered out, and the first display data is data obtained by the cloud application of the data processing end in response to the application request; a display module, used to display the second display data.

[0010] According to a sixth aspect of an embodiment of the present invention, a display data processing device is provided, comprising: a second acquisition module for acquiring first display data generated by a cloud application in processing an application request from a terminal application; a filtering module for filtering out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; and a third acquisition module for sending second display data determined based on the filtered first display data to the terminal application. According to a seventh aspect of an embodiment of the present invention, a data processing device is provided, comprising: a first receiving module for receiving a user's exit operation on a terminal application on a terminal device, generating an application exit request based on the exit operation, and sending the application exit request to a data processing end; a second receiving module for receiving an exit message returned by the data processing end in response to the application exit request, wherein the exit message carries information indicating that the cloud application corresponding to the terminal application has exited; and a first exit module for exiting the terminal application based on the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

[0011] According to the eighth aspect of an embodiment of the present invention, a data processing device is provided, including: a second exit module, used to exit the cloud application corresponding to the terminal application in the data processing end according to an application exit request sent by the terminal application; a message feedback module, used to send an exit message to the terminal application to indicate that the cloud application has exited, so as to instruct the terminal application to perform an exit operation through the exit message, and enable the terminal device where the terminal application is located to display the local desktop.

[0012] According to a ninth aspect of an embodiment of the present invention, there is provided an electronic device comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction enables the processor to perform an operation corresponding to the display data processing method as described in the first aspect or the second aspect, or enables the processor to perform an operation corresponding to the data processing method as described in the third aspect or the fourth aspect.

[0013] According to the tenth aspect of an embodiment of the present invention, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the display data processing method as described in the first aspect or the second aspect, or, when executed, it implements the data processing method as described in the third aspect or the fourth aspect.

[0014] According to the display data processing solution provided by an embodiment of the present invention, first display data generated in the process of processing an application request by a cloud application is obtained, and the first display data is filtered to remove data other than the data requested by the application request. Second display data is then determined based on the filtered first display data, and the second display data is sent to the terminal application, so that the terminal application can display the data requested by the application request to the user by displaying an interface corresponding to the second display data, thereby responding to the operation. In addition, the data processing burden of the terminal device can be reduced, and the user will not be aware of the existence of the cloud application, thereby avoiding a sense of disconnection and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1a This is a flowchart of a method for processing display data according to Embodiment 1 of the present invention;

[0017] Figure 1b is a flowchart of another method for processing display data according to the first embodiment of the present invention;

[0018] Figure 1c A schematic diagram of a usage scenario according to the first embodiment of the present invention;

[0019] Figure 2a This is a flowchart of a method for processing display data according to a second embodiment of the present invention;

[0020] Figure 2b A schematic diagram showing the connection between the terminal device and the data processing terminal;

[0021] Figure 2c A schematic diagram of a terminal application displaying an interface rendered by a data processing terminal in a technical solution;

[0022] Figure 2d A schematic diagram of a usage scenario according to the second embodiment of the present invention;

[0023] Figure 3 This is a flowchart of a display data processing method according to Embodiment 3 of the present invention;

[0024] Figure 4 This is a flowchart of a method for processing display data according to a fourth embodiment of the present invention;

[0025] Figure 5aThis is a flowchart of a data processing method according to a fifth embodiment of the present invention;

[0026] Figure 5b A flowchart of a technical solution for processing an application request of a terminal application by a cloud application at a data processing end and rendering the running result of the cloud application for display by the terminal application;

[0027] Figure 5c A schematic diagram of a process for a user to exit a terminal application that uses a cloud application for data processing in a technical solution;

[0028] Figure 5d A schematic diagram of a process for processing an application request of a terminal application by a cloud application on a data processing end and rendering an operating result of the cloud application for display by the terminal application in a usage scenario according to a fifth embodiment of the present invention;

[0029] Figure 5e This is a schematic diagram of a process for a user to exit a terminal application that uses a cloud application for data processing in a usage scenario according to the fifth embodiment of the present invention;

[0030] Figure 6 This is a flowchart of a data processing method according to a sixth embodiment of the present invention;

[0031] Figure 7 This is a structural block diagram of a display data processing device according to a seventh embodiment of the present invention;

[0032] Figure 8 is a structural block diagram of a display data processing device according to an eighth embodiment of the present invention;

[0033] Figure 9 This is a structural block diagram of a data processing device according to a ninth embodiment of the present invention;

[0034] Figure 10 This is a structural block diagram of a data processing device according to a tenth embodiment of the present invention;

[0035] Figure 11 FIG. 1 is a structural diagram of an electronic device according to an eleventh embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0037] Typically, due to the inherent settings of the cloud operating system, the cloud application interface data generated in the process of the cloud application processing the application request will not only include the data requested by the application request, but also include the status bar data of the cloud application (the status bar includes but is not limited to the notification bar and navigation bar. The notification bar can be used to display the battery level, time and other information of the device, and the navigation bar is used to display the "back button", "desktop button" and "multi-tasking button", etc.). The style of the icons (such as the battery indicator icon and the WIFI indicator icon) contained in the status bar data in the cloud application interface data is preset by the cloud operating system and may be inconsistent with the icon style of the terminal operating system where the terminal application is located. For example, the icon styles of different operating systems are inconsistent, or the icon styles of different versions of the same operating system are also inconsistent. As a result, when the terminal application displays the display data generated according to the cloud application interface data, the user will feel the existence of the cloud application, thereby creating a sense of disconnection, and may even think that the terminal device is damaged, which brings a bad user experience.

[0038] The specific implementation of the embodiment of the present invention is further described below with reference to the accompanying drawings of the embodiment of the present invention.

[0039] Example 1

[0040] Reference Figure 1a , shows a step flow chart of a display data processing method according to embodiment 1 of the present invention.

[0041] In this embodiment, the implementation process of the display data processing method configured in a terminal device is described as an example. The display data processing method includes the following steps:

[0042] Step S102: Generate an application request according to the user's operation on the terminal application on the terminal device, and send the application request to the data processing end.

[0043] The terminal device can be a smart speaker equipped with a display screen, a smart TV, a mobile phone, a PAD, or other appropriate terminal device.

[0044] One or more terminal applications can be installed on the terminal device, such as a video player application, a music player application, and the like. The user's operation on the terminal application can be a triggering operation on an option in the terminal application, a triggering operation on the back button in the navigation bar, and the like. For example, the terminal device receives a click operation by the user on Video 1 displayed on the homepage interface of the terminal application. An application request is generated based on the operation, and the application request is sent to the data processing end, so that the application request is processed by the cloud application in the data processing end to obtain the interface data of the video 1. Subsequently, the data processing end sends the interface data of the video 1 processed by the cloud application to the terminal device, thereby responding to the user operation by displaying the obtained interface data of the video 1 on the display screen of the terminal device.

[0045] Step S104: Acquire the second display data returned by the data processing end in response to the application request.

[0046] In this embodiment, the second display data is data determined by the data processing end based on the first display data after filtering out preset data, and the first display data is data obtained by the cloud application of the data processing end in response to the application request.

[0047] The first display data is data obtained by the cloud application on the data processing end in response to the application request. The first display data includes the status bar data of the cloud application and the data requested by the application. The status bar data of the cloud application does not need to be displayed to the user, so the preset data includes the status bar data of the cloud application. By filtering out the preset data in the first display data and determining the second display data based on the filtered first display data, the second display data sent by the data processing end to the terminal device does not include the status bar data of the cloud application, thereby preventing the user from seeing the status bar data of the cloud application and preventing the user from perceiving the cloud application.

[0048] The data processing end can send the second display data to the terminal device in the form of a video stream. The video stream includes multiple terminal application interface image frames. Of course, in other embodiments, the terminal device can obtain the second display data in other ways, and this embodiment does not limit this.

[0049] Step S106: displaying the second display data.

[0050] In a specific implementation, the second display data is rendered and displayed by the terminal application to generate a display interface of the terminal application.

[0051] For example, the second display data includes multiple terminal application interface image frames, which are drawn and displayed by the terminal application, so that the terminal application displays the data requested by the application request obtained by the cloud application processing the application request (such as the interface for playing video 1). In this way, the application request is processed by the cloud application without the user being aware of the cloud application, thus avoiding a sense of disconnection.

[0052] During drawing and display, drawing and display may be performed according to the time sequence relationship of the terminal application interface image frames in the second display data.

[0053] Alternatively, as Figure 1b As shown, in a specific implementation of this embodiment, in order to improve the applicability of the display data processing method so that the user can select the display mode of the terminal application as needed, the method further includes step S100.

[0054] Step S100: determining whether the terminal application operated by the user displays preset data according to a preset display policy.

[0055] In this embodiment, the user can pre-configure the display policy corresponding to each terminal application, such as the display policy of video applications and music applications is not to display preset data, and the display policy of communication applications is to display preset data, etc. In this way, the user can control the display policy of the terminal application on the terminal device as needed.

[0056] When a user operates an application, a display policy corresponding to the operated application is determined from preset display policies, and then whether the application displays preset data is determined according to the display policy.

[0057] In one scenario, if preset data is displayed, a data processing request can be generated based on the user's operation and sent to the data processing terminal. The cloud application corresponding to the operated terminal application processes the data processing request. The data processing terminal processes the cloud application's interface data into a video stream and sends it to the terminal device, which then displays the video stream.

[0058] In another case, if the preset data is not displayed, the aforementioned step S102 is executed. In this way, different display modes can be set on the terminal device, and different display modes can be adapted for each terminal application, thereby making the display effect better and being able to meet different needs.

[0059] It should be noted that step S100 is an optional step and can be omitted in other embodiments.

[0060] The implementation process of this method is now described in detail based on a specific usage scenario:

[0061] Figure 1c Interface 1 shows the homepage of terminal application A. When a user clicks on video 1, terminal application A receives the user's action, generates an application request based on the action, and sends it to the data processing end. In this scenario, the data processing end can be the server. Of course, in other scenarios, the data processing end can be other IoT devices with strong computing capabilities, such as smart speakers.

[0062] Those skilled in the art can use any appropriate method to generate an application request based on the operation as needed, and the specific content of the application request can also be determined as needed. For example, the application request is used to request to obtain interface data for playing video 1, etc.

[0063] After the data processing end receives the application request, if the cloud application a corresponding to the terminal application A has been started, the application request can be directly sent to the cloud application a, so that it processes the application request and obtains the cloud application interface data formed by the processing process (which can be determined as the first display data). The cloud application interface data includes the status bar data of the cloud application and the data requested by the application request (that is, the data of the display window showing the cloud application).

[0064] In this usage scenario, after obtaining the first display data, the data processing end filters out all data in the first display data except the data requested by the application request, so that the second display data determined based on the filtered first display data does not include the status bar. In this way, the user will not perceive the existence of the cloud application, thereby avoiding misunderstandings caused by the user's perception of the cloud application, and can ensure that the data requested by the application request can be displayed to meet the needs of the user.

[0065] Those skilled in the art may determine the second display data according to the filtered first display data in any appropriate manner, and this usage scenario does not limit this.

[0066] Figure 1cInterface 1 shows the display window of the terminal application. When the user clicks on the option for Video 1, the terminal application sends an application request to the data processing end. The cloud application corresponding to the terminal application in the data processing end processes the application request and obtains first display data. The first display data includes the data requested by the application request (such as the interface image of playing Video 1) and the status bar data of the cloud application. The cloud application status bar data is filtered out to obtain the filtered first display data, and the second display data is determined based on the filtered first display data. The second display data is sent to the terminal device, and the second display data is displayed by the terminal application on the terminal device (as shown in Interface 2 in the figure). That is, the terminal application displays the interface corresponding to the second display data in its own display window to realize the function of playing Video 1.

[0067] In this way, by changing the logic of the interface synthesis service, the problem of the conventional technical solution in which the data processing end directly synthesizes the interface indicated by the data of the cloud application's display window and the interface indicated by the status bar data is solved, and data for display by the terminal application is generated and sent to the terminal application, resulting in the interface displayed by the terminal application including the status bar of the cloud application. If the style of some content elements in the status bar is inconsistent with the style of the content elements of the terminal operating system, it will cause confusion to the user and the user experience will be poor. It can also reduce the data processing burden of the terminal device.

[0068] Through this embodiment, an application request is generated based on the user's operation information, and the application request is sent to the data processing end. The second display data determined based on the filtered first display data returned by the data processing end is obtained, and the second display data is rendered and displayed, so that the user can see the required data, and the status bar of the cloud application will not be displayed, avoiding the user's perception of the existence of the cloud application and causing confusion or a sense of disconnection, thereby improving the user experience and reducing the data processing burden.

[0069] The display data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC.

[0070] Example 2

[0071] Reference Figure 2a , shows a flowchart of the steps of a display data processing method according to embodiment 2 of the present invention.

[0072] In this embodiment, the display data processing method can be configured in a data processing terminal. The data processing terminal is connected to the user's terminal device via a network, so that the cloud application on the data processing terminal provides data processing services for the corresponding terminal application on the terminal device, thereby reducing the computing power and storage space requirements of the terminal device, reducing the cost of the terminal device, and making the terminal device smaller and easier to achieve thinness and lightness.

[0073] For example, Figure 2b As shown, in one example, a terminal device is a Bluetooth speaker with a screen. Of course, the terminal device can also be a mobile phone, a smart TV, a personal computer, etc. The terminal device is connected to the data processing end through a webRTC server configured with the web Real-Time Communication (webRTC) protocol to achieve data interaction.

[0074] The data processing end includes at least one of a server and an IoT device (IoT device) with computing capabilities in the cloud. IoT devices include, but are not limited to, smart speakers, smart TVs, smartphones, and tablets. For example, if the data processing end is in the cloud, one or more containers will run. These containers are used to run an operating system (referred to as the cloud operating system), which provides services to cloud applications through the operating system's underlying framework, enabling cloud applications to rely on the cloud operating system for operation.

[0075] When the terminal application installed in the terminal device, such as video applications, instant messaging applications, music applications, etc., receives a user operation, since the main data processing of the terminal application is implemented through the cloud application on the data processing end, the terminal application generates an application request based on the user's operation information and sends the application request to the data processing end, which is processed by the cloud application on the data processing end.

[0076] In order to solve the problems that users often feel disconnected when using terminal applications and the terminal device has a heavy processing burden in the aforementioned common situations, the display data processing method provided in an embodiment of the present invention includes the following steps: Step S202: Obtain the first display data generated by the cloud application during the processing of the application request from the terminal application.

[0077] The application request carries the user's operation information on the terminal application, and the application request is used to request the cloud application corresponding to the terminal application to process the operation information.

[0078] Operation information includes, for example, information about a user launching a terminal application on a terminal device, triggering an option in a terminal application, or triggering an operation such as the "back button" in a navigation bar in a terminal application. The terminal application generates a corresponding application request based on the operation information and sends the application request to the data processing end, so that the cloud application in the data processing end processes the application request accordingly.

[0079] For example, if a user needs to launch a terminal application, they trigger the terminal application icon displayed on the local desktop of the terminal device. Based on the operation information, a corresponding application request is generated and sent to the data processing end. The data processing end processes the application request through the corresponding cloud application.

[0080] When a cloud application processes an application request from a terminal application, the generated cloud application interface data includes not only the cloud application's display window data but also the cloud application's status bar data. Furthermore, during the processing process, system notification message data from the system hosting the cloud application may also be sent to the cloud application. These data can all be considered the first display data during the processing process.

[0081] If the application request is used to indicate a request to play video 1, the cloud application processes the application request as follows: the cloud application obtains video 1 according to the application request, decodes video 1, and plays video 1 in a display window.

[0082] The data of the display window of the cloud application (ie, the data requested by the application) includes the data of playing the video 1 in the display window.

[0083] The status bar indicated by the status bar data includes, but is not limited to, a notification bar located at the upper portion of the display window of the cloud application and a navigation bar located at the lower portion of the display window of the cloud application.

[0084] If the cloud operating system where the cloud application is located does not generate system notification message data during the process of the cloud application processing the application request from the terminal application, the first display data only includes the aforementioned cloud application interface data.

[0085] If the cloud operating system where the cloud application is located generates system notification message data during the process of the cloud application processing the application request from the terminal application, the first display data includes the cloud application interface data and the system notification message data.

[0086] Step S204: Filtering out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request.

[0087] As mentioned above, if the first display data generated in one processing process includes cloud application interface data and system notification message data, it is necessary to filter out the status bar data in the system notification message data and the cloud application interface data.

[0088] If the first display data generated in one processing process only includes cloud application interface data, it is necessary to filter out the status bar data in the cloud application interface data.

[0089] By filtering out the status bar data from the first display data (or filtering out the status bar data and system notification message data), the display data subsequently sent to the terminal application no longer contains the status bar data and / or system notification message data, and the terminal application will no longer display icons with the style of the cloud operating system (such as the battery indicator icon, the WIFI indicator icon), thereby avoiding the display of a status bar or system notification message that is inconsistent with the style of the terminal operating system on the terminal device, thereby avoiding the user from feeling that the terminal application and the cloud application are not the same application, ensuring that the user has a better experience when using the terminal application without a sense of disconnection.

[0090] Step S206: sending second display data determined according to the filtered first display data to the terminal application.

[0091] In order to reduce the amount of computation of the terminal application and lower the computation load of the terminal device, the data processing end generates second display data for display on the terminal device based on the filtered first display data. Those skilled in the art can determine the second display data in any appropriate manner.

[0092] For example, in this embodiment, the interface synthesis service (surface flinger) of the cloud operating system is used to synthesize the data requested by the application (i.e., the data of the display window of the cloud application) to indicate the interface, and obtain the terminal application interface data for display on the display screen of the terminal device as the second display data, and send it to the terminal application so that it can be displayed through the display screen or other display structures (such as a projection lens).

[0093] The following combines the above Figure 2bThe schematic diagram of the connection between the terminal device and the data processing end is shown. The data processing end is connected to the terminal device through the WebRTC service. When the terminal device 1 receives the user's operation on the terminal application on it and needs to use the data processing end for remote rendering to obtain the interface required by the user, the terminal device 1 sends the application request to the webRTC service, and converts the application request into corresponding input events through the webRTC data transmission channel (data channel) and sends it to the cloud system infrastructure framework of the data processing end, which forwards it to the corresponding cloud application. This achieves the purpose of sending the application request to the cloud application. The cloud application can process the application request accordingly according to its own logic.

[0094] The interface synthesis service (surface flinger) in the basic framework of the cloud operating system is used to obtain data from multiple sources, synthesize the interfaces indicated by these data, form a hardware refresh, and then send it to the terminal device for display through the webRTC service. For example, in a common technical solution, since each cloud application in the cloud operating system has a corresponding status bar (for example, an Android system application includes not only the main window of the application, but also the upper notification bar and the lower navigation bar), when the interface synthesis service is processed, it will not only receive the data of the display window of the cloud application, but also receive the status bar data of the cloud application, and synthesize it into an image and send it to the terminal device. In this way, in addition to displaying the application interface that the user wants (that is, the interface of the main window of the cloud application), the terminal device will also display the status bar of the cloud operating system or the system notification message received during the operation of the cloud application. The schematic diagram of the display window of the terminal application in this common technical solution is shown as follows: Figure 2c shown.

[0095] This results in some content elements that are not available on the terminal operating system being displayed on the terminal device, which not only burdens the terminal device but also affects the user experience, causing a sense of disconnection during use.

[0096] The display data processing method of this embodiment solves the problems in the conventional technical solutions. The following describes the implementation process and principle of the display data processing method in solving the aforementioned problems using a specific usage scenario.

[0097] In this usage scenario, it is taken as an example that a user clicks on video 1 displayed on the homepage interface of terminal application A, causing terminal application A to play video 1.

[0098] Figure 2dInterface 1 in the middle shows the homepage of terminal application A. When a user clicks on video 1, terminal application A receives user operation information indicating that the user has clicked on video 1. Based on this operation information, terminal application A generates an application request and sends it to the data processing end. Those skilled in the art can use any appropriate method to generate an application request based on the operation information, and the specific content of the application request can also be determined as needed. For example, the application request may be used to request interface data for playing video 1.

[0099] After the data processing end receives the application request, if the cloud application a corresponding to the terminal application A has been started, the application request can be directly sent to the cloud application a, so that it processes the application request and obtains the cloud application interface data formed by the processing process (which can be determined as the first display data). The cloud application interface data includes the status bar data of the cloud application and the data requested by the application request (that is, the data of the display window showing the cloud application).

[0100] In this usage scenario, after obtaining the first display data, the data processing end filters out all data in the first display data except the data requested by the application request, so that the second display data determined based on the filtered first display data does not include the status bar. In this way, the user will not perceive the existence of the cloud application, thereby avoiding misunderstandings caused by the user's perception of the cloud application, and can ensure that the data requested by the application request can be displayed to meet the needs of the user.

[0101] Those skilled in the art may determine the second display data according to the filtered first display data in any appropriate manner, and this usage scenario does not limit this.

[0102] Figure 2d The middle interface 2 shows a schematic diagram of implementing the function of playing the video 1 by sending the second display data to the terminal application, and the terminal application displays the interface corresponding to the second display data in its own display window.

[0103] In this way, by changing the logic of the interface synthesis service, the problem of the conventional technical solution in which the data processing end directly synthesizes the interface indicated by the data of the cloud application's display window and the interface indicated by the status bar data is solved, and data for display by the terminal application is generated and sent to the terminal application, resulting in the interface displayed by the terminal application including the status bar of the cloud application. If the style of some content elements in the status bar is inconsistent with the style of the content elements of the terminal operating system, it will cause confusion to the user and the user experience will be poor. It can also reduce the data processing burden of the terminal device.

[0104] Through this embodiment, the first display data generated by the cloud application in the process of processing the application request is obtained, and the first display data is filtered to remove data other than the data requested by the application request. The second display data is then determined based on the filtered first display data, and the second display data is sent to the terminal application, so that the terminal application can display the data requested by the application request to the user by displaying the interface corresponding to the second display data, thereby responding to the operation. It can also reduce the data processing burden of the terminal device, prevent the user from perceiving the existence of the cloud application, avoid a sense of disconnection, and improve the user experience.

[0105] The display data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC.

[0106] Example 3

[0107] Reference Figure 3 , shows a flowchart of the steps of a display data processing method according to embodiment 3 of the present invention.

[0108] The display data processing method of this embodiment includes the aforementioned steps S202 to S206 .

[0109] In a specific implementation, step S202 may be implemented as: obtaining cloud application interface data generated in the process of the cloud application processing the application request, and determining the first display data using the obtained cloud application interface data.

[0110] In this implementation, no system notification message data is received during the cloud application processing of the application request, so only the generated cloud application interface data is determined as the first display data. The cloud application interface data includes the status bar data of the cloud application and the data requested by the application request.

[0111] The data requested by the application request may be interface data of a display window of the cloud application, and the interface data of the display window is used to display interface data of an operation result of processing the application request by the cloud application.

[0112] The status bar data of the cloud application (status bar data) includes but is not limited to the notification bar data located at the upper part of the display window and the navigation bar data located at the lower part of the display window.

[0113] In the case where the cloud application interface data is used as the first display data, the preset data includes status bar data of the cloud application. In this case, step S204 can be implemented as: filtering out the status bar data from the first display data.

[0114] By filtering out the status bar data, the filtered first display data no longer includes data other than the data requested by the application. In this way, in subsequent steps, the interface indicated by the second display data determined based on the filtered first display data will not include the status bar. When the terminal application displays according to the second display data, the status bar will not be displayed in the terminal application interface, thereby avoiding users from noticing the cloud application.

[0115] In this embodiment, in a specific implementation of step S206, step S206 may include the following sub-steps S2061 to S2062.

[0116] Sub-step S2061: Acquire the data requested by the application request contained in the filtered first display data, perform interface synthesis processing on the data requested by the application request, and obtain the second display data including at least one terminal application interface image frame.

[0117] For example, a surface flinger is used to obtain one or more (the multiple in this embodiment can be two or more) interfaces indicated by the data requested by the application (such as the data of the display window of the cloud application), and one or more frames of terminal application interface image frames are formed based on the position, size, occlusion relationship and content displayed in the interface (that is, the running results of the cloud application) of the one or more interfaces, and these terminal application interface image frames are used as the second display data.

[0118] Sub-step S2062: Send the second display data to the terminal application.

[0119] By sending the second display data (multiple second display data form an image frame stream videostream sent to the terminal application) to the terminal application, the terminal application can render and display the terminal application interface image frames in its own window according to the timing of these terminal application interface image frames, and thus display the running results of the cloud application to the user, so that the user can obtain the data he needs, thereby realizing response to the user's operation.

[0120] In this way, the terminal application does not need to perform interface synthesis, which can reduce the load, and can display the data required by the user without displaying the status bar data of the cloud application, so that the user will not be aware of the cloud application. Through this embodiment, the first display data generated by the cloud application in the process of processing the application request is obtained, and the first display data is filtered to remove data other than the data requested by the application request. The second display data is then determined based on the filtered first display data, and the second display data is sent to the terminal application, so that the terminal application can display the data requested by the application request to the user by displaying the interface corresponding to the second display data, thereby responding to the operation. It can also reduce the data processing burden of the terminal device and prevent the user from being aware of the existence of the cloud application, avoiding a sense of disconnection and improving the user experience.

[0121] The display data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC.

[0122] Example 4

[0123] Reference Figure 4 , shows a step flow chart of a display data processing method according to embodiment 4 of the present invention.

[0124] The display data processing method of this embodiment includes the aforementioned steps S202 to S206, wherein step S206 can be implemented in any of the aforementioned embodiments, which is not limited in this embodiment.

[0125] In a specific implementation of this embodiment, if system notification message data is received during the process of the cloud application processing the application request, step S202 can be implemented as: obtaining cloud application interface data and system notification message data generated during the process of the cloud application processing the application request, and determining the obtained cloud application interface data and system notification message data as the first display data.

[0126] When system notification message data is obtained (e.g., when the cloud operating system detects insufficient memory for running the cloud operating system, it will generate system notification message data indicating insufficient memory), the first display data includes not only cloud application interface data but also system notification message data. The cloud application interface data includes the status bar data of the cloud application and the data requested by the application request. The status bar data and the data requested by the application request may be the status bar data and the data requested by the application request described in Example 2, and thus will not be described in detail.

[0127] In this case, both the status bar data and the system notification message data are data that are not desired to be displayed to the user, so the preset data includes the status bar data and the system notification message data. At this time, step S204 can be implemented as: filtering out the status bar data and the system notification message data from the first display data.

[0128] In this way, all data that does not need to be displayed to users can be filtered out, thereby preventing users from being aware of the existence of cloud applications and avoiding a sense of disconnection.

[0129] In addition, in this embodiment, when the cloud application is in the startup state, all received system notification message data can be ignored, thereby preventing the terminal application from displaying the system notification message of the data processing end.

[0130] Through this embodiment, the first display data generated by the cloud application in the process of processing the application request is obtained, and the first display data is filtered to remove data other than the data requested by the application request. The second display data is then determined based on the filtered first display data, and the second display data is sent to the terminal application, so that the terminal application can display the data requested by the application request to the user by displaying the interface corresponding to the second display data, thereby responding to the operation. It can also reduce the data processing burden of the terminal device, prevent the user from perceiving the existence of the cloud application, avoid a sense of disconnection, and improve the user experience.

[0131] The display data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a tablet computer, a mobile phone, etc.) and a PC.

[0132] Example 5

[0133] Reference Figure 5a , shows a step flow chart of a data processing method according to embodiment 5 of the present invention.

[0134] In this embodiment, the implementation process is described by taking the configuration of the data processing method in the terminal device as an example.

[0135] The data processing method in this embodiment includes the following steps:

[0136] Step S502: receiving a user's exit operation on a terminal application on a terminal device, generating an application exit request according to the exit operation, and sending the application exit request to a data processing end.

[0137] In one specific implementation, the user can exit the terminal application by clicking the "back button" in the navigation bar of the terminal application interface displayed on the display screen of the terminal device. Of course, in other embodiments, the terminal application can also be exited by any other appropriate method, and this embodiment does not limit this.

[0138] Those skilled in the art may generate an application exit request based on the exit operation in any appropriate manner, and this embodiment does not limit this. By sending the application exit request to the data processing terminal, the data processing terminal performs the exit operation on the cloud application corresponding to the terminal application. Exiting the cloud application may be implemented using the method described in the fifth embodiment or other appropriate implementation methods, and this embodiment does not limit this.

[0139] It should be noted that the data processing end can be a server, cloud or other terminal devices with computing capabilities (such as smart speakers, smart TVs or smart phones, etc.).

[0140] In this embodiment, the aforementioned cloud application corresponding to the terminal application may be a cloud application started according to a start request of the terminal application, and the cloud application is used to provide a data processing function for the terminal application.

[0141] Optionally, before step S502, a step can also be performed to obtain the user's startup operation on the terminal application on the terminal device, generate an application startup request based on the startup operation, and send the application startup request to the data processing end, so that the data processing end starts the cloud application corresponding to the terminal application to process the request sent by the terminal application.

[0142] For example, the user starts the terminal application by triggering the icon of the terminal application displayed on the display screen of the terminal device, generates an application start request (the application start request can be considered as a type of application request) based on the start operation, and sends it to the data processing end.

[0143] The data processing end launches the corresponding cloud application based on the application launch request, for example, by launching the corresponding cloud application instance and pushing the launched cloud application instance onto a stack. The launched cloud application can process the application request of the terminal application and send the result of the cloud application's processing of the application request to the terminal application for display, using the implementation method described in any of the preceding embodiments 1 to 3.

[0144] Step S504: receiving an exit message returned by the data processing end in response to the application exit request.

[0145] After the cloud application in the data processing end is exited, the data processing end sends a exit message to the terminal device, which carries information indicating that the cloud application corresponding to the terminal application has been exited. In this way, the terminal device can promptly know that the cloud application has been exited through the exit message.

[0146] Step S506: Exit the terminal application according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

[0147] In a specific implementation, step S506 may be implemented as: sending a disconnect request to the data processing end according to the exit message, and closing the display window of the terminal application.

[0148] A disconnect request disconnects the data processing end, preventing it from sending the cloud operating system's desktop launcher interface display data to the terminal device and allowing the user to see it. After disconnecting from the data processing end, the terminal application's display window is closed, allowing the terminal operating system's local desktop to be displayed.

[0149] The following describes how to implement the data processing process based on a specific usage scenario:

[0150] In this scenario, the terminal device can be a smart speaker equipped with a display. To reduce the load on the smart speaker, the terminal application interface can be rendered using remote rendering. This allows the terminal device to integrate terminal applications with cloud applications, requiring only a certain amount of memory and storage space. This allows users to use cloud applications as if they were local applications, improving the user experience.

[0151] But if Figure 5b As shown, in the conventional technical solution, when the cloud application is used on the terminal device, the surface flinger used for data synthesis on the data processing end will synthesize the display window data and status bar data of the cloud application together and send them to the terminal application, causing the status bar (such as StatusBar) of the cloud operating system on the data processing end to be displayed on the terminal device. If the cloud application receives a system notification message from the cloud operating system, the system notification message will also be displayed on the terminal device.

[0152] like Figure 5cAs shown, in a typical technical solution, when a user clicks the back button on the homepage of a terminal application, the cloud operating system displays the desktop launcher interface, and the surface flinger also sends the display data of this interface to the terminal device, resulting in the cloud operating system's desktop launcher interface being displayed on the terminal device's display screen. This can confuse users if the terminal operating system on the terminal device is not the same as the cloud operating system or the system versions are different. Why does opening a terminal application on the terminal operating system cause content elements that are not present in the terminal operating system? This can cause users to perceive the existence of two operating systems and increase the data processing load on the terminal device.

[0153] To solve the above problems, the implementation process of the data processing method for this usage scenario is as follows:

[0154] like Figure 5d and 5e As shown, one or more terminal application icons are displayed on the local desktop. The user can start the terminal application by clicking the icon. When the terminal device obtains the user's trigger operation on the terminal application, it starts the terminal application and generates an application start request. At the same time, it establishes a connection with the data processing end and sends the application start request to the data processing end.

[0155] The data processing end starts a cloud application instance (eg, main activity) corresponding to the terminal application according to an application start request (which is a type of application request).

[0156] On the one hand, the cloud application instance (activity) is pushed into the stack.

[0157] On the other hand, the surface flinger on the data processing end obtains the activated cloud application interface data as the first display data, and filters the first display data to remove all display data (such as status bar data) except for the data requested by the user's application (i.e., the data of the cloud application's display window). The second display data is then determined based on the remaining cloud application display window data after filtering. (As shown in the figure, the video stream may include multiple second display data. Since the display screen has a certain refresh rate, such as 60 Hz, the terminal application interface image frames refreshed two times may be the same. Therefore, to improve efficiency, after obtaining the terminal application interface image frame at time t1, the surface flinger can use it as the terminal application interface image frames at times t2 to tN and send it to the terminal operating system.) This prevents the status bar data and the cloud application display window data from being synthesized together. The second display data sent to the terminal device no longer contains the status bar data. Only the interface displayed in the cloud application display window is displayed in the terminal application's display window, and the status bar and system notification messages on the data processing end are not displayed, thus preventing the user from perceiving the cloud application or the data processing end.

[0158] This method solves the problem in the conventional technical solution that, due to the inherent configuration of the cloud operating system, Surface Flinger receives graphic display data from multiple sources, synthesizes them, and then sends the synthesized results to the terminal device for display. In addition to displaying the terminal application interface required by the user, the terminal device also displays the status bar of the cloud operating system and some system notification messages, causing trouble for the user.

[0159] If the data processing end is connected to the terminal device and receives system notification message data sent by the cloud operating system (such as indicating that the available memory of the container where the cloud operating system is located is low), the surface flinger ignores the system notification message data and does not send the system notification message data to the terminal device, thereby preventing the system notification message data of the cloud operating system from being displayed on the terminal device.

[0160] When a user clicks the "Back" button on the homepage of a terminal application, it is considered that the user has exited the terminal application. Based on this exit operation, an application exit request is generated and sent to the data processing end. Based on the application exit request, the data processing end pops the cloud application instance (activity) of the cloud application corresponding to the terminal application indicated by the application exit request from the stack. If the cloud application's corresponding stack no longer contains the cloud application instance, before displaying the desktop launcher interface of the cloud operating system, an exit message is sent to the terminal device, indicating that the cloud application has exited.

[0161] Upon receiving the logout message, the terminal device sends a disconnect request to the data processing end to disconnect from the data processing end and immediately close the terminal application's display window, thereby displaying the local desktop. After the data processing end disconnects from the terminal device, it can display the desktop launcher interface of the cloud operating system.

[0162] By sending an exit message to the terminal device before displaying the desktop launcher interface, a solution is provided to the conventional technical solution that when the user clicks the back button on the homepage of the terminal application on the terminal device, the data processing end exits the corresponding cloud application and switches to the interface of the desktop launcher of the cloud operating system, causing the surface flinger to send the interface of the desktop launcher to the terminal device, causing the interface of the desktop launcher of the cloud operating system to be displayed on the terminal device, and causing some content elements of the cloud operating system to be displayed on the terminal device, so that the user perceives the existence of two operating systems.

[0163] Through this embodiment, the Surface Flinger on the data processing end only synthesizes the data of the display window of the cloud application that needs to be displayed. Other status bar data (such as StatusBar) and any other system notification message data will not be synthesized, so the status bar data and system notification messages will not be displayed on the terminal device.

[0164] When the terminal application needs to be closed, the data processing end sends an exit message to the terminal device, thereby disconnecting the terminal device from the data processing end and closing the terminal application display window, thereby exiting to the local desktop. In this way, the window management of the data processing end and the terminal device can be integrated.

[0165] Through this embodiment, an exit message can be sent to the terminal device after the cloud application is exited, so that it disconnects from the data processing end and closes the display window of the terminal application, thereby exiting to the local desktop. In this way, the desktop launcher interface of the cloud operating system will not be displayed, so that the user will not perceive the cloud application and cloud operating system. It can also reduce the load of the terminal device during display, so that the use effect is better.

[0166] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.) and PCs, etc.

[0167] Example 6

[0168] Reference Figure 6 , shows a step flow chart of a data processing method according to embodiment 6 of the present invention.

[0169] In this embodiment, the implementation process of the data processing method configured on the data processing end is described as an example. Among them, since the data processing end is connected to the terminal device through a network, and the display data in the data processing end (such as cloud application interface data or desktop launcher interface data, etc.) will be sent to the terminal device and displayed on the display screen of the terminal device, when the user exits the terminal application on the terminal device, the local desktop of the terminal operating system will not be directly displayed in the conventional technical solution. Instead, the screen of the cloud application corresponding to the terminal application on the data processing end and the screen of the desktop launcher on the data processing end will be displayed before the local desktop is displayed, causing the user to perceive the existence of the data processing end, which will cause confusion and a sense of disconnection, resulting in a poor user experience.

[0170] To solve this problem, the display data processing method of this embodiment includes the following steps:

[0171] Step S602: according to the application exit request sent by the terminal application, exit the cloud application corresponding to the terminal application in the data processing end.

[0172] The application exit request may be an application request generated by the terminal device according to the user's exit operation on the terminal application, and sent to the data processing end.

[0173] For example, in a specific implementation, in order to enable the cloud application to provide data processing services for the terminal application, a connection between the cloud application and the terminal application can be established before step S602, and the request from the terminal application can be processed by the cloud application.

[0174] The implementation process of this step may adopt any implementation method described in any one of Examples 1 to 3, or other implementation methods, and this embodiment does not limit this.

[0175] In a specific implementation, step S602 may be implemented as follows: determining, based on the application exit request sent by the terminal application, a cloud application instance corresponding to the cloud application that matches the application exit request; and popping the determined cloud application instance from the stack.

[0176] After receiving the application exit request, the cloud application corresponding to the terminal application indicated by the application exit request may be determined in any appropriate manner, such as by using an identifier or ID in the application exit request.

[0177] Since when a cloud application is started and running, the cloud application instance corresponding to the cloud application (such as the activity in the Android application) will be started and pushed into the stack, when exiting the cloud application, the cloud application can be exited by popping the cloud application instance corresponding to the cloud application from the stack.

[0178] For example, when the cloud application instance corresponding to the cloud application no longer exists in the stack, it can be determined that the cloud application has exited.

[0179] Step S604: Sending an exit message to the terminal application to indicate that the cloud application has exited, so as to instruct the terminal application to perform an exit operation through the exit message, and causing the terminal device where the terminal application is located to display a local desktop.

[0180] At the data processing end, when the cloud application is exited, the data processing end will display the interface of the desktop launcher corresponding to the cloud operating system. In the usual technical solution, since the data processing end and the terminal device are still connected after the cloud application is exited, the interface of the desktop launcher of the data processing end will still be sent to the terminal device and displayed in the display window of the terminal application. This allows the user to see the interface of the desktop launcher of the cloud operating system in the display window of the terminal, which makes the user feel confused and disconnected.

[0181] To address this issue, in this embodiment, once the cloud application has exited, before displaying the cloud operating system's desktop launcher interface, a quit message is sent to the terminal application. This quit message instructs the terminal application to exit and disconnects from the data processing end, thereby preventing the display data of the desktop launcher interface displayed on the data processing end from being sent to the terminal device. In this way, the terminal device disconnects from the data processing end in response to the quit message, closes the terminal application's display window, and displays the local desktop, ensuring that the user does not see the cloud operating system's desktop launcher interface, thereby avoiding user confusion.

[0182] Through this embodiment, when the user exits the terminal application, an application exit request is sent to the data processing end, thereby exiting the cloud application, and returning an exit message to the terminal device instructing it to disconnect from the data processing end, thereby avoiding the desktop launcher interface of the cloud operating system displayed after the cloud application in the data processing end is exited from being sent to the terminal device for display, thereby avoiding user confusion, improving user experience, and reducing the data processing burden.

[0183] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.) and PCs, etc.

[0184] Example 7

[0185] Reference Figure 7 , shows a structural block diagram of a display data processing device according to embodiment 7 of the present invention.

[0186] The display data processing device of this embodiment includes: a sending module 702, which is used to generate an application request based on the user's operation on the terminal application on the terminal device, and send the application request to the data processing end; a first acquisition module 704, which is used to obtain second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data filtered out of the preset data, and the first display data is data obtained by the cloud application of the data processing end in response to the application request; a display module 706, which is used to display the second display data.

[0187] Optionally, the preset data includes status bar data of the cloud application in the data processing end.

[0188] Optionally, the display module 706 is configured to render and display the second display data through the terminal application to generate a display interface of the terminal application.

[0189] Optionally, the device also includes: a policy determination module 708, which is used to determine whether the terminal application operated by the user displays preset data according to a preset display policy before the sending module 702 generates an application request according to the user's operation on the terminal application on the terminal device and sends the application request to the data processing end; if the preset data is not displayed, the operation of generating an application request according to the user's operation on the terminal application on the terminal device and sending the application request to the data processing end is executed.

[0190] The display data processing device of this embodiment is used to implement the corresponding display data processing methods of the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the display data processing device of this embodiment can refer to the corresponding description of the aforementioned method embodiments and will not be described in detail here.

[0191] Example 8

[0192] Reference Figure 8 , shows a structural block diagram of a display data processing device according to embodiment 8 of the present invention.

[0193] The display data processing device of this embodiment includes: a second acquisition module 802, used to obtain first display data generated by the cloud application when processing an application request from a terminal application; a filtering module 804, used to filter out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; and a third acquisition module 806, used to send second display data determined based on the filtered first display data to the terminal application.

[0194] Optionally, the second acquisition module 802 is used to obtain cloud application interface data generated during the process of the cloud application processing the application request, and the cloud application interface data includes the status bar data of the cloud application and the data requested by the application request; the obtained cloud application interface data is determined as the first display data; the filtering module 804 is used to filter out the status bar data from the first display data.

[0195] Optionally, the second acquisition module 802 is used to obtain cloud application interface data and system notification message data generated during the process of the cloud application processing the application request, the cloud application interface data including the status bar data of the cloud application and the data requested by the application request; the obtained cloud application interface data and system notification message data are determined as the first display data; the filtering module 804 is used to filter out the status bar data and the system notification message data from the first display data.

[0196] Optionally, the third acquisition module 806 is used to obtain the data requested by the application request contained in the filtered first display data, perform interface synthesis processing on the data requested by the application request, and obtain the second display data containing at least one terminal application interface image frame; and send the second display data to the terminal application.

[0197] The display data processing device of this embodiment is used to implement the corresponding display data processing methods of the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the display data processing device of this embodiment can refer to the corresponding description of the aforementioned method embodiments and will not be described in detail here.

[0198] Embodiment 9

[0199] Reference Figure 9 , shows a structural block diagram of a data processing device according to embodiment 9 of the present invention.

[0200] The display data processing device of this embodiment includes: a first receiving module 902, configured to receive a user's exit operation on a terminal application on a terminal device, generate an application exit request according to the exit operation, and send the application exit request to a data processing end;

[0201] A second receiving module 904 is configured to receive a quit message returned by the data processing end in response to the application quit request, wherein the quit message carries information indicating that the cloud application corresponding to the terminal application has quit;

[0202] The first exit module 906 is configured to exit the terminal application according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

[0203] Optionally, the first exit module 906 is configured to send a disconnect request to the data processing end according to the exit message, and close the display window of the terminal application.

[0204] Optionally, the device also includes a startup module 900 for obtaining a user's startup operation on the terminal application on the terminal device before receiving the user's exit operation on the terminal application on the terminal device, generating an application exit request based on the exit operation, and sending the application exit request to the data processing end, so that the data processing end starts the cloud application corresponding to the terminal application to process the request sent by the terminal application.

[0205] The data processing device of this embodiment is used to implement the corresponding data processing methods in the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0206] Example 10

[0207] Reference Figure 10 , shows a structural block diagram of a data processing device according to embodiment 10 of the present invention.

[0208] The data processing device of this embodiment includes: a second exit module 1002, configured to exit the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application;

[0209] The message feedback module 1004 is used to send an exit message to the terminal application to indicate that the cloud application has exited, so as to instruct the terminal application to perform an exit operation through the exit message, and enable the terminal device where the terminal application is located to display the local desktop.

[0210] Optionally, the second exit module 1002 is configured to determine, based on the application exit request sent by the terminal application, a cloud application instance corresponding to the cloud application that matches the application exit request; and pop the determined cloud application instance from the stack.

[0211] Optionally, the device also includes a disconnect request receiving module 1006, which is used to receive a disconnect request sent by the terminal application after sending an exit message to the terminal application to indicate that the cloud application has exited, and disconnect from the terminal application according to the disconnect request.

[0212] Optionally, the device also includes a cloud application startup module 1000, which is used to establish a connection between the cloud application and the terminal application before exiting the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application, and process the request from the terminal application through the cloud application.

[0213] The data processing device of this embodiment is used to implement the corresponding data processing methods in the aforementioned multiple method embodiments and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0214] Example 11

[0215] Reference Figure 11 , shows a schematic structural diagram of an electronic device according to embodiment 11 of the present invention. The specific embodiment of the present invention does not limit the specific implementation of the electronic device.

[0216] like Figure 11 As shown, the electronic device may include: a processor (processor) 1102 , a communication interface (Communications Interface) 1104 , a memory (memory) 1106 , and a communication bus 1108 .

[0217] in:

[0218] The processor 1102 , the communication interface 1104 , and the memory 1106 communicate with each other via a communication bus 1108 .

[0219] The communication interface 1104 is used to communicate with other electronic devices such as terminal devices or servers.

[0220] The processor 1102 is configured to execute the program 1110 , and specifically may execute the relevant steps in the above-mentioned display data processing method or data processing method embodiment.

[0221] Specifically, the program 1110 may include program codes, which include computer operation instructions.

[0222] Processor 1102 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in an electronic device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0223] The memory 1106 is used to store the program 1110. The memory 1106 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0224] Program 1110 can specifically be used to enable the processor 1102 to perform the following operations: generate an application request based on the user's operation on the terminal application on the terminal device, and send the application request to the data processing end; obtain the second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data with the preset data filtered out, and the first display data is data obtained by the cloud application of the data processing end in response to the application request; and display the second display data.

[0225] In an optional implementation, the preset data includes status bar data of the cloud application in the data processing end.

[0226] In an optional implementation, the program 1110 is further configured to enable the processor 1102 to render and display the second display data through the terminal application when displaying the second display data, thereby generating a display interface of the terminal application.

[0227] In an optional embodiment, program 1110 is also used to enable the processor 1102 to generate an application request based on the user's operation on the terminal application on the terminal device and send the application request to the data processing end, based on a preset display strategy, to determine whether the terminal application operated by the user displays preset data; if the preset data is not displayed, then generate an application request based on the user's operation on the terminal application on the terminal device and send the application request to the data processing end.

[0228] or,

[0229] Program 1110 can specifically be used to enable processor 1102 to perform the following operations: obtain first display data generated by the cloud application when processing an application request from a terminal application; filter out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; and send second display data determined based on the filtered first display data to the terminal application.

[0230] In an optional embodiment, program 1110 is also used to enable the processor 1102 to obtain cloud application interface data generated in the process of obtaining the cloud application processing the application request from the terminal application, and the cloud application interface data includes the status bar data of the cloud application and the data requested by the application request; determine the obtained cloud application interface data as the first display data; program 1110 is also used to enable the processor 1102 to filter out the status bar data from the first display data when filtering out preset data from the first display data.

[0231] In an optional embodiment, program 1110 is also used to enable the processor 1102 to obtain the cloud application interface data and system notification message data generated in the process of obtaining the first display data generated in the process of obtaining the cloud application processing the application request from the terminal application, and the cloud application interface data includes the status bar data of the cloud application and the data requested by the application request; determine the obtained cloud application interface data and system notification message data as the first display data; program 1110 is also used to enable the processor 1102 to filter out the status bar data and the system notification message data from the first display data when filtering out the preset data from the first display data.

[0232] In an optional embodiment, the program 1110 is also used to enable the processor 1102 to obtain the data requested by the application request contained in the filtered first display data when sending the second display data determined based on the filtered first display data to the terminal application, perform interface synthesis processing on the data requested by the application request, and obtain the second display data containing at least one terminal application interface image frame; and send the second display data to the terminal application.

[0233] or,

[0234] Program 1110 can specifically be used to enable the processor 1102 to perform the following operations: receive a user's exit operation on a terminal application on a terminal device, generate an application exit request based on the exit operation, and send the application exit request to a data processing end; receive an exit message returned by the data processing end in response to the application exit request, wherein the exit message carries information indicating that the cloud application corresponding to the terminal application has exited; exit the terminal application according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

[0235] In an optional embodiment, the program 1110 is further configured to enable the processor 1102 to send a disconnect request to the data processing end according to the exit message and close the display window of the terminal application when exiting the terminal application according to the exit message.

[0236] In an optional embodiment, program 1110 is also used to enable the processor 1102 to obtain the user's startup operation on the terminal application on the terminal device before receiving the user's exit operation on the terminal application on the terminal device, generating an application exit request based on the exit operation, and sending the application exit request to the data processing end, so that the data processing end starts the cloud application corresponding to the terminal application to process the request sent by the terminal application.

[0237] or,

[0238] Program 1110 can specifically be used to enable the processor 1102 to perform the following operations: exit the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application; send an exit message to the terminal application to indicate that the cloud application has exited, so as to instruct the terminal application to perform the exit operation through the exit message, and enable the terminal device where the terminal application is located to display the local desktop.

[0239] In an optional embodiment, program 1110 is also used to enable processor 1102 to determine a cloud application instance corresponding to the cloud application that matches the application exit request sent by the terminal application when exiting the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application; and pop the determined cloud application instance from the stack where it is located.

[0240] In an optional embodiment, the program 1110 is also used to enable the processor 1102 to receive a disconnect request sent by the terminal application after sending an exit message to the terminal application to indicate that the cloud application has exited, and disconnect from the terminal application according to the disconnect request.

[0241] In an optional embodiment, program 1110 is also used to enable the processor 1102 to establish a connection between the cloud application and the terminal application before exiting the cloud application corresponding to the terminal application in the data processing end based on the application exit request sent by the terminal application, and to process the request from the terminal application through the cloud application.

[0242] The specific implementation of each step in program 1110 can be found in the corresponding descriptions of the corresponding steps and units in the above-mentioned display data processing method or data processing method embodiment, and will not be repeated here. Those skilled in the art will clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the above-mentioned method embodiment, and will not be repeated here.

[0243] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present invention can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0244] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium and will be stored in a local recording medium, which is downloaded via a network, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the display data processing method or data processing method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the display data processing method or data processing method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the display data processing method or data processing method shown herein.

[0245] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present invention.

[0246] The above implementation methods are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Ordinary technicians in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims

1. A display data processing method, comprising: Determine whether the terminal application operated by the user displays the preset data according to the preset display strategy; If preset data is displayed, a data processing request is generated based on the user's operation on the terminal application on the terminal device, and the data processing request is sent to the data processing end, so that the cloud application corresponding to the operated terminal application in the data processing end processes the data processing request, and the data processing end processes the interface data of the cloud application into a video stream and sends it to the terminal device, and the terminal application in the terminal device displays the video stream; If the preset data is not displayed, generating an application request according to the user's operation on the terminal application on the terminal device, and sending the application request to the data processing end; Obtaining second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data after filtering out preset data, and the first display data is data obtained by the cloud application of the data processing end in response to the application request; The second display data is displayed.

2. The method according to claim 1, wherein The preset data includes status bar data of the cloud application in the data processing end.

3. The method according to claim 1, wherein The displaying of the second display data includes: The second display data is rendered and displayed by the terminal application to generate a display interface of the terminal application.

4. A display data processing method, comprising: When the cloud application processes the data processing request, the interface data of the cloud application is processed into a video stream and sent to the terminal device. When the terminal device determines, according to a preset display policy, that the terminal application operated by the user is to display preset data, the terminal device sends the data processing request to the cloud application corresponding to the terminal application on the data processing end; Obtaining first display data generated by the cloud application during processing of an application request from the terminal application, wherein the terminal device sends the application request to the cloud application corresponding to the terminal application on the data processing end when determining, according to a preset display policy, that the terminal application operated by the user does not display preset data; filtering out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; Sending second display data determined according to the filtered first display data to the terminal application.

5. The method according to claim 4, wherein The obtaining of first display data generated during the process of the cloud application processing the application request from the terminal application includes: Acquire cloud application interface data generated during the cloud application processing of the application request, the cloud application interface data including status bar data of the cloud application and data requested by the application request; Determining the acquired cloud application interface data as first display data; The filtering out the preset data from the first display data includes: The status bar data is filtered out from the first display data.

6. The method according to claim 4, wherein: The obtaining of first display data generated during the process of the cloud application processing the application request from the terminal application includes: Obtaining cloud application interface data and system notification message data generated during the cloud application processing of the application request, wherein the cloud application interface data includes status bar data of the cloud application and data requested by the application request; Determining the acquired cloud application interface data and system notification message data as the first display data; The filtering out the preset data from the first display data includes: The status bar data and the system notification message data are filtered out from the first display data.

7. The method according to any one of claims 4 to 6, wherein The sending, to the terminal application, second display data determined according to the filtered first display data, includes: Acquire the data requested by the application request contained in the filtered first display data, perform interface synthesis processing on the data requested by the application request, and obtain the second display data including at least one terminal application interface image frame; The second display data is sent to the terminal application.

8. A data processing method, comprising: Obtaining a user's startup operation on a terminal application on a terminal device, generating an application startup request according to the startup operation, and sending the application startup request to the data processing end, so that the data processing end starts a cloud application corresponding to the terminal application to process the request sent by the terminal application, wherein the terminal application sends the request by executing the display data processing method according to any one of claims 1 to 3 above; Receive a user's exit operation on a terminal application on a terminal device, generate an application exit request according to the exit operation, and send the application exit request to a data processing end; receiving a quit message returned by the data processing end in response to the application quit request, wherein the quit message carries information indicating that the cloud application corresponding to the terminal application has been quit; The terminal application is exited according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

9. The method according to claim 8, wherein Exiting the terminal application according to the exit message includes: According to the exit message, a disconnect request is sent to the data processing end, and the display window of the terminal application is closed.

10. A data processing method, comprising: Establishing a connection between a cloud application and a terminal application, and processing a request from the terminal application through the cloud application, wherein processing the request from the terminal application through the cloud application comprises executing the display data processing method according to any one of claims 4 to 7; Exit the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application; An exit message for indicating that the cloud application has exited is sent to the terminal application, so as to instruct the terminal application to perform an exit operation through the exit message, and enable the terminal device where the terminal application is located to display a local desktop.

11. The method according to claim 10, wherein: Exiting the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application includes: Determining, according to the application exit request sent by the terminal application, a cloud application instance corresponding to the cloud application that matches the application exit request; The determined cloud application instance is popped from the stack.

12. The method according to claim 11, wherein After sending a logout message to the terminal application indicating that the cloud application has been logged out, the method further includes: A disconnection request is received from the terminal application, and a connection with the terminal application is disconnected according to the disconnection request.

13. A display data processing device, comprising: A policy determination module, configured to determine whether a terminal application operated by a user displays preset data according to a preset display policy; If preset data is displayed, a data processing request is generated based on the user's operation on the terminal application on the terminal device, and the data processing request is sent to the data processing end, so that the cloud application corresponding to the operated terminal application in the data processing end processes the data processing request, and the data processing end processes the interface data of the cloud application into a video stream and sends it to the terminal device, and the terminal application in the terminal device displays the video stream; a sending module, configured to generate an application request according to a user's operation on a terminal application on the terminal device if the preset data is not displayed, and to send the application request to the data processing end; a first acquisition module, configured to acquire second display data returned by the data processing end in response to the application request, wherein the second display data is data determined by the data processing end based on the first display data after filtering out preset data, and the first display data is data obtained by the cloud application of the data processing end in response to the application request; The display module is configured to display the second display data.

14. A display data processing device, comprising: a second acquisition module, configured to, when the cloud application processes the data processing request, process the interface data of the cloud application into a video stream and transmit it to the terminal device; wherein, when the terminal device determines, based on a preset display policy, that the terminal application operated by the user is displaying preset data, transmit the data processing request to the cloud application corresponding to the terminal application on the data processing end; Obtaining first display data generated by the cloud application during processing of an application request from the terminal application, wherein the terminal device sends the application request to the cloud application corresponding to the terminal application on the data processing end when determining, according to a preset display policy, that the terminal application operated by the user does not display preset data; a filtering module, configured to filter out preset data from the first display data, wherein the preset data is used to indicate data other than the data requested by the application request; A third acquisition module is configured to send second display data determined according to the filtered first display data to the terminal application.

15. A data processing device comprising: a startup module, configured to obtain a user's startup operation on a terminal application on a terminal device, generate an application startup request based on the startup operation, and send the application startup request to the data processing end, so that the data processing end starts a cloud application corresponding to the terminal application to process the request sent by the terminal application, wherein the terminal application sends the request by executing the display data processing method according to any one of claims 1 to 3; A first receiving module is configured to receive a user's exit operation on a terminal application on a terminal device, generate an application exit request according to the exit operation, and send the application exit request to a data processing end; a second receiving module, configured to receive a quit message returned by the data processing end in response to the application quit request, wherein the quit message carries information indicating that the cloud application corresponding to the terminal application has been quit; The first exit module is configured to exit the terminal application according to the exit message, so that the terminal device where the terminal application is located displays the local desktop of the terminal device.

16. A data processing device comprising: a cloud application startup module, configured to establish a connection between a cloud application and a terminal application, and process a request from the terminal application through the cloud application, wherein processing the request from the terminal application through the cloud application comprises executing the display data processing method according to any one of claims 4 to 7; A second exit module is configured to exit the cloud application corresponding to the terminal application in the data processing end according to the application exit request sent by the terminal application; The message feedback module is used to send an exit message to the terminal application to indicate that the cloud application has exited, so as to instruct the terminal application to perform an exit operation through the exit message and enable the terminal device where the terminal application is located to display the local desktop.

17. An electronic device comprising: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, which enables the processor to perform an operation corresponding to the display data processing method described in any one of claims 1-3 or claims 4-7, or to perform an operation corresponding to the data processing method described in any one of claims 8-9 or any one of claims 10-12.

18. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the display data processing method according to any one of claims 1 to 3 or claims 4 to 7, or implements the data processing method according to any one of claims 8 to 9 or any one of claims 10 to 12.

Images