Cloud application interface processing system, method, apparatus, and device

Abstract

The application discloses a cloud application interface processing system, method, device and equipment. The system sends interface content information and interface format information of a cloud application to a terminal device through a first cloud application processing module deployed on a server, and presents the interface content information on a display unit according to the interface format information and display direction information of the terminal device through a second cloud application processing module deployed on the terminal device. In this way, the device can adjust the display of the cloud application interface that is accidentally rotated according to the interface format information, and the problem of inverted display of interface content can be solved without relying on third-party application adaptation, so that a mobile phone application can be directly transplanted into a cloud application scene for use. Therefore, the use experience of a cloud application user can be effectively improved, the expansion scene of the cloud application can be improved, and the development workload of a third-party application can be reduced.

Description

Technical Field

[0001] This application relates to the field of cloud application technology, specifically to cloud application interface processing systems, cloud gaming screen processing systems, cloud application interface processing methods and apparatus, and electronic devices. Background Technology

[0002] Cloud apps are applications that interact between a terminal and a service (cloud). Terminal operations are synchronized with the cloud, and the terminal data is backed up and retained in the cloud while occupying local space. Cloud apps not only help users reduce device costs, but their cross-platform nature also helps users significantly reduce usage costs and improve work efficiency. Therefore, the wave of innovation and transformation of traditional software towards cloud apps is unstoppable.

[0003] Cloud applications run in a virtual environment in the cloud, with the server sending the rendered results in real time to smart devices such as smart speakers and smart TVs for display. In practice, most cloud applications are quickly ported from existing mobile phone ecosystems rather than being specifically designed for cloud use, leading to various anomalies. One common anomaly is that while mobile phones support landscape / portrait switching, devices using cloud applications (such as smart speakers and smart TVs) do not. This results in a landscape / portrait switching issue when displaying the cloud application. In other words, the orientation of the rendered cloud application from the server differs from the orientation of the terminal device, causing the cloud application interface to rotate. This results in images and text appearing horizontally on landscape-oriented smart devices, making the cloud application content inverted and difficult for users to view, thus degrading the user experience. Currently, a typical approach to addressing this anomaly is to rely on third-party applications to adapt and resolve the issue.

[0004] However, in the process of realizing this invention, the inventors found that the technical solution has at least the following problems: relying on third-party applications to adapt and solve this problem will lead to a large number of applications needing to be modified. If the cloud application is not modified in time (such as the cloud application platform being unable to contact the third-party application developers), it will cause the cloud application to display the interface content inverted (such as portrait screen data lying horizontally), which will reduce the user experience of cloud application users and reduce the expansion scenarios of cloud applications. Summary of the Invention

[0005] This application provides a cloud application interface processing system to solve the problem in existing technologies where mobile applications need to be modified to avoid inverted display of interface content when porting them to cloud applications. This application also provides a cloud application interface processing method and apparatus, a cloud gaming screen processing system, a smart speaker, a smart TV, a smart air conditioner, and electronic devices.

[0006] This application provides a cloud application interface processing system, including:

[0007] The first cloud application processing module is used to determine the interface content information and interface layout information of the cloud application interface, and send the interface content information and interface layout information to the terminal device with the second cloud application processing module.

[0008] The second cloud application processing module is used to obtain the display orientation information of the terminal device; and to present the interface content information on the display unit according to the interface layout information and the display orientation information.

[0009] This application also provides a cloud gaming screen processing system, including:

[0010] The first cloud application processing module is used to determine the screen content information and screen layout information of the cloud game screen, and send the screen content information and screen layout information to the terminal device of the second cloud application processing module.

[0011] The second cloud application processing module is used to obtain the display orientation information of the terminal device; and to present the screen content information on the display unit according to the screen layout information and the display orientation information.

[0012] This application also provides a cloud application interface processing method, including:

[0013] Determine the interface content and layout information of the cloud application interface;

[0014] The interface content information and interface layout information are sent to a terminal device with a corresponding cloud application interface processing unit. The interface layout information is used to indicate the display direction of the interface content information to the terminal device.

[0015] Optionally, the method further includes:

[0016] Generate video data frames that include the interface content information;

[0017] Generate supplemental enhancement information (SEI) frames that include the interface layout information;

[0018] The step of sending the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit includes:

[0019] The video data frames and SEI frames are sent to the terminal device.

[0020] Optionally, sending the video data frames and SEI frames to the terminal device includes:

[0021] If the layout of the cloud application interface to be displayed is inconsistent with the preset layout of the cloud application, the SEI frame will be sent to the terminal device.

[0022] Optionally, sending the video data frames and SEI frames to the terminal device includes:

[0023] The SEI frame corresponding to the first frame of the cloud application interface is sent to the terminal device. If the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, the SEI frame corresponding to the cloud application interface to be displayed is sent to the terminal device.

[0024] Optionally, the interface content information and interface layout information can be determined by running the operating system container of the cloud application;

[0025] The video data frames and the SEI frames are generated through the operating system container;

[0026] The video data frames and SEI frames are sent to the terminal device through the operating system container.

[0027] This application also provides a cloud application interface processing method for displaying a cloud application interface provided by a cloud application platform on a terminal device, including:

[0028] Receive interface content and layout information of the cloud application interface sent by the cloud application platform.

[0029] Obtain the display orientation information of the terminal device;

[0030] Based on the interface layout and display orientation information, the interface content information is presented on the display unit.

[0031] Optionally, the interface content information and interface layout information of the cloud application interface sent by the cloud application platform include:

[0032] Receive video data frames including the interface content information and supplementary enhancement information (SEI) frames including the interface layout information;

[0033] The method further includes:

[0034] The interface content information is obtained from the video data frame, and the interface layout information is obtained from the SEI frame.

[0035] Optionally, if the layout of the cloud application interface to be displayed is inconsistent with the default layout of the cloud application, an SEI frame corresponding to the cloud application interface to be displayed will be received.

[0036] Optionally, if the cloud application interface to be displayed is the first frame of the cloud application, or if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, then an SEI frame corresponding to the cloud application interface to be displayed is received.

[0037] Optionally, based on the interface layout information and display orientation information, the interface content information is presented on the display unit, including:

[0038] Based on the interface layout information, determine the interface transition matrix;

[0039] Adjust the orientation of the interface content information according to the interface transformation matrix;

[0040] The adjusted interface content information is displayed in the display unit.

[0041] Optionally, the orientation of the interface content information can be adjusted according to the interface transformation matrix, including:

[0042] The interface content information is transformed and processed using the first translation matrix and rotation matrix to obtain the first interface content information after the orientation is adjusted.

[0043] The content information of the first interface is transformed using the second translation matrix and the scaling matrix to obtain the content information of the second interface after the orientation is adjusted.

[0044] Optional, also includes:

[0045] Obtain the first operation position on the cloud application interface after the orientation adjustment;

[0046] The first operation position is converted into a second operation position for the interface content information, and the second operation position is sent to the cloud platform.

[0047] Optionally, converting the first operation position into a second operation position for the interface content information includes:

[0048] Perform an inverse matrix transformation on the first operation position to obtain the second operation position.

[0049] Optionally, the terminal device does not support automatic switching between portrait and landscape orientations.

[0050] Optional, also includes:

[0051] The terminal device obtains a cloud application processing module provided by the cloud application platform to execute the method;

[0052] Load the cloud application processing module.

[0053] This application also provides a cloud application interface processing device, including:

[0054] The interface content and layout determination unit is used to determine the interface content information and interface layout information of the cloud application interface.

[0055] The interface content and layout sending unit is used to send the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit. The interface layout information is used to indicate the display direction of the interface content information to the terminal device.

[0056] This application also provides a cloud application interface processing device, including:

[0057] The interface content and layout receiving unit is used to receive the interface content information and interface layout information of the cloud application interface sent by the cloud application platform.

[0058] The device display orientation acquisition unit is used to acquire display orientation information of the terminal device.

[0059] The interface display unit is used to present the interface content information on the display unit according to the interface layout information and display orientation information.

[0060] This application also provides an electronic device, including:

[0061] Processor; and

[0062] A memory for storing a program that implements the above method, which is powered on by the device and runs through the processor.

[0063] Optionally, the electronic device includes one of the following: a smart TV, a smart speaker.

[0064] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the various methods described above.

[0065] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to perform the various methods described above.

[0066] Compared with the prior art, this application has the following advantages:

[0067] The cloud application interface processing system provided in this application embodiment sends the interface content information and interface layout information of the cloud application to the terminal device through a first cloud application processing module deployed on the server side. A second cloud application processing module deployed on the terminal device then presents the interface content information on the display unit according to the interface layout information and the display orientation information of the terminal device. This processing method allows the device to correct the display of the cloud application interface that has been accidentally rotated based on the interface layout information, avoiding reliance on third-party application adaptation to solve the problem of inverted interface content display. This allows mobile applications to be directly ported to cloud application scenarios. Therefore, it can effectively improve the user experience of cloud applications, expand the expansion scenarios of cloud applications, and reduce the development workload of third-party applications. Attached Figure Description

[0068] Figure 1 This application provides a schematic diagram of the structure of an embodiment of a cloud application interface processing system;

[0069] Figure 2 This application provides a schematic diagram illustrating an application scenario of an embodiment of a cloud application interface processing system.

[0070] Figure 3 This application provides a device interaction diagram of an embodiment of a cloud application interface processing system;

[0071] Figure 4 This application provides a schematic diagram of the interface display under abnormal conditions according to an embodiment of a cloud application interface processing system;

[0072] Figure 5 This application provides a schematic diagram of the interface operation of an embodiment of a cloud application interface processing system. Detailed Implementation

[0073] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0074] This application provides a cloud application interface processing system, a cloud gaming screen processing system, a cloud application interface processing method and apparatus, and an electronic device. The various solutions are described in detail in the following embodiments.

[0075] First Embodiment

[0076] Please refer to Figure 1 This is a structural diagram of an embodiment of the cloud application interface processing system of this application. The system includes: a first cloud application processing module 1 and a second cloud application processing module 2.

[0077] The first cloud application processing module 1 can be deployed on the server side of the cloud application platform. The server side can be a public cloud server side, providing cloud application services to multiple terminal devices. The server side can also be a private cloud server side, dedicated to providing cloud application services to terminal devices within a certain scope (such as a community, organization, etc.).

[0078] The second cloud application processing module 2 can be deployed on the terminal device of the cloud application user. The terminal device can be a device that does not support automatic switching between portrait and landscape modes, including but not limited to smart TVs, smart air conditioners, smart speakers and other smart devices, as well as smartphones (such as low-performance feature phones for the elderly), tablets and the like.

[0079] Cloud applications refer to applications that interact with a server. The terminal device's operations are synchronized with the server, and the terminal's data is backed up and retained through the server while occupying local space. Cloud applications have cross-platform characteristics; different terminal devices using the same cloud application can run different operating systems, and the terminal devices can have relatively few computing and storage resources.

[0080] Please refer to Figure 2 This is a schematic diagram illustrating the cloud application interface processing system of this application. The server and terminal devices can connect via a network, such as via Wi-Fi. Users can use the cloud application services provided by the server and view the cloud application interface provided by the server through their terminal devices. Figure 2 As shown, the cloud application interface may be a normal interface with a preset layout, or it may be an abnormal interface that has rotated relative to the preset layout. Taking cloud gaming on a smart speaker as an example, the cloud application interface may be the screen of the cloud game, which is a normal screen; the cloud application interface may also be a product promotion interface inserted into the game screen with a rotated layout, which is an abnormal screen. When the cloud application interface to be displayed is an abnormal interface, the interface content information and interface layout information can be sent to the terminal device, such as sending video data frames including interface content information and sending supplementary enhancement information (SEI) frames including interface layout information. The terminal device displays the abnormal interface in a positive orientation based on the interface content information and interface layout information to avoid the problem of images and text in the cloud application interface being horizontally oriented.

[0081] Please refer to Figure 3 This is a device interaction diagram of an embodiment of the cloud application interface processing system of this application. In this embodiment, the first cloud application processing module is used to determine the interface content information and interface layout information of the cloud application interface, and send the interface content information and interface layout information to a terminal device with a second cloud application processing module; the second cloud application processing module is used to obtain the display orientation information of the terminal device; and present the interface content information on the display unit according to the interface layout information and display orientation information.

[0082] The cloud application interface can be an image drawn through graphics processing. In this case, the interface content information is the rendering information of the cloud application interface. Rendering, in computer graphics, refers to the process of generating an image from a model using software. Through rendering, the final display effect of the model and animation can be obtained. In this embodiment, the cloud application platform generates rendering information of the cloud game screen on the server side and sends the rendering result of the cloud game screen to the terminal device in real time. As long as the terminal device has display capabilities, it can display the cloud application interface according to the interface rendering information. For example, the rotating advertisement inserted into the cloud game interface is an advertisement created through animation.

[0083] The cloud application interface can also be a captured image; in this case, the interface content information can be the original image information. For example, an advertisement that rotates and changes in shape inserted into a cloud gaming interface is an advertisement shot in real-life conditions.

[0084] The interface layout information includes, but is not limited to, whether the interface is landscape or portrait, and may also include the interface rotation angle. The interface rotation angle includes the rotation angle of the interface relative to the preset interface layout. For example, if the preset interface layout is landscape, and the interface rotation angle is 0 degrees, it indicates that the interface is a normal interface and does not require special processing for upright display; if the interface rotation angle is 90 degrees, it indicates that the interface is portrait and requires special processing for upright display. In practical applications, the interface rotation angle can also be 180 degrees, 270 degrees, 37 degrees, etc.

[0085] In practice, the cloud application server can determine the interface layout information based on the size of the cloud application interface. For example, if the horizontal size is larger than the vertical size, the interface is in landscape mode; if the horizontal size is smaller than the vertical size, the interface is in portrait mode.

[0086] In this embodiment, the cloud application server sends the interface content information and interface layout information to the terminal device. The terminal device can determine whether the interface layout of the cloud application interface to be displayed is consistent with the terminal device's display orientation based on the interface layout information and the terminal device's display orientation. If they are inconsistent, it indicates that the interface needs to be displayed in a forward orientation, and the cloud application interface is displayed in a forward orientation based on the interface layout information and interface content information. If the determination result is consistent, it indicates that the cloud application interface can be displayed directly based on the interface content information. The display orientation of the terminal device can be either landscape or portrait.

[0087] The rendering information of a single cloud application interface is equivalent to an image; the rendering information of multiple cloud application interfaces constitutes video data, with each interface's rendering information representing a single frame in the video. Therefore, the data volume of rendering information from multiple interfaces of a single cloud application is extremely large. The transmission networks for cloud application interface rendering information (such as 4G or 5G networks) typically cannot tolerate the massive data volume of video data (including rendering information from multiple cloud application interfaces). In this embodiment, the rendering information of the cloud application interfaces is encoded and compressed—essentially video encoding—before transmission. This ensures the normal transmission of the cloud application interface. The terminal device decodes the received video data frames using video decoding technology; the decoded data is the rendering information of the cloud application interface, which is then used to display the cloud application interface.

[0088] In specific implementation, the first cloud application processing module can also be used to generate video data frames including the interface content information; and to generate supplementary enhancement information (SEI) frames including the interface layout information, and specifically to send the video data frames and SEI frames to the terminal device.

[0089] The SEI (Supplemental Enhancement Information) frame belongs to the bitstream category and provides a method for adding extra information to the video bitstream. It is one of the features of video compression standards such as H.264 / H.265. The extra information added to the SEI frame by the cloud application platform server includes interface layout information.

[0090] like Figure 2 As shown, on the cloud application server side, an SEI frame can be inserted into the video data frame corresponding to the cloud application interface. This SEI frame records the interface layout information of the next cloud application interface. When the terminal device receives the SEI frame, it extracts the interface layout information of the next cloud application interface and compares it with the display orientation information of the terminal device to determine whether the interface layout of the next cloud application interface is consistent with the display orientation of the terminal device. If the determination result is inconsistent, it means that the next cloud application interface needs to be displayed in a forward orientation. Therefore, based on the interface layout information and the content information of the next cloud application interface, the next cloud application interface is displayed in a forward orientation.

[0091] In one example, sending the SEI frame to the terminal device can be achieved as follows: if the layout of the cloud application interface to be displayed is inconsistent with the preset interface layout of the cloud application, then the SEI frame is sent to the terminal device. The preset interface layout is the layout of the normal cloud application interface. Taking cloud gaming on a smart speaker as an example, the cloud game screen is a normal screen, while advertisements inserted into the game screen with rotating layouts are abnormal screens. The preset interface layout can be horizontal or vertical. Using this method, whenever the layout of the cloud application interface is inconsistent with the display orientation of the terminal device, the corresponding SEI frame needs to be transmitted based on the video data frames of that interface. That is, if the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, multiple SEI frames need to be sent. Therefore, this method requires the transmission of a large number of SEI frames, consuming significant computing and network resources.

[0092] In another example, the SEI frame can be sent to the terminal device as follows: The SEI frame corresponding to the first frame of the cloud application is sent to the terminal device; and if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, the SEI frame corresponding to the cloud application interface to be displayed is sent to the terminal device. This approach ensures that SEI frames are transmitted only when the first frame of the cloud application or the layout of the cloud application interface changes. If the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, only one SEI frame needs to be sent for the first interface whose layout changed. Thus, even if the display orientation of the terminal device is inconsistent with the preset interface layout, the cloud application interface can still be displayed in a positive orientation, effectively improving the flexibility of the terminal device's display orientation. Furthermore, this method requires fewer SEI frames to be transmitted, effectively saving computing and network resources.

[0093] In this embodiment, the cloud platform determines the interface content information and interface layout information on the server side by running an operating system container (such as an Android container, an iOS container, etc.) of the cloud application, generates the video data frame and the SEI frame, and sends the video data frame and the SEI frame to the terminal device. The operating system container can be an Android container, an iOS container, etc.

[0094] In one example, the terminal device receives a video data frame containing interface content information and an SEI frame containing interface layout information from the server. The terminal device needs to decode the video data frame using video decoding technology to obtain the interface content information. Similarly, it also needs to read the interface layout information from the SEI frame. This processing method can effectively save transmission network resources.

[0095] In another example, the terminal device receives interface content and layout information from the server and can directly determine whether the interface layout of the cloud application interface is consistent with the display orientation of the terminal device based on the received interface content and layout information. This processing method can effectively improve interface display speed.

[0096] If the terminal device determines that the layout of the cloud application interface is inconsistent with the display orientation of the terminal device, it will display the cloud application interface in the forward orientation according to the layout information and content information.

[0097] The terminal device can obtain its display orientation information and compare it with the received interface layout information to determine whether the interface layout of the cloud application interface is consistent with the display orientation of the terminal device. If it is determined that the interface layout of the cloud application interface is inconsistent with the display orientation of the terminal device, it indicates that the cloud application interface is an abnormal interface and needs to be corrected to display in the correct orientation.

[0098] In one example, the terminal device will receive an SEI frame corresponding to the displayed cloud application interface whenever the layout of the displayed interface differs from the default layout of the cloud application. This approach ensures that an SEI frame will be received whenever the layout of the cloud application interface is inconsistent with the display orientation of the terminal device. In other words, if the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, multiple SEI frames will be received, thus consuming significant network resources.

[0099] In another example, if the cloud application interface to be displayed is the first frame of the cloud application, or if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, then the SEI frame corresponding to the cloud application interface to be displayed is received. This processing method ensures that the corresponding SEI frame is only received when the cloud application's first frame or its layout changes. Thus, even if the terminal device's display orientation is inconsistent with the cloud application's preset interface layout, the cloud application interface can still be displayed correctly, effectively improving the flexibility of the terminal device's display orientation. Furthermore, this method requires fewer SEI frames to be transmitted, effectively saving computing and network resources.

[0100] In one example, presenting the interface content information on the display unit based on the interface layout information and display orientation information may include the following sub-steps: determining an interface transformation matrix based on the interface layout information; adjusting the orientation of the interface content information based on the interface transformation matrix; and presenting the adjusted interface content information on the display unit. For example, if the layout of the cloud application interface to be displayed is inconsistent with the display orientation of the terminal device, the orientation of the interface content information can be adjusted according to the interface transformation matrix to ensure that the cloud application interface to be displayed is displayed in a positive orientation.

[0101] The interface conversion matrix refers to a matrix that can adjust the display of cloud application interface content (such as images and text) whose interface layout is inconsistent with the display orientation of the terminal device. The interface conversion matrix may include multiple matrices, such as a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix.

[0102] In one example, adjusting the orientation of the interface content information according to the interface transformation matrix may include the following sub-steps: 1) Transforming the interface content information using a first translation matrix and a rotation matrix to obtain the first interface content information after orientation adjustment; 2) Transforming the first interface content information using a second translation matrix and a scaling matrix to obtain the second interface content information after orientation adjustment.

[0103] like Figure 4As shown, the first translation matrix is ​​used to translate the interface content information, and the rotation matrix is ​​used to rotate the interface content information. After transforming the interface content information using the first translation matrix and the rotation matrix, the following can be obtained: Figure 4 The first interface content information after adjustment when the interface size exceeds the screen size. The second translation matrix is ​​used to translate the first interface content information, and the scaling matrix is ​​used to scale the first interface content information. After transforming the first interface content information through the second translation matrix and the scaling matrix, the following can be obtained: Figure 4 The content information of the second interface after the orientation is adjusted to fit the screen size.

[0104] It should be noted that because the interface content is translated, rotated, and scaled when the interface layout changes, it is impossible to directly obtain the operation position information of the interface content. To solve this problem, the terminal device can also be used to obtain a first operation position of the cloud application interface after the orientation is adjusted and displayed in the forward direction; convert the first operation position into a second operation position of the interface content information; and send the second operation position to the server of the cloud platform.

[0105] The first operation position refers to the user's operation position on the adjusted cloud application interface, such as the position where the user operates the adjusted cloud application interface via touch on a touchscreen device. Figure 5 As shown, the first operation position is the coordinates (x, y) in the upper right corner of the interface.

[0106] The second operation position refers to the position where the user interacts with the interface content information. For example... Figure 5 As shown, the second operation position is the coordinate (x, y) in the upper left corner of the interface.

[0107] In this embodiment, an inverse matrix transformation is performed on the first operation position to obtain a second operation position. The inverse transformation matrix may include multiple matrices, such as the inverse matrix of the first translation matrix, the inverse matrix of the rotation matrix, the inverse matrix of the second translation matrix, and the inverse matrix of the scaling matrix. Converting the first operation position to a second operation position for the interface content information may include the following sub-steps: 1) Transforming the first operation position using the inverse matrix of the second translation matrix and the inverse matrix of the scaling matrix to obtain a third operation position, such as... Figure 5 As shown, the third operation position is the coordinate (x, y) in the interface below; 2) The third operation position is transformed by the inverse matrix of the first translation matrix and the inverse matrix of the rotation matrix to obtain the second operation position.

[0108] As can be seen from the above embodiments, the cloud application interface processing system provided in this application sends the interface rendering information and interface layout information of the cloud application to the terminal device through a first cloud application processing module deployed on the server side, and presents the interface content information on the display unit according to the interface layout information and the display orientation information of the terminal device through a second cloud application processing module deployed on the terminal device. This processing method enables the device to correct the display of the cloud application interface that has been accidentally rotated according to the interface layout information, avoiding the need to rely on third-party application adaptation to solve the problem of inverted display of interface content. In this way, mobile applications can be directly ported to cloud application scenarios. Therefore, it can effectively improve the user experience of cloud applications, expand the expansion scenarios of cloud applications, and reduce the development workload of third-party applications.

[0109] Second Embodiment

[0110] Corresponding to the cloud application interface processing system described above, this application also provides a cloud application interface processing method. The execution entity of this method includes, but is not limited to, a first cloud application processing module deployed on the cloud platform server. The parts of this embodiment that are the same as those in the first embodiment will not be repeated here; please refer to the corresponding parts in Embodiment 1.

[0111] In this embodiment, the method includes the following steps:

[0112] Step 1: Determine the interface content and layout information of the cloud application interface to be displayed.

[0113] Cloud applications refer to applications where terminal devices interact with a service (cloud). Terminal device operations are synchronized with the cloud, and the data used on the local device is also backed up and retained in the cloud. These terminal devices can be IoT devices such as smart speakers, smart TVs, and smart refrigerators, as well as smartphones and tablets. Cloud applications have cross-platform characteristics; different terminal devices using the same cloud application can run different operating systems, and these devices can require relatively few computing and storage resources.

[0114] The cloud application interface may be a normal interface with a preset layout, or it may be an abnormal interface that has been rotated relative to the preset layout. Taking cloud gaming as an example, the cloud application interface may be the screen of the cloud game, which is a normal screen; the cloud application interface may also be a product promotion interface inserted into the game screen with a rotated layout, which is an abnormal screen.

[0115] In this embodiment, the server generates the interface content information of the cloud application and sends it to the terminal device in real time. As long as the terminal device has display capabilities, it can display the cloud application interface according to the interface rendering information. Since the interface content information and its generation method are existing technologies, they will not be described in detail here.

[0116] The interface layout information includes, but is not limited to, whether the interface is landscape or portrait, and may also include the interface rotation angle. The interface rotation angle includes the rotation angle of the interface relative to the preset interface layout. For example, if the preset interface layout is landscape, and the interface rotation angle is 0 degrees, it indicates that the interface is a normal interface and does not require special processing for upright display; if the interface rotation angle is 90 degrees, it indicates that the interface is portrait and requires special processing for upright display. In practical applications, the interface rotation angle can also be 180 degrees, 270 degrees, 37 degrees, etc.

[0117] In practice, the cloud application server can determine the interface layout information based on the size of the cloud application interface. For example, if the horizontal size is larger than the vertical size, the interface is in landscape mode; if the horizontal size is smaller than the vertical size, the interface is in portrait mode.

[0118] Step 2: Send the interface content information and interface layout information to the terminal device with the corresponding cloud application interface processing unit. The interface layout information is used to indicate the display direction of the interface content information to the terminal device.

[0119] In this embodiment, the cloud application server sends the interface content information and interface layout information to the terminal device. The terminal device can determine whether the interface layout of the cloud application interface to be displayed is consistent with the terminal device's display orientation based on the interface layout information and the terminal device's display orientation. If they are inconsistent, it indicates that the interface needs to be displayed in a forward orientation, and the cloud application interface is displayed in a forward orientation based on the interface layout information and interface content information. If the determination result is consistent, it indicates that the cloud application interface can be displayed directly based on the interface content information. The display orientation of the terminal device can be either landscape or portrait.

[0120] In specific implementation, the method may further include the following steps: generating a video data frame including the interface content information; and generating a supplementary enhancement information (SEI) frame including the interface layout information. Accordingly, step 3 is implemented as follows: sending the video data frame and the SEI frame to the terminal device. This processing method allows for the compression of the content information of the cloud application interface, effectively saving network traffic.

[0121] The method provided in this application inserts an SEI frame into the video data frame corresponding to the cloud application interface. This SEI frame records the interface layout information of the next cloud application interface. When the terminal device receives the SEI frame, it extracts the interface layout information of the next cloud application interface and compares it with the display orientation information of the terminal device to determine whether the interface layout of the next cloud application interface is consistent with the display orientation of the terminal device. If the determination result is inconsistent, it indicates that the next cloud application interface needs to be displayed in a forward orientation. Therefore, based on the interface layout information and the interface content information of the next cloud application interface, the next cloud application interface is displayed in a forward orientation.

[0122] In one example, the SEI frame can be sent to the terminal device as follows: if the layout of the cloud application interface to be displayed is inconsistent with the preset interface layout of the cloud application, then the SEI frame is sent to the terminal device. This approach means that whenever the layout of the cloud application interface is inconsistent with the display orientation of the terminal device, a corresponding SEI frame needs to be transmitted based on the video data frames of that interface. In other words, if the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, multiple SEI frames need to be sent. Therefore, this method requires the transmission of a large number of SEI frames, consuming significant computing and network resources.

[0123] In another example, the SEI frame can be sent to the terminal device as follows: The SEI frame corresponding to the first frame of the cloud application is sent to the terminal device; and if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, the SEI frame corresponding to the cloud application interface to be displayed is sent to the terminal device. This approach ensures that SEI frames are transmitted only when the first frame of the cloud application or the layout of the cloud application interface changes. If the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, only one SEI frame needs to be sent for the first interface whose layout changed. Thus, even if the display orientation of the terminal device is inconsistent with the preset interface layout, the cloud application interface can still be displayed in a positive orientation, effectively improving the flexibility of the terminal device's display orientation. Furthermore, this method requires fewer SEI frames to be transmitted, effectively saving computing and network resources.

[0124] In this embodiment, the first cloud application processing module determines the interface content information and interface layout information by running the operating system container (such as an Android container, an iOS container, etc.) of the cloud application, generates the video data frame and the SEI frame, and sends the video data frame and the SEI frame to the terminal device. The operating system container can be an Android container, an iOS container, etc.

[0125] As can be seen from the above embodiments, the cloud application interface processing method provided in this application determines the interface content information and interface layout information of the cloud application interface; sends the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit, and the interface layout information is used to indicate the display direction of the interface content information to the terminal device. This processing method allows the device to correct the display of an unexpectedly rotated cloud application interface based on the interface layout information, avoiding reliance on third-party application adaptation to solve the problem of inverted interface content display. This allows mobile applications to be directly ported to cloud application scenarios; therefore, it can effectively improve the user experience of cloud applications, expand the expansion scenarios of cloud applications, and reduce the development workload of third-party applications.

[0126] Third Embodiment

[0127] In the above embodiments, a cloud application interface processing method is provided. Correspondingly, this application also provides a cloud application interface processing apparatus. This apparatus corresponds to the embodiments of the above method.

[0128] The parts of this embodiment that are the same as those in the first embodiment will not be repeated here; please refer to the corresponding parts in Embodiment 1. A cloud application interface processing device provided in this application includes:

[0129] The interface content and layout determination unit is used to determine the interface content information and interface layout information of the cloud application interface.

[0130] The interface content and layout sending unit is used to send the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit. The interface layout information is used to indicate the display direction of the interface content information to the terminal device.

[0131] Fourth embodiment

[0132] This application also provides an embodiment of an electronic device. Since the device embodiment is substantially similar to the method embodiment, it is described simply; relevant details can be found in the description of the method embodiment. The device embodiment described below is merely illustrative.

[0133] This embodiment provides an electronic device, which includes a processor and a memory. The memory stores a program for implementing a cloud application interface processing method. After the device is powered on and the program for the method is run by the processor, it performs the following steps: determining the interface content information and interface layout information of the cloud application interface; sending the interface content information and interface layout information to a terminal device having a corresponding cloud application interface processing unit, wherein the interface layout information is used to indicate the display direction of the interface content information to the terminal device.

[0134] Fifth embodiment

[0135] Corresponding to the aforementioned cloud application interface processing system, this application also provides a cloud application interface processing method. The execution entity of this method includes, but is not limited to, a second cloud application processing module deployed on a terminal device. The terminal device may be a device that does not support automatic switching between portrait and landscape modes, including but not limited to smart TVs, smart air conditioners, smart speakers, and other smart devices, as well as smartphones (such as low-performance feature phones for the elderly), tablet computers, etc. The parts of this embodiment that are the same as those in the first embodiment will not be repeated here; please refer to the corresponding parts in Embodiment 1.

[0136] In this embodiment, the method includes the following steps:

[0137] Step 1: Receive the interface content and layout information of the cloud application interface sent by the cloud application platform.

[0138] In one example, the terminal device receives a video data frame containing interface information and an SEI frame containing interface layout information from the server. The terminal device needs to decode the video data frame using video decoding technology to obtain the interface information. Similarly, it also needs to read the interface layout information from the SEI frame. This processing method can effectively save transmission network resources.

[0139] In another example, the terminal device receives interface and layout information from the server and can directly determine whether the layout of the cloud application interface matches the display orientation of the terminal device based on this information. This approach can effectively improve interface display speed.

[0140] Step 2: Obtain the display orientation information of the terminal device.

[0141] Step 3: Based on the interface layout information and display orientation information, present the interface content information on the display unit.

[0142] The terminal device can obtain its display orientation information and compare it with the received interface layout information to determine whether the interface layout of the cloud application interface is consistent with the display orientation of the terminal device. If it is determined that the interface layout of the cloud application interface is inconsistent with the display orientation of the terminal device, it indicates that the cloud application interface is an abnormal interface and needs to be corrected to display in the correct orientation. The method of obtaining display orientation information by the terminal device is a relatively mature existing technology, so it will not be described in detail here.

[0143] In practical applications, the display orientation of the terminal device can be consistent with or inconsistent with the preset interface layout of the cloud application.

[0144] In one example, the terminal device will receive an SEI frame corresponding to the displayed cloud application interface whenever the layout of the displayed interface differs from the default layout of the cloud application. This approach ensures that an SEI frame will be received whenever the layout of the cloud application interface is inconsistent with the display orientation of the terminal device. In other words, if the layouts of multiple consecutive interfaces are inconsistent with the display orientation of the terminal device, multiple SEI frames will be received, thus consuming significant network resources.

[0145] In another example, if the cloud application interface to be displayed is the first frame of the cloud application, or if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, then the SEI frame corresponding to the cloud application interface to be displayed is received. This processing method ensures that the corresponding SEI frame is only received when the cloud application's first frame or its layout changes. Thus, even if the terminal device's display orientation is inconsistent with the cloud application's preset interface layout, the cloud application interface can still be displayed correctly, effectively improving the flexibility of the terminal device's display orientation. Furthermore, this method requires fewer SEI frames to be transmitted, effectively saving computing and network resources.

[0146] In one example, presenting the interface content information on the display unit based on the interface layout information and display orientation information may include the following sub-steps: determining an interface transformation matrix based on the interface layout information; adjusting the orientation of the interface content information based on the interface transformation matrix; and presenting the adjusted interface content information on the display unit. For example, if the layout of the cloud application interface to be displayed is inconsistent with the display orientation of the terminal device, the orientation of the interface content information can be adjusted according to the interface transformation matrix to ensure that the cloud application interface to be displayed is displayed in a positive orientation.

[0147] In one example, adjusting the orientation of the interface content information according to the interface transformation matrix may include the following sub-steps: 1) Transforming the interface content information using a first translation matrix and a rotation matrix to obtain the first interface content information after orientation adjustment; 2) Transforming the first interface content information using a second translation matrix and a scaling matrix to obtain the second interface content information after orientation adjustment.

[0148] It should be noted that when the interface layout changes, the interface content information is translated, rotated, and scaled, making it impossible to directly obtain the operation position information of the interface content information. To solve this problem, the method may further include the following steps: the terminal device may also be used to: convert the first operation position into a second operation position of the interface content information, and send the second operation position to the server of the cloud platform.

[0149] Step 4: Obtain the first operation position of the cloud application interface after the orientation is adjusted.

[0150] Step 5: Convert the first operation position into a second operation position for the interface content information, and send the second operation position to the cloud platform.

[0151] The first operation position refers to the user's operation position on the adjusted cloud application interface, such as the position where the user operates the adjusted cloud application interface via touch on a touchscreen device. The second operation position refers to the user's operation position on the interface rendering information.

[0152] In this embodiment, an inverse matrix transformation is performed on the first operation position to obtain a second operation position. The inverse transformation matrix may include multiple matrices, such as the inverse matrix of the first translation matrix, the inverse matrix of the rotation matrix, the inverse matrix of the second translation matrix, and the inverse matrix of the scaling matrix. Converting the first operation position to a second operation position for the interface content information may include the following sub-steps: 1) Transforming the first operation position using the inverse matrix of the second translation matrix and the inverse matrix of the scaling matrix to obtain a third operation position, such as... Figure 5 As shown, the third operation position is the coordinate (x, y) in the interface below; 2) The third operation position is transformed by the inverse matrix of the first translation matrix and the inverse matrix of the rotation matrix to obtain the second operation position.

[0153] In this embodiment, the terminal device obtains a cloud application processing module provided by the cloud application platform to execute the method, and loads the cloud application processing module so that when the user uses the cloud application through the terminal device, the terminal device can display the cloud application interface provided by the cloud application platform with the interface layout rotated in the forward direction.

[0154] As can be seen from the above embodiments, the cloud application interface processing method provided in this application receives interface content information and interface layout information of the cloud application interface sent by the cloud application platform; obtains the display orientation information of the terminal device; and presents the interface content information on the display unit according to the interface layout information and display orientation information. This processing method allows the device to correct the display of an unexpectedly rotated cloud application interface according to the interface layout information, avoiding reliance on third-party application adaptation to solve the problem of inverted interface content display. This allows mobile applications to be directly ported to cloud application scenarios; therefore, it can effectively improve the user experience of cloud applications, expand the expansion scenarios of cloud applications, and reduce the development workload of third-party applications.

[0155] Sixth Embodiment

[0156] In the above embodiments, a cloud application interface processing method is provided. Correspondingly, this application also provides a cloud application interface processing apparatus. This apparatus corresponds to the embodiments of the above method.

[0157] The parts of this embodiment that are the same as those in the first embodiment will not be repeated here; please refer to the corresponding parts in Embodiment 1. A cloud application interface processing device provided in this application includes:

[0158] The interface content and layout receiving unit is used to receive the interface content information and interface layout information of the cloud application interface sent by the cloud application platform.

[0159] The device display orientation acquisition unit is used to acquire display orientation information of the terminal device.

[0160] The interface display unit is used to present the interface content information on the display unit according to the interface layout information and display orientation information.

[0161] Seventh Embodiment

[0162] This application also provides an embodiment of an electronic device. Since the device embodiment is substantially similar to the method embodiment, it is described simply; relevant details can be found in the description of the method embodiment. The device embodiment described below is merely illustrative.

[0163] This embodiment provides an electronic device, which includes a processor and a memory. The memory stores a program for implementing a cloud application interface processing method. After the device is powered on and the program for the method is run by the processor, it performs the following steps: receiving interface content information and interface layout information of the cloud application interface sent by the cloud application platform; obtaining display orientation information of the terminal device; and presenting the interface content information on the display unit according to the interface layout information and display orientation information.

[0164] Eighth embodiment

[0165] Corresponding to the cloud application interface processing system described above, this application also provides a cloud gaming screen processing system. The parts of this embodiment that are the same as those in the first embodiment will not be repeated here; please refer to the corresponding parts in Embodiment 1.

[0166] The cloud gaming screen processing system provided in this application may include: a first cloud application processing module and a second cloud application processing module.

[0167] The first cloud application processing module is used to determine the screen content information and screen layout information of the cloud game screen, and send the screen content information and screen layout information to the terminal device of the second cloud application processing module; the second cloud application processing module is used to obtain the display orientation information of the terminal device; and present the screen content information on the display unit according to the screen layout information and display orientation information.

[0168] The cloud gaming screen processing system provided in this application embodiment sends the screen content information and screen layout information of the cloud gaming screen to the terminal device through a first cloud application processing module deployed on the server side. A second cloud application processing module deployed on the terminal device then presents the screen content information on the display unit according to the screen layout information and the display orientation information of the terminal device. This processing method allows the device to correct the display of the cloud gaming screen that has been accidentally rotated based on the screen layout information, avoiding reliance on third-party cloud gaming application developers to adapt and solve the problem of inverted game screen display. This allows mobile games to be directly ported to cloud application scenarios. Therefore, it can effectively improve the user experience of cloud gaming, expand the expansion scenarios of cloud gaming, and reduce the development workload of third-party cloud gaming applications.

[0169] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0170] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0171] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0172] 1. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0173] 2. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A cloud application interface processing system, characterized in that, include: The first cloud application processing module is used to port applications developed for devices that support landscape and portrait screen switching to cloud applications. The interface content information and interface layout information of the cloud application interface are determined, and the interface content information and interface layout information are sent to a terminal device with a second cloud application processing module. The terminal device does not support landscape or portrait screen switching. The second cloud application processing module is used to obtain the display orientation information of the terminal device; and to present the interface content information on the display unit according to the interface layout information and the display orientation information. The step of presenting the interface content information on the display unit according to the interface layout information and display orientation information includes: Based on the interface layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The interface content information is transformed using the first translation matrix and the rotation matrix to obtain the first interface content information after the orientation is adjusted. The first interface content information is transformed using the second translation matrix and the scaling matrix to obtain the second interface content information after the orientation is adjusted. The content information of the second interface is presented on the display unit.

2. A cloud gaming screen processing system, characterized in that, include: The first cloud application processing module is used to port game applications developed for devices that support landscape and portrait screen switching to cloud gaming. The screen content information and screen layout information of the cloud gaming screen are determined, and the screen content information and screen layout information are sent to the terminal device of the second cloud application processing module. The terminal device does not support landscape or portrait screen switching. The second cloud application processing module is used to obtain the display orientation information of the terminal device; and to present the screen content information on the display unit according to the screen layout information and the display orientation information. The step of presenting the screen content information on the display unit according to the screen layout information and display orientation information includes: Based on the screen layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The image content information is transformed using the first translation matrix and the rotation matrix to obtain the first image content information after the orientation is adjusted. The first screen content information is transformed using the second translation matrix and the scaling matrix to obtain the second screen content information after the orientation is adjusted. The content information of the second screen is presented on the display unit.

3. A cloud application interface processing method, characterized in that, include: Port applications developed for devices that support landscape / portrait mode switching to cloud applications; Determine the interface content and layout information of the cloud application interface; The interface content information and interface layout information are sent to a terminal device with a corresponding cloud application interface processing unit. The interface layout information is used to indicate the display direction of the interface content information to the terminal device. The terminal device does not support landscape or portrait screen switching. Based on the interface layout information and the display orientation information of the terminal device, the interface content information is presented on the display unit; The step of presenting the interface content information on the display unit according to the interface layout information and the display orientation information of the terminal device includes: Based on the interface layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The interface content information is transformed using the first translation matrix and the rotation matrix to obtain the first interface content information after the orientation is adjusted. The first interface content information is transformed using the second translation matrix and the scaling matrix to obtain the second interface content information after the orientation is adjusted. The content information of the second interface is presented on the display unit.

4. The method according to claim 3, characterized in that, The method further includes: Generate video data frames that include the interface content information; Generate supplemental enhancement information (SEI) frames that include the interface layout information; The step of sending the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit includes: The video data frames and SEI frames are sent to the terminal device.

5. The method according to claim 4, characterized in that, The step of sending the video data frames and SEI frames to the terminal device includes: If the layout of the cloud application interface to be displayed is inconsistent with the preset layout of the cloud application, the SEI frame will be sent to the terminal device.

6. The method according to claim 4, characterized in that, The step of sending the video data frames and SEI frames to the terminal device includes: The SEI frame corresponding to the first frame of the cloud application interface is sent to the terminal device. If the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, the SEI frame corresponding to the cloud application interface to be displayed is sent to the terminal device.

7. The method according to claim 4, characterized in that, The interface content information and interface layout information are determined by running the operating system container of the cloud application. The video data frames and the SEI frames are generated through the operating system container; The video data frames and SEI frames are sent to the terminal device through the operating system container.

8. A cloud application interface processing method for displaying a cloud application interface provided by a cloud application platform on a terminal device, characterized in that, include: The cloud application receives interface content information and interface layout information from the cloud application platform provider, and the platform provider will port the application developed for devices that support landscape and portrait screen switching to the cloud application. Obtain the display orientation information of the terminal device, wherein the terminal device does not support landscape or portrait screen switching; Based on the interface layout information and display orientation information, the interface content information is presented in the display unit; The step of presenting the interface content information on the display unit based on the interface layout information and display orientation information includes: Based on the interface layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The interface content information is transformed using the first translation matrix and the rotation matrix to obtain the first interface content information after the orientation is adjusted. The first interface content information is transformed using the second translation matrix and the scaling matrix to obtain the second interface content information after the orientation is adjusted. The content information of the second interface is presented on the display unit.

9. The method according to claim 8, characterized in that, The interface content information and interface layout information of the cloud application interface sent by the cloud application platform include: Receive video data frames including the interface content information and supplementary enhancement information (SEI) frames including the interface layout information; The method further includes: The interface content information is obtained from the video data frame, and the interface layout information is obtained from the SEI frame.

10. The method according to claim 9, characterized in that, If the layout of the cloud application interface to be displayed is inconsistent with the default layout of the cloud application, then the SEI frame corresponding to the cloud application interface to be displayed will be received.

11. The method according to claim 9, characterized in that, If the cloud application interface to be displayed is the first frame of the cloud application interface, or if the layout of the cloud application interface to be displayed is inconsistent with the layout of the previous cloud application interface, then the SEI frame corresponding to the cloud application interface to be displayed is received.

12. The method according to claim 8, characterized in that, Also includes: Obtain the first operation position on the cloud application interface after the orientation adjustment; The first operation position is converted into a second operation position for the interface content information, and the second operation position is sent to the cloud platform.

13. The method according to claim 12, characterized in that, The step of converting the first operation position to a second operation position for the interface content information includes: Perform an inverse matrix transformation on the first operation position to obtain the second operation position.

14. The method according to claim 8, characterized in that, Also includes: The terminal device obtains a cloud application processing module provided by the cloud application platform to execute the method; Load the cloud application processing module.

15. A cloud application interface processing device, characterized in that, include: The Interface Content and Layout Determination Unit is used to port applications developed for devices that support landscape / portrait screen switching to cloud applications. Determine the interface content and layout information of the cloud application interface; The interface content and layout sending unit is used to send the interface content information and interface layout information to a terminal device with a corresponding cloud application interface processing unit. The interface layout information is used to indicate to the terminal device the display direction of the interface content information. The terminal device does not support landscape or portrait orientation switching. The device is also used for: Based on the interface layout information and the display orientation information of the terminal device, the interface content information is presented on the display unit; The step of presenting the interface content information on the display unit according to the interface layout information and the display orientation information of the terminal device includes: Based on the interface layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The interface content information is transformed using the first translation matrix and the rotation matrix to obtain the first interface content information after the orientation is adjusted. The first interface content information is transformed using the second translation matrix and the scaling matrix to obtain the second interface content information after the orientation is adjusted. The content information of the second interface is presented on the display unit.

16. A cloud application interface processing device, characterized in that, include: The interface content and layout receiving unit is used to receive interface content information and interface layout information of the cloud application interface sent by the cloud application platform provider, wherein the platform provider will port the application developed for devices that support landscape and portrait screen conversion to the cloud application. A device display orientation acquisition unit is used to acquire display orientation information of a terminal device, wherein the terminal device does not support landscape or portrait screen switching. The interface display unit is used to present the interface content information on the display unit according to the interface layout information and display orientation information; Specifically, the interface display unit is used for: Based on the interface layout information, an interface transformation matrix is ​​determined, which includes at least a first translation matrix, a rotation matrix, a second translation matrix, and a scaling matrix. The interface content information is transformed using the first translation matrix and the rotation matrix to obtain the first interface content information after the orientation is adjusted. The first interface content information is transformed using the second translation matrix and the scaling matrix to obtain the second interface content information after the orientation is adjusted. The content information of the second interface is presented on the display unit.

17. An electronic device, characterized in that, include: processor; as well as A memory for storing a program for implementing the method according to any one of claims 3 to 14, wherein the device is powered on and the program for running the method is executed by the processor.

18. The electronic device according to claim 17, characterized in that, The electronic device includes one of the following: a smart TV, a smart speaker.

Images