Application picture rendering method, device and equipment and readable storage medium

Abstract

The invention discloses an application picture rendering method, device and equipment and a readable storage medium. The method comprises the following steps: creating a rendering program container corresponding to a target mobile terminal application; obtaining shader operation data of a binary language type pre-stored for the target mobile terminal application, and loading the shader operation data into a rendering program container; wherein the shader operation data comprises binary program data obtained by compiling a local end shader program link, and binary mapping data representing a grammar variable mapping relationship between a mobile end shader program of the target mobile end application and the local end shader program; the local end shader program is obtained through grammar translation based on the mobile end shader program; and executing a corresponding target shader program based on the binary program data and the binary mapping data in the rendering program container so as to render a picture of the target mobile terminal application in the simulator. The rendering efficiency of the application picture is improved, and the security of a shader program is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an application screen rendering method, device, equipment and readable storage medium. Background Art

[0002] The development of information technology has promoted the rapid development of entertainment business, and more and more applications can be run on mobile phones, such as game applications, film and television applications, shopping applications, etc. However, there are certain limitations when using some mobile versions of applications on mobile phones. In this regard, mobile versions of applications can be used on personal computers to broaden the use of applications on mobile phones.

[0003] The related technology creates a rendering program for the mobile version of the application on a personal computer, creates and binds a vertex shader and a fragment shader in the rendering program, links the vertex shader and the fragment shader in the rendering program to form a rendering pipeline, and renders the mobile version of the application according to the rendering pipeline in the rendering program so as to use the mobile version of the application on the personal computer. Thereafter, when the mobile version of the application stops being used on the personal computer, the rendering pipeline in the rendering program is released and the rendering program is deleted.

[0004] During the research and practice of related technologies, the inventors of the present application discovered that when the related technology creates a rendering program for a mobile version of an application on a personal computer, it is necessary to go through a cumbersome process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, which makes the startup time of the application longer. In addition, since the vertex shader and fragment shader programs need to be provided in the form of source code, this can easily lead to the leakage of the shader source code and is not secure. Summary of the invention

[0005] The present application provides an application screen rendering method, device, equipment and readable storage medium, which can simplify the rendering process, improve the rendering efficiency of the application screen, and do not need to provide a shader program in source code form, thereby improving the security of the shader program.

[0006] In order to solve the above technical problems, this application provides the following technical solutions:

[0007] The present application embodiment provides an application screen rendering method, including:

[0008] Creating a rendering program container corresponding to a target mobile application, wherein the target mobile application is a mobile application in the simulator;

[0009] Acquire binary language type shader execution data pre-stored for the target mobile terminal application, and load the binary language type shader execution data into the rendering program container;

[0010] The binary language type shader operation data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a syntax variable mapping relationship between a mobile end shader program of the target mobile end application and the local end shader program, wherein the local end shader program is obtained by syntax translation based on the mobile end shader program;

[0011] Based on the binary program data and the binary mapping data in the rendering program container, a corresponding target shader program is executed to render the screen of the target mobile application in the simulator.

[0012] Accordingly, an embodiment of the present application provides an application screen rendering device, including:

[0013] A creation unit, used to create a rendering program container corresponding to a target mobile application, wherein the target mobile application is a mobile application in the simulator;

[0014] An acquisition unit, configured to acquire binary language type shader execution data pre-stored for the target mobile terminal application, and load the binary language type shader execution data into the rendering program container;

[0015] The binary language type shader operation data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a syntax variable mapping relationship between a mobile end shader program of the target mobile end application and the local end shader program, wherein the local end shader program is obtained by syntax translation based on the mobile end shader program;

[0016] An execution unit is used to execute a corresponding target shader program based on the binary program data and the binary mapping data in the rendering program container to render a screen of the target mobile application in the simulator.

[0017] In some embodiments, the acquisition unit is further used to:

[0018] Acquire the to-be-parsed operation data corresponding to the target mobile application from the storage space, wherein the to-be-parsed operation data at least includes a feature header, a data hash value, and candidate operation data;

[0019] When the feature header is consistent with the preset feature header for the operation data to be parsed, performing a hash check on the candidate operation data according to the data hash value;

[0020] When the hash check performed on the candidate execution data according to the data hash value passes, the candidate execution data is determined to be shader execution data of a binary language type.

[0021] In some embodiments, the acquisition unit is further used to:

[0022] determining binary configuration parameters, binary program data, and binary mapping data from the candidate operational data;

[0023] Acquire target configuration parameters of the local image processing unit, and verify the binary configuration parameters according to the target configuration parameters to obtain verification results;

[0024] When the verification result is that the binary configuration parameter is consistent with the target configuration parameter, the binary program data and the binary mapping data are determined as shader execution data of a binary language type.

[0025] In some implementations, the execution unit is further configured to:

[0026] Deserialize the binary mapping data to obtain a main syntax variable position table and a variable mapping table;

[0027] According to the binary program data, starting a corresponding target shader program;

[0028] The screen of the target mobile terminal application in the simulator is rendered by combining the started target shader program with the main syntax variable position table and the variable mapping table.

[0029] In some implementations, the execution unit is further configured to:

[0030] Determine, according to the variable inverse mapping table in the variable mapping table, a target mobile terminal variable mapped with a real-time target local terminal variable in the binary program data;

[0031] According to the target variable position information associated with the target mobile terminal variable in the main syntax variable position table, adjusting the shader variable information of the target shader;

[0032] The screen of the target mobile terminal application in the simulator is rendered by starting the target shader program in combination with the shader variable information of the target shader.

[0033] In some implementations, the execution unit is further configured to:

[0034] Sending the target mobile terminal variable to the simulator;

[0035] When it is detected that the simulator calls the variable location interface to send a location query request for the target mobile terminal variable, determining the target local terminal variable mapped by the target mobile terminal variable according to the variable positive mapping table in the variable mapping table;

[0036] Query the target variable position information corresponding to the target local terminal variable from the main syntax variable position table;

[0037] The shader variable information of the target shader is adjusted according to the target variable position information.

[0038] In some implementations, the application screen rendering device further includes a pre-storage unit, which is used to:

[0039] According to the target syntax rules, the mobile terminal shader program of the target mobile terminal application is translated into a local terminal shader program;

[0040] Generate binary mapping data according to a syntax variable mapping relationship between the local end shader program and the mobile end shader program;

[0041] Obtaining binary program data for a rendering pipeline formed by linking the local end shader program;

[0042] The binary mapping data and the binary program data are stored.

[0043] In some implementations, the pre-storage unit is further used for:

[0044] Determine a mobile terminal syntax version corresponding to the mobile terminal shader program of the target mobile terminal application, and determine a target syntax version that matches the mobile terminal syntax version according to a target syntax rule;

[0045] Determining the grammar extension attributes supported by the mobile terminal grammar version in the target grammar version;

[0046] Reading the target configuration parameters of the image processing unit of the local end, and filling the syntax constants corresponding to the syntax version of the mobile end according to the target configuration parameters, to obtain the target syntax information corresponding to the target syntax version;

[0047] According to the syntax extension attribute and the target syntax information, the mobile end shader program is translated into a local end shader program corresponding to the target syntax version.

[0048] In some implementations, the pre-storage unit is further used for:

[0049] Obtaining syntax variable information corresponding to the local end shader program;

[0050] Fill the preset variable position table with the position information corresponding to each target local end variable in the grammar variable information to obtain a main grammar variable position table;

[0051] Determine a syntax variable mapping relationship between the local end shader program and the mobile end shader program according to the syntax variable information, and generate a variable forward mapping table and a variable inverse mapping table according to the syntax variable mapping relationship;

[0052] Binary mapping data is generated according to the main syntax variable position table, the variable positive mapping table and the variable inverse mapping table.

[0053] In some implementations, the pre-storage unit is further used for:

[0054] Generate binary language type shader execution data according to the binary mapping data and the binary program data;

[0055] Binding the target configuration parameter, the binary language type, and the shader running data to obtain candidate running data;

[0056] Obtaining a data hash value, data length, and feature header corresponding to the candidate running data;

[0057] The characteristic header, the data length, the data hash value and the candidate operation data are combined in order to obtain target operation data, and the target operation data is stored.

[0058] In some implementations, the application screen rendering device further includes a notification unit, configured to:

[0059] Generate a notification message, and send the notification message to the simulator, wherein the notification message is used to instruct the shader running data supporting the binary language type to run the target mobile terminal application;

[0060] The creation unit is further configured to: when detecting that the simulator requests to run the target mobile application through the target shader program of the binary language type, create a rendering program container corresponding to the target mobile application.

[0061] In some implementations, the application screen rendering device further includes a downloading unit, which is used to:

[0062] Reading the target configuration parameters and operating system parameters of the image processing unit at the local end, and downloading the corresponding binary language type shader running data from the data sharing end according to the target configuration parameters and the operating system parameters;

[0063] The data sharing end at least stores binary language type shader operation data uploaded by other device ends for the target mobile end application;

[0064] The downloaded shader execution data of the binary language type is stored.

[0065] In addition, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned application screen rendering method when executing the computer program.

[0066] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the above-mentioned application screen rendering method.

[0067] In addition, an embodiment of the present application further provides a computer program product, the computer program product includes a computer program, and the computer program can be stored in a storage medium. The processor of the computer device reads the computer program from the storage medium and executes the computer program to implement the above-mentioned application screen rendering method.

[0068] An embodiment of the present application can be achieved by creating a rendering program container corresponding to a target mobile application, where the target mobile application is a mobile application in a simulator; obtaining binary language type shader execution data pre-stored for the target mobile application, and loading the binary language type shader execution data into the rendering program container; wherein the binary language type shader execution data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a grammatical variable mapping relationship between a mobile end shader program of the target mobile end application and a local end shader program, wherein the local end shader program is obtained through grammatical translation based on the mobile end shader program; based on the binary program data and the binary mapping data in the rendering program container, executing the corresponding target shader program to render the screen of the target mobile end application in the simulator.

[0069] Based on this, when the simulator triggers the start of the target mobile application, the present application can first create a rendering program container for loading shader program data on the local side, and then obtain the shader running data of the binary language type pre-stored for the target mobile application, and load the shader running data of the binary language type into the rendering program container. It should be noted that the shader running data of the binary language type refers to the binary program data obtained by linking and compiling the local shader program obtained by translating the mobile shader program of the target mobile application, and the binary mapping data of the grammatical variable mapping relationship of the mobile shader program of the target mobile application before and after the grammatical translation. Finally, the corresponding shader program is directly run on the personal computer based on the binary language type shader running data to realize the rendering of the screen of the target mobile application in the simulator.

[0070] Therefore, compared with the prior art solution that creates a rendering program through the cumbersome process of creating and binding vertex shaders and fragment shaders and linking them to form a rendering pipeline, the present application obtains pre-stored shader execution data of a binary language type and directly loads the shader execution data of the binary language type into the rendering program container. Since the binary language is a machine language, the compiled code can be directly executed without further interpretation or conversion. Therefore, the shader program corresponding to the shader execution data can be directly executed to render the screen of the target mobile application. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader execution data is stored in a binary language type, the leakage of the shader source code in the form of a high-level language is avoided, thereby improving the security of the shader program.

[0071] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0073] Figure 1 A schematic diagram of a scene of an application screen rendering system provided in an embodiment of the present application;

[0074] Figure 2A schematic diagram of the steps of the application screen rendering method provided in an embodiment of the present application;

[0075] Figure 3 A schematic diagram of another step flow of the application screen rendering method provided in an embodiment of the present application;

[0076] Figure 4 A flowchart of rendering a mobile application on a personal computer provided in an embodiment of the present application;

[0077] Figure 5 A flowchart of application screen rendering based on instruction stream provided in an embodiment of the present application;

[0078] Figure 6 A flowchart of the translation between the mobile end shader program and the local end shader program provided in the embodiment of the present application;

[0079] Figure 7 A flowchart of binarization of grammatical variable mapping relationships provided in an embodiment of the present application;

[0080] Figure 8 An example diagram of a synchronous response of a binary shader program acquisition interface between a mobile terminal and a local terminal provided in an embodiment of the present application;

[0081] Fig. 9 An interactive flow chart of the grammatical variable information of the instruction stream during the application screen rendering process provided by an embodiment of the present application;

[0082] Fig.10 A schematic diagram of the structure of an application screen rendering device provided in an embodiment of the present application;

[0083] Fig.11 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0084] Fig.12 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0086] It can be understood that in the specific implementation of the present application, related data such as shader operation data, mobile shader program, local shader program, binary program data, binary mapping data, etc. are involved. When the above embodiments of the present application are applied to specific products or technologies, it is necessary to obtain the object's permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards.

[0087] In addition, when the embodiments of the present application need to obtain shader running data, mobile shader programs, local shader programs, binary program data, binary mapping data and other related data, a separate license or separate consent for the shader running data, mobile shader programs, local shader programs, binary program data, binary mapping data and other related data will be obtained through pop-up windows or jumping to a confirmation page. After clearly obtaining a separate license or separate consent for the shader running data, mobile shader programs, local shader programs, binary program data, binary mapping data and other related data, the necessary shader running data, mobile shader programs, local shader programs, binary program data, binary mapping data and other related data for the normal operation of the embodiments of the present application will be obtained.

[0088] It should be noted that in some processes described in the specification, claims and the above-mentioned drawings, multiple steps appearing in a specific order are included, but it should be clearly understood that these steps may not be executed in the order in which they appear in this document or may be executed in parallel. The step numbers are only used to distinguish different steps, and the numbers themselves do not represent any execution order. In addition, descriptions such as "first", "second" or "target" in this document are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0089] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0090] The embodiments of the present application propose an application screen rendering method, apparatus, device and readable storage medium. When the simulator triggers the start of the target mobile application, the local end first creates a rendering program container for loading shader program data, then obtains the binary language type shader running data pre-stored for the target mobile application, and loads the binary language type shader running data into the rendering program container. It should be noted that the binary language type shader running data refers to the binary program data obtained after linking and compiling the local end shader program translated by the mobile end shader program of the target mobile application, and the binary mapping data of the syntax variable mapping relationship of the mobile end shader program of the target mobile application before and after translation. Finally, the corresponding shader program is directly run based on the binary language type shader running data to achieve rendering of the screen of the target mobile application in the simulator. Please continue to refer to the following specific embodiments for details.

[0091] The application screen rendering method of each embodiment of the present application can be executed by a computer device, and the computer device can be a server or a terminal.

[0092] The terminal can be a personal computer (such as a tablet computer, a laptop computer, a desktop computer, etc.), a smart home appliance, a vehicle terminal, an intelligent voice interaction device, an aircraft, etc.

[0093] A server can be a computer system that provides certain services to terminals. Compared with ordinary terminals, servers have higher requirements in terms of stability, security, performance, etc. For example, a server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0094] The application screen rendering method of the embodiment of the present application can be implemented by the terminal, and the server provides corresponding business data services for the terminal. On the other hand, the application screen rendering method can also be implemented by the server, and the terminal displays the application screen data obtained by processing the server. In addition, the application screen rendering method can also be implemented in part on the terminal and in part on the server, so as to be implemented jointly by the terminal and the server.

[0095] See also Figure 1 , is a scene diagram of an application screen rendering system provided in an embodiment of the present application, and the system includes a terminal 110 and a server 120.

[0096] The terminal 110 may have a target mobile terminal application installed thereon, and may run corresponding application services through the target mobile terminal application. The server 120 may be a service server corresponding to the target mobile terminal application.

[0097] Among them, taking the terminal 110 as a personal computer as an example, the personal computer creates a simulator through virtualization technology. When the target mobile application is started on the simulator, the local end on the personal computer executes the steps of the application screen rendering method. Specifically, a rendering program container corresponding to the target mobile application can be created, and the target mobile application is the mobile application in the simulator; the binary language type shader running data pre-stored for the target mobile application is obtained, and the binary language type shader running data is loaded into the rendering program container; wherein the binary language type shader running data includes binary program data obtained by linking and compiling the local end shader program, and binary mapping data representing the grammatical variable mapping relationship between the mobile end shader program of the target mobile end application and the local end shader program, and the local end shader program is obtained through grammatical translation based on the mobile end shader program; based on the binary program data and the binary mapping data in the rendering program container, the corresponding target shader program is executed to render the screen of the target mobile end application in the simulator.

[0098] Based on the above, the present application can obtain pre-stored shader running data of binary language type and directly load the shader running data of the binary language type into the rendering program container, so as to directly run the shader program to render the screen of the target mobile application. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader running data is stored in binary language type, the leakage of the shader source code is avoided and the security of the shader program is improved.

[0099] It should be noted that the above are only examples and can also be applied to other application screen rendering scenarios, which will not be described one by one here.

[0100] For ease of understanding, each step of the application screen rendering method will be described in detail below. It should be noted that the order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0101] See also Figure 2 , Figure 2 A schematic diagram of the steps of the application screen rendering method provided in the embodiment of the present application. In the embodiment of the present application, the application screen rendering method can be performed by Figure 1 The terminal 110 in the embodiment is executed. In order to better illustrate the embodiment of the present application, the terminal 110 is taken as a personal computer as an example for description. The specific process is as follows:

[0102] 101. Create a rendering program container corresponding to the target mobile application.

[0103] At present, many business service providers have developed mobile version applications for mobile devices (such as mobile phones, smart watches and other computer devices). For example, taking mobile phones as an example, mobile versions of game applications, short video applications, etc. have been developed for mobile devices. However, although users can use mobile version applications on their mobile phones at any time, which is convenient, the hardware characteristics of the mobile device itself may not be conducive to the user's operation of the mobile version application, and limit the rendering performance of the mobile version application. For example, the mobile device may limit the operation and rendering effect of the game application. Based on this, virtualization technology has been introduced, which can be used to run mobile applications on PC devices. In this way, when the user's device conditions permit, the mobile version application can be used through a personal computer.

[0104] For ease of understanding, virtualization technology is described below. Virtualization technology is mainly implemented in mobile simulators used in personal computer devices, which can be understood as a simulated environment implemented through virtualization technology. The core of the simulator is to create a virtual target device environment that is different from the main device in the main device, thereby helping developers test the compatibility of applications in the simulated target device environment and providing users with a way to use applications in the target device environment on the main device without the need for a target device. For example, it allows users to use mobile applications of the Android system on desktop operating systems such as Windows and macOS.

[0105] When running a mobile application on a personal computer through virtualization technology, a rendering program is created for the mobile version of the application, and vertex shaders and fragment shaders are created and bound in the rendering program. The vertex shaders and fragment shaders in the rendering program are linked to form a rendering pipeline, and the mobile version of the application is rendered according to the rendering pipeline in the rendering program, so that the mobile version of the application can be used on the personal computer. However, this requires a cumbersome process of creating and binding vertex shaders and fragment shaders and linking to form a rendering pipeline, which makes the startup time of the application longer, and because the vertex shader and fragment shader programs need to be provided in the form of source code, this can easily lead to the leakage of shader source code, reducing the security of the shader program.

[0106] Based on this, in order to simplify the rendering process, the embodiment of the present application obtains the pre-stored binary language type shader running data, and directly loads the binary language type shader running data into the rendering program container, so as to directly run the shader program to render the screen of the target mobile application, thereby improving the rendering efficiency of the application screen. In addition, in order to avoid the leakage of the shader source code, the shader running data is stored in the binary language type, thereby improving the security of the shader program.

[0107] The embodiment of the present application can abstract and simulate a simulator environment on a personal computer through virtualization technology. The simulator environment can be a virtual environment obtained by simulating a mobile terminal (mobile phone, smart watch, smart wearable device, etc.), which is used to implement the running of the mobile version of the application on the personal computer. It should be noted that the simulator environment can also be understood as a virtual machine. In addition to the simulator environment, the personal computer also includes another operating environment, that is, the local operating system of the personal computer. The simulator environment can be used as the mobile terminal, and the local terminal environment can be used as the local terminal.

[0108] The target mobile terminal application is an application developed for mobile terminals, such as game applications, short video applications, online shopping applications, live broadcast applications, etc. developed for mobile phone operating environments. Furthermore, the target mobile terminal application is a mobile terminal application in the simulator. Specifically, the mobile terminal version application can be run in the simulator environment, and the local terminal on the personal computer provides data support for the operation of the mobile terminal version application.

[0109] Among them, the rendering program container can be a container for loading data related to the shader program. In the computer field, the container can be an application container, which packages the software and all its dependencies (such as code, runtime environment, system tools, system libraries, etc.) into an independent unit. By loading the data related to the shader program in the container, the shader can be run based on the shader running data in the container to render the screen of the target mobile application. It should be understood that the running process of the target mobile application can be understood as the rendering process of the shader.

[0110] In an embodiment of the present application, after an operating environment for a mobile terminal is abstracted and simulated on a personal computer through virtualization technology, the personal computer can be regarded as including at least two operating environments, specifically including a simulator environment for the mobile terminal and a local operating environment. The simulator environment can be understood as a simulation environment of the mobile terminal, which is used to run the target mobile terminal application. When the personal computer detects that the target mobile terminal application is to be started in the simulator environment, it can create a rendering program container on the local side, so that the operating data related to the shader of the target mobile terminal application can be loaded into the rendering program container later, so as to perform screen rendering of the target mobile terminal application based on the operating data in the rendering program container.

[0111] It should be noted that when running the mobile version of the application through virtualization technology, a data interaction process is involved. Specifically, when rendering the screen of the target mobile application, it involves the interaction of instructions between the mobile terminal (i.e., the simulator environment) and the local terminal (i.e., using the local operating environment). The local terminal executes the relevant processes of application screen rendering based on the instructions of the mobile terminal. In order to avoid complex and cumbersome screen rendering processes, the local terminal can support loading binary shader program data in the rendering program container, thereby skipping the program loading, binding, linking and other processes.

[0112] In some embodiments, in order to avoid cumbersome and complicated rendering processes, a notification message indicating that the local end supports the shader running data of the binary language type can be notified to the simulator environment, so that the simulator environment requests the local end to directly run the corresponding target mobile application through the data of the binary language type shader program. For example, before step 101, it can also include: generating a notification message and sending the notification message to the simulator, the notification message is used to indicate that the target mobile application is run by the shader running data supporting the binary language type; then step 101 may include: when it is detected that the simulator requests to run the target mobile application through the target shader program of the binary language type, creating a rendering program container corresponding to the target mobile application.

[0113] The binary language type may be a binary machine language, which is represented by 0 and 1. Specifically, the machine language is a set of binary code instructions that can be directly recognized and executed by a computer. It is the lowest-level computer language, which uses binary numbers (0 and 1) to represent instructions and data. Each computer's central processing unit (CPU) has its own specific machine language instruction set, which directly controls the operation of computer hardware, such as data transmission, arithmetic and logical operations, control transfer, etc.

[0114] It should be noted that since the language type supported by the local operating environment of the personal computer may be different from the shading language type used by the mobile operating environment, the local end needs to translate, load, bind, link, compile and other cumbersome processes for the mobile end shader program. Therefore, in order to avoid cumbersome processes, the mobile end shader program of the target mobile end application can be converted into data of the language type directly supported by the local end in advance, for example, shader-related data of the binary language type, and the data of the binary language type is pre-stored for subsequent loading and use. Based on this, the local end can support directly loading the target shader program of the binary language type to run the target mobile end application and notify the simulator environment.

[0115] After that, when the user triggers the start of the target mobile application in the simulator environment, the simulator environment will call the local binary program interface to send a run request for the target mobile application, which can be understood as a load request for the "binary language type target shader program". At this time, the local end creates a corresponding rendering program container for the target mobile application in the simulator environment through the binary program interface, so that the data related to the binary language type shader corresponding to the target mobile application can be loaded into the rendering program container based on the simplified instruction stream to perform the screen rendering of the target mobile application. In this way, there is no need to go through the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, which simplifies the rendering process.

[0116] In the above manner, when the target mobile application is started in the simulator environment, a rendering program container can be created for the target mobile application, so that the binary language type shader running data can be directly loaded into the rendering program container to perform screen rendering of the target mobile application.

[0117] 102. Obtain binary language type shader execution data pre-stored for the target mobile application, and load the binary language type shader execution data into a rendering program container.

[0118] In an embodiment of the present application, after creating a corresponding rendering program container for a target mobile application, the binary language type shader running data pre-stored for the target mobile application can be obtained, and the binary language type shader running data can be loaded into the rendering program container. Specifically, in order to ensure that the correct, complete, and binary language type shader running data corresponding to the target mobile application is loaded into the rendering program container, the shader-related data needs to be verified, and after the verification is passed, it is loaded into the rendering program container to ensure the accuracy of the binary language type shader running data, so that the screen of the target mobile application can be rendered.

[0119] The shader operation data may be binary operation data related to the shader program of the target mobile application when it is running on the local side, describing how to render a "model" and representing a rendering logic. Specifically, since the target mobile application can only run in the corresponding operating environment, when it runs in the operating environment of a personal computer with the help of virtualization technology, it is necessary to translate the mobile shader program in the shading language of the mobile side into the local shader program in the high-level shading language of the graphics programming of the local side, and obtain the syntax variable difference between the two, as well as the binary language type program data of the local shader program when it is running, so as to generate the shader operation data.

[0120] Among them, the binary language type shader running data includes binary program data obtained by linking and compiling the local shader program, and binary mapping data representing the syntax variable mapping relationship between the mobile shader program of the target mobile application and the local shader program. The local shader program is obtained through syntax translation based on the mobile shader program.

[0121] It should be noted that for applications with real-time data streams (such as game applications, live broadcast applications, etc.), it can be understood as a real-time rendering process of the screen, specifically, the application screen is rendered through the shader program of the target mobile application. Based on this, the mobile shader program can be the shader program of the mobile version of the target mobile application, for example, the shader program of the mobile phone version; and the local shader program can be the shader program corresponding to the mobile shader program on the local side. The syntax of the mobile shader program is different from that of the local shader program, and the two need to be translated and converted.

[0122] Since the mobile shader program and the local shader program use different shading languages, which can be understood as different grammatical descriptions, it is necessary to obtain the grammatical difference between the mobile shader program and the local shader program. Specifically, the grammatical variable difference between the two can be obtained, and the grammatical variable difference can be understood as the different expressions of the same semantic variable between the mobile shader program and the local shader program. For the same semantic variable, the grammatical variable mapping relationship between the mobile shader program and the local shader program can be obtained, as well as the location information of each variable of the shader. The grammatical variable mapping relationship and the location information of each variable of the shader are described in binary form to obtain binary mapping data.

[0123] In addition, in order to obtain binary language type program data of a local shader program that can be directly run, it is necessary to link and compile the local shader program to obtain binary program data, skip the compilation process of the local shader program under the high-level language, and directly run the local shader with the final binary program data.

[0124] In some implementations, in order to avoid the tedious process of creating and binding vertex shaders and fragment shaders and linking to form a rendering pipeline, before rendering, the mobile end shader program can be translated into a local end shader program, and the local end shader program and the syntax variable mapping relationship before and after the translation can be pre-stored in binary form. For example, before step 102, it can also include:

[0125] (1) According to the target syntax rules, the mobile end shader program of the target mobile end application is translated into the local end shader program;

[0126] (2) Generate binary mapping data according to the syntax variable mapping relationship between the local end shader program and the mobile end shader program;

[0127] (3) Obtaining binary program data for a rendering pipeline formed by linking local shader programs;

[0128] (4) Store binary mapping data and binary program data.

[0129] Among them, the target syntax rule may include syntax version correspondence information between the mobile shader program and the local shader program, indicating the syntax version correspondence between different shading languages. It can be understood that different shading languages ​​are used between the mobile shader program and the local shader program, and different shading languages ​​correspond to different language versions during conversion.

[0130] Exemplarily, the mobile end shader program uses an embedded shading language (ESSL), and the local end shader program uses a high-level shading language for graphics programming (OpenGL Shading Language, GLSL). A shading language may have multiple versions. For example, the versions of the embedded shading language include ESSL100, ESSL300, ESSL310, etc., and the versions of the high-level shading language for graphics programming may include GLSL330, GLSL430, GLSL450, etc. Among them, ESSL100 corresponds to GLSL330, ESSL300 corresponds to GLSL430, and ESSL310 corresponds to GLSL450. Based on the version correspondence information between different shading languages, the target grammar rules are constructed.

[0131] The binary mapping data may include data describing the syntax variable mapping relationship between the mobile shader program and the local shader program, indicating the syntax variable difference between the mobile shader program and the local shader program, and further including binary data of a variable position table describing the location information of each syntax variable. Specifically, since the mobile shader program and the local shader program use different shading languages, for any semantic variable (uniform), the syntax variable mapping relationship between the mobile shader program and the local shader program can be obtained, and the syntax variable mapping relationship can be described in binary form.

[0132] The binary program data may be program data in binary form obtained after linking and compiling the local end shader program, which may be understood as basic shader data in binary form that can be directly read and run on the local end.

[0133] In order to avoid the tedious and complicated shader building process during the image rendering process, when the personal computer runs the target mobile application through virtualization technology for the first time or at any time, it can process and generate binary shader related data at the corresponding time based on the instructions of the simulator environment, and store the binary shader related data. Specifically, when it is detected that the simulator environment starts the target mobile application at the first or target time (such as the first startup time after the target mobile application is updated, or other scenarios where binary shader related data needs to be stored), the mobile shader program of the target mobile application is translated into a local shader program according to the target syntax rules to obtain a shader program in a high-level shading language for image programming; then, the syntax variable difference between the mobile shader program and the local shader program is determined, and binary mapping data is generated; further, the local shader program can include a vertex shader program and a fragment shader program, and the vertex shader program and the fragment shader program are linked to form a rendering pipeline, and the program corresponding to the rendering pipeline is compiled to obtain binary program data; finally, the binary program data and the binary mapping data are stored. In this way, the shader running data of the binary language type that can be directly run on the local side can be pre-stored, so that the shader running data of the binary language type can be directly loaded later, avoiding the tedious and complicated shader building process during the picture rendering process.

[0134] In some embodiments, the target syntax version, syntax extension attributes, and target syntax information supported by the mobile terminal shader program of the target mobile terminal application may be determined first, so as to translate the mobile terminal shader program into a local terminal shader program based on the target syntax version, syntax extension attributes, and target syntax information. For example, step (1) may include: determining the mobile terminal syntax version corresponding to the mobile terminal shader program of the target mobile terminal application, and determining the target syntax version that matches the mobile terminal syntax version according to the target syntax rule; determining the syntax extension attributes supported by the mobile terminal syntax version in the target syntax version; reading the target configuration parameters of the local image processing unit, and filling the syntax constants corresponding to the mobile terminal syntax version according to the target configuration parameters to obtain the target syntax information corresponding to the target syntax version; translating the mobile terminal shader program into a local terminal shader program corresponding to the target syntax version according to the syntax extension attributes and the target syntax information.

[0135] Among them, the mobile terminal syntax version can be a shading language version for mobile terminal applications, for example, it can include versions such as ESSL100, ESSL300, and ESSL310.

[0136] The target syntax version may be a version of a high-level shading language for image programming of a local shader program, for example, it may include versions such as GLSL330, GLSL430, and GLSL450.

[0137] Among them, the syntax extension attribute can be the syntax extension attribute function supported by the mobile syntax version after being converted into the target syntax version. Specifically, it can also be understood as the syntax extension attribute supported by the mobile shader program of the mobile syntax version after being converted into the local shader program of the target syntax version.

[0138] Among them, the image processing unit can be a graphics processing unit (GPU) hardware unit on a personal computer. It can be understood that the image processing unit has corresponding characteristic parameters, which can reflect hardware parameters and can reflect the main parameters of the GPU including computing power, video memory size, video memory bandwidth, power consumption control and inter-card interconnection rate. Based on these hardware parameters, characteristic parameters can be determined, such as how many frames are buffered per second, the rendering rate of the shader, and the model corresponding to the image processing unit. Based on the above information, target configuration parameters are generated for the filling process of syntax constants or the subsequent verification process.

[0139] Among them, the target syntax information can be syntax information corresponding to the target syntax version (GLSL), which reflects the ability limitations or characteristics of the graphics card of the personal computer in processing uniform buffers and shader storage buffers. In the GLSL syntax information, these graphics card characteristics may be reflected in some way, such as limiting the scope of syntax usage for certain buffer-related operations or providing syntax structures optimized for specific graphics cards. For example, if the maximum number of uniform buffer bindings supported by the graphics card is limited, the GLSL syntax information may include restriction prompts or optimization suggestions for the number of uniform buffers used to ensure that the shader program running on the graphics card can correctly and efficiently utilize graphics card resources, avoiding performance problems or errors caused by unreasonable buffer usage.

[0140] Specifically, the process of translating the mobile end shader program into the local end shader program is as follows: first, the mobile end syntax version of the mobile end shader program of the target mobile end application is detected, and the target syntax version matching the mobile end syntax version is determined according to the syntax version correspondence of different shading languages ​​in the target syntax rule. Exemplarily, taking the game application as an example, the game platform has launched some cool skins, special effects, etc., which require new versions to use and display. Therefore, before translating the mobile end shader program into the local end shader program, it is necessary to determine the syntax version of the mobile end shader program. It should be noted that during the rendering of the game screen, assuming that there are two "skins" or "special effects" with different syntax versions, different syntax versions of "skins" may also appear in the rendering process. At this time, through the binary language type, only these special effects and skins need to be translated once, and the variable information of the shader program (such as variables and variable positions) is serialized and saved. Subsequently, the model rendering of different versions of "skins" or "special effects" of the same application screen can be directly executed based on the variable information.

[0141] Then, determine the syntax extension attributes supported by the mobile-side syntax version in the target syntax version; then, fill in the syntax constants corresponding to the mobile-side syntax version (ESSL syntax version) according to the target configuration parameters of the image processing unit to obtain the syntax information of the syntax version of the graphics programming high-level shading language. The target syntax information can limit the syntax usage range of buffer-related operations during the rendering process, such as how much buffer is rendered per unit time; finally, translate the mobile-side shader program into the local-side shader program corresponding to the target syntax version according to the syntax extension attributes and the target syntax information. At this point, the mobile-side shader program can be translated into a local-side shader program of the syntax version of the graphics programming high-level shading language that can be supported by the local side of the personal computer, so that the binary program data corresponding to the local-side shader program can be stored later, thereby directly loading the data to run the corresponding target shader program.

[0142] In some embodiments, the syntax variable mapping information between the mobile end shader program and the local end shader program may include syntax variable information and location information of each target local end variable, and binary mapping data is generated according to the syntax variable information and the location information of each target local end variable. For example, step (2) may include: obtaining syntax variable information corresponding to the local end shader program; filling the location information corresponding to each target local end variable in the syntax variable information in the preset variable location table to obtain a main syntax variable location table; determining the syntax variable mapping relationship between the local end shader program and the mobile end shader program according to the syntax variable information, and generating a variable positive mapping table and a variable inverse mapping table according to the syntax variable mapping relationship; generating binary mapping data according to the main syntax variable location table, the variable positive mapping table, and the variable inverse mapping table.

[0143] Among them, the syntax variable information can be the variable information of the local shader program translated into the graphics programming high-level shading language, which includes one or more local variables, which include but are not limited to any defined variables (uniform), variables (varying) used to transfer data between vertex shaders and fragment shaders, vertex attribute variables (attribute), interface blocks (Interface block), etc.

[0144] It should be noted that the variable (uniform) is a modification of the syntax variable, which is used to define the access scope of the shader variable. It does not define the type of the structure of the variable itself (such as int, float, vector, etc.). Vertex shaders and fragment shaders are designed on different hardware units, and data access between different shader programs can be achieved through variables. It is worth noting that the variable value will not change before the entire shader is executed. It can be combined with any basic type variable. Uniform variables can also be used to place environmental data passed by external programs (such as point light position, model transformation matrix, etc.).

[0145] Vertex attribute variables (attribute) are used in vertex shaders and can only be combined with floating-point, vector or matrix variables. Vertex attribute variables can be used to place model vertices, normals, colors, textures and other data passed by the program, and they can access data buffers.

[0146] Variables (varying) are messengers between the vertex shader and the fragment shader. Generally, the variable is modified in the vertex shader and then used in the fragment shader.

[0147] The main syntax variable position table is used to record the position information of each target local variable, and the corresponding data parameters are operated through the position information, such as modifying the data parameters corresponding to the position information to modify the shader data.

[0148] The syntax variable mapping relationship includes a forward mapping relationship between the mobile syntax variables of the mobile shader program and the local syntax variables of the local shader program, and a reverse mapping relationship between the local syntax variables of the local shader program and the mobile syntax variables of the mobile shader program. Therefore, the variable forward mapping table includes a forward mapping relationship between the mobile syntax variables and the local syntax variables, and the variable reverse mapping table includes a reverse mapping relationship between the local syntax variables and the mobile syntax variables.

[0149] Specifically, the formation process of the binary mapping data is as follows: first, the corresponding syntax variable information is obtained from the local end shader program, and the syntax variable information includes one or more target local end variables, and the position information corresponding to each target local end variable is obtained. Then, in a recursive manner, the position information corresponding to each target local end variable is filled in the preset variable position table to obtain the main syntax variable position table. It should be noted that the recursive method can traverse the data structure related to the local end syntax variable (uniform). For example, there are multiple local end syntax variables (uniform) in the shader program, and there may be nested or hierarchical structures (such as uniform variables in the structure). The recursive algorithm can start from the outermost layer and go deep into each internal layer in turn to accurately determine the location of each uniform variable in the memory. Therefore, through recursive traversal, it is ensured that no local end syntax variable is missed, and the complex data structure relationship can be correctly handled, providing a basis for quickly and accurately accessing these variables in the subsequent graphics rendering process.

[0150] Further, the local syntax variable corresponding to the local shader program is determined according to the syntax variable information, and the mobile syntax variable mapped to the local syntax variable in the mobile shader program is determined, and the variable reverse mapping table is generated according to the syntax variable reverse mapping relationship between the local syntax variable and the mobile syntax variable, and the variable forward mapping table is generated according to the syntax variable forward mapping relationship between the mobile syntax variable and the local syntax variable. Finally, according to the binary serialization method, the three tables of the main syntax variable position table, the variable forward mapping table and the variable reverse mapping table are saved to obtain binary mapping data. Among them, the binary serialization can be understood as the process of converting complex data structures (such as objects, structures, etc.) into byte streams. For example, in a vertex data (vertex position, color, texture coordinates, etc.), through binary serialization, this vertex data structure can be converted into a byte sequence, so that the binary mapping data of the binary serialization can be stored later, reducing the occupancy rate of storage space (such as cache, memory, etc.).

[0151] In some embodiments, in order to subsequently verify the integrity of the binary program data and binary mapping data associated with the local shader program, the verification indicator information can be bound to the binary program data and the binary mapping data, so that the binary program data and the binary mapping data can be verified according to the verification indicator information. For example, step (4) may include: generating shader operation data of binary language type according to the binary mapping data and the binary program data; binding the target configuration parameters, the binary language type and the shader operation data to obtain candidate operation data; obtaining the data hash value, data length and feature header corresponding to the candidate operation data; combining the feature header, data length, data hash value and candidate operation data in order to obtain the target operation data, and storing the target operation data.

[0152] In order to ensure that the complete binary language type shader running data can be obtained later, it is necessary to set verification index information for the shader running data of the binary language type. The verification index information includes but is not limited to the target configuration parameters of the image processing unit, the binary version corresponding to the binary language type, data hash, data length and feature header, etc. By binding these verification index information with the shader running data, the binary language type shader running data can be verified according to these verification index information to ensure data integrity.

[0153] Specifically, the storage process of binary mapping data and binary program data is as follows: first, the binary mapping data and binary program data are combined, such as by superposition, splicing, binding, etc., to obtain shader operation data of binary language type. Then, the target configuration parameters corresponding to the image processing unit of the personal computer are obtained, such as obtaining the model, number, configuration, etc. of the graphics card, and obtaining the binary version number corresponding to the binary language type, and these information are bound to the shader operation data to obtain candidate operation data. In this way, when the candidate operation data is obtained later, the shader operation data can be verified through the target configuration information to determine whether the shader operation data supports the image processing unit.

[0154] Furthermore, the corresponding data hash is calculated for the candidate operation data, for example, the data hash value corresponding to the candidate operation data is calculated by using the message digest algorithm (Message-Digest Algorithm, MD5) or the 256-bit hash algorithm (Secure Hash Algorithm 256, SHA256); and the corresponding data length is set, which is used to describe how many bits (bits) after which the candidate operation data appears, and can also be understood as the arrangement position of the candidate operation data in the overall data; in addition, a feature header is set for the candidate operation data, and the feature header can be a value indicating the data type contained in the candidate operation data, or a predefined code, flag, etc., used to identify the candidate operation data. Finally, according to the feature header, data length, data hash value and the specific sequential format of the candidate operation data, these data are combined to obtain the target operation data, and the target operation data is stored.

[0155] Based on this, the mobile end shader program corresponding to the target mobile end application of the simulator environment can be translated into a local end shader program, and the syntax variable mapping relationship between the mobile end syntax variables of the mobile end shader program and the local end syntax variables of the local end shader program, as well as the main syntax variable position table containing the location information of each local end syntax variable, can be determined, so as to generate binary mapping data according to the syntax variable mapping relationship and the main syntax variable position table, and obtain the binary program data obtained after the local end shader program is linked and compiled, so that the shader operation data of the binary language type can be obtained, and further, the verification index information is bound to the shader operation data to obtain the target operation data. In this way, the pre-built target operation data can be stored, and the target operation data is used to describe the shader state and / or variable information of the local shader program at runtime, etc., indicating the binary type data required by the local shader at runtime, and the target operation data can be directly loaded and used after being parsed and verified later, avoiding the cumbersome and complicated processes such as translation between the mobile end shader program and the local end shader program, loading, binding, linking, and compiling of the local shader program, and improving the subsequent application screen rendering efficiency.

[0156] In the embodiment of the present application, in order to avoid complex and cumbersome processes in the application screen rendering process, the binary language type shader operation data can also be obtained by sharing, that is, the binary language type shader operation data associated with the target mobile terminal application can be shared between different personal computers. Specifically, after the personal computer generates the binary language type shader operation data, it can upload the binary language type shader operation data to the data sharing end for any computer device or computer device with sharing authority to obtain and use.

[0157] In some embodiments, after generating binary language type shader operation data according to binary mapping data and binary program data, the binary language type shader operation data can be associated with the target configuration parameters and operating system parameters of the image processing unit on the personal computer, and after the association is completed, the associated binary language type shader operation data is uploaded to the data sharing end. Alternatively, after obtaining the target operation data by combining in the order of feature header, data length, data hash value and candidate operation data, the target operation data can be associated with the target configuration parameters and operating system parameters of the image processing unit on the personal computer, and after the association is completed, the associated target operation data is uploaded to the data sharing end.

[0158] In some embodiments, in order to avoid the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, before rendering, binary shader-related data can be downloaded from the data sharing end to pre-store the shader-related data. For example, before step 102, it can also include: reading the target configuration parameters and operating system parameters of the image processing unit on the local end, and downloading the corresponding binary language type shader running data from the data sharing end according to the target configuration parameters and operating system parameters; wherein the data sharing end at least stores the binary language type shader running data uploaded by other device ends for the target mobile end application; and storing the downloaded binary language type shader running data.

[0159] Among them, the target configuration parameter refers to the configuration parameter of the graphics card of this personal computer, and the operating system parameter refers to the version number of the operating environment (such as win7, win10, etc.). In order to obtain the shader operation data of the binary language type that conforms to the local personal computer from the data sharing end, the target configuration parameters and operating system parameters of the image processing unit of this personal computer can be read first, and then, from the shader operation data of the binary language type shared by other device ends stored in the data sharing end, the shader operation data of the binary language type is searched according to the target configuration parameters and operating system parameters, and the target shader operation data found is downloaded, and finally, the downloaded target shader operation data is stored. It should be noted that when storing the target shader operation data, the target shader operation data can be bound to the verification index information as described above, and stored after the binding is completed.

[0160] In addition, after obtaining the target shader running data of the binary language type, the target shader running data of the binary language type can be directly loaded into the created rendering program container for direct use. It should be noted that when the data sharing end stores the target running data uploaded by other device ends, similarly, the target running data can be directly stored or parsed for use. In this way, through the sharing method of binary language type shader running data, the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline is avoided, thereby improving the subsequent application screen rendering efficiency.

[0161] In an embodiment of the present application, after creating a corresponding rendering program container for the target mobile application, the target running data pre-stored for the target mobile application can be obtained, and the target running data can be parsed and the data integrity can be checked. After the check is passed, the binary language type shader running data can be loaded into the rendering program container to ensure the accuracy of the binary language type shader running data, so that the screen of the target mobile application can be rendered.

[0162] In some implementations, the operation data to be parsed corresponding to the target mobile terminal application may be obtained, and after parsing, the operation data may be verified according to the feature header and the hash method, so as to determine the candidate operation data that has passed the verification as the shader operation data of the binary language type. For example, the "obtaining the shader operation data of the binary language type pre-stored for the target mobile terminal application" in step 102 may include:

[0163] (102.1) Obtaining the to-be-parsed operation data corresponding to the target mobile application from the storage space, where the to-be-parsed operation data at least includes a feature header, a data hash value, and candidate operation data;

[0164] (102.2) When the feature header is consistent with the preset feature header for the running data to be parsed, performing a hash check on the candidate running data according to the data hash value;

[0165] (102.3) When the hash check performed on the candidate execution data according to the data hash value passes, the candidate execution data is determined to be shader execution data of a binary language type.

[0166] Specifically, the operation data to be parsed corresponding to the target mobile application is obtained from the storage space on the local personal computer, and the operation data to be parsed is parsed to obtain a feature header, a data hash value, and candidate operation data. Then, a preset feature header pre-set for the operation data to be parsed is determined, and the preset feature header is compared with the feature header obtained by parsing. If the preset feature header is consistent with the feature header obtained by parsing, a hash check is further performed on the candidate operation data according to the data hash value. Specifically, the hash value to be checked corresponding to the candidate operation data is calculated, and the hash value to be checked is compared with the hash value of the data obtained by parsing. If the hash value to be checked is consistent with the hash value of the data obtained by parsing, it means that the hash check of the candidate operation data according to the data hash value passes. Finally, when the hash check of the candidate operation data according to the data hash value passes, the candidate operation data is determined as shader operation data of a binary language type.

[0167] It should be noted that if the preset feature header is inconsistent with the feature header obtained by parsing, and / or if the hash check of the candidate running data based on the data hash value fails, it means that the data is incomplete, and the running data that fails the check can be refused to be loaded into the rendering program container. Furthermore, the mobile shader program of the target mobile application can be directly translated into a local shader program, and the syntax variable mapping information between the mobile shader program and the local shader program, and the location information of the variables are obtained to generate binary mapping data. In addition, the local shader program is linked and compiled to obtain binary program data, and the binary mapping data and the binary program data are loaded into the rendering program container.

[0168] In some embodiments, the candidate operation data is composed of binary configuration parameters, binary program data, and binary mapping data. According to the binary configuration parameters, the matching between the binary program data and the binary mapping data and the image processing unit of the personal computer is verified. If they match, the binary program data and the binary mapping data are determined as binary language type shader operation data. For example, the "determining the candidate operation data as binary language type shader operation data" in step (102.3) may include: determining the binary configuration parameters, the binary program data, and the binary mapping data from the candidate operation data; obtaining the target configuration parameters of the image processing unit at the local end, and verifying the binary configuration parameters according to the target configuration parameters to obtain a verification result; when the verification result is that the binary configuration parameters are consistent with the target configuration parameters, the binary program data and the binary mapping data are determined as binary language type shader operation data.

[0169] Specifically, after obtaining the candidate operation data, the candidate operation data can be parsed to obtain two mechanism configuration parameters, binary program data and binary mapping data. The two mechanism configuration parameters can be understood as the configuration parameters of the image processing unit supported by the current binary program data and binary mapping data. According to the binary configuration parameters, it is determined whether the binary program data and binary mapping data conform to the image processing unit of the personal computer. Therefore, the target configuration parameters of the image processing unit of the personal computer are obtained, and the target configuration parameters are compared with the binary configuration parameters. If it is verified that the binary configuration parameters are consistent with the target configuration parameters, for example, the graphics card model represented by the binary configuration parameters is consistent with the graphics card model of the target configuration parameters, the binary program data and the binary mapping data are determined as binary language type shader operation data. It should be noted that the rendering process is mainly implemented based on the graphics card of the PC end (local), so the parameters related to the shader (such as the graphics card configuration parameters recorded by the binary configuration parameters) also need to be consistent with the actual configuration parameters of the graphics card to ensure that the graphics card can support the binary program data and binary mapping data.

[0170] Through the above method, the target running data pre-stored for the target mobile application can be obtained, and the target running data can be parsed and the data integrity can be checked. After the verification is passed, the binary language type shader running data can be loaded into the rendering program container to ensure the accuracy and completeness of the binary language type shader running data so that the screen of the target mobile application can be rendered.

[0171] 103. Based on the binary program data and the binary mapping data in the rendering program container, execute the corresponding target shader program to render the screen of the target mobile application in the simulator.

[0172] In an embodiment of the present application, loading binary language type shader running data into a rendering program container can be understood as loading binary program data and binary mapping data into a rendering program container, where the binary program data and binary mapping data are binary data that can be directly recognized and used by a personal computer, and represent data in binary form that supports the running of a local shader, or can be understood as state data of a local shader program at runtime; based on the binary program data and binary mapping data in the rendering program container, the local shader program is executed to render the screen of a target mobile application in the simulator, thereby eliminating the need to go through the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, and running the shader program is implemented based on the directly loaded binary program data and binary mapping data, thereby improving the efficiency of application screen rendering.

[0173] The target shader program may be a local shader program, which may include a local vertex shader program and a fragment shader program. Based on the directly loaded binary program data and binary mapping data, the local vertex shader program and the fragment shader program are successively run to render the screen of the target mobile application in the simulator.

[0174] In some implementations, the binary mapping data may be deserialized to decompose the main syntax variable position table and the variable mapping table, and the target shader program may be executed according to the binary program data, the main syntax variable position table and the variable mapping table to render the screen of the target mobile terminal application in the simulator. For example, step 103 may include:

[0175] (103.1) Deserialize the binary mapping data to obtain a main syntax variable position table and a variable mapping table;

[0176] (103.2) According to the binary program data, start the corresponding target shader program;

[0177] (103.3) The screen of the target mobile application in the simulator is rendered by starting the target shader program in combination with the main syntax variable location table and the variable mapping table.

[0178] Specifically, since the main syntax variable position table, the variable positive mapping table and the variable inverse mapping table are stored in accordance with binary serialization, after obtaining the binary mapping data, the binary mapping data can be deserialized to obtain the main syntax variable position table and the variable mapping table, and the variable mapping table includes two types: the variable positive mapping table and the variable inverse mapping table. Then, according to the binary program data, the corresponding target shader program is started. For example, the vertex shader and fragment shader of the local end can be started successively or simultaneously. Finally, the screen of the target mobile application in the simulator is rendered based on the main syntax variable position table, the variable positive mapping table and the variable inverse mapping table through the started target shader (program). In this way, the screen of the target mobile application in the simulator is rendered directly based on the binary mapping data and the binary program data in binary form, without going through cumbersome processes, thereby improving the efficiency of screen rendering.

[0179] In some embodiments, during the rendering process, the instruction stream interaction between the simulator environment and the local end is involved, and the local end transmits the variable information of the shader based on the instruction stream interaction, so that the shader renders the screen of the target mobile terminal application in the simulator according to the relevant rendering parameters. For example, step (103.3) may include:

[0180] (103.3.1) determining the target mobile terminal variable for real-time target local terminal variable mapping in the binary program data according to the variable inverse mapping table in the variable mapping table;

[0181] (103.3.2) adjusting the shader variable information of the target shader according to the target variable position information associated with the target mobile terminal variable in the main syntax variable position table;

[0182] (103.3.3) The screen of the target mobile application in the simulator is rendered by starting the target shader program and combining the shader variable information of the target shader.

[0183] Specifically, when it is detected that the simulator calls the binary shader program interface of the local end to send a variable acquisition request, the personal computer determines at least one target local end variable currently required to be used according to the binary program data, and queries the target mobile end variable mapped with the target local end variable from the variable inverse mapping table, so as to send each target mobile end variable queried to the simulator. Furthermore, it is also necessary to obtain the target variable position information associated with each target mobile end variable from the main syntax variable position table, and adjust the shader variable information of the target shader according to these target variable position information. Finally, the state of the target shader (such as structure, morphology, etc., not limited here) is determined by combining the shader variable information with the started target shader program, and combined with the rendering parameters (such as illumination, vertex position, vertex texture coordinates and other vertex data, and material attributes), the screen of the target mobile end application in the simulator is rendered. It should be noted that the target mobile end variable, the target variable position information and the shader variable information can be understood as shader-related data, which is not limited here.

[0184] In some embodiments, since the local end transmits the variable information of the shader based on the interaction of the instruction stream, the target mobile end variable can be sent to the simulator environment, and when the simulator requests to obtain the target variable location information associated with the target mobile end variable, the target variable location information is sent to the simulator to adjust the variable information of the shader. For example, step (103.3.2) may include: sending the target mobile end variable to the simulator; when it is detected that the simulator calls the variable location interface to send a location query request for the target mobile end variable, determining the target local end variable mapped by the target mobile end variable according to the variable positive mapping table in the variable mapping table; querying the target variable location information corresponding to the target local end variable from the main syntax variable location table; adjusting the shader variable information of the target shader according to the target variable location information.

[0185] Among them, the target shader can include a vertex shader and a fragment shader. It should be noted that the vertex shader is responsible for determining the shape and position of the graphics, while the fragment shader is responsible for assigning color and other visual attributes to each pixel on the surface of the graphics. Only when the two work together can complete and realistic graphics rendering be achieved. For example, in a 3D game scene, the vertex shader converts the vertices of the 3D model to the correct position, and the fragment shader calculates the appropriate color for each pixel on the surface of the model based on the scene's lighting, material and other factors, thereby presenting a three-dimensional and realistic game screen.

[0186] Specifically, after obtaining the target mobile terminal variable, the target mobile terminal variable can be sent to the simulator environment; then, if it is detected that the simulator calls the variable location interface to send a location query request for the target mobile terminal variable, the target local terminal variable mapped by the target mobile terminal variable is determined according to the variable positive mapping table in the variable mapping table; then, the target variable location information corresponding to the target local terminal variable is queried from the main syntax variable location table; finally, the target variable location information is sent to the simulator environment to adjust the shader variable information of the target shader according to the target variable location information in the simulator environment. At this point, for a certain variable, the location information (location) corresponding to the variable is determined, and by changing the data parameters of the corresponding position of the target shader, the variable data of the target shader is modified. After that, the target shader will output the result according to different input variables (such as lighting, position and other rendering parameters) to achieve rendering of the screen of the target mobile terminal application.

[0187] Through the above method, the shader program can be run based on the directly loaded binary program data and binary mapping data to execute the local shader program to render the screen of the target mobile application in the simulator. In this way, there is no need to go through the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, thereby improving the efficiency of application screen rendering.

[0188] From the above overall description of the embodiment of the present application, it can be seen that the embodiment of the present application can be achieved by creating a rendering program container corresponding to the target mobile application, where the target mobile application is a mobile application in the simulator; obtaining binary language type shader execution data pre-stored for the target mobile application, and loading the binary language type shader execution data into the rendering program container; wherein the binary language type shader execution data includes binary program data obtained by linking and compiling a local shader program, and binary mapping data representing a grammatical variable mapping relationship between the mobile shader program of the target mobile application and the local shader program, and the local shader program is obtained through grammatical translation based on the mobile shader program; based on the binary program data and the binary mapping data in the rendering program container, executing the corresponding target shader program to render the screen of the target mobile application in the simulator.

[0189] Based on this, when the simulator triggers the start of the target mobile application, the present application can first create a rendering program container for loading shader program data on the local side, and then obtain the shader running data of the binary language type pre-stored for the target mobile application, and load the shader running data of the binary language type into the rendering program container. It should be noted that the shader running data of the binary language type refers to the binary program data obtained by linking and compiling the local shader program obtained by translating the mobile shader program of the target mobile application, and the binary mapping data of the grammatical variable mapping relationship of the mobile shader program of the target mobile application before and after the grammatical translation. Finally, the corresponding shader program is directly run on the personal computer based on the binary language type shader running data to realize the rendering of the screen of the target mobile application in the simulator. Therefore, compared with the prior art solution that creates a rendering program through the cumbersome process of creating and binding vertex shaders and fragment shaders and linking them to form a rendering pipeline, the present application obtains pre-stored shader execution data of a binary language type and directly loads the shader execution data of the binary language type into the rendering program container. Since the binary language is a machine language, the compiled code can be directly executed without further interpretation or conversion. Therefore, the shader program corresponding to the shader execution data can be directly executed to render the screen of the target mobile application. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader execution data is stored in a binary language type, the leakage of the shader source code in the form of a high-level language is avoided, thereby improving the security of the shader program.

[0190] In combination with the method described in the above embodiment, the method will be further described in detail with examples below.

[0191] Figure 3 This is another step flow diagram of the application screen rendering method provided by the embodiment of the present application. Figure 3 Give a description.

[0192] In the embodiment of the present application, the application screen rendering method can be performed by Figure 1 The terminal 110 in the application is executed. In order to better illustrate the embodiment of the present application, the terminal 110 is taken as an example of a personal computer. The specific process of the application screen rendering method is as follows:

[0193] 201. According to the target syntax rules, translate the mobile end shader program of the target mobile end application into a local end shader program.

[0194] In the embodiment of the present application, in order to simplify the rendering process, by obtaining the pre-stored binary language type shader running data, and directly loading the binary language type shader running data into the rendering program container, the shader program can be directly run to render the screen of the target mobile application, so that the rendering efficiency of the application screen can be improved. In addition, in order to avoid the leakage of the shader source code, the shader running data is stored in the binary language type, so that the security of the shader program can be improved.

[0195] In order to obtain relevant data of a directly executable local shader program, the local shader program needs to be linked and compiled to obtain program data in binary form, and the binary program data is used to describe the state data of the local shader program at runtime.

[0196] Specifically, the mobile terminal syntax version of the mobile terminal shader program of the target mobile terminal application is detected, and the target syntax version matching the mobile terminal syntax version is determined according to the corresponding relationship between the syntax versions of different shading languages ​​in the target syntax rules; then, the syntax extension attributes supported by the mobile terminal syntax version in the target syntax version are determined; further, according to the target configuration parameters of the image processing unit, the syntax constants corresponding to the mobile terminal syntax version (ESSL syntax version) are filled to obtain the syntax information of the syntax version of the graphics programming high-level shading language, and the target syntax information can limit the syntax usage range of buffer-related operations during the rendering process, such as how much buffer is rendered per unit time; finally, according to the syntax extension attributes and the target syntax information, the mobile terminal shader program is translated into a local terminal shader program corresponding to the target syntax version. At this point, the mobile terminal shader program can be translated into a local terminal shader program of the syntax version of the graphics programming high-level shading language that can be supported by the local terminal of the personal computer, so that the binary program data corresponding to the local terminal shader program can be stored later, so that the data can be directly loaded later to run the corresponding target shader program.

[0197] 202. Generate binary mapping data according to the syntax variable mapping relationship between the local end shader program and the mobile end shader program.

[0198] In an embodiment of the present application, since different shading languages ​​are used between the mobile-end shader program and the local-end shader program, which can be understood as different grammatical descriptions, it is necessary to obtain the grammatical difference between the mobile-end shader program and the local-end shader program. Specifically, the grammatical variable difference between the two can be obtained, and the grammatical variable difference can be understood as the different expressions of the same semantic variable between the mobile-end shader program and the local-end shader program. For the same semantic variable, the grammatical variable mapping relationship between the mobile-end shader program and the local-end shader program can be obtained, as well as the location information of each variable of the shader. The grammatical variable mapping relationship and the location information of each variable of the shader are described in binary form to obtain binary mapping data.

[0199] Specifically, first, the corresponding syntax variable information is obtained from the local end shader program, and the syntax variable information includes one or more target local end variables, and the position information corresponding to each target local end variable is obtained. Then, in a recursive manner, the position information corresponding to each target local end variable is filled in the preset variable position table to obtain the main syntax variable position table. Further, the local end syntax variable corresponding to the local end shader program is determined according to the syntax variable information, and the mobile end syntax variable mapped to the local end syntax variable in the mobile end shader program is determined, and the variable reverse mapping table is generated according to the syntax variable reverse mapping relationship between the local end syntax variable and the mobile end syntax variable, and the variable forward mapping table is generated according to the syntax variable forward mapping relationship between the mobile end syntax variable and the local end syntax variable. Finally, in accordance with the binary serialization method, the three tables of the main syntax variable position table, the variable forward mapping table and the variable reverse mapping table are saved to obtain binary mapping data.

[0200] 203. Obtain binary program data for a rendering pipeline formed by linking local shader programs.

[0201] In an embodiment of the present application, the local shader program includes a vertex shader program and a fragment shader program. The vertex shader program and the fragment shader program need to be linked to form a rendering pipeline, and the program corresponding to the rendering pipeline is compiled to obtain binary program data.

[0202] 204. Store the binary mapping data and the binary program data.

[0203] In an embodiment of the present application, in order to subsequently verify the integrity of the binary program data and binary mapping data associated with the local shader program, the verification indicator information can be bound to the binary program data and the binary mapping data, so that the binary program data and the binary mapping data can be subsequently verified according to the verification indicator information.

[0204] Specifically, first, the binary mapping data and the binary program data are combined, such as by superposition, splicing, binding, etc., to obtain the shader operation data of the binary language type. Then, the target configuration parameters corresponding to the image processing unit of the personal computer are obtained, such as obtaining the model, number, configuration, etc. of the graphics card, and obtaining the binary version number corresponding to the binary language type, and these information are bound to the shader operation data to obtain the candidate operation data. In this way, when the candidate operation data is obtained later, the shader operation data can be verified by the target configuration information to determine whether the shader operation data supports the image processing unit. Furthermore, the corresponding data hash is calculated for the candidate operation data, for example, the data hash value corresponding to the candidate operation data is calculated by using the message digest algorithm (Message-Digest Algorithm, MD5) or the 256-bit hash algorithm (Secure Hash Algorithm 256, SHA256); and the corresponding data length is set, which is used to describe how many bits (bits) after which the candidate operation data appears, and can also be understood as the arrangement position of the candidate operation data in the overall data; in addition, a feature header is set for the candidate operation data, and the feature header can be a value indicating the data type contained in the candidate operation data, or a predefined code, flag, etc., used to identify the candidate operation data. Finally, according to the feature header, data length, data hash value and the specific sequential format of the candidate operation data, these data are combined to obtain the target operation data, and the target operation data is stored.

[0205] 205. Create a rendering program container corresponding to the target mobile application.

[0206] In the embodiment of the present application, after an operating environment for a mobile terminal is abstracted and simulated on a personal computer through virtualization technology, the personal computer can be regarded as including at least two operating environments, specifically including a simulator environment for the mobile terminal and a local operating environment. The simulator environment can be understood as a simulation environment of the mobile terminal, which is used to run the target mobile terminal application. When the personal computer detects that the target mobile terminal application is to be started in the simulator environment, it can create a rendering program container on the local side so that the shader-related data of the target mobile terminal application can be subsequently loaded into the rendering program container.

[0207] 206 . Obtain binary language type shader execution data pre-stored for the target mobile application, and load the binary language type shader execution data into the rendering program container.

[0208] In an embodiment of the present application, after creating a corresponding rendering program container for a target mobile application, the binary language type shader running data pre-stored for the target mobile application can be obtained, and the binary language type shader running data can be loaded into the rendering program container. Specifically, in order to ensure that the correct, complete, and binary language type shader running data corresponding to the target mobile application is loaded into the rendering program container, the shader-related data needs to be verified, and after the verification is passed, it is loaded into the rendering program container to ensure the accuracy of the binary language type shader running data, so that the screen of the target mobile application can be rendered.

[0209] Specifically, the running data to be parsed corresponding to the target mobile application is obtained from the storage space on the local personal computer, the running data to be parsed is parsed to obtain a feature header, a data hash value and candidate running data, and then, a preset feature header pre-set for the running data to be parsed is determined, and the preset feature header is compared with the feature header obtained by parsing. If the preset feature header is consistent with the feature header obtained by parsing, a hash check is further performed on the candidate running data according to the data hash value. Specifically, the hash value to be checked corresponding to the candidate running data is calculated, and the hash value to be checked is compared with the data hash value obtained by parsing. If the hash value to be checked is consistent with the data hash value obtained by parsing, it means that the hash check of the candidate running data according to the data hash value passes.

[0210] Further, when the hash check of the candidate operation data according to the data hash value passes, the candidate operation data is parsed to obtain two mechanism configuration parameters, binary program data and binary mapping data. The two mechanism configuration parameters can be understood as the configuration parameters of the image processing unit supported by the current binary program data and binary mapping data. According to the binary configuration parameters, it is determined whether the binary program data and binary mapping data conform to the image processing unit of the personal computer. Therefore, the target configuration parameters of the image processing unit of the personal computer are obtained, and the target configuration parameters are compared with the binary configuration parameters. If the binary configuration parameters are verified to be consistent with the target configuration parameters, for example, the graphics card model represented by the binary configuration parameters is consistent with the graphics card model of the target configuration parameters, the binary program data and the binary mapping data are determined as binary language type shader operation data. It should be noted that the rendering process is mainly implemented based on the graphics card of the PC end (local), so the parameters related to the shader (such as the graphics card configuration parameters recorded by the binary configuration parameters) also need to be consistent with the actual configuration parameters of the graphics card to ensure that the graphics card can support the binary program data and binary mapping data.

[0211] 207. Based on the binary program data and the binary mapping data in the rendering program container, execute the corresponding target shader program to render the screen of the target mobile application in the simulator.

[0212] In an embodiment of the present application, loading binary language type shader running data into a rendering program container can be understood as loading binary program data and binary mapping data into a rendering program container, where the binary program data and binary mapping data are binary data that can be directly recognized and used by a personal computer, and represent data in binary form that supports the running of a local shader, or can be understood as state data of a local shader program at runtime; based on the binary program data and binary mapping data in the rendering program container, the local shader program is executed to render the screen of a target mobile application in the simulator, thereby eliminating the need to go through the tedious process of creating and binding vertex shaders and fragment shaders, and linking to form a rendering pipeline, and running the shader program is implemented based on the directly loaded binary program data and binary mapping data, thereby improving the efficiency of application screen rendering.

[0213] In order to facilitate the understanding of the embodiment of the present application, the embodiment of the present application will be described with a specific application scenario example. Specifically, by executing the above steps 201-207, the application scenario example is described.

[0214] It should be noted that the application screen rendering method is suitable for application screen rendering scenarios when running mobile terminal (such as mobile phones, smart watches, etc.) versions of applications on PC devices using virtualization technology, such as mobile game applications, short video applications, online shopping applications, etc. Specifically, when triggering the start of the target mobile terminal application, a rendering program container is created for loading shader program data, and the pre-stored binary language type shader running data is obtained, and the binary language type shader running data is directly loaded into the rendering program container, so as to directly run the shader program to render the target mobile terminal application screen. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader running data is stored in a binary language type, the leakage of the shader source code is avoided, and the security of the shader program is improved. The following introduces the application screen rendering method using the application screen rendering scenario example, as follows:

[0215] 1. The application screen rendering scene example is briefly introduced as follows:

[0216] With the development of mobile information technology, many business service providers have developed mobile applications for mobile terminals (such as mobile phones), such as mobile game applications, short video applications, etc. However, although users can use mobile applications on their mobile phones at any time, which is convenient, the hardware characteristics of the mobile device itself may not be conducive to the user's operation of the mobile application, and limit the rendering of the mobile application. For example, the mobile device may limit the operation and rendering effect of the game application. Based on this, virtualization technology can be used to run mobile applications on PC devices.

[0217] Among them, the mobile terminal simulator used in PC devices is a simulation environment realized by virtualization technology. Among them, the core of the simulator is to create a virtual target device environment that is different from the main device in the main device, thereby helping developers to test the compatibility of applications in the simulated target device environment, and providing users with a way to use applications in the target device environment on the main device without the target device. For example, it allows users to use mobile terminal applications on desktop operating systems such as Windows and macOS. Among them, the main device is a PC device and the target device is a mobile device.

[0218] When a mobile application is run on a PC device through virtualization technology, a rendering program is created for the mobile version of the application, and vertex shaders and fragment shaders are created and bound in the rendering program. The vertex shaders and fragment shaders in the rendering program are linked to form a rendering pipeline, and the mobile version of the application is rendered according to the rendering pipeline in the rendering program so that the mobile version of the application can be used on the personal computer. Thereafter, when the mobile version of the application is stopped being used on the personal computer, the rendering pipeline in the rendering program is released and the rendering program is deleted.

[0219] For example, Figure 4 A flowchart of the rendering of a mobile application on a personal computer provided in an embodiment of the present application, combined with Figure 4 As shown in the figure, when running a mobile application on a personal computer through virtualization technology, the rendering process of the shader is involved, as follows:

[0220] (A.1) Create Program. When the mobile application running in virtualization technology calls the "glCreateProgram" interface, the PC creates a program, which is equivalent to the program used by the rendering pipeline and is a container for shader objects.

[0221] (A.2) Load Shader: First determine whether the shader is written. If not, execute it through the following three steps:

[0222] (A.2.1) Write shader source code.

[0223] (A.2.2) Create Shader. When the mobile application calls the "glCreateShader" interface, a shader object is created. The shader object can include a vertex shader (GL_VERTEX_SHADER) and a fragment shader (GL_FRAGMENT_SHADER), referred to as vs and fs shaders.

[0224] (A.2.3) Initialize the shader object source code. When the mobile application calls the "glShaderSource" interface, the PC provides the shader source program in the shader object.

[0225] (A.2.4) Compile shader source code (Compile Shader). When the mobile application calls the "glCompileShader" interface, the PC compiles the source code in the shader object.

[0226] (A.2.5) Create the shader object.

[0227] (A.3) Attach Shader. When the mobile application calls the "glAttachShader" interface, the PC side binds the vs and fs shaders to the Program object.

[0228] (A.4) Linking shader program. When the mobile application calls the "glLinkProgram" interface, the PC side links the vs shader and the fs shader in the Program object, so that the output of the vs shader becomes the input of the fs shader, thus forming a rendering pipeline.

[0229] (A.5) Run the shader program (Use Program). When the mobile application calls the "glUseProgram" interface, the PC runs the shader program and starts rendering the contents of the Program object, that is, it executes the vs and fs shader programs bound to the Program object in sequence.

[0230] (A.6) Detach Shader. When the mobile application calls the "glDetachShader" interface, it has the opposite effect of the "glAttachShader" interface, and releases the binding relationship between the vs and fs shaders and the Program object.

[0231] (A.7) Delete Shader. When the mobile application calls the "glDeleteShader" interface, it has the opposite effect of the "glCreateShader" interface and is used to delete the Shader object.

[0232] (A.8) Delete Program. When the mobile application calls the "glDeleteProgram" interface, it has the opposite effect of the "glCreateProgram" interface and is used to delete the Program object.

[0233] As can be seen from the above process, the relevant technology needs to go through the tedious process of creating and binding vertex shaders and fragment shaders, and linking them to form a rendering pipeline, which makes the application startup time longer. In addition, since the vertex shader and fragment shader programs need to be provided in the form of source code, this can easily lead to the leakage of shader source code, reducing the security of the shader program.

[0234] In order to address the above problems existing in related technologies, this application screen rendering scene is based on the instruction stream architecture, using OpenGL as the backend on the computer device, that is, the local end, and simulating the operating environment of the mobile phone to run the "GL_OES_get_program_binary" function.

[0235] Figure 5 The application screen rendering flow chart based on the instruction stream provided in the embodiment of the present application is combined with Figure 5 As shown, the application screen rendering scene is as follows:

[0236] (B.1) Create Program. When the mobile application calls the "glCreateProgram" interface, Program is equivalent to the program used by the rendering pipeline and is a container for shader objects.

[0237] (B.2) Load binary shader operation data (Load Program Binary). When the mobile application calls the binary shader program interface (glProgramBinary), the binary data is loaded into the shader program object. The binary data can be understood as the offline binary serialized data after the shader is compiled and linked.

[0238] (B.3) Run the shader program (Use Program). When the mobile application calls the "glUseProgram" interface, it starts rendering the content of the Program object, that is, it executes the vs and fs shader programs bound to the Program object in turn.

[0239] (B.4) Delete Program: When the mobile application calls the "glDeleteProgram" interface, it has the opposite effect of the "glCreateProgram" interface and is used to delete the Program object.

[0240] In the game engine, the engine default terminal (such as personal computer, PC) supports getting the binary type shader program "GL_OES_get_program_binary" to run, and uses the shader loading process.

[0241] 2. The specific implementation process of the application screen rendering scene instance is as follows:

[0242] (1) Translate the Android ESSL shader program into a PC GLSL shader program.

[0243] Figure 6 A flowchart of the translation between the mobile end shader program and the local end shader program provided in the embodiment of the present application. Figure 6 As shown, the mobile end shader program of the target mobile end application is translated (transferred) into the local end shader program, and the translation process is as follows:

[0244] (1.1) When the mobile terminal calls the shader object initialization interface, the PC terminal (local terminal, or backend) detects the ESSL syntax version corresponding to the original mobile terminal shader program through the shader object initialization interface, and determines the GLSL syntax version corresponding to the translated local terminal shader program based on the original ESSL syntax version. For example, the ESSL100 syntax version corresponds to the GLSL330 local terminal shader program. Similarly, ESSL300 corresponds to GLSL430, and ESSL310 corresponds to GLSL450. The above is just an example.

[0245] (1.2) According to different ESSL syntax versions, determine the ESSL syntax extensions to be supported. For example, ESSL310 needs to support syntax extensions such as "GL_OES_texture_buffer" and "GL_OES_shader_image_atomic".

[0246] (1.3) According to the characteristics of the main graphics card on the PC device, fill in the global ESSL constant information and obtain the GLSL syntax information. For example, fill in the "maximum number of buffer bindings" constant (GL_MAX_UNIFORM_BUFFER_BINDINGS) and the maximum number of shader buffer bindings (GL_MAX_SHADER_STORAGE_BUFFER_BINDINGS) in the global ESSL constants, and so on.

[0247] (1.4) According to the GLSL syntax version, ESSL syntax extension, and filled GLSL syntax information, the PC side calls the shader translation module through the shader program translation interface (glShaderSource) to complete the corresponding syntax conversion of the mobile side shader program and obtain the local side shader program.

[0248] (1.5) According to the translated result, the translated grammatical variable information is obtained. The grammatical variable information may include any defined variable (uniform), a variable (varying) used to transfer data between the vertex shader and the fragment shader, a vertex number variable (attribute), an interface block, etc.

[0249] (2) Map the grammatical variables before and after the translation of the above two programs one by one, and serialize the mapping relationship into binary form.

[0250] Figure 7 A flowchart of the binarization of the grammatical variable mapping relationship provided in the embodiment of the present application. Figure 7 As shown, the binarization process of the grammatical variable mapping relationship is as follows:

[0251] (2.1) When the Android end calls the program link interface (glLinkProgram) about the shader, the PC end executes the shader program link, such as linking the vertex shader program and the fragment shader program to create a rendering pipeline with the shader. Specifically, according to the syntax variable information (uniform, varying, attribute, Interfaceblock) obtained previously, the location table is filled with the location corresponding to each uniform variable in a recursive way to obtain the host variable location table, referred to as the "host uniform loc table".

[0252] (2.2) Based on the obtained grammatical variable information, a name forward mapping table (referred to as uniform name table) and a name reverse mapping table (referred to as uniform name reverse table) for uniform variables between the mobile end shader program and the local end shader program before and after translation are further filled in a recursive manner.

[0253] (2.3) The three tables "host uniform loc&&uniform name&&uniform name reverse" are serialized and saved using a certain binary serialization method. For example, the data serialization format (Protocol Buffers) can be used to implement binary serialization.

[0254] (3) Binaryize the PC GLSL shader program.

[0255] After the PC side obtains the GLSL syntax information and the binary serialization information of the syntax variable mapping of the "ESSL->GLSL" translation, it is further necessary to obtain the binary serialization information of the GLSL shader program after linking and compilation, for example, through the binary retrieval method (PROGRAM_BINARY_RETRIEVABLE_HINT) from the PC's back-end driver (such as the driver corresponding to the graphics card).

[0256] (4) After the mobile terminal (Android terminal) obtains the above binary serialized data, it stores it in a certain format.

[0257] Figure 8 This is an example diagram of a synchronous response of a binary shader program acquisition interface between a mobile terminal and a local terminal provided in an embodiment of the present application. Figure 8 As shown, the storage process of the binary serialized data is as follows:

[0258] (4.1) The PC side (local side, or backend) has obtained the binary serialization information of the "ESSL->GLSL" translation mapping and the binary serialization information of the GLSL shader program after linking and compilation. At this time, it needs to be returned to the mobile side (for example, Android side) in a certain data format.

[0259] (4.2) The entire data is stored in the format of "feature header (length 2) + data Len (length 4) + data (Binary data hash (length 8) + Binary data)". Among them, "feature header and Binary data hash" are used to verify the integrity of binary serialized data to prevent inconsistency in data content before and after storage. It should be noted that "Binarydata" is binary serialized information.

[0260] (4.3) For binary serialization information (Binary data), the binary serialization data of the three tables "Binary Ver+Binary VENDOR+host uniform loc&&uniform name&&uniform name reverse" and the binary data of the GLSL shader program are stored in this format, where Binary Ver is used for function expansion to maintain the extensibility of the binary serialization information of the "ESSL->GLSL" translation mapping. For example, a new table can be added later to store the "attribute" variable information; and "Binary VENDOR" is used to check the validity of the binary data of the GLSL shader program to prevent the binary serialization information from being unsuitable due to changes in the PC machine hardware graphics card.

[0261] (5) The PC side parses the binary data sent from the Android side according to the corresponding format, first loads and parses the binary program data of the GLSL shader program, and deserializes the binary mapping data, so as to return the correct syntax variable information to the mobile side based on the three deserialized tables.

[0262] Fig. 9 The interactive flow chart of the grammatical variable information of the instruction stream in the application screen rendering process provided by the embodiment of the present application. Fig. 9 As shown, during the application screen rendering process, the mobile terminal (simulated Android terminal, front end) on the personal computer and the local terminal (back end) interact with each other in terms of syntax variable information. The specific process is as follows:

[0263] (5.1) When the mobile terminal calls the "glProgramBinary" interface of the local terminal, the PC terminal (local terminal) first verifies the stored data according to the "feature header and binary data hash", and then loads and parses the binary data corresponding to the GLSL shader program. Furthermore, it deserializes the "host uniform loc&&uniform name&&uniform namereverse" binary mapping data and restores the corresponding three table structures.

[0264] (5.2) When the mobile terminal calls the "glGetActiveUniform" interface to obtain uniform information, the PC terminal (local terminal) returns the corresponding mobile terminal uniform variable name information based on the information in the uniform name reverse table.

[0265] (5.3) When the mobile terminal calls the "glGetUniformLocation" interface to query the location information of the corresponding variable according to the uniform variable name information, the PC terminal queries the PC terminal uniform variable name according to the information in the uniform name table, uses the PC terminal uniform variable name to query the "host uniform loc table", and returns the corresponding location information to the mobile terminal. In this way, the data of the corresponding uniform variable can be changed according to the "location information" operation to change the shader data, so as to draw the corresponding content during the rendering process.

[0266] By executing the above application screen rendering scene example, the following effects can be achieved: the process is simple, you only need to provide binary data to load it into the shader program object for use, and the shader object is no longer explicitly required in the process, reducing the burden on developers; secondly, fast loading and parsing, the binary shader program does not need to be compiled and linked, and is directly loaded into the graphics card driver for processing; in addition, the source code can be protected, and the binary shader program does not contain readable source code.

[0267] From the above, it can be seen that in the embodiment of the present application, when the simulator triggers the start of the target mobile application, the local side first creates a rendering program container for loading shader program data, and then obtains the shader running data of the binary language type pre-stored for the target mobile application, and loads the shader running data of the binary language type into the rendering program container. It should be noted that the shader running data of the binary language type refers to the binary program data obtained by linking and compiling the local shader program obtained by translating the mobile shader program of the target mobile application, and the binary mapping data of the grammatical variable mapping relationship of the mobile shader program of the target mobile application before and after the grammatical translation. Finally, the corresponding shader program is directly run on the personal computer based on the binary language type shader running data to realize the rendering of the screen of the target mobile application in the simulator. Therefore, compared with the prior art solution that creates a rendering program through the cumbersome process of creating and binding vertex shaders and fragment shaders and linking them to form a rendering pipeline, the present application obtains pre-stored shader execution data of a binary language type and directly loads the shader execution data of the binary language type into the rendering program container. Since the binary language is a machine language, the compiled code can be directly executed without further interpretation or conversion. Therefore, the shader program corresponding to the shader execution data can be directly executed to render the screen of the target mobile application. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader execution data is stored in a binary language type, the leakage of the shader source code in the form of a high-level language is avoided, thereby improving the security of the shader program.

[0268] The specific implementation of the above steps can be found in the previous embodiments, which will not be described in detail here.

[0269] In order to facilitate better implementation of the application screen rendering method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above application screen rendering method. The meanings of the terms are the same as those in the above application screen rendering method, and the specific implementation details can refer to the description in the method embodiment.

[0270] See also Fig.10 , Fig.10 This is a schematic diagram of the structure of an application screen rendering device provided in an embodiment of the present application. The application screen rendering device is integrated into a computer device of the present application, such as Figure 1 On the terminal 110, the application screen rendering device may include a creating unit 401, an acquiring unit 402 and an executing unit 403.

[0271] A creating unit 401 is used to create a rendering program container corresponding to a target mobile application, where the target mobile application is a mobile application in the simulator;

[0272] An acquisition unit 402 is used to acquire binary language type shader execution data pre-stored for a target mobile terminal application, and load the binary language type shader execution data into a rendering program container;

[0273] The binary language type shader operation data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a syntax variable mapping relationship between a mobile end shader program of a target mobile end application and a local end shader program, wherein the local end shader program is obtained by syntax translation based on the mobile end shader program;

[0274] The execution unit 403 is used to execute the corresponding target shader program based on the binary program data and the binary mapping data in the rendering program container to render the screen of the target mobile application in the simulator.

[0275] In some implementations, the acquisition unit 402 is further configured to:

[0276] Obtaining the to-be-parsed running data corresponding to the target mobile application from the storage space, where the to-be-parsed running data at least includes a feature header, a data hash value, and candidate running data;

[0277] When the feature header is consistent with the preset feature header for the running data to be parsed, a hash check is performed on the candidate running data according to the data hash value;

[0278] When the hash check performed on the candidate execution data according to the data hash value passes, the candidate execution data is determined to be shader execution data of the binary language type.

[0279] In some implementations, the acquisition unit 402 is further configured to:

[0280] determining binary configuration parameters, binary program data, and binary mapping data from the candidate operational data;

[0281] Obtaining target configuration parameters of the local image processing unit, and verifying the binary configuration parameters according to the target configuration parameters to obtain verification results;

[0282] When the verification result is that the binary configuration parameters are consistent with the target configuration parameters, the binary program data and the binary mapping data are determined as shader execution data of a binary language type.

[0283] In some implementations, the execution unit 403 is further configured to:

[0284] Deserialize the binary mapping data to obtain a main grammar variable position table and a variable mapping table;

[0285] According to the binary program data, start the corresponding target shader program;

[0286] The screen of the target mobile application in the simulator is rendered by starting the target shader program in combination with the main syntax variable position table and the variable mapping table.

[0287] In some implementations, the execution unit 403 is further configured to:

[0288] Determine the target mobile terminal variable for real-time target local terminal variable mapping in the binary program data according to the variable inverse mapping table in the variable mapping table;

[0289] According to the target variable position information associated with the target mobile terminal variable in the main syntax variable position table, adjusting the shader variable information of the target shader;

[0290] The screen of the target mobile application in the simulator is rendered by starting the target shader program and combining the shader variable information of the target shader.

[0291] In some implementations, the execution unit 403 is further configured to:

[0292] Send the target mobile variables to the simulator;

[0293] When it is detected that the simulator calls the variable location interface to send a location query request for the target mobile terminal variable, the target local terminal variable mapped to the target mobile terminal variable is determined according to the variable positive mapping table in the variable mapping table;

[0294] Query the target variable position information corresponding to the target local variable from the main syntax variable position table;

[0295] Adjust the shader variable information of the target shader according to the target variable location information.

[0296] In some implementations, the application screen rendering device further includes a pre-storage unit for:

[0297] According to the target syntax rules, translate the mobile end shader program of the target mobile end application into the local end shader program;

[0298] Generate binary mapping data according to the syntax variable mapping relationship between the local end shader program and the mobile end shader program;

[0299] Obtain binary program data for the rendering pipeline formed by linking local side shader programs;

[0300] Binary mapping data and binary program data are stored.

[0301] In some embodiments, the pre-storage unit is further used for:

[0302] Determine a mobile terminal syntax version corresponding to a mobile terminal shader program of a target mobile terminal application, and determine a target syntax version that matches the mobile terminal syntax version according to a target syntax rule;

[0303] Determine the syntax extension attributes supported by the mobile terminal syntax version in the target syntax version;

[0304] Read the target configuration parameters of the image processing unit of the local end, and fill the syntax constants corresponding to the syntax version of the mobile end according to the target configuration parameters to obtain the target syntax information corresponding to the target syntax version;

[0305] According to the syntax extension attributes and the target syntax information, the mobile end shader program is translated into the local end shader program corresponding to the target syntax version.

[0306] In some embodiments, the pre-storage unit is further used for:

[0307] Get the syntax variable information corresponding to the local shader program;

[0308] Fill the position information corresponding to each target local variable in the grammar variable information in the preset variable position table to obtain the main grammar variable position table;

[0309] Determine the syntax variable mapping relationship between the local end shader program and the mobile end shader program according to the syntax variable information, and generate a variable forward mapping table and a variable inverse mapping table according to the syntax variable mapping relationship;

[0310] Binary mapping data is generated according to the main syntax variable position table, the variable positive mapping table and the variable inverse mapping table.

[0311] In some embodiments, the pre-storage unit is further used for:

[0312] Generate shader running data of binary language type according to the binary mapping data and the binary program data;

[0313] Binding target configuration parameters, binary language type, and shader running data to obtain candidate running data;

[0314] Obtain the data hash value, data length and feature header corresponding to the candidate running data;

[0315] The characteristic header, data length, data hash value and candidate running data are combined in order to obtain target running data, and the target running data is stored.

[0316] In some implementations, the application screen rendering device further includes a notification unit, configured to:

[0317] Generate a notification message, and send the notification message to the simulator, wherein the notification message is used to instruct the shader running data supporting the binary language type to run the target mobile terminal application;

[0318] The creation unit is also used to: when it is detected that the simulator requests to run the target mobile application through a target shader program of a binary language type, create a rendering program container corresponding to the target mobile application.

[0319] In some implementations, the application screen rendering device further includes a downloading unit, which is used to:

[0320] Read the target configuration parameters and operating system parameters of the local image processing unit, and download the corresponding binary language type shader running data from the data sharing end according to the target configuration parameters and the operating system parameters;

[0321] The data sharing end at least stores binary language type shader operation data uploaded by other device ends for the target mobile end application;

[0322] The downloaded binary language type shader execution data is stored.

[0323] From the above, it can be seen that in the embodiment of the present application, when the simulator triggers the start of the target mobile application, the local side first creates a rendering program container for loading shader program data, and then obtains the shader running data of the binary language type pre-stored for the target mobile application, and loads the shader running data of the binary language type into the rendering program container. It should be noted that the shader running data of the binary language type refers to the binary program data obtained by linking and compiling the local shader program obtained by translating the mobile shader program of the target mobile application, and the binary mapping data of the grammatical variable mapping relationship of the mobile shader program of the target mobile application before and after the grammatical translation. Finally, the corresponding shader program is directly run based on the shader running data of the binary language type to realize the rendering of the screen of the target mobile application in the simulator. Therefore, compared with the prior art solution that requires going through the cumbersome process of creating and binding vertex shaders and fragment shaders and linking to form a rendering pipeline when creating a rendering program, the present application obtains pre-stored shader execution data of a binary language type and directly loads the shader execution data of the binary language type into the rendering program container. Since the binary language is a machine language, the compiled code can be directly executed without further interpretation or conversion. Therefore, the shader program corresponding to the shader execution data can be directly executed to render the screen of the target mobile application. In this way, the rendering process can be simplified and the rendering efficiency of the application screen can be improved. Moreover, since the shader execution data is stored in a binary language type, the leakage of the shader source code in the form of a high-level language is avoided, thereby improving the security of the shader program.

[0324] The specific implementation of each of the above units can be found in the previous embodiments, which will not be described in detail here.

[0325] See also Fig.11 , Fig.11 The schematic diagram of the structure of the terminal provided in the embodiment of the present application includes a structural frame of a part of the terminal 110 implementing the embodiment of the present application. The terminal may be Figure 1 The terminal 110 shown in the figure specifically includes: a radio frequency (RF) circuit 510, a memory 515, an input unit 520, a display unit 540, a sensor 550, an audio circuit 560, a wireless fidelity (WiFi) module 570, a processor 580, and a power supply 590. Those skilled in the art will appreciate that the structure of the terminal 110 shown in the figure does not limit the mobile phone or computer, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0326] The RF circuit 510 may be used for receiving and sending signals during information transmission or communication. In particular, after receiving downlink information from the base station, the information is sent to the processor 580 for processing. In addition, the uplink data is sent to the base station.

[0327] The memory 515 may be used to store software programs and modules. The processor 580 executes various functional applications and application screen rendering of the terminal by running the software programs and modules stored in the memory 515 .

[0328] The input unit 520 may be used to receive input digital or character information and generate key signal input related to the terminal's settings and function control. Specifically, the input unit 520 may include a touch panel 531 and other input devices 532 .

[0329] The display unit 540 may be used to display input information or provided information and various menus of the terminal. The display unit 540 may include a display panel 541 .

[0330] The audio circuit 560 , the speaker 561 , and the microphone 562 may provide an audio interface.

[0331] In this embodiment, the processor 580 included in the terminal 110 can execute the application screen rendering method of the previous embodiment.

[0332] See also Fig.12 , Fig.12 The schematic diagram of the structure of the server provided in the embodiment of the present application includes a structural frame of the server 120 implementing the embodiment of the present application. The server may be Figure 1 The server 120 shown in the figure may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 622 (e.g., one or more processors) and memory 632, and one or more storage media 630 (e.g., one or more mass storage devices) storing application programs 642 or data 644. Among them, the memory 632 and the storage medium 630 may be temporary storage or permanent storage. The program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server 600. Furthermore, the central processing unit 622 may be configured to communicate with the storage medium 630 to execute a series of instruction operations in the storage medium 630 on the server 600.

[0333] The server 600 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658, and / or one or more operating systems 641, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0334] The central processing unit 622 in the server 600 can be used to execute the application screen rendering method of the embodiment of the present application.

[0335] An embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute the application screen rendering method of each of the aforementioned embodiments.

[0336] The embodiment of the present application further provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, so that the computer device executes and implements the above-mentioned application screen rendering method.

[0337] In addition, the terms "comprises" and "includes" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product or apparatus.

[0338] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0339] It should be understood that in the description of the embodiments of the present application, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood to not include the number, and above, below, within, etc. are understood to include the number.

[0340] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0341] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0342] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0343] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0344] It should also be understood that the various implementations provided in the embodiments of the present application can be combined arbitrarily to achieve different technical effects.

[0345] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0346] The above is a specific description of the implementation method of the present application, but the present application is not limited to the above-mentioned implementation method. Technical personnel familiar with the field can also make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A method for rendering an application screen, characterized in that: include: Creating a rendering program container corresponding to a target mobile application, wherein the target mobile application is a mobile application in the simulator; Acquire binary language type shader execution data pre-stored for the target mobile terminal application, and load the binary language type shader execution data into the rendering program container; The binary language type shader operation data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a syntax variable mapping relationship between a mobile end shader program of the target mobile end application and the local end shader program, wherein the local end shader program is obtained by syntax translation based on the mobile end shader program; Based on the binary program data and the binary mapping data in the rendering program container, a corresponding target shader program is executed to render the screen of the target mobile application in the simulator.

2. The method according to claim 1, characterized in that The obtaining of binary language type shader operation data pre-stored for the target mobile terminal application includes: Acquire the to-be-parsed operation data corresponding to the target mobile application from the storage space, wherein the to-be-parsed operation data at least includes a feature header, a data hash value, and candidate operation data; When the feature header is consistent with the preset feature header for the operation data to be parsed, performing a hash check on the candidate operation data according to the data hash value; When the hash check performed on the candidate execution data according to the data hash value passes, the candidate execution data is determined to be shader execution data of a binary language type.

3. The method according to claim 2, characterized in that The step of determining the candidate running data as shader running data of a binary language type comprises: determining binary configuration parameters, binary program data, and binary mapping data from the candidate operational data; Acquire target configuration parameters of the local image processing unit, and verify the binary configuration parameters according to the target configuration parameters to obtain verification results; When the verification result is that the binary configuration parameter is consistent with the target configuration parameter, the binary program data and the binary mapping data are determined as shader execution data of a binary language type.

4. The method according to any one of claims 1 to 3, characterized in that: The executing a corresponding target shader program based on the binary program data and the binary mapping data in the rendering program container to render a screen of the target mobile terminal application in the simulator includes: Deserialize the binary mapping data to obtain a main syntax variable position table and a variable mapping table; According to the binary program data, starting a corresponding target shader program; The screen of the target mobile terminal application in the simulator is rendered by combining the started target shader program with the main syntax variable position table and the variable mapping table.

5. The method according to claim 4, characterized in that The starting target shader program is combined with the main syntax variable position table and the variable mapping table to render the screen of the target mobile terminal application in the simulator, including: Determine, according to the variable inverse mapping table in the variable mapping table, a target mobile terminal variable mapped with a real-time target local terminal variable in the binary program data; According to the target variable position information associated with the target mobile terminal variable in the main syntax variable position table, adjusting the shader variable information of the target shader; The screen of the target mobile terminal application in the simulator is rendered by starting the target shader program in combination with the shader variable information of the target shader.

6. The method according to claim 5, characterized in that The adjusting the shader variable information of the target shader according to the target variable position information associated with the target mobile terminal variable in the main syntax variable position table includes: Sending the target mobile terminal variable to the simulator; When it is detected that the simulator calls the variable location interface to send a location query request for the target mobile terminal variable, determining the target local terminal variable mapped by the target mobile terminal variable according to the variable positive mapping table in the variable mapping table; Query the target variable position information corresponding to the target local terminal variable from the main syntax variable position table; The shader variable information of the target shader is adjusted according to the target variable position information.

7. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the binary language type shader execution data pre-stored for the target mobile terminal application, the method further includes: According to the target syntax rules, the mobile terminal shader program of the target mobile terminal application is translated into a local terminal shader program; Generate binary mapping data according to a syntax variable mapping relationship between the local end shader program and the mobile end shader program; Obtaining binary program data for a rendering pipeline formed by linking the local end shader program; The binary mapping data and the binary program data are stored.

8. The method according to claim 7, characterized in that The translating the mobile terminal shader program of the target mobile terminal application into a local terminal shader program according to the target grammar rule includes: Determine a mobile terminal syntax version corresponding to the mobile terminal shader program of the target mobile terminal application, and determine a target syntax version that matches the mobile terminal syntax version according to a target syntax rule; Determining the grammar extension attributes supported by the mobile terminal grammar version in the target grammar version; Reading the target configuration parameters of the image processing unit of the local end, and filling the syntax constants corresponding to the syntax version of the mobile end according to the target configuration parameters, to obtain the target syntax information corresponding to the target syntax version; According to the syntax extension attribute and the target syntax information, the mobile end shader program is translated into a local end shader program corresponding to the target syntax version.

9. The method according to claim 7 or 8, characterized in that: The generating binary mapping data according to the syntax variable mapping relationship between the local end shader program and the mobile end shader program includes: Obtaining syntax variable information corresponding to the local end shader program; Fill the preset variable position table with the position information corresponding to each target local end variable in the grammar variable information to obtain a main grammar variable position table; Determine a syntax variable mapping relationship between the local end shader program and the mobile end shader program according to the syntax variable information, and generate a variable forward mapping table and a variable inverse mapping table according to the syntax variable mapping relationship; Binary mapping data is generated according to the main syntax variable position table, the variable positive mapping table and the variable inverse mapping table.

10. The method according to any one of claims 7 to 9, characterized in that: The storing of the binary mapping data and the binary program data comprises: Generate binary language type shader execution data according to the binary mapping data and the binary program data; Binding the target configuration parameter, the binary language type, and the shader running data to obtain candidate running data; Obtaining a data hash value, data length, and feature header corresponding to the candidate running data; The characteristic header, the data length, the data hash value and the candidate operation data are combined in order to obtain target operation data, and the target operation data is stored.

11. The method according to any one of claims 7 to 10, characterized in that: Before creating a rendering program container corresponding to the target mobile application, the method further includes: Generate a notification message, and send the notification message to the simulator, wherein the notification message is used to instruct the shader running data supporting the binary language type to run the target mobile terminal application; The step of creating a rendering program container corresponding to the target mobile application includes: When it is detected that the simulator requests to run the target mobile application through the target shader program of the binary language type, a rendering program container corresponding to the target mobile application is created.

12. The method according to any one of claims 1 to 6, characterized in that: Before obtaining the binary language type shader execution data pre-stored for the target mobile terminal application, the method further includes: Reading the target configuration parameters and operating system parameters of the image processing unit at the local end, and downloading the corresponding binary language type shader running data from the data sharing end according to the target configuration parameters and the operating system parameters; The data sharing end at least stores binary language type shader operation data uploaded by other device ends for the target mobile end application; The downloaded shader execution data of the binary language type is stored.

13. An application screen rendering device, characterized in that: include: A creation unit, used to create a rendering program container corresponding to a target mobile application, wherein the target mobile application is a mobile application in the simulator; An acquisition unit, configured to acquire binary language type shader execution data pre-stored for the target mobile terminal application, and load the binary language type shader execution data into the rendering program container; The binary language type shader operation data includes binary program data obtained by linking and compiling a local end shader program, and binary mapping data representing a syntax variable mapping relationship between a mobile end shader program of the target mobile end application and the local end shader program, wherein the local end shader program is obtained by syntax translation based on the mobile end shader program; An execution unit is used to execute a corresponding target shader program based on the binary program data and the binary mapping data in the rendering program container to render a screen of the target mobile application in the simulator.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the application screen rendering method according to any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the application screen rendering method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the application screen rendering method according to any one of claims 1 to 12 is implemented.

Images