WebGL display optimization method and system for multi-device dynamic adaptation

Abstract

The invention provides a WebGL display optimization method and system for multi-device dynamic adaptation, and belongs to the technical field of WebGL display.The method comprises the steps that display parameters of target devices are collected, and device display feature vectors are constructed; a texture image in the WebGL model is preprocessed; training and generating a mapping relation library on the basis of historical adaptation data, dynamically adapting the device feature display vector of the target device, obtaining required WebGL model operation parameters, and dynamically adjusting the calculation graph of the preprocessed WebGL model; executing a vector graphic rendering algorithm on the fixed layout of the WebGL model, and dynamically sampling and optimizing the dynamic content of the WebGL model to realize multi-resolution adaptation; and monitoring the state of the target equipment in real time, pausing the rendering task of the WebGL model when detecting that the display mode is switched, and carrying out equipment parameter acquisition again. According to the invention, optimal visual effects and performance optimization on different devices are realized.

Description

Technical Field

[0001] This application belongs to the technical field of WebGL display, and specifically relates to a WebGL display optimization method and system for multi-device dynamic adaptation. Background Art

[0002] With the development of the Internet, WebGL (Web Graphics Library), as a technology for rendering high-performance 3D graphics in web browsers, has been widely used. WebGL is based on the OpenGL ES 2.0 specification, allowing developers to create and manipulate 3D graphics in the browser through JavaScript without installing any plugins. However, the existing WebGL model deployment methods have the following limitations: Firstly, traditional WebGL model deployment is usually optimized for a fixed-resolution display screen and a specific display mode. When deployed to other display devices, due to differences in aspects such as screen size, pixel density, and color display characteristics, the original display effect will be greatly reduced, resulting in problems such as blurred images, disproportionate ratios, and color distortion. Secondly, when the display mode changes, such as switching from landscape to portrait or adjusting the screen refresh rate, the functions of traditional WebGL models may be severely affected, resulting in problems such as operation response delays, incorrect interface layouts, and even program crashes. Finally, in order to achieve good display effects on different devices, developers need to perform separate adaptation debugging for each device, which not only consumes a large amount of time and manpower but also makes it difficult to ensure the unity of the adaptation effect.

[0003] In summary, the strong dependence on specific scenarios greatly limits the generality and adaptability of WebGL models and cannot meet the actual application needs of diversification and dynamism. Summary of the Invention

[0004] In a first aspect, an embodiment of this application provides a WebGL display optimization method for multi-device dynamic adaptation, including the following steps: S1. Collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; S2. Preprocess the texture images in the WebGL model using an adaptive resolution algorithm; S3. Based on the mapping relationship library between the device display features and the WebGL model operation parameters generated by training with historical adaptation data, dynamically adapt the device feature display vector of the target device, obtain the required WebGL model operation parameters, and dynamically adjust the computational graph of the preprocessed WebGL model; S4. Start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and optimize it using the required WebGL model running parameters to achieve multi-resolution adaptation; S5. Monitor the status of the target device in real time. When a display mode switch is detected, pause the rendering task of the WebGL model and return to step S1.

[0005] Furthermore, the specific steps of step S1 are as follows: S11. Obtain the display parameters of the target device according to the device interface of the target device type; S12. Standardize the collected display parameters and uniformly convert them into a predefined data format and vector to obtain a device display feature vector; Specifically, the device feature vector ; where R represents the resolution, A represents the display ratio, C represents the color space, F represents the refresh rate, and DPR represents the device pixel ratio.

[0006] Furthermore, the specific steps of step S2 are as follows: S21. Calculate the target rendering size according to the device pixel ratio and resolution of the target device; S22. Use the bicubic interpolation scaling algorithm to scale the texture image in the WebGL model to the target rendering size to obtain an adapted texture object; S23. Pass the scaled texture image through an edge-preserving filter to eliminate the edge jaggedness after scaling; S24. If the color space is P3, pre-compute the gamut conversion matrix.

[0007] Furthermore, the specific steps of step S21 are as follows: S211. Obtain the resolution R of the target device and split the resolution R into a logical width Hw and a logical height Hh; S212. The target rendering size H×W is calculated by the following formula: W = Hw×DPR H = Hh×DPR where DPR is the device pixel ratio; The specific steps of step S22 are as follows: S221. Obtain the original size Hy×Wy of the texture image in the WebGL model; S222. Calculate the scaling ratio ; S223. Use bicubic interpolation according to the scaling ratio Scale the texture image in the WebGL model;

[0008] Among them, is the original texture image, is the scaled texture image, and W is the bicubic weight function; ( x i , y i ) represents the 4×4 neighborhood pixel coordinates around the target position ( in the original texture image x , y ); S224. Crop or fill the part that exceeds the target rendering size to generate an adapted texture object.

[0009] Furthermore, the specific steps of step S3 are as follows: S31. Collect the display parameters of different devices and the corresponding WebGL model running parameters to construct a historical adaptation dataset ; The WebGL model running parameters include LOD level, anti-aliasing level, shader parameters, weight offset, activation function threshold, and computational graph structure adjustment flag;

[0010] Among them, is the display feature vector of the i-th device, is the corresponding adapted WebGL model running parameter; S32. Construct a feature embedding model and a parameter mapping model, and use the historical adaptation dataset to train the feature embedding model through contrastive loss, and use the historical adaptation dataset to train the parameter mapping model through meta-learning; S33. Use the feature embedding model to map the device feature display vector of the target device into an embedding vector, then map the embedding vector into WebGL model running parameters through the parameter mapping model, and save the device feature display vector with a usage frequency higher than the frequency threshold and the WebGL model running parameters to the mapping relationship library; S34. Dynamically adjust the computational graph of the WebGL model that has completed the texture image preprocessing according to the WebGL model running parameters matched by the target device.

[0011] Furthermore, the specific steps of step S32 are as follows: S321. Use a lightweight neural network to construct a feature embedding model; S322. Embed vector annotation is performed on the historical matching dataset. Using the display feature vector as the input and the embedded vector annotation result as the output, the feature embedding model is pre-trained using the contrastive loss method:

[0012] where, is the contrastive loss function, is the display feature vector of the i-th device, is the display feature vector of the j-th device, is the indicator function, which is 1 when is similar to , otherwise 0; represents the output embedded vector ; S323. A parameter mapping model is constructed based on the model-agnostic meta-learning framework. An offline support set and a query set are constructed based on the historical adaptation dataset, where the number of samples in the offline support set is less than the quantity threshold; S324. The parameter mapping model is fine-tuned using the offline support set to obtain the initial parameters of the parameter mapping model, and the parameter mapping model is optimized using the query set to optimize the initial parameters and obtain the optimized parameter mapping model; The specific steps of step S33 are as follows: S331. Match the device feature vector of the target device with the mapping relationship library; If the device feature vector is matched, the corresponding WebGL model running parameters are used as the WebGL model running parameters matched by the target device, and step S34 is entered; If the device feature vector cannot be matched, step S332 is entered; S332. Compare the device feature vector of the target device with the historical adaptation dataset, and select the TOP-K samples with the highest similarity as the online support set; S333. Update the optimized parameter mapping model based on the online support set to obtain the updated parameter mapping model; S334. Input the device feature vector of the target device into the updated parameter mapping model, and output the WebGL model running parameters matched by the target device; The specific steps of step S34 are as follows: S341. Determine whether the WebGL model running parameters matched by the target device contain weight offsets; If so, adjust the convolution kernel weights of the computational graph of the WebGL model according to the channels:

[0013] where, Represents the original convolutional kernel weight, Represents the weight offset used to adjust the convolutional kernel weight, Represents the adjusted convolutional kernel weight, Represents the scaling factor adjusted according to the device pixel ratio DPR; If not, go to step S342; S342. Calibrate the activation function threshold in the WebGL model running parameters according to the color space C; S343. Adjust the computational graph topology of the WebGL model according to the display ratio A.

[0014] Furthermore, the specific steps of constructing the offline support set and query set based on the historical adaptation dataset in step S323 are as follows: The historical adaptation dataset Is clustered according to the combination of display parameters; Randomly select M clusters, and for each cluster, select K samples from the historical adaptation dataset To construct the offline support set And select N samples to construct the query set Where ; In step S332, the device feature vector of the target device Is input into the feature embedding model To obtain the embedding vector ; Calculate the cosine similarity between the embedding vector Of the target device and the historical devices in the historical adaptation dataset ;

[0015] Select the TOP-K samples with the highest similarity as the historical device samples to construct the offline support set.

[0016] Furthermore, the specific steps of step S4 are as follows: S41. Start the rendering task of the WebGL model for the target device and initialize the rendering pipeline: Set the viewport according to the target rendering size, enable the adapted anti-aliasing level, and when the color space is P3, perform color space conversion through the gamut conversion matrix; S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable; S43. For the fixed layout part, use the vector graph form, bind the display ratio to the viewport in the vector graph form, export it as a bitmap texture map, and then draw it as a full-screen quadrilateral using the fixed layer fragment shader; S44. Perform supersampling on the dynamic content part. When the device pixel ratio is greater than the pixel ratio threshold, bind the adapted texture object to the frame buffer object, call the LOD level to render the scene, and then downsample the supersampled texture and output it to the screen; S45. Render the fixed layout part and the dynamic content part together according to the refresh rate setting and execute to complete frame rate synchronization.

[0017] Further, the specific steps of step S5 are as follows: S51. Monitor the changes in the resolution R, display ratio, and device pixel ratio of the target device, and determine whether any one of them has changed; If so, enter step S52; If not, wait for a set period of time and return to step S51; S52. Freeze the thread of the current rendering task, re-perform step S1 on the target device to collect display parameters and construct a device feature display vector, and then regenerate new WebGL model running parameters through step S3 to update the computational graph of the WebGL model; S53. Restart the rendering pipeline with the new WebGL model running parameters and continue to execute the rendering task.

[0018] In a second aspect, an embodiment of the present application further provides a WebGL display optimization system for multi-device dynamic adaptation, including: A device display parameter collection module, configured to collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; A WebGL model preprocessing module, configured to preprocess the texture images in the WebGL model using an adaptive resolution algorithm; A dynamic adaptation module, configured to dynamically adapt the device feature display vector of the target device based on the mapping relationship library between the device display features and the WebGL model running parameters generated by training with historical adaptation data, obtain the required WebGL model running parameters, and dynamically adjust the computational graph of the preprocessed WebGL model; A hierarchical rendering module, configured to start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, perform dynamic sampling on the dynamic content part based on the device pixel ratio, and optimize it using the required WebGL model running parameters to achieve multi-resolution adaptation; A status monitoring module, configured to monitor the status of the target device in real time, and when a display mode switch is detected, pause the rendering task of the WebGL model and re-collect device parameters.

[0019] As can be seen from the above technical solutions, the present application has the following advantages: In the WebGL display optimization method and system for multi-device dynamic adaptation provided by the present application, by dynamically adapting the display parameters of the target device, the rendering parameters of the WebGL model can be automatically adjusted, so as to achieve a consistent display effect on different devices, improving the generality and adaptability of the WebGL model; through the adaptive resolution algorithm and dynamic sampling, the rendering resolution and sampling rate can be dynamically adjusted according to the performance and display characteristics of the device, so as to significantly reduce the rendering load and improve the performance while ensuring the visual effect; by real-time monitoring the device status and dynamically adjusting the rendering parameters, it can ensure that when the display mode is switched or the screen refresh rate changes, the display effect and response speed of the WebGL model are not affected, thus providing a smooth and consistent user experience; through automatic adaptation and dynamic adjustment, the present application reduces the development and maintenance costs and improves the development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the present application, the drawings required for description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a schematic flowchart of the WebGL display optimization method for multi-device dynamic adaptation of the present invention.

[0022] Figure 2 It is a schematic diagram of the WebGL display optimization system for multi-device dynamic adaptation of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] In the following, the specific steps of the WebGL display optimization method for multi-device dynamic adaptation will be described in detail, and various embodiments of the present disclosure will be described more comprehensively. The present disclosure may have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents, and / or alternative solutions that fall within the spirit and scope of the various embodiments of the present disclosure.

[0024] Exemplarily, with the continuous progress of Internet technology, WebGL (Web Graphics Library), as a technology for rendering high-performance 3D graphics in web browsers, has been widely used. WebGL is based on the OpenGLES 2.0 specification, allowing developers to create and manipulate 3D graphics in the browser through JavaScript without installing any additional plugins. However, there are some significant limitations in existing WebGL model deployment methods, which are mainly reflected in the following aspects: First of all, traditional WebGL model deployment methods are usually optimized for a specific fixed-resolution display screen and specific display mode. When these models are deployed to other display devices with different screen sizes, pixel densities, and color display characteristics, the original display effect is often severely affected. Specifically, problems such as blurred images, disproportionate scales, and color distortion occur, which seriously affect the user experience. Secondly, when the display mode changes, such as switching from landscape to portrait or adjusting the screen refresh rate, the functions of traditional WebGL models may be severely affected. These problems may manifest as operation response delays, interface layout errors, and may even lead to program crashes. These problems not only affect the user's operation experience but also may result in the inavailability of the application. Finally, in order to achieve good display effects on different devices, developers usually need to perform separate adaptation and debugging for each device. This process not only consumes a large amount of time and manpower but also is difficult to ensure the unity and consistency of the adaptation effects. This strong dependence on specific scenarios greatly limits the generality and adaptability of WebGL models, making it difficult to meet the growing diverse and dynamic actual application requirements.

[0025] In summary, existing WebGL model deployment methods have obvious deficiencies when facing different devices and display modes. These problems not only affect the user experience but also increase the development and maintenance costs.

[0026] To address the above problems, this embodiment provides a WebGL display optimization method for multi-device dynamic adaptation, improving the generality and adaptability of WebGL models and meeting the application requirements under different devices and display modes.

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0028] Please refer to Figure 1The figure shows a flowchart of a WebGL display optimization method for multi-device dynamic adaptation in a specific embodiment. The method includes the following steps: S1. Collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; It should be noted that by collecting the display parameters of the target device and constructing a device display feature vector, it provides the basic data for subsequent dynamic adaptation, ensuring the accuracy and real-time nature of the adaptation; S2. Preprocess the texture images in the WebGL model using an adaptive resolution algorithm; It should be noted that by preprocessing the texture images using an adaptive resolution algorithm, it can dynamically adjust the resolution of the texture images according to the display characteristics of the device, thereby achieving a consistent visual effect on different devices while optimizing performance; S3. Based on the mapping relationship library between the device display features generated by training with historical adaptation data and the WebGL model operation parameters, dynamically adapt the device feature display vector of the target device, obtain the required WebGL model operation parameters, and dynamically adjust the computational graph of the preprocessed WebGL model; It should be noted that through the mapping relationship library generated by training with historical adaptation data, it can quickly and accurately generate the adapted WebGL model operation parameters for new devices, improving the adaptation efficiency and accuracy; S4. Start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and optimize it using the required WebGL model operation parameters to achieve multi-resolution adaptation; It should be noted that by splitting the display content into a fixed layout part and a dynamic content part and optimizing them separately, multi-resolution adaptation is achieved, ensuring the best visual effect on different devices; S5. Real-time monitor the state of the target device. When a display mode switch is detected, pause the rendering task of the WebGL model and return to step S1; It should be noted that by real-time monitoring the device state and dynamically adjusting the rendering parameters, it can ensure that when the display mode switches or the screen refresh rate changes, the display effect and response speed of the WebGL model are not affected, thereby providing a smooth and consistent user experience.

[0029] This embodiment realizes the best visual effect and performance optimization on different devices through collecting device display parameters, preprocessing texture images, dynamically adjusting the computational graph, and hierarchical rendering.

[0030] Further, as a refinement and extension of the specific implementation of the above embodiments, in order to fully illustrate the specific implementation process in this embodiment, another WebGL display optimization method for multi-device dynamic adaptation is provided. The method includes the following steps: S1. Collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; the specific steps of step S1 are as follows: S11. Obtain the display parameters of the target device according to the device interface of the target device type; Exemplarily, on Android system devices, use the display management API provided by the system to periodically poll and obtain the current resolution and refresh rate display parameters of the screen; on iOS devices, obtain the corresponding display parameters through private frameworks or other compliant methods; for computer devices with Windows and Mac systems, obtain the respective display parameter values by accessing the system registry or using the display configuration functions provided by the system; S12. Standardize the collected display parameters and uniformly convert them into a predefined data format and vector to obtain a device display feature vector; Specifically, the device feature vector ; where R represents the resolution, A represents the display ratio, C represents the color space, F represents the refresh rate, and DPR represents the device pixel ratio; It should be noted that the resolution R is used to describe the total number of physical pixels of the screen; the display ratio A is used to control the viewport ratio of the fixed layout layer and is the geometric aspect ratio, such as 16:9, 9:16, or an unconventional ratio; the device pixel ratio DPR is used to dynamically calculate the rendering resolution, such as the actual canvas size = logical size × DPR; the color space C affects color mapping, such as sRGB, P3, NTSC; the refresh rate F is, for example, 60Hz, 120Hz, or a variable refresh rate; S2. Preprocess the texture images in the WebGL model using an adaptive resolution algorithm; the specific steps of step S2 are as follows: S21. Calculate the target rendering size according to the device pixel ratio and resolution of the target device; the specific steps of step S21 are as follows: S211. Obtain the resolution R of the target device and split the resolution R into a logical width Hw and a logical height Hh; S212. The target rendering size H×W is calculated by the following formula: W = Hw×DPR H = Hh×DPR where DPR is the device pixel ratio; S22. Scale the texture image in the WebGL model to the target rendering size using the bicubic interpolation scaling algorithm to obtain an adapted texture object. The specific steps of step S22 are as follows: S221. Obtain the original size Hy×Wy of the texture image in the WebGL model; S222. Calculate the scaling ratio ; S223. Use bicubic interpolation to scale the texture image in the WebGL model according to the scaling ratio ;

[0031] where, is the original texture image, is the scaled texture image, and W is the bicubic weight function; ( x i , y i ) represents the 4×4 neighborhood pixel coordinates in the original texture image around the target position ( x , y ); x i , y i are selected according to the following rules:

[0032]

[0033] where, is the floor function, and i, j = 0, 1, 2, 3 Exemplarily, ; Specifically, the bicubic weight function is as follows:

[0034] S224. Crop or fill the part that exceeds the target rendering size to generate an adapted texture object; S23. Apply an edge-preserving filter to the scaled texture image to eliminate the edge aliasing after scaling; Specifically, the edge-preserving filter template is as follows:

[0035] where, c 1, c 2, c 3 are the filtering coefficients at different positions and are dynamically adjusted according to the edge detection results; S24. If the color space is P3, pre-compute the gamut conversion matrix; S3. Based on the mapping relationship library between the device display characteristics and the WebGL model running parameters generated by training with historical adaptation data, dynamically adapt the device feature display vector of the target device, obtain the required WebGL model running parameters, and dynamically adjust the computation graph of the pre-processed WebGL model; The specific steps of step S3 are as follows: S31. Collect the display parameters of different devices and the corresponding WebGL model running parameters, and construct a historical adaptation data set ; The WebGL model running parameters include LOD level, anti-aliasing level, shader parameters, weight offset, activation function threshold, and computation graph structure adjustment flag;

[0036] Among them, is the display feature vector of the i-th device, is the corresponding adapted WebGL model running parameter; It should be noted that LOD is the level of detail. For example, the LOD level can be divided into three levels: high, medium, and low; The anti-aliasing level MSAA = 2x / 4x / 8x; Shader parameters such as exposure value, gamma value; S32. Construct a feature embedding model and a parameter mapping model, and use the historical adaptation data set to train the feature embedding model through contrast loss, and use the historical adaptation data set to train the parameter mapping model through meta-learning; The specific steps of step S32 are as follows: S321. Use a lightweight neural network to construct a feature embedding model; Exemplarily, use a lightweight neural network of a 4-layer MLP model or a micro Transformer model as the feature embedding model; S322. Perform embedding vector annotation on the historical matching data set, use the display feature vector as the input, and use the embedding vector annotation result as the output, and pre-train the feature embedding model using the contrast loss method:

[0037] Among them, is the contrast loss function, is the display feature vector of the i-th device, is the display feature vector of the j-th device, is the indicator function, when is similar to it is 1, otherwise it is 0; represents the output embedding vector ; S323. Construct a parameter mapping model based on the model-agnostic meta-learning framework, and construct an offline support set and a query set based on the historical adaptation dataset, where the number of samples in the offline support set is less than the number threshold; The specific steps of constructing the offline support set and the query set based on the historical adaptation dataset in step S323 are as follows: Cluster the historical adaptation dataset According to the combination of display parameters; Randomly select M clusters, and for each cluster, select K samples from the historical adaptation dataset To construct the offline support set , and select N samples to construct the query set , where ; Exemplarily, stratify the historical dataset by the device pixel ratio DPR and the resolution R, and randomly select 5 samples as the support set and 20 samples as the query set for each layer to ensure the balanced parameter distribution of each type of device; S324. Use the offline support set to fine-tune the parameter mapping model to obtain the initial parameters of the parameter mapping model, and use the query set to optimize the training of the parameter mapping model to optimize the initial parameters and obtain the optimized parameter mapping model; S33. Use the feature embedding model to map the device feature display vector of the target device into an embedding vector, and then map the embedding vector into the WebGL model running parameters through the parameter mapping model, and save the device feature display vector with a usage frequency higher than the frequency threshold and the WebGL model running parameters to the mapping relationship library; The specific steps of step S33 are as follows: S331. Match the device feature vector of the target device with the mapping relationship library; If the device feature vector is matched, use the corresponding WebGL model running parameters as the WebGL model running parameters matched by the target device, and enter step S34; If the device feature vector cannot be matched, enter step S332; S332. Compare the device feature vector of the target device with the historical adaptation dataset, and screen out the TOP-K samples with the highest similarity as the online support set; In step S332, the device feature vector of the target device Is input into the feature embedding model To obtain the embedding vector ; Calculate the cosine similarity between the embedding vector of the target device And the historical devices in the historical adaptation dataset ;

[0038] Select the TOP-K samples with the highest similarity as historical device samples to construct an offline support set; S333. Update the optimized parameter mapping model based on the online support set to obtain an updated parameter mapping model; S334. Input the device feature vector of the target device into the updated parameter mapping model and output the WebGL model running parameters matched by the target device; S34. Dynamically adjust the computational graph of the WebGL model that has completed texture image preprocessing according to the WebGL model running parameters matched by the target device; The specific steps of step S34 are as follows: S341. Determine whether the WebGL model running parameters matched by the target device contain a weight offset; If so, adjust the convolution kernel weights of the computational graph of the WebGL model according to the channels:

[0039] Among them, represents the original convolution kernel weight, represents the weight offset, which is used to adjust the convolution kernel weight, represents the adjusted convolution kernel weight, represents the scaling factor adjusted according to the device pixel ratio DPR. For example, for high-DPI devices (DPR≥2), the scaling factor can be increased to enhance edge detection; If not, go to step S342; S342. Calibrate the activation function threshold in the WebGL model running parameters according to the color space C; Exemplarily,

[0040] S343. Adjust the computational graph topology of the WebGL model according to the display ratio A; Exemplarily, when the display ratio A is an unconventional ratio (such as 18:9), remove redundant pooling layers to avoid feature map size mismatches, and insert an adaptive spatial transformation layer STN in the computational graph to maintain geometric consistency; It should be noted that the parameters of the inserted spatial transformation layer STN are dynamically calculated by the display ratio A to maintain the geometric ratio of the content; S4. Start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and optimize it using the required WebGL model running parameters to achieve multi-resolution adaptation; The specific steps of step S4 are as follows: S41. Start the rendering task of the WebGL model for the target device and initialize the rendering pipeline: Set the viewport according to the target rendering size, enable the appropriate anti-aliasing level, and perform color space conversion through the color gamut conversion matrix when the color space is P3; S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable; S43. Use a vector graphic form for the fixed layout part, bind the display ratio to the viewport in vector graphic form, export it as a bitmap texture map, and then draw it as a full-screen quadrilateral using the fixed layer fragment shader; S44. Perform supersampling on the dynamic content part. When the device pixel ratio is greater than the pixel ratio threshold, bind the appropriate texture object to the frame buffer object, call the LOD level to render the scene, and then downsample the supersampled texture and output it to the screen; For example, when DPR > 2, bind the texture object to the frame buffer object; S45. Set up and execute the rendering loop for the fixed layout part and the dynamic content part together according to the refresh rate to complete frame rate synchronization; S5. Monitor the status of the target device in real time. When a display mode switch is detected, pause the rendering task of the WebGL model and return to step S1; The specific steps of step S5 are as follows: S51. Monitor changes in the resolution R, display ratio, and device pixel ratio of the target device, and determine whether any of them have changed; If so, enter step S52; If not, wait for a set period of time and return to step S51; S52. Freeze the thread of the current rendering task, re-execute step S1 for the target device to collect display parameters and construct the device feature display vector, and then regenerate new WebGL model running parameters through step S3 to update the computational graph of the WebGL model; S53. Restart the rendering pipeline with the new WebGL model running parameters and continue to execute the rendering task.

[0041] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0042] Such as Figure 2As shown below, the following is an embodiment of a WebGL display optimization system for multi-device dynamic adaptation provided by the embodiments of the present disclosure. This system belongs to the same inventive concept as the multi-device dynamic adaptation WebGL display optimization method in the above embodiments. Details not described in detail in the embodiments of the WebGL display optimization system for multi-device dynamic adaptation can refer to the embodiments of the above multi-device dynamic adaptation WebGL display optimization method.

[0043] The system includes: A device display parameter acquisition module, configured to acquire the display parameters of a target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate. A WebGL model preprocessing module, configured to preprocess the texture images in the WebGL model using an adaptive resolution algorithm. A dynamic adaptation module, configured to dynamically adapt the device feature display vector of the target device based on a mapping relationship library between device display features and WebGL model operation parameters generated by training with historical adaptation data, obtain the required WebGL model operation parameters, and dynamically adjust the computational graph of the preprocessed WebGL model. A hierarchical rendering module, configured to start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, perform dynamic sampling on the dynamic content part based on the device pixel ratio, and optimize it using the required WebGL model operation parameters to achieve multi-resolution adaptation. A status monitoring module, configured to monitor the status of the target device in real time. When a display mode switch is detected, the rendering task of the WebGL model is paused, and device parameter acquisition is restarted.

[0044] Through the interaction and cooperation of the device display parameter acquisition module, the WebGL model preprocessing module, the dynamic adaptation module, the hierarchical rendering module, and the status monitoring module, this embodiment realizes the best visual effect and performance optimization on different devices.

[0045] The WebGL display optimization method for multi-device dynamic adaptation provided by the embodiments of this application can be applied to electronic devices. Those skilled in the art can understand that the structure of the electronic device involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. In the embodiments of the present invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0046] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a key, a camera, a display screen, and a SIM card interface, etc.

[0047] It can be understood that the structure schematically shown in the embodiments of this application does not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0048] The processor may include one or more processing units. For example, the processor may include a central processing unit (CPU), etc., an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0049] Among them, the processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0050] A memory can also be set in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can save the instructions or data that the processor has just used or recycled. If the processor needs to use the instruction or data again, it can directly call it from this memory. This avoids repeated accesses, reduces the waiting time of the processor, and thus improves the system efficiency.

[0051] The above-mentioned electronic device realizes the technical solution of collecting the display parameters of the target device in the WebGL display optimization method for multi-device dynamic adaptation of the present application, constructing a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; preprocessing the texture images in the WebGL model using an adaptive resolution algorithm; dynamically adapting the device feature display vector of the target device based on the mapping relationship library between the device display features generated by training based on historical adaptation data and the WebGL model operation parameters, obtaining the required WebGL model operation parameters, and dynamically adjusting the computational graph of the preprocessed WebGL model; starting the rendering task of the WebGL model on the target device, splitting the display content of the WebGL model into a fixed layout part and a dynamic content part, performing a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and performing dynamic sampling on the dynamic content part based on the device pixel ratio and optimizing it using the required WebGL model operation parameters to achieve multi-resolution adaptation; real-time monitoring the state of the target device, and when a display mode switch is detected, pausing the rendering task of the WebGL model and re-collecting device parameters, achieving the beneficial effect of realizing the best visual effects and performance optimization on different devices.

[0052] In the storage medium provided by the present application, there is a program product capable of realizing the WebGL display optimization method for multi-device dynamic adaptation.

[0053] The WebGL display optimization method for multi-device dynamic adaptation includes: collecting the display parameters of the target device and constructing a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; using an adaptive resolution algorithm to preprocess the texture images in the WebGL model; based on the mapping relationship library between the device display features generated by training with historical adaptation data and the WebGL model running parameters, dynamically adapting the device feature display vector of the target device, obtaining the required WebGL model running parameters, and dynamically adjusting the computational graph of the preprocessed WebGL model; starting the rendering task of the WebGL model on the target device, splitting the display content of the WebGL model into a fixed layout part and a dynamic content part, performing a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and performing dynamic sampling on the dynamic content part based on the device pixel ratio and optimizing it using the required WebGL model running parameters to achieve multi-resolution adaptation; monitoring the state of the target device in real time, and when a display mode switch is detected, pausing the rendering task of the WebGL model and re-collecting the device parameters.

[0054] In some possible implementation manners, the WebGL display optimization method for multi-device dynamic adaptation of the present disclosure may be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section above in this specification.

[0055] The storage medium of the present disclosure may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0056] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A WebGL display optimization method for multi-device dynamic adaptation, characterized in that, It includes the following steps: S1. Collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; S2. Preprocess the texture images in the WebGL model using an adaptive resolution algorithm; S3. Based on the mapping relationship library between the device display features and the WebGL model running parameters generated by training with historical adaptation data, dynamically adapt the device feature display vector of the target device, obtain the required WebGL model running parameters, and dynamically adjust the computational graph of the preprocessed WebGL model; S4. Start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and optimize it using the required WebGL model running parameters to achieve multi-resolution adaptation; S5. Monitor the status of the target device in real time. When a display mode switch is detected, pause the rendering task of the WebGL model and return to step S1.

2. The WebGL display optimization method for multi-device dynamic adaptation according to claim 1, wherein The specific steps of step S1 are as follows: S11. Obtain the display parameters of the target device according to the device interface of the target device type; S12. Standardize the collected display parameters and uniformly convert them into a predefined data format and vector to obtain a device display feature vector; Specifically, the device feature vector ; Among them, R represents the resolution, A represents the display ratio, C represents the color space, F represents the refresh rate, and DPR represents the device pixel ratio.

3. The WebGL display optimization method for multi-device dynamic adaptation according to claim 2, wherein The specific steps of step S2 are as follows: S21. Calculate the target rendering size according to the device pixel ratio and resolution of the target device; S22. Use the bicubic interpolation scaling algorithm to scale the texture images in the WebGL model to the target rendering size to obtain an adapted texture object; S23. Pass the scaled texture images through an edge-preserving filter to eliminate the edge jaggedness after scaling; S24. If the color space is P3, pre-compute the gamut conversion matrix.

4. The WebGL display optimization method for multi-device dynamic adaptation according to claim 3, wherein, The specific steps of step S21 are as follows: S211. Obtain the resolution R of the target device and split the resolution R into a logical width Hw and a logical height Hh; S212. The target rendering size H×W is calculated by the following formula: W = Hw×DPR H = Hh×DPR where DPR is the device pixel ratio; The specific steps of step S22 are as follows: S221. Obtain the original size Hy×Wy of the texture images in the WebGL model; S222. Calculate the scaling ratio ; S223. Scale the texture image in the WebGL model according to the scaling ratio using bicubic interpolation ; Scale the texture image in the WebGL model Among them, is the original texture image, is the scaled texture image, and W is the bicubic weight function; ( x i , y i ) represents the original texture image in the 4×4 neighborhood pixel coordinates around the target position ( x , y ); S224. Crop or fill the part that exceeds the target rendering size to generate an adapted texture object.

5. The method for optimizing WebGL display with multi-device dynamic adaptation according to claim 3, wherein The specific steps of step S3 are as follows: S31. Collect the display parameters of different devices and the corresponding WebGL model running parameters, and construct a historical adaptation data set ; The WebGL model running parameters include LOD level, anti-aliasing level, shader parameters, weight offset, activation function threshold, and computational graph structure adjustment flag; Among them, is the display feature vector of the i-th device, is the running parameter of the corresponding adapted WebGL model; S32. Construct a feature embedding model and a parameter mapping model, and use the historical adaptation dataset to train the feature embedding model through contrastive loss, and use the historical adaptation dataset to train the parameter mapping model through meta-learning; S33. Use the feature embedding model to map the device feature display vector of the target device into an embedding vector, then map the embedding vector into the WebGL model running parameters through the parameter mapping model, and save the device feature display vector with a usage frequency higher than the frequency threshold and the WebGL model running parameters to the mapping relationship library; S34. Dynamically adjust the computational graph of the WebGL model that has completed texture image preprocessing according to the WebGL model running parameters matched by the target device.

6. The method for optimizing WebGL display with multi-device dynamic adaptation according to claim 5, characterized in that, The specific steps of step S32 are as follows: S321. Use a lightweight neural network to construct a feature embedding model; S322. Perform embedding vector annotation on the historical matching dataset. Using the display feature vector as the input and the embedding vector annotation result as the output, pre-train the feature embedding model using the contrast loss method: Among them, is the contrastive loss function, is the display feature vector of the i-th device, is the display feature vector of the j-th device, is the indicator function, which is 1 when is similar to and 0 otherwise; represents the output embedding vector ; S323. Construct a parameter mapping model based on the model-agnostic meta-learning framework, and construct an offline support set and a query set based on the historical adaptation dataset, where the number of samples in the offline support set is less than the number threshold; S324. Use the offline support set to fine-tune and train the parameter mapping model to obtain the initial parameters of the parameter mapping model, and use the query set to optimize and train the parameter mapping model to optimize the initial parameters and obtain the optimized parameter mapping model; The specific steps of step S33 are as follows: S331. Match the device feature vector of the target device with the mapping relationship library; If the device feature vector is matched, use the corresponding WebGL model running parameters as the WebGL model running parameters matched by the target device, and go to step S34; If the device feature vector cannot be matched, go to step S332; S332. Compare the device feature vector of the target device with the historical adaptation dataset, and screen out the TOP-K samples with the highest similarity as the online support set; S333. Update the optimized parameter mapping model based on the online support set to obtain the updated parameter mapping model; S334. Input the device feature vector of the target device into the updated parameter mapping model, and output the WebGL model running parameters matched by the target device; The specific steps of step S34 are as follows: S341. Determine whether the WebGL model running parameters matched by the target device include weight offsets; If so, adjust the convolution kernel weights of the computational graph of the WebGL model according to the channels: Among them, represents the original convolution kernel weight, represents the weight offset, which is used to adjust the convolution kernel weight, represents the adjusted convolution kernel weight, represents the scaling factor adjusted according to the device pixel ratio DPR; If not, go to step S342; S342. Calibrate the activation function threshold in the WebGL model running parameters according to the color space C; S343. Adjust the computational graph topology of the WebGL model according to the display ratio A.

7. The WebGL display optimization method for multi-device dynamic adaptation according to claim 6, characterized in that, The specific steps of constructing the offline support set and the query set based on the historical adaptation dataset in step S323 are as follows: Adapt the historical dataset Cluster according to the combination of display parameters; Randomly select M clusters, and for each cluster, from the historical adaptation dataset Select K samples to construct an offline support set , and select N samples to construct a query set , where ; In step S332, the device feature vector of the target device is input into the feature embedding model , and an embedding vector is obtained; Calculate the embedding vector of the target device with the historical adaptation dataset the cosine similarity of the historical devices in it; Screen out the TOP-K samples with the highest similarity as the historical device samples to construct the offline support set.

8. The WebGL display optimization method for multi-device dynamic adaptation according to claim 5, wherein The specific steps of step S4 are as follows: S41. Start the rendering task of the WebGL model for the target device and perform rendering pipeline initialization: Set the viewport according to the target rendering size, enable the adapted anti-aliasing level, and perform color space conversion through the color gamut conversion matrix when the color space is P3; S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable; S43. For the fixed layout part, use the vector graph form, bind the display ratio to the viewport in the vector graph form, export it as a bitmap texture map, and then use the fixed layer fragment shader to draw it as a full-screen quadrilateral; S44. Perform supersampling on the dynamic content part. When the device pixel ratio is greater than the pixel ratio threshold, bind the adapted texture object to the frame buffer object, call the LOD level to render the scene, and then downsample the supersampled texture and output it to the screen; S45. Set up a rendering loop for the fixed layout part and the dynamic content part together according to the refresh rate and execute it to complete frame rate synchronization.

9. The method for optimizing WebGL display with multi-device dynamic adaptation according to claim 8, wherein The specific steps of step S5 are as follows: S51. Monitor the changes in the resolution R, display ratio, and device pixel ratio of the target device, and determine whether any one of them has changed; If so, go to step S52; If not, wait for a set period of time and return to step S51; S52. Freeze the thread of the current rendering task, re-perform step S1 on the target device to collect display parameters and construct a device feature display vector, and then regenerate new WebGL model running parameters through step S3 to update the computational graph of the WebGL model; S53. Restart the rendering pipeline with the new WebGL model running parameters and continue to execute the rendering task.

10. A WebGL display optimization system for multi-device dynamic adaptation, characterized in that, Including: A device display parameter collection module for collecting the display parameters of the target device and constructing a device display feature vector; The display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; A WebGL model preprocessing module for preprocessing the texture images in the WebGL model using an adaptive resolution algorithm; A dynamic adaptation module for dynamically adapting the device feature display vector of the target device based on the mapping relationship library between the device display features and the WebGL model running parameters generated by training with historical adaptation data, obtaining the required WebGL model running parameters, and dynamically adjusting the computational graph of the preprocessed WebGL model; A hierarchical rendering module for starting the rendering task of the WebGL model on the target device, splitting the display content of the WebGL model into a fixed layout part and a dynamic content part, performing a vector graphics rendering algorithm on the fixed layout part based on the resolution and display ratio, dynamically sampling the dynamic content part based on the device pixel ratio, and optimizing it using the required WebGL model running parameters to achieve multi-resolution adaptation; A status monitoring module for real-time monitoring of the status of the target device. When a display mode switch is detected, pause the rendering task of the WebGL model and re-collect device parameters.

Images