A multi-device dynamic adaptation WebGL display optimization method and system

Abstract

The application provides a multi-device dynamic adaptation WebGL display optimization method and system, and belongs to the technical field of WebGL display. The method comprises the following steps: collecting display parameters of a target device to construct a device display characteristic vector; performing preprocessing on a texture image in a WebGL model; training and generating a mapping relationship library based on historical adaptation data, dynamically adapting the device characteristic display vector of the target device, obtaining required WebGL model running parameters, and dynamically adjusting a calculation graph of the preprocessed WebGL model; executing a vector graphics rendering algorithm on a fixed layout of the WebGL model, dynamically sampling and optimizing dynamic content of the WebGL model, and realizing multi-resolution adaptation; and monitoring the state of the target device in real time, pausing the rendering task of the WebGL model when detecting display mode switching, and re-collecting device parameters. The application realizes optimal visual effects and performance optimization on different devices.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of WebGL display, and particularly relates to a WebGL display optimization method and system for multi-device dynamic adaptation. BACKGROUND

[0002] With the development of the Internet, WebGL (Web Graphics Library) is widely used as a technology for rendering high-performance 3D graphics in web browsers. WebGL is based on the OpenGL ES 2.0 specification and allows developers to create and manipulate 3D graphics in browsers through JavaScript without the need to install any plug-ins. However, the existing WebGL model deployment method has the following limitations:

[0003] Firstly, traditional WebGL model deployment is usually optimized for a fixed resolution display screen and a specific display mode. When deployed on other display devices, due to differences in screen size, pixel density, color display characteristics, etc., the original display effect will be greatly compromised, with problems such as blurred pictures, proportion misadjustment, and color distortion. Secondly, when the display mode changes, such as switching from landscape to portrait or adjusting the screen refresh rate, the traditional WebGL model function may be severely affected, with problems such as operation response delay, interface layout error, and even program crash. Finally, in order to achieve good display effect on different devices, developers need to perform separate adaptation debugging for each device, which not only consumes a lot of time and manpower, but also makes it difficult to ensure the uniformity of the adaptation effect.

[0004] In summary, the strong dependence on specific scenarios greatly limits the universality and adaptability of WebGL models, and cannot meet the diversified and dynamic actual application requirements. SUMMARY

[0005] In a first aspect, the embodiments of the application provide a WebGL display optimization method for multi-device dynamic adaptation, including the following steps:

[0006] S1. Collecting display parameters of a target device to construct a device display feature vector; the display parameters include resolution, display proportion, device pixel ratio, color space, and refresh rate;

[0007] S2. Preprocessing texture images in the WebGL model using an adaptive resolution algorithm;

[0008] S3. Training a mapping relationship library between the generated device display features and WebGL model running parameters based on historical adaptation data, dynamically adapting the device feature display vector of the target device, obtaining the required WebGL model running parameters, and dynamically adjusting the computation graph of the preprocessed WebGL model.

[0009] S4. In the target device, start the rendering task of the WebGL model, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display scale, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and optimization using the required WebGL model running parameters, to achieve multi-resolution adaptation;

[0010] S5. Real-time monitor the state of the target device, and when detecting display mode switching, pause the rendering task of the WebGL model, and return to step S1.

[0011] Further, the step S1 is specifically as follows:

[0012] S11. Obtain the display parameters of the target device according to the device interface of the target device type;

[0013] S12. Standardize the collected display parameters and uniformly convert them into a predefined data format and vector to obtain a device display feature vector;

[0014] Specifically, the device feature vector ;

[0015] Wherein, R represents resolution, A represents display scale, C represents color space, F represents refresh rate, and DPR represents device pixel ratio.

[0016] Further, the step S2 is specifically as follows:

[0017] S21. Calculate the target rendering size according to the device pixel ratio and resolution of the target device;

[0018] S22. Use a bicubic interpolation scaling algorithm to scale the texture image in the WebGL model to the target rendering size to obtain an adapted texture object;

[0019] S23. Eliminate the edge jaggies of the scaled texture image through an edge-preserving filter;

[0020] S24. If the color space is P3, pre-calculate the gamut conversion matrix.

[0021] Further, the step S21 is specifically as follows:

[0022] S211. Obtain the resolution R of the target device, and split the resolution R into a logical width Hw and a logical height Hh;

[0023] S212. The target rendering size HxW is calculated by the following formula:

[0024] W=Hw×DPR

[0025] H = Hh x DPR

[0026] wherein DPR is device pixel ratio;

[0027] The step S22 includes the following steps:

[0028] S221. Obtain the original size Hy x Wy of the texture image in the WebGL model;

[0029] S222. Calculate the scaling ratio ;

[0030] S223. Scale the texture image in the WebGL model by the scaling ratio using bicubic interpolation;

[0031]

[0032] wherein, is the original texture image, is the scaled texture image, and W is a bicubic weight function; x i , y i represents the 4 x 4 neighborhood pixel coordinates of the original texture image around the target position x , y ;

[0033] S224. Crop or fill the part exceeding the target rendering size to generate an adapted texture object.

[0034] Further, the step S3 includes the following steps:

[0035] S31. Collect the display parameters of different devices and the corresponding WebGL model running parameters to construct a historical adaptation dataset ; the WebGL model running parameters include LOD level, anti-aliasing level, shader parameter, weight offset, activation function threshold, and calculation graph structure adjustment flag;

[0036]

[0037] wherein, is the display feature vector of the i-th device, is the corresponding adapted WebGL model running parameter;

[0038] S32. Constructing a feature embedding model and a parameter mapping model, and training the feature embedding model by contrastive loss using a historical fitting dataset, and training the parameter mapping model by meta-learning using the historical fitting dataset;

[0039] S33. Mapping the device feature display vector of the target device into an embedding vector using the feature embedding model, and mapping the embedding vector into the WebGL model running parameter using the parameter mapping model, and saving the device feature display vector with a frequency higher than a frequency threshold and the WebGL model running parameter to a mapping relationship library;

[0040] S34. Dynamically adjusting the calculation graph of the WebGL model with preprocessed texture images according to the WebGL model running parameter matched by the target device.

[0041] Further, the specific steps of step S32 are as follows:

[0042] S321. Constructing a feature embedding model using a lightweight neural network;

[0043] S322. Embedding vector annotation is performed on a historical matching dataset, the display feature vector is used as input, and the embedding vector annotation result is used as output, and the feature embedding model is pre-trained using a contrastive loss method:

[0044]

[0045] wherein, is a contrastive loss function, is the display feature vector of the i-th device, is the display feature vector of the j-th device, is an indicator function, when is 1, otherwise 0; represents the output embedding vector ;

[0046] S323. Constructing a parameter mapping model based on a model-independent meta-learning framework, constructing an offline support set and a query set based on a historical fitting dataset, wherein the number of samples of the offline support set is less than a quantity threshold;

[0047] S324. Fine-tuning the parameter mapping model using the offline support set to obtain initial parameters of the parameter mapping model, and optimizing the training of the parameter mapping model using the query set to optimize the initial parameters to obtain an optimized parameter mapping model;

[0048] The specific steps of step S33 are as follows:

[0049] S331. Matching the device feature vector of the target device with the mapping relationship library;​

[0050] If the device feature vector is matched, the corresponding WebGL model running parameters are taken as the WebGL model running parameters matched by the target device, and step S34 is entered;

[0051] If the device feature vector cannot be matched, step S332 is entered;

[0052] S332. The device feature vector of the target device is compared with the historical adaptation dataset, and the TOP-K samples with the highest similarity are selected as the online support set;

[0053] S333. The updated parameter mapping model is obtained by updating the optimized parameter mapping model based on the online support set;

[0054] S334. The device feature vector of the target device is input into the updated parameter mapping model, and the WebGL model running parameters matched by the target device are output;

[0055] Step S34 includes the following specific steps:

[0056] S341. It is judged whether the WebGL model running parameters matched by the target device contain a weight offset;

[0057] If yes, the convolution kernel weight of the calculation graph of the WebGL model is adjusted according to the channel:

[0058]

[0059] wherein, represents the original convolution kernel weight, represents the weight offset, which is used to adjust the convolution kernel weight, represents the adjusted convolution kernel weight, represents a scaling factor adjusted according to the device pixel ratio DPR;

[0060] If no, step S342 is entered;

[0061] S342. The activation function threshold in the WebGL model running parameters is calibrated according to the color space C;

[0062] S343. The calculation graph topology of the WebGL model is adjusted according to the display ratio A.

[0063] Further, the specific steps of constructing the offline support set and the query set based on the historical adaptation dataset in step S323 are as follows:

[0064] The historical adaptation dataset is clustered according to the combination of display parameters;

[0065] Randomly select M clusters, and for each cluster, select K samples from the historical adaptation dataset Select K samples to construct an offline support set , and select N samples to construct a query set , wherein ;

[0066] In step S332, the device feature vector of the target device is input into the feature embedding model , to obtain an embedding vector ; ;

[0067] The embedding vector of the target device is calculated , and the cosine similarity between the embedding vector and the historical device in the historical adaptation dataset is calculated.

[0068]

[0069] The TOP-K samples with the highest similarity are selected as the historical device samples to construct the offline support set.

[0070] Further, the specific steps of step S4 are as follows:

[0071] S41. Start the rendering task of the WebGL model for the target device, and perform rendering pipeline initialization:

[0072] Set the viewport according to the target rendering size, enable the adaptive anti-aliasing level, and perform color space conversion through the color gamut conversion matrix when the color space is P3;

[0073] S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable;

[0074] S43. Use a vector graph form for the fixed layout part, bind the display ratio to the window of the vector graph form, and export it as a bitmap texture map, and then use a fixed layer fragment shader to draw it as a full-screen quadrilateral;

[0075] S44. Perform supersampling on the dynamic content part, bind the adaptive texture object to the frame buffer object when the device pixel ratio is greater than the pixel ratio threshold, call the LOD level to render the scene, and then downsample the supersampled texture and output it to the screen;

[0076] S45. Render the fixed layout part and the dynamic content part together according to the refresh rate setting, and perform rendering loop to complete frame rate synchronization.

[0077] Further, the specific steps of step S5 are as follows:

[0078] S51. Monitor the changes of the target device resolution R, display scale and device pixel ratio, and determine whether any of them has changed;

[0079] If yes, go to step S52;

[0080] If no, wait for a set period of time and return to step S51;

[0081] S52. Freeze the thread of the current rendering task, re-collect the display parameters of the target device according to step S1, construct the device feature display vector, and re-generate the new WebGL model running parameters according to step S3, and update the calculation graph of the WebGL model;

[0082] S53. Restart the rendering pipeline using the new WebGL model running parameters and continue to execute the rendering task.

[0083] In a second aspect, the embodiments of the present application also provide a multi-device dynamic adaptation WebGL display optimization system, comprising:

[0084] A device display parameter collection module is configured to collect the display parameters of the target device and construct a device display feature vector; the display parameters include resolution, display scale, device pixel ratio, color space and refresh rate;

[0085] A WebGL model preprocessing module is configured to preprocess the texture images in the WebGL model using an adaptive resolution algorithm;

[0086] A dynamic adaptation module is configured to dynamically adapt the device feature display vector of the target device based on the mapping relationship library between the device display features and the WebGL model running parameters trained and generated by the historical adaptation data, obtain the required WebGL model running parameters, and dynamically adjust the calculation graph of the preprocessed WebGL model;

[0087] A hierarchical rendering module is configured to start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on the resolution and display scale, perform dynamic sampling on the dynamic content part based on the device pixel ratio, and optimize using the required WebGL model running parameters to achieve multi-resolution adaptation;

[0088] A state monitoring module is configured to monitor the state of the target device in real time, pause the rendering task of the WebGL model when a display mode switch is detected, and re-collect the device parameters.

[0089] As can be seen from the above technical solutions, the present application has the following advantages:

[0090] The multi-device dynamic adaptation WebGL display optimization method and system provided by the application can automatically adjust the rendering parameters of the WebGL model by dynamically adapting the display parameters of the target device, thereby realizing consistent display effects on different devices and improving the universality and adaptability of the WebGL model; by using the adaptive resolution algorithm and dynamic sampling, the rendering resolution and sampling rate can be dynamically adjusted according to the performance and display characteristics of the device, thereby significantly reducing the rendering load while ensuring the visual effect and improving the performance; by monitoring the device state in real time and dynamically adjusting the rendering parameters, the display effect and response speed of the WebGL model can be ensured when the display mode is switched or the screen refresh rate is changed, thereby providing smooth and consistent user experience; the application reduces the development and maintenance cost and improves the development efficiency by automatic adaptation and dynamic adjustment. BRIEF DESCRIPTION OF DRAWINGS

[0091] In order to more clearly illustrate the technical solutions of the application, the drawings needed to be used in the description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0092] Figure 1 The flowchart of the multi-device dynamic adaptation WebGL display optimization method of the application.

[0093] Figure 2 The schematic diagram of the multi-device dynamic adaptation WebGL display optimization system of the application. DETAILED DESCRIPTION

[0094] In the following detailed description of the specific steps of the multi-device dynamic adaptation WebGL display optimization method, various embodiments of the present disclosure will be described more fully. The present disclosure can have various embodiments, and adjustments and changes can be made therein. However, it should be understood that there is no intention to limit various embodiments of the present disclosure to the specific embodiments disclosed herein, but the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives falling within the spirit and scope of various embodiments of the present disclosure.

[0095] Illustratively, with the continuous progress of Internet technology, WebGL (Web Graphics Library) has been widely used as a technology for rendering high-performance 3D graphics in web browsers. WebGL is based on the OpenGLES 2.0 specification, allowing developers to create and manipulate 3D graphics in the browser through JavaScript without the need to install any additional plugins. However, the existing WebGL model deployment method has some significant limitations, mainly reflected in the following aspects:

[0096] Firstly, the traditional WebGL model deployment method is usually optimized for a specific fixed resolution display screen and a specific display mode. When these models are deployed to other display devices with different screen sizes, pixel densities, and color display characteristics, the original display effect is often severely affected. Specifically, there are problems such as blurred pictures, proportion misadjustment, and color distortion, which seriously affect the user experience. Secondly, when the display mode changes, such as switching from landscape to portrait or adjusting the screen refresh rate, the traditional WebGL model function may be severely affected. These problems may manifest as operation response delay, interface layout errors, or even program crashes. These problems not only affect the user's operation experience, but also may lead to application unavailability. Finally, in order to achieve good display effect on different devices, developers usually need to perform separate adaptation and debugging for each device. This process not only consumes a lot of time and manpower, but also is difficult to guarantee the uniformity and consistency of the adaptation effect. This strong dependence on specific scenarios greatly limits the universality and adaptability of WebGL models, making it difficult to meet the growing demand for diversification and dynamicity of practical applications.

[0097] In summary, the existing WebGL model deployment method has obvious shortcomings when facing different devices and display modes. These problems not only affect the user experience, but also increase the cost of development and maintenance.

[0098] To address the above problems, the present embodiment provides a multi-device dynamic adaptation WebGL display optimization method to improve the universality and adaptability of WebGL models and meet the application requirements under different devices and display modes.

[0099] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0100] Please refer toFigure 1 Fig. 1 shows a flowchart of a method for dynamically adapting WebGL display optimization in a multi-device embodiment, the method comprising the following steps:

[0101] S1. Collect display parameters of the target device to construct a device display feature vector; the display parameters include resolution, display scale, device pixel ratio, color space, and refresh rate;

[0102] It should be noted that by collecting the display parameters of the target device to construct a device display feature vector, the method provides basic data for subsequent dynamic adaptation, ensuring the accuracy and real-time performance of the adaptation;

[0103] S2. Preprocess the texture images in the WebGL model using an adaptive resolution algorithm;

[0104] It should be noted that by preprocessing the texture images using an adaptive resolution algorithm, the method can dynamically adjust the resolution of the texture images according to the display characteristics of the device, thereby achieving consistent visual effects on different devices while optimizing performance;

[0105] S3. Based on the mapping relationship library generated by training the historical adaptation data between the device display features and the WebGL model running parameters, dynamically adapt the device feature display vector of the target device to obtain the required WebGL model running parameters, and dynamically adjust the computation graph of the preprocessed WebGL model;

[0106] It should be noted that the mapping relationship library generated by training the historical adaptation data can quickly and accurately generate adapted WebGL model running parameters for new devices, improving the efficiency and accuracy of the adaptation;

[0107] S4. Start the rendering task of the WebGL model on the target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform vector graphics rendering algorithm on the fixed layout part based on resolution and display scale, and perform dynamic sampling on the dynamic content part based on the device pixel ratio and use the required WebGL model running parameters for optimization, achieving multi-resolution adaptation;

[0108] It should be noted that by splitting the display content into a fixed layout part and a dynamic content part and optimizing them separately, the method achieves multi-resolution adaptation, ensuring the best visual effect on different devices;

[0109] S5. Real-time monitoring of the status of the target device, when the display mode is detected to be switched, the rendering task of the WebGL model is paused, and the method returns to step S1;

[0110] It should be noted that by monitoring the device state in real time and dynamically adjusting the rendering parameters, the display effect and response speed of the WebGL model can be ensured not to be affected when the display mode is switched or the screen refresh rate is changed, thereby providing a smooth and consistent user experience.

[0111] The embodiment realizes the best visual effect and performance optimization on different devices by collecting device display parameters, preprocessing texture images, dynamically adjusting calculation graphs and layered rendering.

[0112] Further, as a refinement and extension of the above embodiment, in order to fully describe the specific implementation process in the embodiment, another multi-device dynamic adaptation WebGL display optimization method is provided, which comprises the following steps:

[0113] S1. Collect the display parameters of the target device to construct a device display feature vector; the display parameters include resolution, display scale, device pixel ratio, color space and refresh rate; the specific steps of step S1 are as follows:

[0114] S11. Obtain the display parameters of the target device according to the device interface of the target device type;

[0115] Exemplarily, on an Android system device, the current resolution and refresh rate display parameters of the screen are obtained by regularly polling using the display management API provided by the system; on an iOS device, the corresponding display parameters are obtained through a private framework or other compliance methods; for computer devices of Windows and Mac systems, the display parameter values are obtained by accessing the system registry or using the display configuration function provided by the system;

[0116] S12. Standardize the collected display parameters and uniformly convert them into a predefined data format and vector to obtain a device display feature vector;

[0117] Specifically, the device feature vector ;

[0118] Wherein, R represents resolution, A represents display scale, C represents color space, F represents refresh rate, and DPR represents device pixel ratio;

[0119] It should be noted that the resolution R is used to describe the total number of physical pixels of the screen; the display scale A is used for viewport scale control of fixed layout layers, which is a geometric aspect ratio, such as 16:9, 9:16 or an irregular scale; the device pixel ratio DPR is used to dynamically calculate the rendering resolution, such as actual canvas size = logical size x DPR; the color space C affects color mapping, such as sRGB, P3, NTSC; the refresh rate F is, for example, 60Hz, 120Hz or a variable refresh rate;

[0120] S2. Preprocess the texture image in the WebGL model by using an adaptive resolution algorithm; the specific steps of step S2 are as follows:

[0121] S21. Calculate the target rendering size according to the device pixel ratio and the resolution of the target device; the specific steps of step S21 are as follows:

[0122] S211. Obtain the resolution R of the target device, and split the resolution R into a logical width Hw and a logical height Hh;

[0123] S212. The target rendering size HxW is calculated by the following formula:

[0124] W = Hw x DPR

[0125] H = Hh x DPR

[0126] wherein DPR is the device pixel ratio;

[0127] S22. Scale the texture image in the WebGL model to the target rendering size by using a bicubic interpolation scaling algorithm to obtain an adapted texture object; the specific steps of step S22 are as follows:

[0128] S221. Obtain the original size HyxWy of the texture image in the WebGL model;

[0129] S222. Calculate the scaling ratio ;

[0130] S223. Scale the texture image in the WebGL model by the scaling ratio by using bicubic interpolation;

[0131]

[0132] wherein, is the original texture image, is the scaled texture image, and W is a bicubic weight function; x i , y i indicate 4x4 neighborhood pixel coordinates of the original texture image around the target position x , y ;

[0133] x i , y i are selected according to the following rules:

[0134]

[0135]

[0136] wherein, is the floor symbol, i, j = 0, 1, 2, 3

[0137] Exemplarily, ;

[0138] Specifically, the bicubic weight function is as follows:

[0139]

[0140] S224. Cropping or padding the part exceeding the target rendering size to generate an adaptive texture object;

[0141] S23. Eliminating the edge jaggies of the scaled texture image through an edge-preserving filter;

[0142] Specifically, the edge-preserving filter template is as follows:

[0143]

[0144] wherein, c 1, c 2, c 3 are filter coefficients at different positions, which are dynamically adjusted according to the edge detection result;

[0145] S24. If the color space is P3, a color gamut conversion matrix is pre-calculated;

[0146] S3. Based on the mapping relationship library of the generated device display characteristics and the WebGL model running parameters, the device characteristic display vector of the target device is dynamically adapted, the required WebGL model running parameters are obtained, and the calculation graph of the preprocessed WebGL model is dynamically adjusted; the specific steps of step S3 are as follows:

[0147] S31. Collecting display parameters of different devices and corresponding WebGL model running parameters to construct a historical adaptation data set ; the WebGL model running parameters include LOD level, anti-aliasing level, shader parameter, weight offset, activation function threshold and calculation graph structure adjustment mark;

[0148]

[0149] wherein, is the display characteristic vector of the i-th device, is the corresponding adaptive WebGL model running parameter;

[0150] It should be noted that LOD is a level of detail, for example, LOD levels can be divided into three levels of high, medium and low; Anti-aliasing levels MSAA = 2x / 4x / 8x; Shader parameters such as exposure value, gamma value;

[0151] S32. Construct a feature embedding model and a parameter mapping model, and train the feature embedding model using a historical adaptation dataset through a contrastive loss, and train the parameter mapping model using the historical adaptation dataset through meta-learning; The specific steps of step S32 are as follows:

[0152] S321. Construct a feature embedding model using a lightweight neural network;

[0153] Exemplarily, a 4-layer MLP model or a lightweight neural network of a micro Transformor model is used as the feature embedding model;

[0154] S322. Embedding vector labeling is performed on the historical matching dataset, the feature vector is used as input, and the embedding vector labeling result is used as output, and the feature embedding model is pre-trained using a contrastive loss method:

[0155]

[0156] Wherein, is a contrastive loss function, is the display feature vector of the i-th device, is the display feature vector of the j-th device, is an indicator function, when is 1, otherwise 0; represents the output embedding vector ;

[0157] S323. Construct a parameter mapping model based on a model-independent meta-learning framework, and construct an offline support set and a query set based on a historical adaptation dataset, wherein the number of samples of the offline support set is less than a number threshold;

[0158] The specific steps of constructing the offline support set and the query set based on the historical adaptation dataset in step S323 are as follows:

[0159] The historical adaptation dataset is clustered according to the combination of display parameters;

[0160] M clusters are randomly selected, and K samples are selected from the historical adaptation dataset for each cluster to construct an offline support set , and N samples are selected to construct a query set , wherein ; ​

[0161] Exemplarily, the historical data set is layered by device pixel ratio DPR and resolution R, 5 samples are randomly selected as a support set and 20 samples are randomly selected as a query set in each layer, and the parameter distribution of each type of device is ensured to be balanced;

[0162] S324. Fine-tuning training is performed on the parameter mapping model using the offline support set to obtain initial parameters of the parameter mapping model, and optimization training is performed on the parameter mapping model using the query set to optimize the initial parameters and obtain an optimized parameter mapping model;

[0163] S33. The device feature display vector of the target device is mapped into an embedding vector using the feature embedding model, and then the embedding vector is mapped into the WebGL model running parameter through the parameter mapping model, and the device feature display vector with a frequency higher than the frequency threshold and the WebGL model running parameter are saved to the mapping relationship library; step S33 comprises the following steps:

[0164] S331. The device feature vector of the target device is matched with the mapping relationship library;

[0165] If the device feature vector is matched, the corresponding WebGL model running parameter is taken as the WebGL model running parameter matched by the target device, and step S34 is entered;

[0166] If the device feature vector cannot be matched, step S332 is entered;

[0167] S332. The device feature vector of the target device is compared with the historical adaptation data set, and the TOP-K samples with the highest similarity are selected as an online support set;

[0168] In step S332, the device feature vector of the target device Input the feature embedding model to obtain an embedding vector ;

[0169] The cosine similarity of the embedding vector of the target device and the historical device in the historical adaptation data set is calculated;

[0170]

[0171] The TOP-K samples with the highest similarity are selected as historical device samples to construct an offline support set;

[0172] S333. The optimized parameter mapping model is updated based on the online support set to obtain an updated parameter mapping model;

[0173] S334. input the device feature vector of the target device into the updated parameter mapping model, and output the WebGL model running parameters matched by the target device;

[0174] S34. dynamically adjust the computational graph of the WebGL model with the preprocessed texture image according to the WebGL model running parameters matched by the target device; the specific steps of step S34 are as follows:

[0175] S341. determine whether the WebGL model running parameters matched by the target device contain a weight offset;

[0176] If yes, adjust the convolution kernel weight of the computational graph of the WebGL model according to the channel:

[0177]

[0178] wherein, represents the original convolution kernel weight, represents the weight offset, used for adjusting the convolution kernel weight, represents the adjusted convolution kernel weight, represents a scaling factor adjusted according to the device pixel ratio DPR, for example, for a high DPI device (DPR≥2), the scaling factor can be increased to enhance edge detection;

[0179] If no, go to step S342;

[0180] S342. calibrate the activation function threshold in the WebGL model running parameters according to the color space C;

[0181] Exemplarily,

[0182] S343. adjust the computational graph topology of the WebGL model according to the display ratio A;

[0183] Exemplarily, when the display ratio A is an irregular ratio (such as 18:9), the redundant pooling layer is removed to avoid mismatch of the feature map size, and an adaptive spatial transformation layer STN is inserted in the computational graph to maintain geometric consistency;

[0184] It should be noted that the parameters of the inserted spatial transformation layer STN are dynamically calculated by the display ratio A to maintain the content geometric ratio;

[0185] S4. In the target device, start the rendering task of the WebGL model, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on resolution and display scale, perform dynamic sampling on the dynamic content part based on device pixel ratio and optimization using required WebGL model running parameters, and achieve multi-resolution adaptation; the specific steps of step S4 are as follows:

[0186] S41. Start the rendering task of the WebGL model for the target device, and perform rendering pipeline initialization:

[0187] Set the viewport according to the target rendering size, enable the adaptive anti-aliasing level, and perform color space conversion through a color gamut conversion matrix when the color space is P3;

[0188] S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable;

[0189] S43. Use a vector graph form for the fixed layout part, bind the display scale to the vector graph form window, export it as a bitmap texture map, and then draw it into a full-screen quadrilateral using a fixed layer fragment shader;

[0190] S44. Perform oversampling on the dynamic content part, bind the adaptive texture object to the frame buffer object when the device pixel ratio is greater than the pixel ratio threshold, call the LOD level to render the scene, and then output the oversampled texture to the screen after downsampling;

[0191] For example, when DPR>2, bind the texture object to the frame buffer object;

[0192] S45. Set and execute the rendering loop according to the refresh rate for the fixed layout part and the dynamic content part together, and complete the frame rate synchronization;

[0193] S5. Monitor the state of the target device in real time, and when a display mode switch is detected, pause the rendering task of the WebGL model and return to step S1; the specific steps of step S5 are as follows:

[0194] S51. Monitor the changes in the resolution R, display scale, and device pixel ratio of the target device, and determine whether any of them has changed;

[0195] If yes, go to step S52;

[0196] If no, wait for a set period of time and return to step S51;

[0197] S52. Freeze the thread of the current rendering task, perform display parameter collection of step S1 again on the target device, construct the device feature display vector, and regenerate the new WebGL model running parameter through step S3 again to update the calculation graph of the WebGL model;

[0198] S53. Restart the rendering pipeline using the new WebGL model running parameter, and continue to execute the rendering task.

[0199] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0200] As Figure 2 shown below is an embodiment of a multi-device dynamic adaptation WebGL display optimization system provided by the embodiment of the present disclosure. The system and the multi-device dynamic adaptation WebGL display optimization method described above belong to the same inventive concept. Details not described in the embodiment of the multi-device dynamic adaptation WebGL display optimization system can be referred to the embodiment of the multi-device dynamic adaptation WebGL display optimization method described above.

[0201] The system comprises:

[0202] A device display parameter collection module is configured to collect display parameters of a target device and construct a device display feature vector. The display parameters include resolution, display scale, device pixel ratio, color space, and refresh rate.

[0203] A WebGL model preprocessing module is configured to preprocess texture images in a WebGL model using an adaptive resolution algorithm.

[0204] A dynamic adaptation module is configured to dynamically adapt the device feature display vector of the target device based on a mapping relationship library between the device display feature trained and generated by historical adaptation data and WebGL model running parameters, obtain the required WebGL model running parameters, and dynamically adjust the calculation graph of the preprocessed WebGL model.

[0205] A hierarchical rendering module is configured to start a rendering task of a WebGL model on a target device, split the display content of the WebGL model into a fixed layout part and a dynamic content part, perform a vector graphics rendering algorithm on the fixed layout part based on resolution and display scale, dynamically sample the dynamic content part based on a device pixel ratio, and optimize using the required WebGL model running parameter to achieve multi-resolution adaptation.

[0206] The state monitoring module is configured to monitor the state of the target device in real time, pause the rendering task of the WebGL model when the display mode switching is detected, and re-collect the device parameters.

[0207] The embodiment realizes the best visual effect and performance optimization on different devices through the interaction and cooperation of the device display parameter collection module, the WebGL model preprocessing module, the dynamic adaptation module, the layered rendering module, and the state monitoring module.

[0208] The multi-device dynamic adaptation WebGL display optimization method provided by the embodiment of the application can be applied to an electronic device. Those skilled in the art can understand that the electronic device structure involved in the embodiment of the application does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the illustration, or combine certain components, or different component arrangements. In the embodiment of the application, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the application described and / or claimed herein.

[0209] The electronic device can include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, a key, a camera, a display screen, and a SIM card interface, etc.

[0210] It can be understood that the structure illustrated in the embodiment of the application does not constitute a specific limitation on the electronic device. In other embodiments of the application, the electronic device can include more or fewer components than the illustration, or combine certain components, or split certain components, or different component arrangements. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0211] The processor can include one or more processing units, such as: the processor can include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video code, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices or integrated in one or more processors.

[0212] The processor can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0213] The processor can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory can save instructions or data that the processor has just used or repeatedly uses. If the processor needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor, thereby improving system efficiency.

[0214] The electronic device realizes the collection of display parameters of a target device in the multi-device dynamic adaptation WebGL display optimization method, and constructs a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; an adaptive resolution algorithm is used to preprocess texture images in a WebGL model; a mapping relationship library of device display features and WebGL model running parameters generated based on historical adaptation data is trained, a device feature display vector of the target device is dynamically adapted, WebGL model running parameters required are obtained, a calculation graph of the preprocessed WebGL model is dynamically adjusted; when the target device starts a rendering task of the WebGL model, the display content of the WebGL model is split into a fixed layout part and a dynamic content part, a vector graphics rendering algorithm is executed based on resolution and display ratio for the fixed layout part, dynamic sampling is performed based on the device pixel ratio for the dynamic content part, and optimization is performed using the required WebGL model running parameters, multi-resolution adaptation is realized; the state of the target device is monitored in real time, and when display mode switching is detected, the rendering task of the WebGL model is paused, and the technical scheme of device parameter collection is performed again, thereby achieving the beneficial effects of realizing the best visual effect and performance optimization on different devices.

[0215] In the storage medium provided in the application, a program product capable of realizing the multi-device dynamic adaptation WebGL display optimization method is stored.

[0216] The multi-device dynamic adaptation WebGL display optimization method comprises: collecting display parameters of a target device, and constructing a device display feature vector; the display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; an adaptive resolution algorithm is used to preprocess texture images in a WebGL model; a mapping relationship library of device display features and WebGL model running parameters generated based on historical adaptation data is trained, a device feature display vector of the target device is dynamically adapted, WebGL model running parameters required are obtained, a calculation graph of the preprocessed WebGL model is dynamically adjusted; when the target device starts a rendering task of the WebGL model, the display content of the WebGL model is split into a fixed layout part and a dynamic content part, a vector graphics rendering algorithm is executed based on resolution and display ratio for the fixed layout part, dynamic sampling is performed based on the device pixel ratio for the dynamic content part, and optimization is performed using the required WebGL model running parameters, multi-resolution adaptation is realized; the state of the target device is monitored in real time, and when display mode switching is detected, the rendering task of the WebGL model is paused, and the technical scheme of device parameter collection is performed again, thereby achieving the beneficial effects of realizing the best visual effect and performance optimization on different devices.

[0217] In some possible implementation manners, the multi-device dynamic adaptation WebGL display optimization method of the present disclosure can be implemented in the form of a program product, which includes program codes for causing an end device to perform the steps described in the above "Exemplary Method" section according to various exemplary embodiments of the present disclosure when the program product is run on the end device.

[0218] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0219] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for multi-device dynamic adaptation of WebGL display optimization, the method comprising: Comprising the following steps: S1. Collecting display parameters of the target device to construct a device display feature vector; the display parameters include resolution, display scale, device pixel ratio, color space, and refresh rate; S2. Preprocessing the texture images in the WebGL model using an adaptive resolution algorithm; S3. Training a mapping relationship library between the generated device display feature and the WebGL model running parameters based on historical adaptation data, dynamically adapting the device feature display vector of the target device, obtaining the required WebGL model running parameters, and dynamically adjusting the computation graph of the preprocessed WebGL model; step S3 comprises the following steps: S31. Collect display parameters of different devices and corresponding WebGL model running parameters to build a historical adaptation dataset ; the WebGL model running parameters include LOD level, anti-aliasing level, shader parameter, weight offset, activation function threshold, and calculation graph structure adjustment mark wherein, is a display feature vector of the i-th device, is a corresponding adapted WebGL model run parameter; S32. Constructing a feature embedding model and a parameter mapping model, and training the feature embedding model using the historical adaptation dataset through a contrastive loss, and training the parameter mapping model using the historical adaptation dataset through meta-learning; S33. Mapping the device feature display vector of the target device to an embedding vector using the feature embedding model, and then mapping the embedding vector to the WebGL model running parameters using the parameter mapping model, and saving the device feature display vector and the WebGL model running parameters with a frequency higher than a frequency threshold to the mapping relationship library; S34. Dynamically adjusting the computation graph of the WebGL model with texture image preprocessing according to the WebGL model running parameters matched by the target device; S4. Starting the rendering task of the WebGL model on the target device, splitting the display content of the WebGL model into a fixed layout part and a dynamic content part, performing a vector graphics rendering algorithm on the fixed layout part based on resolution and display scale, dynamically sampling the dynamic content part based on the device pixel ratio, and optimizing using the required WebGL model running parameters to achieve multi-resolution adaptation; S5. Real-time monitoring of the state of the target device, pausing the rendering task of the WebGL model when a display mode switch is detected, and returning to step S1.

2. The method of claim 1, wherein, Step S1 comprises the following steps: S11. Obtaining the display parameters of the target device according to the device interface of the target device type; S12. Standardizing the collected display parameters and uniformly converting them into a predefined data format and vector to obtain the device display feature vector; In particular, the device feature vector ; Wherein, R represents resolution, A represents display scale, C represents color space, F represents refresh rate, and DPR represents device pixel ratio.

3. The method of claim 2, wherein, Step S2 comprises the following steps: S21. Calculating the target rendering size according to the device pixel ratio and resolution of the target device; S22. Scaling the texture images in the WebGL model to the target rendering size using a bicubic interpolation scaling algorithm to obtain an adapted texture object; S23. Eliminating the edge jaggies of the scaled texture images through an edge-preserving filter; S24. If the color space is P3, precomputing the gamut conversion matrix.

4. The method of claim 3, wherein, Step S21 comprises the following steps: S211. Obtaining the resolution R of the target device, and splitting the resolution R into a logical width Hw and a logical height Hh; S212. The target rendering size HxW is calculated by the following formula: W=Hw×DPR H=Hh×DPR Wherein, DPR is the device pixel ratio; The specific steps of step S22 are as follows: S221. Obtain the original size Hy*Wy of the texture image in the WebGL model; S222. Compute scaling ratio ; S223. Scale using bicubic interpolation Scaling the texture image in the WebGL model; wherein, is the original texture image, is the scaled texture image, and W is a bicubic weight function. ( x i , y i ) denotes the original texture image around the target position ( x , y ) 4x4 neighborhood pixel coordinates; S224. Crop or fill the part exceeding the target rendering size to generate an adaptive texture object.

5. The method of claim 3, wherein, The specific steps of step S32 are as follows: S321. Construct a feature embedding model using a lightweight neural network; S322. Perform embedding vector labeling on the historical matching data set, use the feature vector as input, and use the embedding vector labeling result as output, and pre-train the feature embedding model using the contrastive loss method: wherein, is a contrastive loss function, is a display feature vector of the i-th device, is a display feature vector of the j-th device, is an indicator function, when is 1, otherwise 0; denotes the output embedding vector ;​ S323. Construct a parameter mapping model based on the model-agnostic meta-learning framework, and construct an offline support set and a query set based on the historical adaptation data set, wherein the number of samples of the offline support set is less than a number threshold; S324. Fine-tune the parameter mapping model using the offline support set to obtain the initial parameters of the parameter mapping model, and optimize the training of the parameter mapping model using the query set to optimize the initial parameters and obtain the optimized parameter mapping model; The specific steps of step S33 are as follows: S331. Match the device feature vector of the target device with the mapping relationship library; If the device feature vector is matched, the corresponding WebGL model running parameter is taken as the WebGL model running parameter matched by the target device, and step S34 is entered; If the device feature vector cannot be matched, step S332 is entered; S332. Compare the device feature vector of the target device with the historical adaptation data set, and select the TOP-K samples with the highest similarity as the online support set; S333. Update the optimized parameter mapping model based on the online support set to obtain the updated parameter mapping model; S334. Input the device feature vector of the target device into the updated parameter mapping model to output the WebGL model running parameter matched by the target device; The specific steps of step S34 are as follows: S341. Determine whether the WebGL model running parameter matched by the target device contains a weight offset; If yes, adjust the convolution kernel weight of the calculation graph of the WebGL model according to the channel: wherein, denotes the original convolution kernel weight, denotes a weight offset for adjusting the convolution kernel weight, denotes the adjusted convolution kernel weight, denotes a scaling factor adjusted according to a device pixel ratio DPR; If no, step S342 is entered; S342. Calibrate the activation function threshold in the WebGL model running parameter according to the color space C; S343. Adjust the calculation graph topology of the WebGL model according to the display ratio A.

6. The method of claim 5, wherein, The specific steps of constructing the offline support set and the query set based on the historical adaptation data set in step S323 are as follows: historical adaptation dataset clustering according to a combination of display parameters; M clusters are randomly selected, for each cluster a historical fitness dataset is selected K samples are selected to construct an offline support set and N samples are selected to construct a query set where ; In step S332, the device feature vector of the target device is obtained input feature embedding model , to obtain an embedding vector ; computing an embedding vector for a target device with a historical adaptation dataset cosine similarity of the historical device; Select the TOP-K samples with the highest similarity as the historical device samples to construct the offline support set.

7. The method of claim 3, wherein, The specific steps of step S4 are as follows: S41. Start the rendering task of the WebGL model for the target device, and perform rendering pipeline initialization: Set the viewport according to the target rendering size, enable the adaptive anti-aliasing level, and perform color space conversion through the color gamut conversion matrix when the color space is P3; S42. Divide the display content of the WebGL model into a fixed layout part and a dynamic content part according to whether it is variable; S43. The fixed layout part is in the form of a vector graph, the display ratio is bound to the window in the form of a vector graph, and is exported to a bitmap texture map, and is drawn into a full-screen quadrilateral using a fixed layer fragment shader; S44. The dynamic content part is super-sampled, when the device pixel ratio is greater than the pixel ratio threshold, the adaptive texture object is bound to the frame buffer object, the scene is rendered by calling the LOD level, and the super-sampled texture is down-sampled and output to the screen; S45. The fixed layout part and the dynamic content part are rendered together according to the refresh rate setting and are executed to complete the frame rate synchronization.

8. The method of claim 7, wherein, The specific steps of step S5 are as follows: S51. Monitor the changes of the target device resolution R, the display ratio, and the device pixel ratio, and determine whether any of them changes; If yes, go to step S52; If no, wait for a set period of time and return to step S51; S52. Freeze the thread of the current rendering task, re-collect the display parameters of the target device according to step S1 to construct the device feature display vector, and re-generate the new WebGL model running parameters according to step S3 to update the calculation graph of the WebGL model; S53. Restart the rendering pipeline using the new WebGL model running parameters and continue to execute the rendering task.

9. A multi-device dynamically adapted WebGL display optimization system, applying the multi-device dynamically adapted WebGL display optimization method of any one of claims 1-8, characterized in that, Comprise: A device display parameter acquisition module for acquiring display parameters of a target device and constructing a device display feature vector; The display parameters include resolution, display ratio, device pixel ratio, color space, and refresh rate; A WebGL model preprocessing module for preprocessing texture images in a WebGL model using an adaptive resolution algorithm; A dynamic adaptation module for dynamically adapting the device feature display vector of a target device based on a mapping relationship library between device display features and WebGL model running parameters generated by training historical adaptation data, obtaining the required WebGL model running parameters, and dynamically adjusting the calculation graph of the preprocessed WebGL model; A layered rendering module for starting a rendering task of a WebGL model on a target device, splitting the display content of the WebGL model into a fixed layout part and a dynamic content part, performing a vector graphics rendering algorithm on the fixed layout part based on resolution and display ratio, dynamically sampling the dynamic content part based on the device pixel ratio, and optimizing using the required WebGL model running parameters to achieve multi-resolution adaptation; A state monitoring module for monitoring the state of a target device in real time, pausing the rendering task of a WebGL model when a display mode switch is detected, and re-collecting device parameters.

Images