Real-time rendering method based on WASM
By adopting a WASM-based real-time rendering method in the graphics rendering technology, the problems of low computing efficiency, poor performance and insufficient security of intensive matrices in the prior art are solved, and more efficient and secure rendering performance is achieved.
Patent Information
- Application Number
- CN202411746357.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-03-25
AI Technical Summary
The prior art is inefficient, poor performance and insufficient security when performing intensive matrix calculations, resulting in inefficient efficiency when dealing with complex scenarios and prone to memory leakage or resource management problems.
Using the WASM-based real-time rendering method, by obtaining the basic structural information of the 3D object to be rendered, starting the web engine and initializing the rendering environment, loading the WASM memory module asynchronously, saving the matrix data into the matrix buffer, and performing matrix calculation and uploading to the GPU for rendering.
Improve rendering performance and efficiency, reduce performance losses in data copying and passing, provide a more stringent memory management mechanism, avoid memory leaks and resource management problems, and make the rendering process safer and more efficient.
Smart Images
Figure CN119693218A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of graphics rendering technology, and in particular to a real-time rendering method based on WASM. Background Art
[0002] In the traditional graphics rendering mechanism, there is a lack of high-level memory management mechanism. The data organization of the rendering object is based on the object, and the developer needs to manually manage the GPU resources for each rendering object. Because it relies on the implicit state machine mode, for matrix data, it is necessary to bind and manage the matrix data of each object separately, and perform explicit binding and unbinding operations, which increases the difficulty of development and may become inefficient when processing complex scenes. It is also easy to cause memory leaks or resource management problems, and cannot fully utilize all the capabilities of modern GPUs.
[0003] Therefore, there is an urgent need for a real-time rendering method based on WASM. Summary of the invention
[0004] 1. Technical issues to be resolved
[0005] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a real-time rendering method based on WASM, which solves the technical problems of low efficiency, poor performance and poor security in the prior art when performing intensive matrix calculations.
[0006] (II) Technical solution
[0007] In order to achieve the above object, the main technical solutions adopted by the present invention include:
[0008] An embodiment of the present invention provides a WASM-based real-time rendering method, comprising:
[0009] S10, obtaining basic structural information of the 3D object to be measured;
[0010] S20, start the Web engine, initialize the rendering environment, asynchronously load the WASM memory module, dynamically add scripts, construct the basic structure information of the 3D object to be tested into an initial matrix, and store the constructed initial matrix into the matrix buffer of the WASM memory module;
[0011] S30, translating, rotating, scaling, and confirming the parent-child relationship of the initial matrix in the matrix buffer to obtain an updated model matrix;
[0012] S40, performing MVP matrix calculation on the updated model matrix to obtain the MVP matrix, uploading the MVP matrix to the GPU for rendering at one time to obtain a final 2D image.
[0013] Optionally, the basic structural information of the 3D object to be measured includes: vertex information, texture coordinate information, transformation information and hierarchical relationship information.
[0014] Optionally, the matrix buffer includes:
[0015] A first matrix buffer, used to store matrix data;
[0016] The second matrix buffer is used to store the scaling, rotation, and translation data of each matrix;
[0017] The third matrix buffer is used to store the continuous transformation data of the matrix;
[0018] The fourth matrix buffer is used to store the status information of each matrix.
[0019] Optionally, the state information of each matrix includes: one-time transformation state information, continuous transformation state information, parent matrix index information and depth order information.
[0020] Optionally, the S20 further includes:
[0021] When the reserved matrix buffer space is insufficient, continue to open up matrix space until the constructed initial matrix can be accommodated.
[0022] Optionally, the S30 specifically includes:
[0023] S31, determining a translation vector, a rotation axis, a rotation angle, and a scaling factor when updating a matrix, and constructing a translation matrix, a rotation matrix, and a scaling matrix according to the translation vector, the rotation axis, the rotation angle, and the scaling factor;
[0024] S32, inputting the translation matrix, rotation matrix and scaling matrix into the following formula to obtain a model matrix;
[0025] Model matrix = translation matrix × rotation matrix × scaling matrix.
[0026] Optionally, the S30 further includes:
[0027] S33, according to the SetParent method, determining the parent-child relationship of different model matrices;
[0028] When there is a parent-child relationship, the updated model matrix is:
[0029] Updated parent matrix = child matrix × model matrix;
[0030] Updated sub-matrix = parent matrix -1 × model matrix.
[0031] Optionally, the S40 specifically includes:
[0032] S41, obtaining the view matrix and projection matrix of the 3D object to be measured, and inputting the view matrix and projection matrix into the following formula to obtain the MVP matrix:
[0033] MVP matrix = view matrix × projection matrix × updated model matrix;
[0034] S42, uploading the MVP matrix to the GPU for rendering at one time to obtain a final 2D image.
[0035] Optionally, the S30 further includes:
[0036] Use the updateAllChangedTranslate method to update multiple initial matrices at one time.
[0037] Optionally, in S41, obtaining the view matrix and the projection matrix of the 3D object to be measured includes:
[0038] S411, determining the position, direction and field of view angle of the observation point corresponding to the 3D object to be measured, and dynamically constructing a view matrix using the lookAt function according to the position, direction and field of view angle of the observation point;
[0039] S412. Obtain the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point corresponding to the 3D object to be measured, and use the gl-matrix function to construct a projection matrix according to the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point.
[0040] (III) Beneficial effects
[0041] The beneficial effects of the present invention are as follows: a real-time rendering method based on WASM of the present invention, due to the use of a matrix calculation and management module based on the efficient execution capability of WASM, uniformly processes and binds buffer objects, and compared with the prior art, it avoids the performance damage caused by frequent data copying and transmission, thereby improving rendering performance and rendering efficiency; in addition, WASM is subject to strict verification when it is designed and run, reducing the possibility of running malicious code; further, WASM provides a more rigorous memory management mechanism, avoiding common memory leaks and management problems, and is safer and more efficient when performing a large number of matrix operations and graphics rendering. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 This is a flowchart of a WASM-based real-time rendering method according to Embodiment 1 of the present invention;
[0043] Figure 2This is a schematic diagram of memory sharing of a WASM-based real-time rendering method in Embodiment 2 of the present invention;
[0044] Figure 3 This is a schematic diagram of memory allocation in a matrix buffer area taking 100 matrices as an example in Embodiment 2 of the present invention;
[0045] Figure 4 This is an overall flow chart of a WASM-based real-time rendering method in Embodiment 2 of the present invention. DETAILED DESCRIPTION
[0046] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.
[0047] A real-time rendering method based on WASM proposed in an embodiment of the present invention is to solve the problem that the existing graphics rendering mechanism lacks a high-level memory management mechanism, which leads to inefficiency in processing complex scenes, and is prone to memory leaks or resource management problems, and cannot fully utilize the capabilities of the GPU. The embodiment of the present invention uses WASM and constructs a matrix buffer at the same time. It has efficient execution capabilities, can uniformly process and bind the buffer object matrix, avoids the performance loss of frequent data copying and transmission, and improves rendering efficiency; on the other hand, WASM has been strictly verified in design, reducing the possibility of running malicious code; further, the WASM of the present invention provides a more stringent memory management mechanism, avoiding common memory leaks and management problems, and is safer and more efficient for complex applications that process a large number of matrix operations and graphics rendering.
[0048] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.
[0049] Example 1
[0050] See also Figure 1 , a WASM-based real-time rendering method according to an embodiment of the present invention includes:
[0051] Step S10, obtaining basic structural information of the 3D object to be measured;
[0052] Step S20: Start the Web engine, initialize the rendering environment, asynchronously load the WASM memory module, dynamically add scripts, construct the basic structure information of the 3D object to be tested into an initial matrix, and store the constructed initial matrix in the matrix buffer of the WASM memory module;
[0053] Step S30, translating, rotating, scaling, and confirming the parent-child relationship of the initial matrix in the matrix buffer to obtain an updated model matrix;
[0054] Step S40: perform MVP matrix calculation on the updated model matrix to obtain the MVP matrix, and upload the MVP matrix to the GPU for rendering at one time to obtain the final 2D image.
[0055] In this embodiment, a matrix buffer is set in the WASM memory module, and the buffer object matrix is processed uniformly. After the MVP matrix is obtained, the MVP matrix is uploaded to the GPU for rendering at one time, avoiding the performance loss of frequent data copying and transmission, and improving rendering efficiency.
[0056] Example 2
[0057] See also Figure 4 , a WASM-based real-time rendering method according to an embodiment of the present invention includes:
[0058] Step S10, obtaining basic structural information of the 3D object to be measured;
[0059] Step S20: Start the Web engine, initialize the rendering environment, asynchronously load the WASM memory module, dynamically add scripts, construct the basic structure information of the 3D object to be tested into an initial matrix, and store the constructed initial matrix in the matrix buffer of the WASM memory module;
[0060] Step S30, translating, rotating, scaling, and confirming the parent-child relationship of the initial matrix in the matrix buffer to obtain an updated model matrix;
[0061] Step S40: perform MVP matrix calculation on the updated model matrix to obtain the MVP matrix, and upload the MVP matrix to the GPU for rendering at one time to obtain the final 2D image.
[0062] The basic structural information of the 3D object to be tested includes vertex information, texture coordinate information, transformation information, and hierarchical relationship information.
[0063] In a specific implementation process, basic structural information of the 3D object to be measured is obtained through a model editing tool, such as 3dMax, C4D or Blander.
[0064] In this embodiment, the matrix buffer includes:
[0065] A first matrix buffer, used to store matrix data;
[0066] The second matrix buffer is used to store the scaling, rotation, and translation data of each matrix;
[0067] The third matrix buffer is used to store the continuous transformation data of the matrix;
[0068] The fourth matrix buffer is used to store the status information of each matrix.
[0069] The state information of each matrix includes: one-time transformation state information, continuous transformation state information, parent matrix index information and depth order information.
[0070] See also Figure 3 , each entry stored in the first matrix buffer is a 4×4×4 floating-point matrix representing the world space transformation of an object, which can be stored in the form of a data or structure array.
[0071] The second matrix buffer stores 3×3×4 scaling, rotation, and translation data.
[0072] The third matrix buffer stores continuous transformation data of a 1×3×4 matrix, such as displacement changes and rotation changes of each frame.
[0073] The one-time transformation state information stored in the fourth matrix buffer refers to whether a one-time transformation operation needs to be performed; the continuous transformation state information refers to whether there is an ongoing continuous transformation, such as animation; the parent matrix index information points to the index of the parent object matrix, which is used to build a hierarchical relationship; the depth order information refers to the specific rendering order, especially the rendering order of transparent objects.
[0074] In a specific implementation process, step S20 also includes:
[0075] When the reserved matrix buffer space is insufficient, continue to open up matrix space until the constructed initial matrix can be accommodated.
[0076] Furthermore, when the space of the matrix buffer required by the constructed initial matrix reaches the maximum limit of the browser, an error occurs and the browser issues an error alarm.
[0077] The maximum limit of the browser is 2 28 -1 numeric value.
[0078] In this embodiment, step S30 specifically includes:
[0079] Step S31, determining the translation vector, rotation axis, rotation angle and scaling factor when updating the matrix, and constructing the translation matrix, rotation matrix and scaling matrix according to the translation vector, rotation axis, rotation angle and scaling factor;
[0080] Step S32: Input the translation matrix, rotation matrix and scaling matrix into the following formula to obtain the model matrix;
[0081] Model matrix = translation matrix × rotation matrix × scaling matrix.
[0082] In a specific implementation process, the translation matrix is used to change the position of an object, the rotation matrix is used to change the direction of an object, and the scaling matrix is used to change the size of an object.
[0083] The translation matrix, rotation matrix, and scaling matrix determine the specific position and posture of the object in the scene by transforming the local coordinates of the object into the world coordinate system.
[0084] Step S30 also includes:
[0085] Step S33, determining the parent-child relationship of different model matrices according to the SetParent method;
[0086] When there is a parent-child relationship, the updated model matrix is:
[0087] Updated parent matrix = child matrix × model matrix;
[0088] Updated sub-matrix = parent matrix -1 × model matrix.
[0089] The SetParent method is used to define the hierarchical relationship between objects. This method allows one object to be set as a child of another object, thereby affecting the position, rotation, and scale of the child object.
[0090] When an object becomes the child of another object, its position, rotation, and scale are calculated relative to its parent. This means that all transformations of the child object are performed based on the parent's world coordinate system.
[0091] That is, the child's local transform (relative to the parent's transform) is applied to the parent's world transform to get the child's world transform.
[0092] In this embodiment, step S40 specifically includes:
[0093] Step S41, obtain the view matrix and projection matrix of the 3D object to be measured, input the view matrix and projection matrix into the following formula to obtain the MVP matrix:
[0094] MVP matrix = view matrix × projection matrix × updated model matrix;
[0095] Step S42: Upload the MVP matrix to the GPU for rendering at one time to obtain the final 2D image.
[0096] In this embodiment, step S30 further includes:
[0097] Use the updateAllChangedTranslate method to update multiple initial matrices at one time.
[0098] The updateAllChangedTranslate method can iterate over all objects marked as "to be updated" and update their world transformation matrices. In this way, multiple objects can be updated efficiently in batches, ensuring that all objects in the scene maintain the latest transformation state.
[0099] In step S41, obtaining the view matrix and projection matrix of the 3D object to be measured includes:
[0100] Step S411, determine the position, direction and field of view angle of the observation point corresponding to the 3D object to be measured, and use the lookAt function to dynamically construct a view matrix according to the position, direction and field of view angle of the observation point;
[0101] Step S412: Obtain the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point corresponding to the 3D object to be measured, and use the gl-matrix function to construct a projection matrix according to the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point.
[0102] Example 3
[0103] A WASM-based real-time rendering method according to an embodiment of the present invention includes:
[0104] Step S10, obtaining basic structural information of the 3D object to be measured;
[0105] Step S20: Start the Web engine, initialize the rendering environment, asynchronously load the WASM memory module, dynamically add scripts, construct the basic structure information of the 3D object to be tested into an initial matrix, and store the constructed initial matrix in the matrix buffer of the WASM memory module;
[0106] Step S30, translating, rotating, scaling, and confirming the parent-child relationship of the initial matrix in the matrix buffer to obtain an updated model matrix;
[0107] Step S40: perform MVP matrix calculation on the updated model matrix to obtain the MVP matrix, and upload the MVP matrix to the GPU for rendering at one time to obtain the final 2D image.
[0108] In the specific implementation process, Figure 2 As shown in the figure, the matrix buffer in the WASM memory module is a linear memory block, which is represented as an ArrayBuffer and can be accessed by both JavaScript and WASM. Memory is allocated in pages, 1 page = 64KB.
[0109] Among them, JavaScript directly reads or writes WASM's matrix buffer through standard TypedArray (such as Float32Array, Int32Array, etc.); WASM accesses data through memory pointers. Usually, functions in WASM need to pass in pointers to memory. Since WASM cannot directly process complex data structures, data needs to be stored in reallocated memory space.
[0110] Through the above operations, both JavaScript and WASM can directly access and operate data, avoiding frequent data exchange between JavaScript and WebAssembly and greatly improving efficiency.
[0111] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0112] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0113] In the present invention, unless otherwise clearly specified and limited, when a first feature is “on” or “below” a second feature, it may be that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediate medium. Moreover, when a first feature is “above”, “above” or “above” a second feature, it may be that the first feature is directly above or obliquely above the second feature, or it may simply mean that the first feature is higher in level than the second feature. When a first feature is “below”, “below” or “below” a second feature, it may be that the first feature is directly below or obliquely below the second feature, or it may simply mean that the first feature is lower in level than the second feature.
[0114] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0115] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may alter, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A real-time rendering method based on WASM, characterized in that: include: S10, obtaining basic structural information of the 3D object to be measured; S20, start the Web engine, initialize the rendering environment, asynchronously load the WASM memory module, dynamically add scripts, construct the basic structure information of the 3D object to be tested into an initial matrix, and store the constructed initial matrix into the matrix buffer of the WASM memory module; S30, translating, rotating, scaling, and confirming the parent-child relationship of the initial matrix in the matrix buffer to obtain an updated model matrix; S40, performing MVP matrix calculation on the updated model matrix to obtain the MVP matrix, uploading the MVP matrix to the GPU for rendering at one time to obtain a final 2D image.
2. The WASM-based real-time rendering method according to claim 1, characterized in that: The basic structural information of the 3D object to be measured includes: vertex information, texture coordinate information, transformation information and hierarchical relationship information.
3. The WASM-based real-time rendering method according to claim 1, characterized in that: The matrix buffer includes: A first matrix buffer, used to store matrix data; The second matrix buffer is used to store the scaling, rotation, and translation data of each matrix; The third matrix buffer is used to store the continuous transformation data of the matrix; The fourth matrix buffer is used to store the status information of each matrix.
4. The WASM-based real-time rendering method according to claim 3, characterized in that: The state information of each matrix includes: one-time transformation state information, continuous transformation state information, parent matrix index information and depth order information.
5. The WASM-based real-time rendering method according to claim 1, characterized in that: The S20 further includes: When the reserved matrix buffer space is insufficient, continue to open up matrix space until the constructed initial matrix can be accommodated.
6. The WASM-based real-time rendering method according to claim 1, characterized in that: The S30 specifically includes: S31, determining a translation vector, a rotation axis, a rotation angle, and a scaling factor when updating a matrix, and constructing a translation matrix, a rotation matrix, and a scaling matrix according to the translation vector, the rotation axis, the rotation angle, and the scaling factor; S32, inputting the translation matrix, rotation matrix and scaling matrix into the following formula to obtain a model matrix; Model matrix = translation matrix × rotation matrix × scaling matrix.
7. The WASM-based real-time rendering method according to claim 1, characterized in that: The S30 further includes: S33, according to the SetParent method, determining the parent-child relationship of different model matrices; When there is a parent-child relationship, the updated model matrix is: Updated parent matrix = child matrix × model matrix; Updated sub-matrix = parent matrix -1 × model matrix.
8. The WASM-based real-time rendering method according to claim 1, characterized in that: The S40 specifically includes: S41, obtaining the view matrix and projection matrix of the 3D object to be measured, and inputting the view matrix and projection matrix into the following formula to obtain the MVP matrix: MVP matrix = view matrix × projection matrix × updated model matrix; S42, uploading the MVP matrix to the GPU for rendering at one time to obtain a final 2D image.
9. The WASM-based real-time rendering method according to claim 1, characterized in that: The S30 further includes: Use the updateAllChangedTranslate method to update multiple initial matrices at one time.
10. The WASM-based real-time rendering method according to claim 8, characterized in that: In the step S41, obtaining the view matrix and the projection matrix of the 3D object to be measured includes: S411, determining the position, direction and field of view angle of the observation point corresponding to the 3D object to be measured, and dynamically constructing a view matrix using the lookAt function according to the position, direction and field of view angle of the observation point; S412. Obtain the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point corresponding to the 3D object to be measured, and use the gl-matrix function to construct a projection matrix according to the field of view angle, aspect ratio, near clipping plane distance and far clipping plane distance of the observation point.