A web data processing method based on collaborative rendering

Abstract

The application belongs to the field of image processing, and provides a WEB data processing method based on cooperative rendering, which comprises the following steps: a front-end cooperative rendering machine pre-stores a plurality of image reconstruction templates and graphic resources; a cloud-end cooperative rendering machine performs complete image rendering of a current frame target picture; the cloud-end cooperative rendering machine extracts structural features of a current image frame, and selects a template most suitable for reconstruction of the current image frame from a plurality of preset image reconstruction templates; the cloud-end cooperative rendering machine generates a corresponding template parameter set according to the selected image reconstruction template; the cloud-end cooperative rendering machine packages a unique identification of the selected image reconstruction template and the template parameter set into a structured data packet, and transmits the structured data packet to the front-end cooperative rendering machine; and the front-end cooperative rendering machine loads required graphic data from a local resource library according to the received image reconstruction template identification, and completes image frame splicing and display according to template logic and the parameter set.

Description

Technical Field

[0001] This invention belongs to the field of image processing, and more specifically relates to a web data processing method based on collaborative rendering. Background Technology

[0002] With the rapid development of web-based 3D graphics display technology, more and more online applications rely on high-quality image rendering and dynamic visualization capabilities, such as virtual scenic spots, architectural visualization, online museums, and industrial remote monitoring. Due to the general limitations of terminal devices in terms of performance, power consumption, and network conditions, traditional all-local rendering solutions are no longer sufficient to meet the demands for high-precision, low-latency 3D image interaction.

[0003] Existing technologies employ a "cloud rendering + image streaming" model, where image rendering tasks are centralized on the server side and then pushed to the front end as image frames, video streams, or differential maps. While this approach reduces the computational burden on the terminal to some extent, it still has the following limitations: First, the large data transmission volume of image frames can easily lead to stuttering in mobile or weak network environments; second, the front end only has image decoding capabilities and cannot reconstruct or extend based on structured information, thus limiting interactive performance; third, the cloud-based image content generation and distribution lacks a content splitting mechanism, making it difficult to adapt to diverse terminal configurations.

[0004] Therefore, existing web-based 3D rendering systems based on image streaming cannot achieve rapid response and efficient updates in low-bandwidth, high-interaction-frequency scenarios. Technical problems such as high latency, heavy resource dependence, and weak adaptability on the terminal side urgently need to be solved. Summary of the Invention

[0005] To address the problems in the existing technology, this invention provides a web data processing method based on collaborative rendering, comprising the following steps:

[0006] The front-end collaborative rendering machine pre-stores multiple image reconstruction templates and graphics resources. The image reconstruction templates are used to assemble the current image frame after receiving specified template parameters.

[0007] After receiving an interaction request or viewpoint change information from the front-end user, the cloud-based collaborative rendering machine performs complete image rendering of the target image in the current frame.

[0008] The cloud-based collaborative rendering machine extracts structural features from the current image frame and selects the template most suitable for reconstructing the current image frame from a set of preset image reconstruction templates. The selection criteria include the execution complexity of the template on the front-end device, the reconstruction latency, and the image region change characteristics.

[0009] The cloud-based collaborative rendering machine generates a corresponding template parameter set based on the selected image reconstruction template. The template parameter set includes at least one of texture index, mask number, transformation parameters, channel blending factor, and spatial adjustment matrix.

[0010] The cloud-based collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits it to the front-end collaborative rendering machine.

[0011] The front-end collaborative rendering machine loads the required graphic data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of image frames according to the template logic and parameter set.

[0012] Furthermore, the step of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources includes: constructing an image reconstruction template library, wherein each template in the template library has a unique number and defines the template type, input resource structure and execution logic, wherein the template type includes at least one of mask blending template, channel combination template, block replacement template, mesh deformation template and progressive refinement template.

[0013] Furthermore, the step of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources further includes: loading graphics resources and initializing the cache, wherein the graphics resources include texture images, mask images, transformation matrices and material channel information, and the resources are stored in the local resource manager with a unique identifier and support version verification.

[0014] Furthermore, the step of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources also includes: establishing a parameter mapping relationship between templates and resources. The parameter mapping structure binds the input fields required for template definition with local resource indexes for quick resource matching during subsequent template execution.

[0015] Furthermore, the cloud-based collaborative rendering machine performs the following steps to render the complete image of the target scene in the current frame after receiving the interaction request or viewpoint change information from the front-end user: parsing user input data to generate camera parameters, constructing the current frame view matrix, and determining the set of 3D scene objects to be rendered based on the view frustum culling technique.

[0016] Furthermore, the cloud-based collaborative rendering machine's step of extracting structural features from the current image frame includes: calculating texture density, number of region boundaries, brightness distribution, and pixel variation regions based on the current image frame data, and constructing a structural feature vector, which is used to guide subsequent template selection.

[0017] Furthermore, the step of the cloud-based collaborative rendering machine selecting the most suitable template for the reconstruction of the current image frame from a plurality of preset image reconstruction templates includes: scoring the candidate templates based on a template fit function, wherein the parameters of the template fit function include structural feature matching degree, template execution complexity and front-end reconstruction latency, and the template with the highest score is selected as the optimal template for the current frame.

[0018] Furthermore, the steps of generating the template parameter set by the cloud-based collaborative rendering machine include: extracting texture index, mask number, channel blending factor and spatial transformation matrix information, and organizing them into a structured template parameter set according to the template definition format.

[0019] Furthermore, the step of the cloud-based collaborative rendering machine packaging the template identifier and template parameter set into a structured data packet and transmitting it to the front end includes: constructing a data packet structure, the structure including a template number, parameter fields, version number and verification information, and transmitting it to the front-end collaborative rendering machine via a network protocol.

[0020] Furthermore, the steps of the front-end collaborative rendering machine to assemble and display image frames according to template logic and parameter set include: parsing the template number and parameter content in the structured data packet, calling the corresponding template logic and loading local resources, performing image reconstruction operation in the Canvas or WebGL rendering context, and drawing the result image to the display area to complete the image frame display.

[0021] This invention structurally separates image rendering and image reconstruction tasks, migrates the computationally intensive image rendering process to the cloud, and deploys various structured image reconstruction templates on the front end. With the help of template parameter sets, it enables local assembly and rapid display of image frames, thereby significantly reducing network bandwidth usage and improving the system's operability and graphics response speed in low-resource environments.

[0022] The image reconstruction template provided by this invention possesses high structural reusability and execution path consistency, supporting efficient reconstruction in various image variation scenarios. Through a dynamic template matching mechanism based on image features, the system can adaptively select the optimal template according to the current image frame content and the capabilities of the front-end device, effectively improving the terminal image processing efficiency and the overall stability of the system.

[0023] Furthermore, the structured template parameters and resource indexing mechanism adopted in this invention enable the front end to complete rapid image combination on the resource cache structure without re-decoding or rendering, thereby achieving millisecond-level perspective switching or screen updates. It is widely applicable to application scenarios with high response latency requirements, such as mobile terminals, low-power devices, and edge rendering platforms. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is the main flowchart of the method of the present invention;

[0026] Figure 2 This is a schematic diagram of the end-to-cloud collaborative rendering principle of the present invention;

[0027] Figure 3 This is a flowchart illustrating the structure of template selection and parameter generation.

[0028] Figure 4 This is a schematic diagram of the logical composition of a structured data packet;

[0029] Figure 5 This is a flowchart of the front-end image frame assembly process. Detailed Implementation

[0030] The invention will now be described in preferred form with reference to the accompanying drawings and specific embodiments.

[0031] This embodiment solves the above problems through the following steps:

[0032] In one embodiment, such as Figure 1 As shown, this invention discloses a web data processing method based on collaborative rendering, which utilizes, as... Figure 2 The information interaction and task allocation between the front-end collaborative rendering machine and the cloud collaborative rendering machine shown enable dynamic separation of image rendering resources, template-driven reconstruction, and low-bandwidth visualization output. This allows for efficient display and real-time interaction of complex 3D images in a terminal environment with limited bandwidth and computing resources, which is a WEB data processing method.

[0033] To achieve the above objectives, the web data processing method based on collaborative rendering specifically includes the following steps:

[0034] Step S10: The front-end collaborative rendering machine pre-stores multiple image reconstruction templates and graphics resources. The image reconstruction templates are used to assemble the current image frame after receiving specified template parameters.

[0035] In collaborative rendering-based web data processing methods, to achieve rapid reconstruction and efficient display of image frames on terminal devices, the computationally intensive image rendering process needs to be migrated to the cloud, while the image reconstruction operation is deployed to the front end in a modular and templated manner. Since front-end devices typically have limited computing power, if the front end performs rendering and image construction operations independently, it will result in significant response latency, reduced interactive performance, and a severe impact on user experience. Therefore, this method pre-stores multiple image reconstruction templates and graphic resources on the front end. Upon receiving template parameter information returned from the cloud, it can directly call the corresponding template and resources to complete the assembly and output of image frames, thereby significantly reducing the computational burden of image generation and network transmission bandwidth.

[0036] In this invention, an image reconstruction template refers to a structured template data structure used to describe the combination and transformation rules for generating image frames based on graphic resources. Graphic resources refer to elements such as texture data, mask maps, channel mixing information, and transformation matrices that can be used for image assembly. These resources can be uniformly loaded into the local cache by the content delivery network upon the first page visit, or prefetched based on user behavior. Template parameters refer to a set of control parameters generated by the cloud-based collaborative rendering machine, used to specify the resource index, transformation method, overlay weight, and other operational details that the front-end needs to select during template application.

[0037] In one optional implementation, step S10 includes the following sub-steps:

[0038] Step S101: Construct an image reconstruction template library

[0039] This step is completed by the front-end system during the initialization phase, specifically as follows:

[0040] Define the template set structure and identify it with a unique template number (such as T_001, T_002);

[0041] Each template definition includes: template type, input resource structure, and execution logic (encapsulated as a JS module or GLSL fragment);

[0042] Template types include, but are not limited to:

[0043] Mask blending template (T_001): Performs local blending based on texture and mask;

[0044] Channel Combination Template (T_002): Combine RGB channels separately;

[0045] Block Replacement Template (T_003): Replaces the specified block area;

[0046] Mesh Deformation Template (T_004): Controls texture bending or translation via a 2D transformation matrix;

[0047] Progressive refinement template (T_005): Prioritizes filling with low resolution and replaces high-resolution layers frame by frame.

[0048] Template definitions are encapsulated using JSON or JS module structures and registered with the local template dispatch engine.

[0049] Step S102: Load graphics resources and initialize cache.

[0050] This step is performed during the page loading phase and includes:

[0051] Preload texture atlases, mask images, transformation matrix libraries, and material channel configurations from the Content Delivery Network (CDN);

[0052] All graphic resources are coded and managed using unique IDs (such as R_007, M_003, L_004);

[0053] Once loaded, the data is stored in the local graphics resource manager, using IndexedDB or an in-memory dictionary structure for caching.

[0054] Supports version number and hash verification to ensure consistency between resources and templates.

[0055] Step S103: Establish template parameter structure mapping relationship

[0056] This step describes the logical binding between the input parameters required by the template and local resources, including:

[0057] Iterate through all templates and extract the input parameter structure;

[0058] Generate a template-parameter mapping description table, for example:

[0059]

[0060] The structure can be used for parameter parsing during template scheduling;

[0061] If resources are missing, the system will automatically trigger a prefetching or placeholder replacement strategy.

[0062] Step S104: Register the template and resource index to the execution engine.

[0063] This step pre-binds the template and resource execution logic to the front-end rendering controller, including:

[0064] Register the execution module for each template;

[0065] Define a template call interface, such as renderFrame(T_001, params);

[0066] Use JS object pools or WebGL context caching units to ensure low-latency calls;

[0067] If the template contains transformation logic (such as T_004 containing a rotation matrix), its linear algebra operation library is loaded during the registration phase.

[0068] By pre-storing multiple structured image reconstruction templates in the front-end collaborative rendering machine and loading matching graphics resource sets, the execution of complex image rendering tasks can be shifted from runtime to the initialization stage, significantly reducing the execution overhead during image frame assembly. Simultaneously, the template structure of this invention possesses high modularity and composability, capable of adapting to the reconstruction needs of different types of image frames, and providing a unified basic structure for subsequent operations such as incremental image updates, local occlusion repair, and multi-view switching.

[0069] In a specific example, taking a smart scenic area 3D display system as an example, when a user first enters the scenic area entrance view page, the front-end collaborative rendering machine loads the following content:

[0070] Template T_001 (mask blending template) has the following structure:

[0071] Input: Main texture ID, mask image ID, blending factor f

[0072] Logical reasoning: Final(x,y) = f × Main(x,y) + (1-f) × Mask(x,y)

[0073] Template T_004 (mesh deformation template), structure is as follows:

[0074] Input: Texture ID, Transformation matrix M (2×2)

[0075] Logic: Resample and redraw after transforming texture UV coordinates.

[0076] Meanwhile, the front end preloaded the following resources:

[0077] Main texture R_017 (visitor center exterior), mask M_005 (shadow area), texture R_022 (glass reflection), spatial matrix G_011 (left view rotation matrix), etc.

[0078] When the user switches from the main entrance to a side view, the cloud-detected image change features indicate a slight local viewpoint shift. Template T_004 is determined to be the most suitable for this reconstruction operation, and a data packet is sent out.

[0079]

[0080] After receiving the data packet, the front end calls the transformation execution logic of template T_004, takes the R_017 texture and G_011 matrix as input, and completes the output of the image frame after mesh deformation. The reconstruction delay is controlled within 40ms. The system does not need to transmit the whole frame image or video stream, which significantly saves network bandwidth and response time.

[0081] In step S20, after receiving the interaction request or viewpoint change information from the front-end user, the cloud-based collaborative rendering machine performs full image rendering of the target image in the current frame.

[0082] To ensure that web-based terminal devices can display complex 3D scenes with low latency and high efficiency, this invention places the image rendering task in the cloud and performs it in collaboration with a rendering machine. Since image rendering involves computationally intensive operations such as complex lighting calculations, occlusion handling, texture mapping, and spatial transformations, performing these operations on the front-end device would severely limit system performance, especially in application scenarios with frequent image frame switching and intensive user interaction. Deploying the image rendering process to the cloud allows for large-scale parallel rendering tasks to be completed using high-performance graphics processing units (GPUs), dynamically generating high-precision image frames based on user interaction input or changes in viewpoint, while simultaneously providing the front-end with structured output data for assembly, thereby improving the overall system's graphics response speed and image continuity.

[0083] In this invention, the current frame target image refers to the 3D perspective image corresponding to the scene state triggered by user interaction at a certain moment. It includes the input information required for rendering, such as viewpoint position, viewing direction, field of view angle, observed objects, and occlusion relationships. Image rendering refers to the process of completing rasterization, shading, texture mapping, ambient lighting, post-processing, and other graphics pipeline operations under given scene data and camera parameters, and outputting the final image frame data. Complete image rendering refers to a comprehensive rendering task that includes viewpoint subdivision, material sampling, layer overlay, and global illumination, which is different from local updates or incremental frame interpolation operations.

[0084] In one optional implementation, step S20 specifically includes the following sub-steps:

[0085] Step S201: Receive interactive event data, including but not limited to mouse clicks, touch swipes, keyboard input, gravity sensing, and head-mounted device pose information. The system performs type parsing and behavior recognition on this data and extracts geometric information parameters for 3D scene mapping. These geometric information parameters include viewpoint position vector, viewing direction vector, camera near and far plane distances, and field of view angles.

[0086] Step S202: Construct the current frame view matrix. Based on the geometric information parameters extracted in step S201, construct a 3D camera matrix structure. The camera matrix includes three parts: a view matrix, a projection matrix, and a camera intrinsic parameter matrix, used to define the mapping relationship between the viewpoint and the 3D world coordinates in the scene. The matrix calculation method can adopt the OpenGL standard right-handed coordinate system definition, or be constructed using the glMatrix library according to WebGL conventions.

[0087] Step S203: Perform 3D scene partitioning. Based on the scene structure diagram and spatial partitioning algorithm, determine the set of 3D objects that need to be rendered within the current field of view. Remove scene elements that are not within the view frustum or are completely occluded to reduce redundant graphics processing computations. The spatial partitioning algorithm can use an octree, a BVH bounding box hierarchy structure, or a view frustum culling strategy based on a Z-buffer.

[0088] Step S204: Perform the complete image rendering process. The system executes the following graphics rendering steps in sequence according to the selected rendering engine (such as Unreal Engine, Unity, custom WebGPU renderer, etc.): load material and texture resources, execute geometry shader and fragment shader programs, sample various texture resources, complete ambient lighting, shadow processing and global illumination models, perform post-processing operations such as anti-aliasing, HDR compositing and tone mapping, and finally generate a complete image frame output.

[0089] Step S205: Output structured rendering results. Based on the reconstruction template features of the front-end collaborative rendering machine, the content of the rendered image frame is converted into a structured data packet. The data packet content includes image region description information, reconstructable template type selection information, template parameter generation results, and resource call index. The data structure meets the logical requirements of the front-end to quickly assemble and reconstruct according to the template.

[0090] Through the above steps, the cloud-based collaborative rendering machine can efficiently complete the rendering task of the current image frame based on the interactive behavior triggered by the front end, and encode and output the rendering results in a structured and templated manner, which significantly reduces the burden of front-end image generation, while improving the system's interactive response capability and screen consistency. It is suitable for the multi-view rapid switching and asynchronous assembly requirements of complex 3D scenes.

[0091] In a specific example, a user swipes the screen from the "front view of the visitor center" to the "side view." The front-end collaborative renderer uploads the camera orientation change, offset angle, and the observed target number to the cloud-based collaborative renderer. Upon receiving this request, the cloud generates a new camera matrix based on the viewpoint change, calls the scene graph structure to perform octree frustum pruning, determines the set of geometric objects to be rendered, loads the corresponding textures and materials based on the Unreal Engine rendering engine, performs PBR lighting rendering, and generates the target frame image.

[0092] In step S30, the cloud-based collaborative rendering machine extracts structural features from the current image frame and selects the template most suitable for reconstructing the current image frame from a set of preset image reconstruction templates. The selection criteria include the execution complexity of the template on the front-end device, the reconstruction latency, and the image region change characteristics.

[0093] In collaborative rendering methods for web data processing, different image frames exhibit significant differences in structural complexity, image variation range, texture distribution, and local occlusion levels. Therefore, a single, fixed image reconstruction template cannot be used for uniform processing. Without precise analysis of the current frame's image features, template selection may mismatch the actual image structure, leading to increased front-end assembly overhead, rendering distortion, or response delays. Therefore, this invention proposes using a cloud-based collaborative rendering machine to extract structural features from the current image frame and dynamically select the optimal template based on the adaptability of multiple templates. This ensures a balance between efficiency and quality in image reconstruction, while also adapting to multi-terminal and multi-environment operating scenarios.

[0094] In this invention, structural features refer to the key graphic information features in an image frame that influence the selection of the reconstruction method, specifically including texture distribution density, number of region boundaries, range of motion changes, proportion of occlusion areas, and brightness gradient distribution. Template fit refers to the comprehensive efficiency index of the current image frame in terms of front-end reconstruction overhead, rendering latency, and image quality when using a certain reconstruction template. Execution complexity refers to the average execution resource consumption of a certain template on the target front-end device, including CPU time, number of GPU calls, and memory usage. Image region change features refer to the distribution of differences between the current frame and the previous frame in terms of spatial location, local content, and color channels.

[0095] like Figure 3 As shown, in one optional implementation, step S30 specifically includes the following sub-steps:

[0096] Step S301: Extract image structural feature data. The cloud-based collaborative rendering machine, based on the currently rendered image frame data, uses image analysis algorithms to calculate the structural feature indicators of the image frame. Specifically, this includes texture density gradient calculation, Sobel edge detection to extract the number of region boundaries, inter-frame pixel difference calculation to obtain the ratio of changing regions, and brightness channel standard deviation statistics to obtain the global contrast level. The above structural feature calculation can be implemented based on a GPU-parallel accelerated image processing framework, and the output is a structural feature vector F, in the format F = [f1, f2, ..., fn], where each fi represents the numerical value of a structural dimension.

[0097] Step S302: Establish a candidate template set. The cloud-based collaborative rendering machine selects all adaptable image reconstruction templates from a preset template library and obtains the performance indicators of the template on the target terminal device. The performance indicators include average execution time, resource consumption level, device compatibility identifier, etc. All information is stored in the template performance database. For each candidate template Tj, its execution complexity Cj and applicable feature range Rj are recorded.

[0098] Step S303: Evaluate template suitability. The system matches the structural feature vector F extracted in step S301 with the applicable scope Rj of each template in step S302, and comprehensively considers the execution complexity Cj of the template on the target device, the front-end rendering path length Lj, and the front-end reconstruction delay Ej to construct a multi-factor evaluation function.

[0099] Score(Tj)=α×Sim(F,Rj)+β×(1 / Cj)+γ×(1 / Ej)

[0100] Where Sim(F,Rj) represents the structural feature matching function, α, β, and γ are weight coefficients, Cj represents the average execution complexity of template Tj, Ej represents the average front-end reconstruction time, and a larger Score(Tj) indicates a higher degree of adaptation.

[0101] Step S304: Select the optimal template. The system sorts the candidate templates Score(Tj) and selects the template T* with the highest score as the optimal reconstruction template for the current image frame, and records its template number, parameter structure, and execution logic. If multiple templates have similar scores, possible implementation schemes include: selecting the one with the lowest execution complexity, prioritizing the template of the same type as the previous frame, or randomly selecting one of the top two scores.

[0102] Step S305: Generate template parameter framework. Based on the definition structure of the selected template T* and the rendering result of the current frame image, the system automatically extracts the texture number, mask index, transformation matrix parameters, channel blending factor, position offset, etc. required by the current template to form a complete template parameter set θ*, and encapsulates it together with the template number into a structured output data packet.

[0103] Through the above steps, the cloud-based collaborative rendering machine can efficiently screen and evaluate the preset template set based on both image structure features and front-end execution performance after the complete image rendering is completed. It can also dynamically select the optimal image reconstruction template for the current frame, thereby ensuring that the front-end collaborative rendering machine completes image reconstruction with minimal overhead within a unit of time, improving system response speed and image consistency, and adapting to the differentiated reconstruction needs of various terminal environments.

[0104] In a specific example, within the aforementioned 3D display scene of the visitor center, the current frame image displays multiple complex glass reflection structures, characterized by a high number of edges, large brightness gradients, and strong local occlusion. The cloud-based collaborative rendering machine, through structural feature extraction, determines that the current frame image has high texture complexity and concentrated areas of change. The system retrieves three candidate templates from the template library: T_001 (mask blending template), T_004 (mesh deformation template), and T_005 (progressive refinement template). Combining this with the front-end device's performance parameters, the system calculates the execution complexity of these three templates on that device to be 5.2, 3.1, and 2.6, respectively, with expected response times of 80ms, 55ms, and 40ms. Finally, based on the template scoring function, the system selects T_005 as the optimal template.

[0105] In step S40, the cloud-based collaborative rendering machine generates a corresponding template parameter set based on the selected image reconstruction template. The template parameter set includes at least one of texture index, mask number, transformation parameters, channel blending factor, and spatial adjustment matrix.

[0106] In collaborative rendering-based web data processing, the front-end collaborative rendering machine lacks complete image content and rendering context during image assembly. Therefore, it relies on a cloud-based collaborative rendering machine to provide explicit template numbers and parameter sets to drive the execution of local image reconstruction templates. The template parameter set not only indicates the resource objects the front-end should select but also determines key execution behaviors such as fusion strategies, spatial positioning, and geometric transformations during image reconstruction. Without a complete template parameter set, the front-end struggles to reconstruct the content and structure of the target image frame, easily leading to image distortion, positional shifts, or reconstruction failure. Therefore, this step generates and transmits a structured template parameter set via the cloud to achieve resource-level and operation-level reconstruction control, ensuring the front-end can accurately execute template logic and obtain high-fidelity image reconstruction results.

[0107] In this invention, the template parameter set refers to a structured data set extracted by the cloud-based collaborative rendering machine based on the defined structure of the reconstructed template from the selected image, within the context of the current image frame, used to guide the execution of the front-end image assembly logic. The texture index is a unique number used to identify a texture in the front-end resource library. The mask number is a resource identifier pointing to a mask image used for local occlusion or region control. Transformation parameters refer to the spatial transformation information applied to a layer or region in the image, typically including scaling factors, rotation angles, translation vectors, etc. Channel blending factors refer to the weight allocation values ​​of each color or texture channel during image overlay or fusion. The spatial adjustment matrix is ​​a two-dimensional or three-dimensional linear transformation matrix used to map textures to the target image region, precisely controlling the geometric position and structural alignment of layers.

[0108] In one optional implementation, step S40 specifically includes the following sub-steps:

[0109] Step S401: Parse the template parameter structure. The cloud-based collaborative rendering machine reconstructs the template number based on the selected image, retrieves the template's definition structure, and extracts the required input parameter fields, including resource type, input format, data structure, and calling method. The template parameter structure definition can adopt a predefined JSON structure or a data table structure, which is used for subsequent parameter filling operations after the system loads it.

[0110] Step S402: Extract texture resource index. The system locates the basic texture resource used for image reconstruction from the current image frame rendering data, according to the template input requirements, extracts its corresponding texture code number or file path information, and stores it in the texture field of the template parameter set. The texture number can be generated based on resource path hash, file ID, or CDN distribution code.

[0111] Step S403: Match mask control information. If the selected template involves local area processing or layer occlusion logic, the system extracts the corresponding mask image resource from the image frame and obtains its mask image number or occlusion area definition rule. The mask number is an index number in the front-end cache resource library, which can be directly called by the mask blending template or transparent texture template.

[0112] Step S404: Calculate the transformation parameters or spatial matrix. If the selected template involves geometric transformations, tile repositioning, or layer affine adjustment operations, the system constructs a two-dimensional affine transformation matrix or a 3D transformation matrix based on the coordinate mapping relationship between the current viewpoint, the target texture area, and the output frame image. The transformation matrix can be represented as a 3×3 or 4×4 linear algebra matrix structure, specifically including combinations of rotation, scaling, and translation matrices. For example, if a two-dimensional affine transformation matrix A is used, its structure is as follows:

[0113] A = [[a11,a12,a13],

[0114] [a21,a22,a23],

[0115] [0,0,1]]

[0116] Where a11 to a23 represent the linear transformation coefficients of the image, the first two rows are the transformation content, and the last row is the homogeneous coordinate retention term.

[0117] Step S405: Generate channel blending factors. If the template supports multi-texture channel blending display logic, the system calculates the blending factor based on layer order, pixel coverage, or transparency information. For example, if the main texture channel blending factor is set to α, then the brightness value L of the synthesized image is:

[0118] L=α×L1+(1-α)×L2

[0119] Where L1 is the pixel value of the main texture, L2 is the pixel value of the blended texture, and α is the blending weight parameter, with a value between 0 and 1.

[0120] Step S406: Integrate and structure the parameters. The system encapsulates all the template parameter fields mentioned above into a unified data structure. The data structure includes the template number, parameter field name, field value, field type, and validation value. After structuring, the data package is ready to be sent to the front-end collaborative rendering machine. The data structure can be encoded using JSON objects, Protocol Buffers, or binary compression format.

[0121] Through the above steps, the cloud-based collaborative rendering machine can accurately extract and calculate the parameters required for the current image frame according to the structural requirements of the selected image reconstruction template, and package and output them in a structured data manner, ensuring that the front end can quickly and accurately execute the image frame assembly task according to the template logic, thereby improving the image restoration quality and terminal response speed.

[0122] In a specific example, using template T_004 (mesh deformation template) from the previous example, after the user's viewpoint switches to the "visitor center side view," the cloud completes rendering and selects this template for image reconstruction. The system extracts the main texture R_017 from the current frame's rendering data and constructs a two-dimensional affine matrix G_011 based on the viewpoint offset angle. This matrix corresponds to the transformation requirement of a slight rotation of the left-side mesh and a 5-pixel translation to the right. Simultaneously, this template does not involve masking or channel blending operations; therefore, the masking and blending factor fields are empty. Finally, the system generates the following structured parameter set:

[0123] {

[0124] "template_id":"T_004",

[0125] "parameters":{

[0126] "main_texture":"R_017",

[0127] "transform_matrix":[[0.98,0.17,5.0],[-0.17,0.98,0.0],[0,0,1]]

[0128] }

[0129] }

[0130] After being encoded, the parameter set is sent to the front-end collaborative rendering machine along with the template number. The front-end calls the execution logic of template T_004 and performs deformation mapping on the main texture R_017 according to the matrix, and finally completes the reconstruction output of the current image frame. The whole process does not depend on the transmission of the complete image, thus achieving efficient image display.

[0131] In step S50, the cloud-based collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits it to the front-end collaborative rendering machine.

[0132] In collaborative rendering-based web data processing, the front-end collaborative rendering machine needs to perform image reconstruction operations based on information transmitted from the cloud. Since the front-end lacks complete rendering capabilities, its image generation relies entirely on the correct invocation of templates and parameters. To achieve decoupling between the front-end and cloud and minimize data transmission, the cloud-based collaborative rendering machine needs to organize the unique identifier of the selected image reconstruction template and its associated template parameter set into a structured data packet and send it to the front-end. This structured data packet serves as the sole basis for the front-end to assemble the image; its structure must meet the requirements of easy parsing, mappability, and low latency, and it must also possess template invocation paths and resource location capabilities to ensure that the front-end can complete stable and efficient image generation tasks on resource-constrained devices.

[0133] In this invention, the unique identifier of an image reconstruction template refers to a number or tag used for quick retrieval and invocation of the corresponding template logic in the front-end system, typically encoded as a string or integer. The template parameter set refers to a structured data set generated in the cloud based on the template structure, containing specific control values ​​such as texture indexes, transformation parameters, mask numbers, and fusion weights. The structured data packet refers to a transmission data unit that encapsulates the template identifier and parameter set in a standard format; its encapsulation method can use JSON, Protocol Buffer, or other cross-platform formats. The organization of the data packet must support high parsing efficiency, clear field semantics, controllable size, and ease of cross-language and cross-engine parsing.

[0134] like Figure 4 As shown, in one optional implementation, step S50 specifically includes the following sub-steps:

[0135] Step S501: Construct the template data structure. The cloud-based collaborative rendering machine initializes the template data structure based on the template parameter set generated in step S40 and the unique identifier of the currently selected template. This structure includes a "template_id" field to identify the current template number and a "parameters" field to encapsulate the corresponding template parameters. Each parameter field should correspond to one of the input interfaces defined in the template, including but not limited to texture numbers, mask numbers, transformation matrices, and channel coefficients. To ensure data consistency, the system adds a field type definition and a data validity check value to each field.

[0136] Step S502: Perform data format encapsulation. The system selects an appropriate data encapsulation protocol based on the characteristics of the target platform and the front-end parsing capabilities. Common encapsulation protocols include, but are not limited to:

[0137] 1) JSON text format, clear structure, suitable for browser-side rendering engines;

[0138] 2) Protocol Buffers use a binary format, resulting in small data size, making them suitable for bandwidth-sensitive scenarios;

[0139] 3) Custom compressed byte stream format, suitable for scenarios with frequent image changes and high concurrency.

[0140] The data encapsulation process includes operations such as field sorting, keyword encoding, Boolean field compression, and optional field truncation to further compress data volume and improve transmission efficiency.

[0141] Step S503: Add structure verification and version control information. To prevent template call anomalies or parameter interpretation failures, the system appends a verification segment and a version identifier field to the data packet. The verification field is the hash value or checksum of the template parameter set, and the front-end performs data integrity verification before unpacking. The version identifier field is used to identify the version number of the current template structure definition, so that the front-end can perform compatibility matching and automatic template structure switching.

[0142] Step S504: Execute structured data packet transmission. The system sends the encapsulated structured data packet to the target front-end collaborative rendering machine via the network transmission module. Transmission methods may include HTTP POST requests, WebSocket push, MQTT message channels, or proprietary transmission protocols. To improve robustness, the system supports basic packet loss retransmission, breakpoint resumption, and encrypted transmission mechanisms. Optional implementation schemes include using an HTTP / 2 multiplexed channel to push data packets, transmitting JSON data in a TLS encrypted channel, or implementing an end-to-end image control signaling channel via WebRTC.

[0143] Through the above steps, the cloud-based collaborative rendering machine can accurately transmit the template identifiers and execution parameters required for image reconstruction to the front end with minimal data volume and optimal structure, ensuring that the front end can complete the image reconstruction task without decoding video frames or image streams, greatly improving network efficiency, data security, and system scalability.

[0144] In a specific example, the cloud-based collaborative rendering machine completes the full rendering of the visitor center side view frame and selects T_004 as the current frame reconstruction template. Based on the defined structure of the mesh deformation template, the system constructs the following structured data package:

[0145]

[0146] The data packet is encapsulated in JSON format and sent to the front-end collaborative rendering machine in real time via a WebSocket channel.

[0147] In step S60, the front-end collaborative rendering machine loads the required graphic data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of image frames according to the template logic and parameter set.

[0148] In collaborative rendering-based web data processing, to achieve efficient and low-latency image display, the front-end collaborative rendering machine, upon receiving structured data packets transmitted from the cloud, immediately invokes a pre-loaded image reconstruction template based on the template identifier. Then, according to the accompanying template parameter set, it extracts relevant textures, masks, matrices, and other data resources from the local resource library, and performs image frame assembly and display operations. This process does not rely on real-time video stream decoding or pixel transmission; instead, it completes image reconstruction through an instruction-based template invocation method, effectively reducing the computational burden on terminal devices and improving graphics response speed and availability under network-constrained conditions.

[0149] Image reconstruction template identifiers are unique identifiers assigned by the cloud to the front-end system, used to uniquely identify and invoke image assembly logic. Graphical data refers to image materials contained in the local resource library, including texture images, mask images, color channel information, spatial matrices, and other texture data. Template logic refers to the predefined image assembly rules in each template, typically implemented by script modules or graphics rendering instructions. The parameter set is a structured execution parameter generated by the cloud after template selection, containing specific control instructions such as texture indices, blending weights, and affine matrices required for reconstruction. Image frame assembly refers to the process by which the front-end constructs the final image output frame in Canvas, WebGL, or other graphics interfaces based on the template logic and local resources.

[0150] like Figure 5 As shown, in one optional implementation, step S60 specifically includes the following sub-steps:

[0151] Step S601: Parse the structured data packet. After receiving the data packet from the cloud, the front-end collaborative rendering machine calls the local parsing engine to parse it, extracting the template number field and parameter field set. The system verifies the integrity and version information of the data packet to ensure that the template structure and parameter set definition are consistent. If parsing fails, a degradation strategy or error message mechanism is triggered.

[0152] Step S602: Match and load template execution logic. The front-end system loads the corresponding template processing module from the registered template library based on the parsed template number. The module is an executable object, internally encapsulating image reconstruction logic, an input field mapping table, and output buffer management functions. Loading methods include dynamic module import or static script binding. Optional implementation schemes include directly mapping the template logic as a JavaScript object, or calling a high-performance rendering subroutine as a WebAssembly module.

[0153] Step S603: Load graphics resources. The system retrieves the corresponding resource file from the local graphics resource library based on the texture number, mask number, material channel, and other fields in the template parameter set. The resource library is organized in a dictionary structure, with the resource number as the key and the memory object or resource path identifier as the value. Resource formats include image files, Canvas cache, GPU texture objects, etc. If the resource is not found, the system can initiate a resource prefetching request according to preset rules or use placeholder resources for temporary display.

[0154] Step S604: Execute the template assembly logic. The system passes the loaded graphics resources as input parameters to the template logic module, and constructs image frames step by step according to the processing order defined by the template. For example, when the template is a mask blending template T_001, the system performs a linear blending operation on the texture image R and the mask image M according to the blending factor α. The image brightness value L is calculated by the following formula:

[0155] L(x,y)=α×R(x,y)+(1-α)×M(x,y)

[0156] Where L(x,y) represents the pixel brightness value at position (x,y) in the final image, R(x,y) and M(x,y) are the corresponding pixel values ​​of the main texture and the mask image, respectively, and α is the fusion factor, ranging from 0 to 1. If the template is a mesh deformation template T_004, the system uses the transformation matrix T to remap the UV coordinates of the texture, performs texture sampling, and draws it onto the target layer.

[0157] Step S605: Output the image frame and submit it to the display buffer. After the template execution is complete, the system will draw the generated image frame to the target Canvas, WebGL rendering context, or other image display area, and refresh the page image content. To ensure a stable frame rate, the system can call the requestAnimationFrame function to schedule the display timing, or trigger interactive response logic after drawing is complete. Optional implementation schemes include using an off-screen Canvas for cached drawing, adopting a double buffering mechanism to improve the smoothness of frame switching, or using a FrameBuffer object in WebGL to complete the frame result output.

[0158] Through the above steps, the front-end collaborative rendering machine can quickly complete the image frame construction and output process based on local cached resources and registered templates after receiving the image reconstruction instruction. This significantly reduces the amount of data transmitted over the network, lowers the consumption of terminal computing resources, and improves the user interaction response speed and system stability under multi-view switching.

[0159] In a specific example, within the aforementioned 3D display system for the visitor center, the front-end collaborative renderer receives a data packet from the cloud, indicating template number T_004, main texture R_017, and spatial transformation matrix G_011. The system first parses the data packet, loads the mesh deformation module corresponding to template T_004, and extracts the R_017 texture and matrix G_011 from the local resource library. Subsequently, the template logic performs an affine transformation on the texture coordinates, rendering the deformed image onto the front-end Canvas context. Once the image is rendered, the system automatically submits the frame buffer and updates the display viewpoint in the user interface. This entire process achieves high-fidelity image reconstruction and rapid response without transmitting image data, allowing users to continuously and effortlessly switch between 3D viewing angles on their terminal devices.

[0160] For any module structures not specifically defined in this invention, the existing technical descriptions shall prevail. The prior art mentioned in the foregoing background and specific embodiments sections can be considered part of this invention and used to understand the meaning of certain technical features or parameters.

Claims

1. A web data processing method based on collaborative rendering, characterized in that, Includes the following steps: The front-end collaborative rendering machine pre-stores multiple image reconstruction templates and graphics resources. The image reconstruction templates are used to assemble the current image frame after receiving specified template parameters. After receiving an interaction request or viewpoint change information from the front-end user, the cloud-based collaborative rendering machine performs complete image rendering of the target image in the current frame. The cloud-based collaborative rendering machine extracts structural features from the current image frame and selects the template most suitable for reconstructing the current image frame from a set of preset image reconstruction templates. The selection criteria include the execution complexity of the template on the front-end device, the reconstruction latency, and the image region change characteristics. The cloud-based collaborative rendering machine generates a corresponding template parameter set based on the selected image reconstruction template. The template parameter set includes at least one of texture index, mask number, transformation parameters, channel blending factor, and spatial adjustment matrix. The cloud-based collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits it to the front-end collaborative rendering machine. The front-end collaborative rendering machine loads the required graphics data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of image frames according to the template logic and parameter set; The step of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources also includes: establishing a parameter mapping relationship between templates and resources. The parameter mapping structure binds the input fields required for template definition with local resource indexes for quick resource matching during subsequent template execution. The cloud-based collaborative rendering machine selects the template most suitable for the reconstruction of the current image frame from a set of preset image reconstruction templates. The steps include: scoring the candidate templates based on the template fit function, the parameters of which include structural feature matching degree, template execution complexity and front-end reconstruction latency, and the template with the highest score is selected as the optimal template for the current frame.

2. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources include: constructing an image reconstruction template library, wherein each template in the template library has a unique number and defines the template type, input resource structure and execution logic. The template type includes at least one of the following: mask blending template, channel combination template, block replacement template, mesh deformation template and progressive refinement template.

3. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The step of the front-end collaborative rendering machine pre-storing multiple image reconstruction templates and graphics resources further includes: loading graphics resources and initializing the cache. The graphics resources include texture images, mask images, transformation matrices and material channel information. The resources are stored in the local resource manager with a unique identifier and support version verification.

4. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps of the cloud-based collaborative rendering machine to perform complete image rendering of the target image in the current frame after receiving the interaction request or view change information from the front-end user include: parsing user input data to generate camera parameters, constructing the view matrix of the current frame, and determining the set of 3D scene objects to be rendered based on the view frustum culling technique.

5. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps of the cloud-based collaborative rendering machine to extract structural features from the current image frame include: calculating texture density, number of region boundaries, brightness distribution, and pixel variation regions based on the current image frame data, and constructing a structural feature vector, which is used to guide subsequent template selection.

6. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps for generating a template parameter set using the cloud-based collaborative rendering machine include: extracting texture indexes, mask numbers, channel blending factors, and spatial transformation matrix information, and organizing them into a structured template parameter set according to the template definition format.

7. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps of the cloud-based collaborative rendering machine packaging template identifiers and template parameter sets into structured data packets and transmitting them to the front end include: constructing a data packet structure, the structure including template number, parameter fields, version number and verification information, and transmitting it to the front-end collaborative rendering machine via a network protocol.

8. The web data processing method based on collaborative rendering according to claim 1, characterized in that, The steps of the front-end collaborative rendering machine to assemble and display image frames according to template logic and parameter set include: parsing the template number and parameter content in the structured data packet, calling the corresponding template logic and loading local resources, performing image reconstruction operation in Canvas or WebGL rendering context, and drawing the result image to the display area to complete the image frame display.

Images