WEB data processing method based on collaborative rendering

Abstract

The invention belongs to the field of image processing, and provides a WEB data processing method based on collaborative rendering, which comprises the following steps of: pre-storing a plurality of image reconstruction templates and graphic resources by a front-end collaborative rendering machine; the cloud collaboration rendering machine executes complete image rendering of the current frame of target picture; the cloud collaborative rendering machine performs structural feature extraction on the current image frame, and selects a template most suitable for reconstruction of the current image frame from a plurality of preset image reconstruction templates; the cloud collaborative rendering machine generates a corresponding template parameter set according to the selected image reconstruction template; the cloud collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits the structured data packet to the front-end collaborative rendering machine; and the front-end collaborative rendering machine reconstructs a template identifier according to the received image, loads required graphic data from a local resource library, and completes splicing and display of image frames according to template logic and a parameter set.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a WEB data processing method based on collaborative rendering. Background Art

[0002] With the rapid development of 3D graphics display technology on the web, an increasing number of online applications rely on high-quality image rendering and dynamic visualization capabilities, such as virtual scenic spots, architectural visualization, online museums, and industrial remote monitoring. Due to widespread limitations in terminal device performance, power consumption, and network conditions, traditional fully local rendering solutions are unable to meet the demands for high-precision, low-latency 3D image interaction.

[0003] Existing technologies employ a "cloud rendering + image streaming" model, centralizing image rendering tasks on the server side and then pushing them to the front-end in the form of image frames, video streams, or differential images. While this approach reduces the computing burden on the terminal to some extent, it still has the following limitations: First, the image frame data transmission volume is large, which can easily cause lag in mobile networks or weak network environments; second, the front-end only has image decoding capabilities and cannot reconstruct and extend structured information, limiting interactive performance; third, cloud-based image content generation and distribution lacks a content splitting mechanism, making it difficult to adapt to diverse terminal configurations.

[0004] Therefore, the existing WEB 3D rendering system based on image streaming transmission cannot achieve fast response and efficient update in low-bandwidth, high-interaction frequency scenarios. Its technical problems such as high presentation latency, heavy resource dependence, and weak adaptability on the terminal side need to be solved urgently. Summary of the Invention

[0005] In order to solve the problems in the prior art, the present invention provides a WEB data processing method based on collaborative rendering, comprising the following steps:

[0006] The front-end collaborative rendering machine pre-stores multiple image reconstruction templates and graphic resources, wherein the image reconstruction template is used to assemble the current image frame after receiving the specified template parameters;

[0007] After receiving the front-end user's interactive request or perspective change information, the cloud-based collaborative rendering machine performs full image rendering of the current frame target screen;

[0008] The cloud-based collaborative rendering machine extracts structural features from the current image frame and selects the template that is most suitable for reconstructing the current image frame from a plurality of preset image reconstruction templates. The selection is based on the template's execution complexity on the front-end device, reconstruction latency, and image region change characteristics.

[0009] The cloud collaborative rendering machine generates a corresponding template parameter set according to the selected image reconstruction template, wherein the template parameter set includes at least one of a texture index, a mask number, a transformation parameter, a channel mixing factor, and a space adjustment matrix;

[0010] The cloud-side collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits it to the front-end collaborative rendering machine;

[0011] The front-end collaborative rendering machine loads the required graphic data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of the image frame according to the template logic and parameter set.

[0012] Furthermore, the step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine includes: constructing an image reconstruction template library, each template in the template library has a unique number, and defining the template type, input resource structure and execution logic, and the template type includes at least one of a mask fusion template, a channel combination template, a block replacement template, a grid deformation template and a progressive refinement template.

[0013] Furthermore, the step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine further includes: loading graphic resources and initializing the cache, the graphic resources including texture images, mask images, transformation matrices and material channel information, the resources are stored in the local resource manager with a unique identifier and support version verification.

[0014] Furthermore, the step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine also includes: establishing a parameter mapping relationship between the template and the resource, and the parameter mapping structure binds the input fields required for the template definition with the local resource index for rapid resource matching during subsequent template execution.

[0015] Furthermore, the cloud-based collaborative rendering machine performs the steps of rendering the complete image of the current frame target screen after receiving the front-end user's interaction request or perspective change information, including: parsing the user input data to generate camera parameters, constructing the current frame view matrix, and determining the set of three-dimensional scene objects that need to be rendered based on the frustum culling technology.

[0016] Furthermore, the step of extracting structural features of the current image frame by the cloud-based collaborative rendering machine includes: calculating texture density, number of region boundaries, brightness distribution and pixel change area based on the current image frame data, and constructing a structural feature vector, which is used to guide subsequent template selection.

[0017] Furthermore, the step of the cloud-based collaborative rendering machine selecting the template that is most suitable for reconstructing the current image frame from multiple preset image reconstruction templates includes: scoring the candidate templates based on the template fitness function, the parameters of the template fitness function include structural feature matching, template execution complexity and front-end reconstruction delay, and the template with the highest score is selected as the optimal template for the current frame.

[0018] Furthermore, the step of generating a template parameter set by the cloud collaborative rendering machine includes: extracting texture index, mask number, channel fusion factor and space transformation matrix information, and organizing them into a structured template parameter set according to the template definition format.

[0019] Furthermore, the step of the cloud-based collaborative rendering machine packaging the template identifier and template parameter set into a structured data packet and transmitting it to the front end includes: constructing a data packet structure, which includes a template number, parameter field, version number and verification information, and transmitting it to the front-end collaborative rendering machine through a network protocol.

[0020] Furthermore, the front-end collaborative rendering machine completes the steps of assembling and displaying the image frame according to the template logic and parameter set, including: parsing the template number and parameter content in the structured data packet, calling the corresponding template logic and loading local resources, performing image reconstruction operations in the Canvas or WebGL rendering context, and drawing the resulting image into the display area to complete the image frame display.

[0021] The present invention structurally separates the image rendering and image reconstruction tasks, migrates the computationally intensive image rendering process to the cloud for execution, and deploys a variety of structured image reconstruction templates on the front end. In conjunction with the template parameter sets, local assembly and rapid display of image frames are achieved, thereby significantly reducing network bandwidth usage and improving the system's operability and graphics response speed in low-resource environments.

[0022] The image reconstruction templates provided by this invention boast a high degree of structural reusability and consistent execution paths, supporting efficient reconstruction in diverse image-changing scenarios. Through a dynamic template matching mechanism based on image features, the system adaptively selects the optimal template based on the current image frame content and the capabilities of the front-end device, effectively improving terminal image processing efficiency and overall system stability.

[0023] In addition, the structured template parameters and resource indexing mechanism adopted by the present invention enable the front-end to complete rapid image combination on the resource cache structure without re-decoding or rendering, thereby achieving millisecond-level perspective switching or screen updates, and is widely applicable to application scenarios with high response delay requirements such as mobile terminals, low-power devices, and edge rendering platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 It is the main flow chart of the method of the present invention;

[0026] Figure 2 This is a schematic diagram of the end-cloud collaborative rendering principle of the present invention;

[0027] Figure 3 It is a structural flow chart of template selection and parameter generation;

[0028] Figure 4 It is a schematic diagram of the logical composition of a structured data packet;

[0029] Figure 5 It is the front-end image frame assembly flow chart. DETAILED DESCRIPTION

[0030] Below, the invention is preferably described with reference to the accompanying drawings and specific embodiments.

[0031] This embodiment solves the above problem through the following steps:

[0032] In one embodiment, if Figure 1 As shown, the present invention discloses a WEB data processing method based on collaborative rendering, which is as follows Figure 2 The information interaction and task allocation between the front-end collaborative rendering machine and the cloud collaborative rendering machine shown in the figure realize the dynamic separation of image rendering resources, template-driven reconstruction and low-bandwidth visualization output, thereby realizing a WEB data processing method for efficient display and real-time interaction of complex three-dimensional images in a terminal environment with limited bandwidth and computing resources.

[0033] In order to achieve the above-mentioned purpose, the WEB data processing method based on collaborative rendering specifically includes the following steps:

[0034] In step S10 , the front-end collaborative rendering machine pre-stores a plurality of image reconstruction templates and graphic resources, wherein the image reconstruction template is used to assemble the current image frame after receiving designated template parameters.

[0035] In the collaborative rendering WEB data processing method, in order to achieve rapid reconstruction and efficient display of image frames on terminal devices, it is necessary to migrate the computationally intensive image rendering process to the cloud for execution, and deploy the image reconstruction operation to the front end in a modular and templated manner. Since the front-end devices usually have limited computing power, if the front end completes the rendering and image construction operations by itself, it will lead to significant response delays and reduced interactive performance, seriously affecting the user experience. Therefore, this method pre-stores multiple image reconstruction templates and graphic resources on the front end. After receiving the template parameter information returned by the cloud, it can directly call the corresponding templates and resources to complete the assembly and output of the image frames, thereby greatly reducing the computational burden of image generation and the network transmission bandwidth.

[0036] In the present invention, an image reconstruction template refers to a structured template data structure that describes the combination and transformation rules of how to generate image frames based on graphic resources. Graphic resources refer to elements such as texture data, mask maps, channel mixing information, and transformation matrices that can be used for image assembly. The resources can be uniformly loaded into the local cache by the content distribution network when the page is first accessed, or pre-fetched based on user behavior. Template parameters refer to a set of control parameters generated by a cloud-based collaborative rendering machine, which are used to specify operational details such as resource indexes, transformation methods, and overlay weights that the front-end needs to select during the template application process.

[0037] In an optional specific implementation, step S10 includes the following sub-steps:

[0038] Step S101: Build an image reconstruction template library

[0039] This step is completed by the front-end system during the initialization phase, specifically:

[0040] Define the template set structure and use unique template numbers (such as T_001, T_002) to identify it;

[0041] Each template definition includes: template type, input resource structure, execution logic (encapsulated in JS module or GLSL fragment);

[0042] Template types include but are not limited to:

[0043] Mask fusion template (T_001): local blending based on texture and mask;

[0044] Channel combination template (T_002): load RGB channels separately and then synthesize;

[0045] Block replacement template (T_003): replaces the specified block area;

[0046] Mesh deformation template (T_004): controls the bending or translation of the texture through a 2D transformation matrix;

[0047] Progressive refinement template (T_005): Prioritize low-resolution filling and replace high-definition layers frame by frame.

[0048] Template definitions are encapsulated in JSON or JS module structures and registered with the local template scheduling engine.

[0049] Step S102: Load graphic resources and initialize cache

[0050] This step is performed during the page loading phase and includes:

[0051] Preload texture atlases, mask maps, transformation matrix libraries, and material channel configurations from the Content Delivery Network (CDN);

[0052] All graphic resources are coded and managed with unique IDs (such as R_007, M_003, L_004);

[0053] After loading is complete, it is stored in the local graphics resource manager and cached using IndexedDB or memory dictionary structure;

[0054] Supports version number and hash verification to ensure consistency between resources and templates.

[0055] Step S103: Establishing a template parameter structure mapping relationship

[0056] This step describes the logical binding between the template's required input parameters and local resources, including:

[0057] Traverse all templates and extract input parameter structure;

[0058] Generate a template-parameter mapping description table, for example:

[0059]

[0060] The structure can be used for parameter parsing during template scheduling;

[0061] If resources are missing, the system automatically triggers prefetching or placeholder replacement strategies.

[0062] Step S104: Register template and resource index to execution engine

[0063] This step pre-binds the template and resource execution logic to the front-end rendering controller, including:

[0064] Register the execution module for each template;

[0065] Define the template calling interface, such as renderFrame(T_001,params);

[0066] Use JS object pools or WebGL context cache processing units to ensure low-latency calls;

[0067] If the template has transformation logic (such as T_004 contains a rotation matrix), its linear algebra operation library is loaded during the registration phase.

[0068] By pre-storing multiple structured image reconstruction templates in the front-end collaborative rendering engine and loading matching sets of graphics resources, the execution of complex image rendering tasks can be shifted from runtime to the initialization phase, significantly reducing the execution overhead of assembling image frames. Furthermore, the template structure of the present invention is highly modular and combinable, adapting to the reconstruction requirements of different image frame types and providing a unified infrastructure for subsequent operations such as incremental image updates, local occlusion repair, and multi-view switching.

[0069] In a specific example, taking a smart scenic spot 3D display system as an example, when a user first enters the scenic spot entrance perspective page, the front-end collaborative rendering machine loads the following content:

[0070] Template T_001 (mask fusion template), the structure is:

[0071] Input: main texture ID, mask ID, blending factor f

[0072] Logic: Final(x,y)=f×Main(x,y)+(1-f)×Mask(x,y)

[0073] Template T_004 (grid deformation template), the structure is:

[0074] Input: Texture ID, transformation matrix M (2×2)

[0075] Logic: Transform the texture UV coordinates and then resample and redraw

[0076] At the same time, the front-end preloads the following resources:

[0077] Main texture R_017 (appearance of the visitor center), mask map M_005 (shadow area), texture R_022 (glass reflection), space matrix G_011 (left view rotation matrix), etc.

[0078] When the user switches from the main entrance to the side view, the cloud detects that the image change is a slight local view switch, determines that template T_004 is the most suitable for this reconstruction operation, and sends a data packet:

[0079]

[0080] After receiving the data packet, the front end calls the transformation execution logic of template T_004, takes the R_017 map and G_011 matrix as input, and completes the image frame output after mesh deformation. The reconstruction delay is controlled within 40ms. The system does not need to transmit the entire frame image or video stream, which significantly saves network bandwidth and response time.

[0081] In step S20 , the cloud collaborative rendering machine performs complete image rendering of the current frame target screen after receiving the front-end user's interactive request or perspective change information.

[0082] In order to ensure that WEB terminal devices can display complex three-dimensional scenes with low latency and high efficiency, the present invention completes the image rendering task by placing it in a cloud-based collaborative rendering machine. Since image rendering involves complex computationally intensive operations such as lighting calculations, occlusion processing, texture mapping, and space transformation, if it is executed on the front-end device, it will seriously restrict the system performance, especially in application scenarios where image frames are frequently switched and user interaction is intensive. By deploying the image rendering process to the cloud, it can rely on a high-performance graphics processing unit (GPU) to complete large-scale parallel rendering tasks, and dynamically generate high-precision image frames based on user interactive input or changes in perspective, while providing the front end with structured output data for assembly, thereby improving the overall system's graphics response speed and picture continuity.

[0083] In this invention, the current frame target image refers to the 3D perspective image corresponding to the scene state triggered by user interaction at a certain moment, which contains the input information required for rendering, such as the viewpoint position, viewing direction, field of view angle, observed object, and occlusion relationship. Image rendering refers to the process of completing graphics pipeline operations such as rasterization, shading, texture mapping, ambient lighting, and post-processing under given scene data and camera parameters to output the final image frame data. Complete image rendering refers to the comprehensive rendering task that includes viewpoint segmentation, material sampling, layer overlay, and global illumination, as distinguished from local update or incremental frame filling operations.

[0084] In an optional specific implementation, step S20 specifically includes the following sub-steps:

[0085] Step S201: Receive interaction event data, including but not limited to mouse clicks, touch slides, keyboard input, gravity sensing, and head-mounted device posture information. The system parses the data and identifies the behavior, extracting geometric information parameters for 3D scene mapping. These geometric information parameters include the viewpoint position vector, viewing direction vector, camera near and far plane distance, and field of view angle.

[0086] Step S202 constructs the current frame's view matrix. This constructs a 3D camera matrix structure based on the geometric information parameters extracted in step S201. The camera matrix consists of a view matrix, a projection matrix, and a camera intrinsic parameter matrix, which define the mapping between the viewpoint in the scene and the 3D world coordinates. Matrix calculations can be defined using the OpenGL standard right-handed coordinate system or constructed using the glMatrix library according to WebGL conventions.

[0087] Step S203 performs 3D scene segmentation. Based on the scene structure graph and a spatial segmentation algorithm, the set of 3D objects that need to be rendered within the current field of view is determined. Scene elements that are not within the view frustum or are completely occluded are eliminated to reduce redundant graphics processing. The spatial segmentation algorithm can use an octree, a BVH bounding box hierarchy, or a Z-buffer-based view frustum culling strategy.

[0088] In step S204, a complete image rendering process is performed. The system executes the following graphics rendering steps in sequence according to the selected rendering engine (such as Unreal Engine, Unity, custom WebGPU renderer, etc.): loading materials and map resources, executing geometry shader and fragment shader programs, sampling various texture resources, completing ambient lighting, shadow processing and global illumination model, performing post-processing operations such as anti-aliasing, HDR synthesis and tone mapping, and finally generating a complete image frame output.

[0089] Step S205, output the structured rendering result, and convert the rendered image frame content into a structured data packet according to the reconstruction template characteristics of the front-end collaborative rendering machine. The data packet content includes image area description information, reconfigurable template type selection information, template parameter generation results and resource call index. The data structure meets the logical requirements of the front-end for rapid assembly and reconstruction according to the template.

[0090] Through the above steps, the cloud-based collaborative rendering machine can efficiently complete the complete rendering task of the current image frame based on the interactive behavior triggered by the front-end, and encode and output the rendering results in a structured and templated manner, significantly reducing the front-end image generation burden. At the same time, it improves the system's interactive response capabilities and picture consistency, and is suitable for the multi-perspective rapid switching and asynchronous assembly requirements of complex three-dimensional scenes.

[0091] In a specific example, a user swipes from the "front view" of the visitor center to the "side view." The front-end collaborative renderer uploads the camera's direction change, offset angle, and observation target number to the cloud-based collaborative renderer. Upon receiving this request, the cloud generates a new camera matrix based on the viewpoint change, invokes the scene graph to perform octree frustum culling, determines the set of geometric objects to be rendered, loads the corresponding textures and materials using the Unreal Engine rendering engine, performs PBR lighting rendering, and generates the target frame image.

[0092] In step S30, the cloud-based collaborative rendering machine extracts structural features of the current image frame and selects the template that is most suitable for reconstructing the current image frame from multiple preset image reconstruction templates. The selection criteria include the execution complexity of the template on the front-end device, the reconstruction delay, and the image area change characteristics.

[0093] In the collaborative rendering WEB data processing method, due to the significant differences in structural complexity, image variation range, texture distribution, and local occlusion degree between different image frames, a single fixed image reconstruction template cannot be used for unified processing. If the image features of the current frame are not accurately analyzed, the template selection may not match the actual structure of the image, resulting in increased front-end assembly overhead, rendering distortion, or response delay. Therefore, the present invention proposes to extract the structural features of the current image frame through a cloud-based collaborative rendering machine, and dynamically select the optimal template based on the adaptability of multiple templates to ensure the efficiency and quality balance of image reconstruction, while adapting to multi-terminal and multi-environment operation scenarios.

[0094] In the present invention, structural features refer to key graphic information features in the image frame that affect the selection of reconstruction methods, specifically including texture distribution density, number of region boundaries, range of motion change, proportion of occlusion area, brightness gradient distribution, etc. Template adaptability refers to the comprehensive efficiency index of the current image frame in terms of front-end reconstruction overhead, rendering delay and image quality when using a certain reconstruction template. Execution complexity refers to the average execution resource consumption of a template on the target front-end device, including CPU time, number of GPU calls and memory usage, etc. Image area change characteristics refer to the distribution of differences between the current frame and the previous frame in terms of spatial position, local content, color channels, etc.

[0095] like Figure 3 As shown, in an optional specific implementation, step S30 specifically includes the following sub-steps:

[0096] Step S301 extracts image structural feature data. The cloud-based collaborative rendering engine uses an image analysis algorithm to calculate structural feature metrics based on the currently rendered image frame data. Specifically, this includes calculating texture density gradients, performing Sobel edge detection to extract the number of regional boundaries, calculating inter-frame pixel differences to obtain the ratio of variable regions, and using luminance channel standard deviation statistics to obtain the global contrast level. This structural feature calculation can be implemented using a GPU-accelerated image processing framework, and the output is a structural feature vector F in the format F = [f1, f2, ..., fn], where each fi represents a numerical value for a structural dimension.

[0097] In step S302, a candidate template set is established. The cloud-based collaborative rendering engine selects all adaptable image reconstruction templates from a preset template library and obtains the template's performance metrics on the target terminal device. These metrics include average execution time, resource consumption level, and device compatibility. All this information is stored in a template performance database. For each candidate template Tj, its execution complexity Cj and applicable feature range Rj are recorded.

[0098] Step S303: Evaluate the template adaptability. The system performs a matching calculation based on the structural feature vector F extracted in step S301 and the applicable range Rj of each template in step S302. It also considers the execution complexity Cj of the template on the target device, the front-end rendering path length Lj, and the front-end reconstruction delay Ej to construct a multi-factor evaluation function:

[0099] Score(Tj)=α×Sim(F,Rj)+β×(1 / Cj)+γ×(1 / Ej)

[0100] Among them, Sim(F,Rj) represents the structural feature matching function, α, β, and γ are weight coefficients, Cj represents the average execution complexity of template Tj, Ej represents the average front-end reconstruction time, and the larger the Score(Tj), the higher the adaptability.

[0101] In step S304, the system selects the optimal template. The system sorts the candidate templates by Score(Tj) and selects the template with the highest score, T*, as the optimal reconstruction template for the current image frame. The template's template number, parameter structure, and execution logic are recorded. If multiple templates have similar scores, optional implementation options include selecting the one with the lowest execution complexity, prioritizing the template of the same type as the previous frame, or randomly selecting one of the top two scoring templates.

[0102] Step S305, generate a template parameter framework. The system automatically extracts the texture number, mask map index, transformation matrix parameters, channel mixing factor, position offset, etc. required for the current template based on the definition structure of the selected template T* and the current frame image rendering result to form a complete template parameter set θ*, and encapsulates it together with the template number into a structured output data packet.

[0103] Through the above steps, the cloud-based collaborative rendering machine can efficiently screen and evaluate the preset template set after the complete image rendering based on the dual indicators of image structure characteristics and front-end execution performance, and dynamically select the optimal image reconstruction template for the current frame, thereby ensuring that the front-end collaborative rendering machine completes image reconstruction with minimal overhead within unit time, improving system response rate and image consistency, and adapting to differentiated reconstruction needs in multiple terminal environments.

[0104] In a specific example, in the aforementioned three-dimensional display scene of the visitor center, the current frame image display area involves multiple complex glass reflective structures, with a high number of edges, large brightness gradients, and strong local occlusion. The cloud-based collaborative rendering machine extracts structural features and determines that the texture complexity of the current frame image is high and the change area is concentrated. The system retrieves three candidate templates, T_001 (mask fusion template), T_004 (grid deformation template), and T_005 (progressive refinement template), from the template library. Combined with the performance parameters of the front-end device, the execution complexity of the three on the device is calculated to be 5.2, 3.1, and 2.6, respectively, and the expected response delay is 80ms, 55ms, and 40ms. Finally, based on the template scoring function, the system selects T_005 as the optimal template.

[0105] In step S40 , the cloud-based collaborative rendering machine generates a corresponding template parameter set according to the selected image reconstruction template, where the template parameter set includes at least one of a texture index, a mask number, a transformation parameter, a channel mixing factor, and a space adjustment matrix.

[0106] In the collaborative rendering WEB data processing method, the front-end collaborative rendering machine does not have complete image content and rendering context during the image assembly process, so it needs to rely on the cloud-based collaborative rendering machine to provide a clear template number and parameter set to drive the execution of the local image reconstruction template. The template parameter set not only indicates the resource object that the front-end should select, but also determines the key execution behaviors such as fusion strategy, spatial positioning, and geometric transformation in the image reconstruction process. If a complete template parameter set is not provided, it will be difficult for the front-end to restore the content and structure of the target image frame, which can easily cause image distortion, position offset, or reconstruction failure. Therefore, this step generates and transmits a structured template parameter set through the cloud to achieve reconstruction control at the resource level and operation level, ensuring that the front-end can accurately execute the template logic and obtain high-fidelity image reconstruction results.

[0107] In the present invention, the template parameter set refers to a structured data set extracted by the cloud-based collaborative rendering machine in the context of the current image frame based on the definition structure of the selected image reconstruction template, and is used to guide the execution of the front-end image assembly logic. The texture index refers to a unique number used to identify a texture in the front-end resource library. The mask number refers to the resource identifier pointing to the mask image used for local occlusion or area control. The transformation parameters refer to the spatial transformation information applied to a layer or area in the image, usually including the scaling factor, rotation angle, translation vector, etc. The channel mixing factor refers to the weight distribution value of each color or texture channel during the image superposition or fusion process. The spatial adjustment matrix refers to a two-dimensional or three-dimensional linear transformation matrix used to map the texture to the target image area, which is used to accurately control the geometric position and structural alignment of the layer.

[0108] In an optional specific implementation, step S40 specifically includes the following sub-steps:

[0109] In step S401, the cloud-based collaborative rendering engine parses the template parameter structure. Based on the selected image reconstruction template number, the cloud-based collaborative rendering engine retrieves the template's definition structure and extracts the required input parameter fields, including the resource type, input format, data structure, and call method. The template parameter structure definition can use a predefined JSON structure or a data table structure, which is loaded into the system for subsequent parameter filling operations.

[0110] In step S402, the system extracts the texture resource index. Based on the template input requirements, the system locates the base texture resource for image reconstruction from the current frame's rendered data. The system then extracts the corresponding texture ID or file path information and stores it in the texture field of the template parameter set. The texture ID can be generated based on a resource path hash, file ID, or CDN distribution code.

[0111] Step S403 matches the mask control information. If the selected template involves local area processing or layer occlusion logic, the system extracts the corresponding mask image resource from the image frame and obtains its mask image number or occlusion area definition rules. The mask number is an index number in the front-end cache resource library and can be directly called by the mask fusion template or transparent map template.

[0112] Step S404, calculate the transformation parameters or spatial matrix. If the selected template involves geometric transformation, tile repositioning or layer affine adjustment operations, the system constructs a two-dimensional affine transformation matrix or a 3D transformation matrix based on the coordinate mapping relationship between the current viewpoint, the target mapping area and the output frame image. The transformation matrix can be represented as a 3×3 or 4×4 linear algebraic matrix structure, specifically including a combination of a rotation matrix, a scaling matrix, and a translation matrix. For example, if a two-dimensional affine transformation matrix A is used, its structure is:

[0113] A=[[a11,a12,a13],

[0114] [a21,a22,a23],

[0115] [0,0,1]]

[0116] Among them, a11 to a23 represent the linear transformation coefficients of the image, the first two lines are the transformation content, and the last line is the homogeneous coordinate preservation item.

[0117] Step S405: Generate a channel fusion factor. If the template supports the fusion display logic of multiple texture channels, the system calculates the fusion factor based on the layer order, pixel coverage, or transparency information. For example, if the main texture channel fusion factor is set to α, the brightness value L of the synthesized image is:

[0118] L=α×L1+(1-α)×L2

[0119] Where L1 is the main map pixel value, L2 is the fused map pixel value, and α is the fusion weight parameter, which ranges from 0 to 1.

[0120] In step S406, the system packages all the template parameter fields into a unified data structure, including the template number, parameter field name, field value, field type, and checksum. The structured data packet is then sent to the front-end collaborative rendering server. This data structure can be encoded in JSON objects, Protocol Buffers, or binary compression formats.

[0121] Through the above steps, the cloud-based collaborative rendering machine can reconstruct the structural requirements of the template for the selected image, accurately extract and calculate the various parameters required for the current image frame, and package and output them in the form of structured data, ensuring that the front-end can quickly and accurately perform image frame assembly tasks based on template logic, thereby improving image restoration quality and terminal response speed.

[0122] In a specific example, still taking the template T_004 (grid deformation template) in the previous example as an example, after the user's perspective switches to the "side view of the visitor center", the cloud completes the rendering and selects the template for image reconstruction. The system extracts the main map R_017 from the current frame rendering data and constructs a two-dimensional affine matrix G_011 based on the perspective offset angle. This matrix corresponds to the transformation requirements of a slight rotation of the grid on the left side of the image and a 5-pixel translation to the right. At the same time, this template does not involve mask maps or channel fusion operations, so the mask field and fusion factor field are empty. Finally, the system generates the following structured parameter set:

[0123] {

[0124] "template_id":"T_004",

[0125] "parameters":{

[0126] "main_texture":"R_017",

[0127] "transform_matrix":[[0.98,0.17,5.0],[-0.17,0.98,0.0],[0,0,1]]

[0128] }

[0129] }

[0130] After being encoded, the parameter set is sent to the front-end collaborative rendering machine along with the template number. The front-end calls the execution logic of template T_004 and deforms and maps the main texture R_017 according to the matrix, finally completing the reconstruction and output of the current image frame. The entire process does not rely on complete image transmission, achieving efficient image display.

[0131] In step S50 , the cloud-side collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits the data packet to the front-end collaborative rendering machine.

[0132] In the collaborative rendering WEB data processing method, the front-end collaborative rendering machine needs to perform image reconstruction operations based on the information transmitted from the cloud. Since the front-end does not have complete rendering capabilities, its image generation is completely dependent on the correct calling of templates and parameters. In order to achieve end-cloud decoupling and minimize data transmission, the cloud-based collaborative rendering machine needs to organize the unique identifier of the selected image reconstruction template and the corresponding template parameter set into a structured data packet and send it to the front-end. As the only basis for the front-end to assemble the image, the structure of this structured data packet must meet the requirements of easy parsing, mappability, and low latency, and must have template calling paths and resource location capabilities to ensure that the front-end can complete stable and efficient image generation tasks on resource-constrained devices.

[0133] In the present invention, the unique identifier of the image reconstruction template refers to a number or label used to quickly retrieve and call the corresponding template logic in the front-end system, usually encoded in the form of a string or integer. The template parameter set refers to a structured data set generated by the cloud based on the template structure, which contains specific control values ​​such as map index, transformation parameters, mask number, fusion weight, etc. A structured data packet refers to a transmission data unit that encapsulates the template identifier and parameter set in a standard format, and the encapsulation method can adopt JSON, Protocol Buffer or other cross-platform formats. The organization of the data packet must support high parsing efficiency, clear field semantics, controllable size, and facilitate cross-language and cross-engine parsing.

[0134] like Figure 4 As shown, in an optional specific implementation, step S50 specifically includes the following sub-steps:

[0135] Step S501: Construct a template data structure. The cloud-based collaborative rendering machine initializes the template data structure based on the template parameter set generated in step S40 and the unique identifier of the currently selected template. This structure includes a "template_id" field, which is used to identify the current template number, and a "parameters" field, which is used to encapsulate the corresponding parameters of the template. The parameter fields should correspond to the input interfaces defined by the template one by one, including but not limited to the texture number, mask number, transformation matrix, and channel coefficients. To ensure data consistency, the system adds a field type definition and a data validity check value to each field.

[0136] Step S502: Data format encapsulation is performed. The system selects an appropriate data encapsulation protocol based on the target platform characteristics and front-end parsing capabilities. Common encapsulation protocols include but are not limited to:

[0137] 1) JSON text format, with clear structure, suitable for browser rendering engines;

[0138] 2) Protocol Buffer binary format, small data size, suitable for bandwidth-sensitive scenarios;

[0139] 3) Customized compressed byte stream format, suitable for scenarios with frequent image changes and high concurrency.

[0140] The data encapsulation process includes operations such as field order sorting, keyword encoding, Boolean field compression, and optional field truncation to further compress data volume and improve transmission efficiency.

[0141] Step S503: Adding structure verification and version control information. To prevent template call anomalies or parameter interpretation failures, the system appends a verification segment and version identification field to the data packet. The verification field is a hash value or checksum of the template parameter set, and the front-end performs data integrity verification before unpacking. The version identification field is used to identify the version number of the current template structure definition, allowing the front-end to perform compatibility matching and automatically switch template structures.

[0142] Step S504 executes structured data packet transmission. The system sends the encapsulated structured data packet to the target front-end collaborative rendering machine via the network transmission module. Transmission methods may include HTTP POST requests, WebSocket push, MQTT message channels, or proprietary transmission protocols. To improve robustness, the system supports basic packet retransmission, breakpoint resumption, and encrypted transmission mechanisms. Optional implementation solutions include using HTTP2 multiplexing channels to push data packets, transmitting JSON data in TLS encrypted channels, or implementing end-to-end image control signaling channels through WebRTC.

[0143] Through the above steps, the cloud-based collaborative rendering machine can accurately transmit the template identifiers and execution parameters required for image reconstruction to the front-end with the smallest data volume and optimal structure, ensuring that the front-end can complete the image reconstruction task without decoding video frames or image streams, greatly improving network efficiency, data security and system scalability.

[0144] In a specific example, the cloud-based collaborative rendering engine completes the rendering of the visitor center side view frame and selects T_004 as the reconstruction template for the current frame. Based on the definition structure of the mesh deformation template, the system constructs the following structured data package:

[0145]

[0146] The data packet is encapsulated in JSON format and sent to the front-end collaborative rendering machine in real time through the WebSocket channel.

[0147] In step S60 , the front-end collaborative rendering machine loads the required graphic data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of the image frame according to the template logic and parameter set.

[0148] In collaborative rendering web data processing, to achieve efficient and low-latency image display, the front-end collaborative rendering machine, upon receiving a structured data packet transmitted from the cloud, must immediately call a locally preloaded image reconstruction template based on the template identifier. Based on the accompanying template parameter set, it extracts relevant textures, masks, matrices, and other data resources from the local resource library to assemble and display the image frames. This process does not rely on real-time video stream decoding or pixel transmission, but instead completes image restoration through an imperative template call method, effectively reducing the computational burden on the terminal device and improving graphics responsiveness and usability under limited network conditions.

[0149] The image reconstruction template identifier refers to a number assigned by the cloud that is used to uniquely identify and call the image assembly logic in the front-end system. Graphic data refers to the image material contained in the local resource library, including texture images, mask images, color channel information, spatial matrices and other texture data. Template logic refers to the image assembly rules predefined in each template, usually implemented by a script module or graphics rendering instructions. The parameter set is the structured execution parameter generated by the cloud after the template is selected, which contains specific control instructions such as texture index, fusion weight, affine matrix, etc. required for reconstruction. Image frame assembly refers to the process by which the front-end constructs the final image output frame in Canvas, WebGL or other graphics interfaces based on the template logic and local resources.

[0150] like Figure 5 As shown, in an optional specific implementation, step S60 specifically includes the following sub-steps:

[0151] Step S601: Parsing a structured data packet. After receiving the data packet from the cloud, the front-end collaborative rendering engine invokes a local parsing engine to parse it, extracting the template number field and parameter field set. The system verifies the data packet integrity and version information to ensure that the template structure and parameter set definitions are consistent. If the parsing fails, a downgrade strategy or error prompt mechanism is triggered.

[0152] In step S602, the template execution logic is matched and loaded. Based on the parsed template number, the front-end system loads the corresponding template processing module from the registered template library. This module is an executable object that encapsulates image reconstruction logic, an input field mapping table, and output buffer management functions. Loading methods include dynamic module import or static script binding. Optional implementations include directly mapping the template logic as a JavaScript object or calling a high-performance rendering subroutine as a WebAssembly module.

[0153] Step S603: Graphics resources are loaded. The system retrieves the corresponding resource file from the local graphics resource library based on the contents of fields such as the texture number, mask number, and material channel in the template parameter set. The resource library is organized as a dictionary, with the key being the resource number and the value being a memory object or resource path identifier. Resources can take the form of image files, Canvas caches, GPU texture objects, and more. If a resource is not found, the system can initiate a resource prefetch request based on pre-set rules or use a placeholder resource for temporary display.

[0154] Step S604: Execute the template assembly logic. The system passes the loaded graphic resources as input parameters to the template logic module, and gradually constructs the image frame according to the processing order defined by the template. For example, when the template is the mask fusion template T_001, the system linearly blends the texture image R and the mask image M according to the fusion factor α. The image brightness value L is calculated by the following formula:

[0155] L(x,y)=α×R(x,y)+(1-α)×M(x,y)

[0156] Where L(x,y) represents the brightness of the pixel at position (x,y) in the final image, R(x,y) and M(x,y) are the corresponding pixel values ​​of the main texture and mask image, respectively, and α is the fusion factor, ranging from 0 to 1. If the template is the mesh deformation template T_004, the system uses the transformation matrix T to remap the UV coordinates of the texture, performs texture sampling, and draws it to the target layer.

[0157] Step S605: Output the image frame and submit it to the display buffer. After the template is executed, the system draws the generated image frame to the target Canvas, WebGL rendering context, or other image display area, refreshing the page image content. To ensure a stable frame rate, the system can call the requestAnimationFrame function to schedule the display timing, or trigger the interactive response logic after the drawing is completed. Optional implementation solutions include using an off-screen Canvas for cached drawing, adopting a double buffering mechanism to improve the smoothness of frame switching, or using the FrameBuffer object in WebGL to complete the frame result output.

[0158] Through the above steps, after receiving the image reconstruction instruction, the front-end collaborative rendering machine can quickly complete the image frame construction and output process based on local cache resources and registered templates, thereby significantly reducing the amount of network transmission data, reducing terminal computing resource consumption, and improving user interaction response speed and system stability under multi-perspective switching.

[0159] In a specific example, in the aforementioned visitor center 3D display system, the front-end collaborative rendering machine receives a data packet from the cloud, indicating that the template number is T_004, the main map is R_017, and the spatial transformation matrix is ​​G_011. The system first parses the data packet, loads the mesh deformation module corresponding to template T_004, and extracts the R_017 map and matrix G_011 from the local resource library. Subsequently, the template logic deforms the map coordinates using an affine transformation and draws the deformed image into the front-end Canvas context. After the image is drawn, the system automatically submits the frame buffer and updates the display perspective in the user interface. The entire process achieves high-fidelity image reconstruction and fast response without transmitting image data. Users can switch 3D viewing angles continuously and without delay on the terminal device.

[0160] The prior art mentioned in the above background technology section and specific embodiments section of the present invention can be regarded as part of the present invention and used to understand the meaning of some technical features or parameters.

Claims

1. A WEB data processing method based on collaborative rendering, characterized in that: The steps include: The front-end collaborative rendering machine pre-stores multiple image reconstruction templates and graphic resources, wherein the image reconstruction template is used to assemble the current image frame after receiving the specified template parameters; After receiving the front-end user's interactive request or perspective change information, the cloud-based collaborative rendering machine performs full image rendering of the current frame target screen; The cloud-based collaborative rendering machine extracts structural features from the current image frame and selects the template that is most suitable for reconstructing the current image frame from a plurality of preset image reconstruction templates. The selection is based on the template's execution complexity on the front-end device, reconstruction latency, and image region change characteristics. The cloud collaborative rendering machine generates a corresponding template parameter set according to the selected image reconstruction template, wherein the template parameter set includes at least one of a texture index, a mask number, a transformation parameter, a channel mixing factor, and a space adjustment matrix; The cloud-side collaborative rendering machine packages the unique identifier of the selected image reconstruction template and the template parameter set into a structured data packet and transmits it to the front-end collaborative rendering machine; The front-end collaborative rendering machine loads the required graphic data from the local resource library according to the received image reconstruction template identifier, and completes the assembly and display of the image frame according to the template logic and parameter set.

2. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine includes: constructing an image reconstruction template library, each template in the template library has a unique number, and defining the template type, input resource structure and execution logic, and the template type includes at least one of a mask fusion template, a channel combination template, a block replacement template, a grid deformation template and a progressive refinement template.

3. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine further includes: loading graphic resources and initializing the cache, the graphic resources including texture images, mask images, transformation matrices and material channel information, the resources are stored in the local resource manager with a unique identifier and support version verification.

4. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of pre-storing multiple image reconstruction templates and graphic resources in the front-end collaborative rendering machine also includes: establishing a parameter mapping relationship between the template and the resource, and the parameter mapping structure binds the input fields required for template definition with the local resource index for rapid resource matching during subsequent template execution.

5. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The cloud-based collaborative rendering machine performs the steps of rendering the complete image of the current frame target screen after receiving the front-end user's interaction request or perspective change information, including: parsing user input data to generate camera parameters, constructing the current frame view matrix, and determining the set of three-dimensional scene objects that need to be rendered based on the frustum culling technology.

6. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of extracting structural features of the current image frame by the cloud-based collaborative rendering machine includes: calculating texture density, number of region boundaries, brightness distribution and pixel change area based on the current image frame data, and constructing a structural feature vector, which is used to guide subsequent template selection.

7. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of the cloud-based collaborative rendering machine selecting the template that is most suitable for reconstructing the current image frame from multiple preset image reconstruction templates includes: scoring the candidate templates based on a template fitness function, the parameters of which include structural feature matching, template execution complexity, and front-end reconstruction delay, where the template with the highest score is selected as the optimal template for the current frame.

8. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of generating a template parameter set by the cloud collaborative rendering machine includes: extracting a texture index, a mask number, a channel fusion factor and a space transformation matrix information, and organizing them into a structured template parameter set according to a template definition format.

9. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The step of the cloud-based collaborative rendering machine packaging the template identifier and template parameter set into a structured data packet and transmitting it to the front end includes: constructing a data packet structure, which includes a template number, parameter field, version number and verification information, and transmitting it to the front-end collaborative rendering machine through a network protocol.

10. The WEB data processing method based on collaborative rendering according to claim 1, characterized in that: The steps of the front-end collaborative rendering machine completing the assembly and display of image frames according to the template logic and parameter set include: parsing the template number and parameter content in the structured data packet, calling the corresponding template logic and loading local resources, performing image reconstruction operations in the Canvas or WebGL rendering context, and drawing the resulting image into the display area to complete the image frame display.

Images