Tree structure big data loading method and device based on double-engine cooperation

Abstract

The application relates to a tree structure big data loading method and device based on double-engine cooperation, and belongs to the computer field.The method comprises the following steps: preprocessing original tree-shaped data; initializing and starting monitoring; predicting and preloading an engine to complete interaction and prediction based on an operation probability model; a progressive rendering engine completes frame-by-frame rendering based on a first rendering load balancing model; the double engines feedback to each other, and the expansion probability threshold value and the rendering load threshold value are adjusted.The device comprises the following modules: a data preprocessing module; an initialization and monitoring module; an interaction and prediction module; a frame-by-frame rendering module; and a feedback adjustment module.Through the closed-loop cooperative architecture of the prediction preloading engine and the progressive rendering engine, the data loading process of the tree structure is reconstructed, the dynamic adaptation of the rendering performance demand is realized, and the smoothness of interaction of more than 100,000 nodes is met.

Description

Technical Field

[0001] This invention belongs to the field of computer science and relates to a method, apparatus, computer device, and computer storage medium for loading large data in a tree structure based on dual-engine collaboration. Background Technology

[0002] Tree structures, as the most fundamental data organization form in application systems, achieve hierarchical representation and navigation of information through nested parent-child node relationships. They are widely used in core business scenarios such as access control, organizational structure, and file systems. Big data loading refers to the end-to-end processing of dynamic requests, memory management, visual rendering, and interactive responses for ultra-large-scale datasets containing 100,000 to 1 million nodes in a single tree instance. With the deepening of enterprise-level digital transformation, the node scale of a single tree structure is rapidly evolving from the traditional thousands to hundreds of thousands and even millions, posing disruptive challenges to front-end runtime rendering performance, interaction smoothness, and resource consumption.

[0003] To alleviate the rendering pressure of large-scale tree-like data, most mainstream front-end frameworks currently use virtual scrolling as a standard solution. It's important to understand that virtual scrolling is not a single rendering method, but rather a comprehensive optimization built upon a specific data loading pattern: after the browser detects a scroll event, it calculates the coordinates of the currently visible area, requests the corresponding data subset as needed, and finally renders the returned data as DOM nodes. This process works well in scenarios with linear lists scrolling at a constant speed, enabling basic pagination and smooth scrolling.

[0004] However, when this mechanism is applied to the expand / collapse operation of a tree structure, its internal workflow exposes the following performance defects:

[0005] 1. Only visible nodes are rendered and clipped. However, when the user expands deeper nodes, the DOM tree depth increases sharply, which will force the browser to complete the entire rendering pipeline of "layout calculation → style repaint → layer composition", causing the main thread to be blocked for a long time and resulting in obvious lag.

[0006] 2. Before loading, it is necessary to synchronously traverse all nodes to determine visibility. In scenarios with 100,000 nodes, the time taken for a single calculation will far exceed the browser's smoothness threshold, directly causing a sharp drop in scrolling frame rate and interaction delay.

[0007] 3. Virtual scrolling uses a "request first, render later" sequence. After the user clicks the expand icon, they have to wait for the network to return before rendering. During this period, there is a significant loading blank period, which leads to operation feedback delays and visual inconsistencies, making it difficult to meet the stringent requirements of enterprise-level applications for smooth interaction.

[0008] In summary, the performance bottleneck of existing technologies in tree structures is essentially a mismatch between the data loading process and the dynamic interactive characteristics of the tree structure. Therefore, in scenarios with more than 100,000 nodes, how to reconstruct the data loading process of the tree structure to adapt to the rendering performance requirements and achieve smooth interaction is an urgent technical problem to be solved. Summary of the Invention

[0009] In order to solve the technical problems in the background art, the present invention provides a method, apparatus, computer device and computer storage medium for loading tree-structured big data based on dual-engine collaboration.

[0010] The technical solution of the present invention to solve the above-mentioned technical problems is as follows:

[0011] The first aspect provides a tree-structured big data loading method based on dual-engine collaboration, which sets up a parallel running and resident prediction preloading engine and a progressive rendering engine. The method includes the following steps:

[0012] Receive the raw tree data, build a flat index, and load the configuration parameters;

[0013] Create a rendering container, start status monitoring, instantiate dual engines and establish a shared parameter channel;

[0014] By monitoring scrolling behavior and predicting the unfolding probability of each node based on the operation probability model of the preloading engine and the shared rendering duration per frame of the progressive rendering engine, it determines whether to trigger asynchronous loading. The operation probability model is as follows:

[0015] Where σ and β are weighting coefficients, and α is the baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. The total height of the scrolling container is γ, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame.

[0016] In response to the expand operation command, the progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendered nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering. The first rendering load balancing model is as follows:

[0017] ,in This is the estimated total rendering time. This represents the total number of rendered frames. The current node depth, The maximum depth of the tree, This represents the current available CPU utilization. The node complexity coefficient is . This is the rolling state coefficient;

[0018] The predictive preloading engine and the progressive rendering engine feed back to each other through shared parameter channels, adjusting the unfolding probability threshold and the rendering load threshold.

[0019] Secondly, a tree-structured big data loading device based on dual-engine collaboration is provided, the device comprising:

[0020] The data preprocessing module is used to receive raw tree data, build a flattened index, and load configuration parameters;

[0021] The initialization and listening module is used to create the rendering container, start state monitoring, instantiate the dual engines, and establish a shared parameter channel.

[0022] The interaction and prediction module is used to monitor scrolling behavior. The prediction preloading engine, based on an operation probability model and combined with the rendering duration shared by the progressive rendering engine per frame, calculates and shares the unfolding probability of each node, and then determines whether to trigger asynchronous loading. The operation probability model is as follows:

[0023] Where σ and β are weighting coefficients, and α is the baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. The total height of the scrolling container is γ, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame.

[0024] The frame-by-frame rendering module responds to expand operation commands. The progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendered nodes per frame based on the first rendering load balancing model, and then executes frame-by-frame rendering. The first rendering load balancing model is as follows:

[0025] ,in This is the estimated total rendering time. This represents the total number of rendered frames. The current node depth, The maximum depth of the tree, This represents the current available CPU utilization. The node complexity coefficient is . This is the rolling state coefficient;

[0026] The feedback adjustment module is used to predict that the preloading engine and the progressive rendering engine will provide feedback to each other through shared parameter channels, and adjust the unfolding probability threshold and the rendering load threshold.

[0027] Thirdly, a computer device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the tree-structured big data loading method based on dual-engine collaboration as described above.

[0028] Fourthly, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the tree-structured big data loading method based on dual-engine collaboration as described above.

[0029] The beneficial effects of this invention are:

[0030] (1) The progressive rendering engine of the present invention calculates the rendering time of each frame based on the first rendering load balancing model and performs depth-aware frame scheduling, which changes the DOM depth growth from a sudden increase to a gradual increase per frame, thus avoiding the rendering pipeline blockage caused by the sudden increase in DOM tree depth in a single frame.

[0031] (2) The present invention constructs a flat node index table and stores the node status in a hash structure to achieve O(1) positioning; the dual engines quickly read the status through the index table, reducing the computational overhead of synchronous traversal of the main thread, reducing blocking, and reducing interaction latency;

[0032] (3) The predictive preloading engine and the progressive rendering engine of this invention are decoupled and parallel through the Time parameter. Data loading and rendering are deeply coordinated through the shared parameter channel, eliminating the interaction blank period caused by the "request first and then render" serial mechanism.

[0033] (4) This invention reconstructs the data loading process of the tree structure through the closed-loop collaborative architecture of the predictive preloading engine and the progressive rendering engine, so that it can dynamically adapt to the rendering performance requirements and thus meet the interactive smoothness of more than 100,000 nodes. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the tree-structured big data loading method based on dual-engine collaboration provided in Embodiment 1 of the present invention.

[0036] Figure 2 This is a schematic diagram illustrating the interaction principle of the dual-engine and collaborative state pool in Embodiment 1 of the present invention.

[0037] Figure 3 This is a schematic diagram of the bidirectional feedback principle of the dual engines through a closed-loop coupler in Embodiment 1 of the present invention.

[0038] Figure 4 This is a schematic diagram of the tree-structured big data loading device based on dual-engine collaboration provided in Embodiment 2 of the present invention.

[0039] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention.

[0040] The attached diagram lists the components represented by each number as follows:

[0041] 2001. Data preprocessing module; 2002. Initialization and monitoring module; 2003. Interaction and prediction module; 2004. Frame rendering module; 2005. Feedback adjustment module; 310. Processor; 320. Communication interface; 330. Memory; 340. Communication bus. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0043] Example 1

[0044] To alleviate the rendering pressure of large-scale tree-like data, most mainstream front-end frameworks currently use virtual scrolling as a standard solution. Its working principle is as follows: after the browser listens for scroll events, it calculates the coordinate range of the currently visible area, requests the corresponding data subset as needed, and finally renders the returned data as DOM nodes. This process works well in scenarios with linear lists scrolling at a constant speed, enabling basic pagination and smooth scrolling.

[0045] However, when this mechanism is applied to the expand / collapse operation of a tree structure, its internal workflow exposes the following performance defects:

[0046] 1. Only visible nodes are rendered and clipped. However, when the user expands deeper nodes, the DOM tree depth increases sharply, which will force the browser to complete the entire rendering pipeline of "layout calculation → style repaint → layer composition", causing the main thread to be blocked for a long time and resulting in obvious lag.

[0047] 2. Before loading, it is necessary to synchronously traverse all nodes to determine visibility. In scenarios with 100,000 nodes, the time taken for a single calculation will far exceed the browser's smoothness threshold, directly causing a sharp drop in scrolling frame rate and interaction delay.

[0048] 3. Virtual scrolling uses a "request first, render later" sequence. After the user clicks the expand icon, they have to wait for the network to return before rendering. During this period, there is a significant loading blank period, which leads to operation feedback delays and visual inconsistencies, making it difficult to meet the stringent requirements of enterprise-level applications for smooth interaction.

[0049] In summary, the performance bottleneck of existing technologies in tree structures is essentially a mismatch between the data loading process and the dynamic interactive characteristics of the tree structure. Therefore, in scenarios with more than 100,000 nodes, how to reconstruct the data loading process of the tree structure to adapt to the rendering performance requirements and achieve smooth interaction is an urgent technical problem to be solved.

[0050] To address the aforementioned issues, this invention provides a tree-structured big data loading method based on dual-engine collaboration. Figure 1 This is a schematic diagram of the tree-structured big data loading method based on dual-engine collaboration provided in an embodiment of the present invention, as shown below. Figure 1 As shown, this method includes:

[0051] Step S101: Receive the original tree data, build a flattened index, and load the configuration parameters.

[0052] Preferably, step S101 further includes:

[0053] Step S1011: Construct a collaborative state pool in memory. The collaborative state pool space is logically divided into a shared data access layer and a global configuration space.

[0054] Step S1012: Receive the original tree data, convert it into node data with a unified structure through a data standardization function, and then store the unified node data in the shared data access layer for real-time operation and updating by the dual engines. Each node includes at least the following fields: node ID field, parent node ID field, display label field, node depth field, node type field, whether there are child nodes field, whether it has been expanded field, and whether it has been rendered field.

[0055] Step S1013: Based on the original tree data, construct a flattened node index table. The index table includes a parent-child node mapping hash table and a node lookup hash table. The index table also records the unique identifier, depth, parent node reference, and visibility status of each node. Store the flattened node index table in the shared data access layer so that the prediction preloading engine and the progressive rendering engine can perform node location, status query, and rendering mark update in O(1) time complexity.

[0056] Step S1014: Receive the set of configuration parameters passed in by the business layer. The set of configuration parameters is loaded into the global configuration space once during system initialization for dual engines to perform read-only operations. The set of configuration parameters includes at least: the maximum depth threshold of the tree, the basic rendering time of a single node, and the node complexity mapping table.

[0057] It's important to understand that the sources of raw tree-structured data are diverse, including database query results, API responses, file parsing, and user-defined configurations. Therefore, some data may be nested JSON / XML, while others may be CSV or YAML. A unified structure can hide these differences, forming a stable data model for both engines to read. Simultaneously, the operational probability model calculation requires the current node depth. Maximum tree depth Data such as current CPU availability and utilization are not runtime dynamic indicators that can be provided by raw data. A unified structure can lay the foundation for accurate calculation of operational probability models.

[0058] It is also understood that the data normalization functions used in this embodiment include, but are not limited to: structural normalization functions, data cleaning functions, derived calculation functions, and data derivation functions. Structural normalization functions are used for field mapping and generating a unified structure; data cleaning functions are used to handle missing values ​​and type conversions; derived calculation functions are used to pre-calculate commonly used derived fields; and data derivation functions are used to automatically fill in missing but necessary fields. Those skilled in the art can combine these functions according to business needs when selecting which to use. However, structural normalization functions are necessary because they directly affect the calculation accuracy and stability of the model in this embodiment. Additionally, as a preferred option, loop detection can be added to prevent infinite recursion. Therefore, by using data normalization functions, the normalization layer becomes the only reliable source of data. The dual-engine system is read-only, and all write operations are written back through the normalization layer, ensuring data consistency.

[0059] It is worth noting that, unlike traditional techniques that synchronously traverse all nodes to determine visibility, the main thread needs to recursively query the expanded state, depth level, and sibling node positions of each node, resulting in a linear increase in computational load with the node size. For scenarios with hundreds of thousands of nodes, this embodiment constructs a flattened node index table during the data preprocessing stage. Key states such as node ID, depth, parent node ID, and visibility marker are stored in the shared data access layer using a hash structure, achieving O(1) complexity for node information location. Both the prediction preloading engine and the progressive rendering engine read node states through this index table during runtime, transforming the synchronous traversal overhead of visibility determination into an O(1) lookup, reducing the computational load of the main thread, and changing from blocking traversal to non-blocking reading, effectively reducing interaction latency.

[0060] It is understandable that O(1) time complexity means that the algorithm execution time does not change with the growth of data size and is constant. The patent emphasizes that after constructing a flat node index table through a hash table, the dual-engine operations of locating nodes, querying status, and updating tags do not need to traverse all nodes. Regardless of whether the tree structure has 1,000 nodes or 100,000 nodes, the time consumption of a single access remains constant at the microsecond level, which is different from the O(n) linear growth time consumption of traditional traversal.

[0061] It is also worth noting that the system constructs a collaborative state pool as the unified resource base for the dual engines. This space is logically divided into two layers: (1) The shared data access layer stores dynamic readable and writable data, covering high-frequency read and write content such as deep copy of the original tree data, flattened node index table, node rendering status marker, preloaded task queue, and rendering duration of each frame output by the progressive rendering engine. The dual engines update the data in this layer in real time through atomic operations to achieve state synchronization; (2) The global configuration space stores read-only parameters injected by the business layer, stores business rules and constants that do not change due to operation, including initialization constants such as node basic rendering time, target frame rate, maximum tree depth, and node complexity mapping table. The dual engines are loaded into memory at once when they start up, and only read operations are performed during runtime to ensure configuration consistency.

[0062] Step S102: Create a rendering container, start status monitoring, instantiate dual engines, and establish a shared parameter channel.

[0063] Preferably, step S102 further includes:

[0064] Step S1021: Create a tree-structured scrolling container with a fixed viewport height and a fixed scrolling style, render the root node to the container, and establish a mapping index table between node IDs and DOM elements in memory for the dual engines to read node position information.

[0065] Step S1022: Instantiate the scroll history manager to listen for scroll events and record the displacement, timing, and direction to the circular buffer;

[0066] Step S1023: Create a visual status monitor instance and bind its root node parameters to the tree scroll container, and configure multi-threshold monitoring of node visual status changes;

[0067] Step S1024: Bind node click events to the tree-shaped scrolling container through the event delegation mechanism to capture user expand / collapse operations;

[0068] Step S1025: Start the progressive rendering engine and the predictive preloading engine in parallel, read the rendering baseline parameters and performance target parameters in the shared configuration respectively, and initialize the rendering queue and the preloading task queue.

[0069] Step S1026: Establish a memory communication channel. The two engines complete state synchronization and instruction transmission by reading and writing real-time load parameters, task queues and node state indexes in the shared state object.

[0070] It is understood that in this embodiment, during the initialization phase of the rendering container, a scrolling container with a unique identifier #tree-scroll-container is created in the DOM tree. This scrolling container is configured with a fixed viewport height and a fixed scrolling style. The viewport height of the scrolling container is dynamically adapted according to the screen resolution of the terminal device. It is set to 300 pixels by default on mobile devices and 600 pixels by default on desktop devices. The height value can be set to a range of 300 to 800 pixels. The scrolling style enables the vertical scrolling mechanism through the CSS property overflow-y:auto.

[0071] During initialization, the system synchronously performs the first rendering of the root node data, drawing the root-level nodes of the original tree data synchronously into the scrolling container to form the initial visual interface. After the root node rendering is complete, the system injects a node reference mapping table into the scrolling container, recording the correspondence between the root node DOM element instance and its data ID. This provides an index basis for subsequent node positioning and update operations of the dual engines.

[0072] It is also understood that in the state monitoring mechanism startup phase, this embodiment instantiates the ScrollHistoryManager, which is configured as a circular buffer structure and can be set to continuously record, such as the displacement, timestamp and direction vector of the most recent 40-50 scroll events, for predicting the scroll trend analysis of the preloading engine.

[0073] Create a visual state observer instance, whose root node parameter is bound to the scroll container. Specific monitoring thresholds can be set, such as [0, 0.1, 0.5, 1], to accurately monitor the entry, partial visibility, and full visibility states of nodes. When the visibility of a node changes, the dual-engine rendering or recycling callback is triggered.

[0074] For scrolling events, node click events are bound to the event delegation mechanism of the scrolling container. The click behavior of the target node can be captured through event bubbling, and after parsing the node ID, it is passed to the dual engines to execute expand or collapse commands. In addition, as a preferred option, the system can be configured to achieve millisecond-level debouncing optimization by combining requestAnimationFrame with a timeout call to setTimeout. This ensures that the event callback is triggered only a few milliseconds after scrolling stops, avoiding high-frequency triggering that could block the main thread and balancing responsiveness and performance overhead.

[0075] It is also understood that in this embodiment, during the dual-engine initialization phase, the prediction preloading engine and the progressive rendering engine start in parallel without dependencies, and each completes the memory allocation and initial value setting of its internal state variables. During the initialization of the prediction preloading engine, the loading probability calculation weight coefficient, preloading probability threshold, etc. are performed; during the initialization of the progressive rendering engine, the global configuration space is read to obtain the basic rendering time of a single node, the target frame rate value, and the node complexity mapping table, etc., and a queue of nodes to be rendered and a set of nodes currently being rendered are created.

[0076] Communication between the two engines is achieved through a shared memory object. Specifically, a shared object is created in the collaborative state pool, which contains key-value attributes such as the current frame rendering time, the preloaded task queue, and the node state index. The two engines complete state synchronization and instruction transmission by reading and writing the attributes of this object. After the shared parameter channel is established, the system broadcasts the ENGINE_READY event.

[0077] It is worth noting that, such as Figure 2 As shown, the system constructs a collaborative state pool located in memory, logically divided into a read-only global configuration space and a read-write shared data layer. After the dual engines start, they connect to each other through a shared parameter channel that encapsulates a thread-safe interface. All cross-engine parameter reading, state writing, and event notifications are completed through this channel. The predictive preloading engine reads the configuration and rendering duration based on the operation probability model, calculates the unfolding probability, and updates the task queue; the progressive rendering engine reads the node complexity based on the first rendering load balancing model, calculates the rendering duration, and writes it to the shared layer. The two engines share the memory context while maintaining logical isolation, and achieve bidirectional feedback and closed-loop collaboration through the channel, forming a loosely coupled and highly collaborative runtime architecture.

[0078] Step S103: Monitor scrolling behavior, predict the preloading engine based on the operation probability model, combine the rendering duration of each frame shared by the progressive rendering engine, calculate and share the expansion probability of each node, and then determine whether to trigger asynchronous loading.

[0079] In step S103, the operation probability model is:

[0080] Where σ and β are weighting coefficients, and α is the baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. γ is the total height of the scrolling container, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame.

[0081] Preferably, step S103 further includes:

[0082] Step S1031: When a scroll event is triggered, the scroll event callback is executed. The scroll history manager records the scroll position and timestamp, calculates the current scroll speed and acceleration, and the visible node list is updated in real time by the visible status monitor.

[0083] Step S1032: The prediction preloading engine calculates the unfolding probability of each node based on the operation probability model and the rendering duration of each frame output by the progressive rendering engine.

[0084] Step S1033: Based on the unfold probability result, determine whether the unfold probability exceeds the preset unfold probability threshold and whether the target node is within the preload distance. If yes, trigger asynchronous loading of unloaded node data near the visible area; otherwise, release all logically invisible node data and resources. The logically invisible nodes include nodes that are outside the visible area and child nodes that are folded and invisible within the visible area.

[0085] It's worth noting that traditional virtual scrolling architectures are based on the physical visible area, i.e., screen pixel coordinates, for node pruning, and cannot perceive the logical visibility of the tree structure, i.e., the collapsed state. This embodiment records whether each node has been expanded or not through a flattened node index table, enabling the resource reclamation strategy to simultaneously perceive both physical location and logical state, achieving dual-judgment reclamation.

[0086] It's worth noting that the predictive preloading engine analyzes the user's scrolling trajectory to understand their intentions and preloads potential data, eliminating loading gaps and providing a smoother user experience. Its working principle involves real-time monitoring of the user's scrolling behavior, including scrolling speed, acceleration, and direction, using this data to predict the nodes the user might explore next. The core output of the progressive rendering engine is used as a key input parameter to the operation probability model, ensuring that the "aggressiveness" of preloading—such as the number and priority of preloaded nodes—adapts to the current rendering performance. This avoids rendering blockage due to excessive preloading or blank periods caused by insufficient preloading, achieving a closed-loop collaboration where "loading prepares for rendering, and rendering guides loading."

[0087] Below, we will introduce the key parameters involved in the operational probability model in detail:

[0088] (1) σ is the acceleration weight coefficient, preferably in the range of 0.7-0.9. The upper limit is constrained to within 0.9 to prevent the acceleration weight from being too high and causing the preloading to be triggered erroneously during fast scrolling; the lower limit is constrained to above 0.7 to ensure that the acceleration can effectively identify the user's operation intention and distinguish the difference between uniform scrolling and accelerated scrolling.

[0089] (2) α is the speed reference constant term, and the value range of α is 1.5 - 2.5. The upper limit is constrained within 2.5, which can avoid the preloading probability approaching 0 and complete failure during high-speed rolling caused by too high speed reference. The lower limit is constrained above 1.5 to prevent frequent and blind preloading during slow rolling caused by excessive speed interference.

[0090] (3) β is the speed suppression weight coefficient, and the preferred range is 0.4 - 0.6. The upper limit is constrained within 0.6, which can avoid the intention recognition function of acceleration being masked by too high speed weight. The lower limit is constrained above 0.4 to ensure the suppression effect of speed on blind preloading and reduce the ineffective loading of users quickly skipping nodes.

[0091] (4) is the rolling acceleration, with the unit of px / ms². Since the acceleration needs to be derived from the change of the displacement change rate (Velocity), at least two sets of displacement differences are required to calculate it. For example, t0, t1, and t2 are the rolling data at three time points, (Δs1 = t1 - t0, Δs2 = t2 - t1), and the acceleration is obtained by Δs2 - Δs1. Generally, it is calculated based on the displacement differences and time differences of the last three rolls, with a relatively balanced accuracy and performance. Of course, increasing it by one or two more times can reduce the influence of accidental fluctuations, but it will occupy more memory and extend the calculation time. Moreover, the rolling behavior is real-time, and too much historical data may lag behind the user's latest behavior.

[0092] (5) is the rolling speed, with the unit of px / ms, calculated based on the displacement difference of the last two rolls divided by the time difference.

[0093] (6) is the minimum distance between the node and the edge of the visible area, with the unit of px, taking the smaller absolute value of the distance between the node and the upper and lower edges of the visible area.

[0094] (7) is the total height of the rolling container, with the unit of px, obtained in real-time through the shared parameter channel.

[0095] (8) γ(Time_1) is the rendering capacity adaptation coefficient, which dynamically adjusts the preloading strategy based on the rendering duration Time of each frame of progressive rendering. Specifically:

[0096] When Time ≤ 4ms, γ(Time_1) = 1.6, representing extremely low rendering pressure and ultra-aggressive preloading.

[0097] When 4ms < Time ≤ 6ms, γ(Time_1) = 1.2, representing low rendering pressure and aggressive preloading.

[0098] When 6ms < Time ≤ 8ms, γ(Time_1) = 0.9, representing medium rendering pressure and conventional preloading;

[0099] When Time > 8ms, γ(Time_1) = 0.2, representing high rendering pressure and only core node preloading.

[0100] For the sake of easy understanding, we provide a calculation example of an operation probability model:

[0101] Suppose the current scroll acceleration is 5px / ms², the scroll speed is 10px / ms, the distance of the current node from the edge of the visible area is 100px, the total height of the scroll container is 1000px, the weight coefficient σ = 0.8, α = 2, β = 0.5, and the Time for progressive rendering calculation is 4.2ms (corresponding to γ(Time_1) = 1.2). Then, through the calculation of the operation probability model, we can get:

[0102]

[0103] The preset expansion probability critical value is 0.05, and this critical value is the "probability baseline" determined based on a large amount of user behavior data and actual measurements in business scenarios. When the calculated is 0.068 and exceeds this critical value, it means that the probability of the user expanding this node has reached the level of "worth preloading". It is predicted that the preloading engine will preload the sub-node data in advance. That is, when the user really clicks to expand the node, the data is already ready in advance and can be immediately rendered, completely eliminating the loading blank period and improving the interaction fluency.

[0104] Of course, if is lower than the critical value, it is determined that the probability of the user expanding this node in the short term is extremely low, and preloading is not triggered to avoid wasting network and memory resources.

[0105] Step S104, in response to the expansion operation instruction, the progressive rendering engine reads the list of unrendered sub-nodes, calculates and shares the rendering duration per frame and the number of rendered nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering.

[0106] In step S104, the first rendering load balancing model is:

[0107] , where is the total estimated rendering time, is the total number of rendering frames, is the current node depth, is the maximum depth of the tree, is the current available CPU usage rate, is the node complexity coefficient, is the scroll state coefficient.

[0108] Preferably, step S104 further includes:

[0109] In step S1041, in response to the expand operation command, the progressive rendering engine reads the list of unrendered child nodes and simultaneously reads the shared state parameters;

[0110] Step S1042: Based on the first rendering load balancing model, calculate the rendering duration per frame and the number of rendering nodes per frame corresponding to the current rendering batch.

[0111] Step S1043: Write the rendering duration of each frame into the shared data access layer for the prediction preloading engine to read;

[0112] Step S1044: Arrange rendering tasks through the frame scheduling interface, traverse unrendered child nodes according to the number of rendering nodes per frame, execute the node rendering function to create DOM elements, bind scroll events and add them to their parent node container, and update the node rendering status flag at the same time.

[0113] In step S1045, the progressive rendering engine submits the node tasks to be rendered in batches to the request animation frame callback function. After each frame is rendered, the remaining number of nodes is checked. If it is greater than zero, the next batch is submitted until all tasks are processed.

[0114] It is worth noting that the progressive rendering engine in this embodiment calculates the rendering time of each frame based on the first rendering load balancing model. This model comprehensively considers core factors such as rendering status, node depth, CPU load, node complexity, and user scrolling status. The dynamic and accurate calculation method is more adaptable to different CPU loads and node levels, helping the engine to perform depth-aware frame scheduling, turning the DOM depth growth from a sudden increase to a gradual increase per frame, and avoiding rendering pipeline blockage and long-term main thread occupation caused by a sudden increase in DOM tree depth within a single frame.

[0115] Below, we will introduce the key parameters involved in the first rendering load balancing model in detail:

[0116] (1) The total estimated rendering time is calculated based on the base rendering time per node multiplied by the total number of nodes to be rendered.

[0117] (2) For the total number of rendering frames, a value of 3-8 is recommended to balance smoothness and total rendering time.

[0118] (3) This represents the depth of the current node, counted starting from the root node, where the root node's depth is 1.

[0119] (4) The maximum depth of the tree is extracted from the tree metadata or preset by the business layer.

[0120] (5) The current CPU utilization rate ranges from 0 to 100, and is calculated in real time based on hardware concurrency and main thread blocking detection.

[0121] (6) This is the node complexity coefficient, ranging from 1.0 to 2.0, quantifying the DOM structure complexity of a node. Plain text nodes are assigned a value of 1.0, nodes containing icons / action buttons are assigned 1.2-1.5, and custom template nodes are assigned 1.8-2.0. The coefficient can be calculated by pre-setting the node type or dynamically parsing the number of DOM child elements. Based on this node complexity parameter, the rendering strategy can dynamically adapt to different node structures. For example, in simple node scenarios, it can automatically reduce the amount of rendering tasks per frame to avoid resource waste; in complex node scenarios, it can precisely control rendering batches to prevent intra-frame overload stuttering, achieving a precise match between rendering overhead and node complexity.

[0122] (7) This is a scrolling state coefficient, ranging from 0.7 to 1.3, adapted to user scrolling behavior. For example, when the scrolling speed is ≤0.2px / ms, scrolling is considered to have stopped, and a value of 1.3 is used; when the scrolling speed is 0.2-1.0px / ms, it is considered to be constant scrolling, and a value of 1.0 is used; when the scrolling speed is >1.0px / ms or the acceleration is >0.3px / ms², it is considered to be accelerated scrolling, and a value of 0.7 is used. This automatically shrinks during scrolling to ensure scrolling response priority, and automatically expands when scrolling stops to improve rendering resource utilization. This dynamic adjustment mechanism deeply adapts the rendering rhythm to the scrolling behavior, avoiding rendering from competing for the main thread and causing frame drops when the user scrolls, while making full use of the main thread's idle window when scrolling stops to accelerate rendering.

[0123] As a more preferred embodiment, step S104 further includes: replacing the first rendering load balancing model with a second rendering load balancing model, calculating the rendering duration per frame and the number of rendering nodes per frame for the current rendering batch, and increasing the calculation factor window density in the second rendering load balancing model based on the first rendering load balancing model. The second rendering load balancing model is as follows:

[0124] ,in , This represents the number of nodes rendered in the currently visible area. The height is the viewport height.

[0125] It is worth noting that, The window density, measured in pixels per 100px, reflects the current layout pressure of the visible area, thereby enhancing the protection capability of the main thread. This means that by sensing the density of the visible nodes in real time, the upper limit of rendering resource allocation per frame is dynamically adjusted to avoid blocking caused by rendering task overload in high-density node areas. Compared with the fallback strategy of a fixed maximum number of nodes, this optimizes fine-grained frame rate stability.

[0126] It is important to note that CPU availability, window density, and other metrics are stored in the shared data access layer, which stores dynamically updated status and metrics after each frame or operation.

[0127] To facilitate understanding, we provide a more concrete calculation example:

[0128] In a tree structure with a maximum depth of 10, a current node depth of 5, an estimated total rendering time of 100ms, a total of 5 rendering frames, a current CPU utilization of 80%, and a node complexity coefficient of 1.4 (containing 1 icon + 1 action button), a viewport height of 500px and 10 nodes already rendered (viewport density = 10 / 500 × 100 = 2.0), and the user scrolling at a constant speed (scrolling state coefficient of 1.0), then, according to the second rendering load balancing model, the following can be calculated:

[0129]

[0130] This means that the rendering time for each frame will be controlled to around 4.2ms, far below the browser's smoothness threshold of 16ms / frame. Afterward, rendering time can be dynamically allocated to avoid overloading a single frame's rendering task, thus effectively preventing main thread blocking. During each frame rendering, the progressive rendering engine calls the browser's `requestAnimationFrame` interface to register callback tasks and uses the calculated rendering time (`Time`) as the scheduling basis to precisely control the number of rendering nodes for the current frame. When the rendering time approaches the `Time` value, the engine actively suspends the task and releases the main thread to ensure the next frame can be scheduled on time. By strictly adhering to the `Time` constraint in its frame-by-frame execution mechanism, it avoids rendering drops caused by single-frame overload, ensuring smooth and stable interface scrolling and interaction.

[0131] As a more preferred method, step S1044 further includes: arranging rendering tasks through the frame scheduling interface, traversing unrendered child nodes according to the number of rendering nodes per frame, dynamically adding CSS hardware acceleration properties to create independent layers when executing the node rendering function, and simultaneously completing DOM element creation, scroll event binding, element mounting to parent container, and node state marker update.

[0132] Step S1045 further includes: splitting the set of nodes to be rendered into sub-batches, submitting rendering tasks frame by frame through the requestAnimationFrame callback, processing only the nodes of the current batch in each frame, detecting the number of remaining nodes after rendering is completed, and continuing to submit the next batch of tasks if not all nodes have been processed, so that the layer creation and composition of each batch of nodes are synchronized with the browser screen refresh rate.

[0133] When rendering nodes, CSS hardware-accelerated properties are dynamically added to the node DOM elements, and the nodes are promoted to independent layers. After rendering, the page is drawn through the browser's layer compositing mechanism, which achieves GPU acceleration optimization and avoids global reflow.

[0134] In step S105, the prediction preloading engine and the progressive rendering engine feed back to each other through shared parameter channels to adjust the unfolding probability threshold and the rendering load threshold.

[0135] Preferably, step S105 further includes:

[0136] Step S1051: After each frame of the progressive rendering engine is rendered, the rendering performance index is written to the shared data access layer. The closed-loop coupler reads the rendering performance index in real time and compares it with the preset rendering load threshold. If the rendering performance index is determined to be lower than the preset rendering load threshold, a negative adjustment signal is broadcast to the prediction preloading engine through the shared parameter channel to automatically lower the unfolding probability threshold.

[0137] In step S1052, when the prediction preloading engine executes an asynchronous loading task, it writes the preloading progress to the shared data access layer. When the closed-loop coupler detects that the rendering load threshold has a recovery margin and the preloading progress is lagging, it sends a positive adjustment signal to the progressive rendering engine, allowing the rendering engine to moderately increase the rendering load threshold.

[0138] Understandably, the closed-loop coupler is a memory-resident logic component located within the shared parameter channel, serving as the decision-making center for dual-engine collaboration. Its physical location is within the same memory heap space as the collaborative state pool, but functionally it is independent of the data storage layer: the collaborative state pool is only responsible for data access, while the closed-loop coupler is responsible for the monitoring-determination-adjustment logic execution.

[0139] It is worth noting that, such as Figure 3 As shown, the dual engines form a continuous bidirectional feedback loop with a closed-loop coupler through a shared parameter channel: the load output of the rendering engine continuously adjusts the unfolding probability threshold of the preloading engine to avoid excessive preloading and resource contention; the progress feedback of the preloading engine dynamically optimizes the rendering load threshold of the rendering engine to ensure efficient utilization of cached data. Under the real-time monitoring of the coupler, the two thresholds constrain and promote each other, maintaining a dynamic balance between high throughput and low latency in the system.

[0140] In this embodiment, the data loading process of the tree structure is reconstructed through a closed-loop collaborative architecture of the predictive preloading engine and the progressive rendering engine, so that it can dynamically adapt to the rendering performance requirements and thus meet the interactive smoothness of more than 100,000 nodes.

[0141] Example 2

[0142] like Figure 4 As shown, in one embodiment, a tree-structured big data loading system device based on dual-engine collaboration is provided. This device includes:

[0143] Data preprocessing module 2001 is used to receive raw tree data, build a flattened index, and load configuration parameters;

[0144] The initialization and listening module 2002 is used to create rendering containers, start state monitoring, instantiate dual engines, and establish a shared parameter channel.

[0145] The Interaction and Prediction Module 2003 is used to monitor scrolling behavior. The prediction preloading engine, based on an operation probability model and combined with the shared rendering duration per frame from the progressive rendering engine, calculates and shares the unfolding probability of each node, then determines whether to trigger asynchronous loading. The operation probability model is as follows:

[0146] Where σ and β are weighting coefficients, and α is the baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. The total height of the scrolling container is γ, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame.

[0147] The frame-by-frame rendering module 2004 responds to expand operation commands. The progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendered nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering. The first rendering load balancing model is as follows:

[0148] ,in This is the estimated total rendering time. This represents the total number of rendered frames. The current node depth, The maximum depth of the tree, This represents the current available CPU utilization. The node complexity coefficient is . This is the rolling state coefficient;

[0149] The feedback adjustment module 2005 is used to predict that the preload engine and the progressive rendering engine will provide feedback to each other through shared parameter channels, and adjust the unfolding probability threshold and the rendering load threshold.

[0150] Below, we provide some comparative experiments to further illustrate this embodiment, as follows:

[0151] (1) Experimental setup and subjects

[0152] To verify the performance advantages of this embodiment in handling tree structures with 100,000 or more nodes, three typical test scenarios were set up:

[0153] Test environment configuration: The hardware platform covers high-performance desktops (Intel Core i7-12700H / DDR5 16GB / RTX 3060) and mid-to-low-end desktops (Intel Core i5-8400 / DDR4 8GB / UHD 630); Chrome 120.0.6099.224 browser is used uniformly to eliminate engine differences; the test dataset covers a basic scenario with 100,000 nodes, a complex scenario with 500,000 nodes, and a scenario adapted to low-configuration hardware. The node types include plain text, icons, buttons, and custom templates, which are mixed to simulate the real load of enterprise-level applications.

[0154] Performance metrics are defined as follows: Core observation metrics include rendering frame time (per frame), memory usage (after stabilization), data loading latency, stuttering rate, first-screen rendering time, node expansion response time, memory growth rate over 30 minutes of continuous scrolling, JS heap memory usage (after garbage collection), and total number of DOM nodes. Each metric is automatically collected via the Performance API and Chrome DevTools Timeline. The stuttering rate is defined as the percentage of long tasks where the main thread is blocked for more than 50ms in a single instance. Node expansion response time measures the end-to-end latency from user click to the completion of the first-screen node rendering.

[0155] (2) Comparison method

[0156] The comparison method uses the default virtual scrolling configuration (including lazy loading) of current mainstream front-end frameworks (Element UI, Ant Design) to reflect the performance baseline of the real production environment.

[0157] (3) Experimental procedure

[0158] Before each scenario test, browser cache and compilation artifacts were cleared, and browser extensions were disabled to ensure a clean environment. Automated test scripts were used to control the operation rhythm, with each metric measured 100 times. Outliers (exceeding 3 standard deviations) caused by browser garbage collection or system background processes were removed, and the median and 95% confidence interval were used to eliminate random bias. The testing process covered four stages: initial rendering baseline determination, 200 consecutive node expansion stress tests, 800px / s high-speed scrolling smoothness tests, and 30-minute continuous scrolling memory leak monitoring. The timing of metric collection was clearly defined: rendering frame time was sampled once per second during continuous scrolling; node expansion response time was recorded after each click to expand; memory usage and the total number of DOM nodes were sampled after 30 seconds of stable state following initial rendering; and JS heap memory was sampled immediately after garbage collection. Performance panel traces and memory snapshots were recorded throughout the process to quantify and compare performance differences.

[0159] (4) Test data

[0160] Scenario 1: Basic scenario with 100,000 nodes (high-performance hardware + mixed node types)

[0161] Hardware: Intel Core i7-12700H + 16GB DDR5 + RTX 3060;

[0162] Browser: Chrome 120.0.6099.224;

[0163] Dataset: 102,400 nodes, plain text 40% + icons 30% + buttons 20% + custom 10%;

[0164] The test results are shown in Table 1.

[0165] Table 1. Performance Comparison of Hybrid Rendering Types with 100,000 Nodes on High-Performance Hardware

[0166] Test metrics Original values ​​for the example (3 times) Average value in this embodiment Original values ​​for comparison (3 times) Comparative average Performance improvement rate Rendering frame time (per frame) 3.8 / 4.2 / 4.0ms 4ms 15.7 / 16.2 / 16.1ms 16ms 75%↓ Memory usage 70 / 73 / 73MB 72MB 118 / 121 / 121MB 120MB 40%↓ Data loading delay 48 / 52 / 50ms 50ms 195 / 203 / 202ms 200ms 75%↓ stuttering rate 3.0 / 3.3 / 3.3% 3.2% 34.5 / 35.3 / 35.2% 35% 91%↓ First screen rendering time 81 / 78 / 81ms 80ms 235 / 242 / 243ms 240ms 67%↓ Node deployment response time 58 / 61 / 61ms 60ms 315 / 323 / 322ms 320ms 81%↓ Memory growth rate over 30 minutes 4.9 / 5.1 / 5.0% 5% 27.5 / 28.3 / 28.2% 28% 82%↓ JS heap memory usage 69.2 / 72.8 / 73.0MB 72MB 117.3 / 120.8 / 121.9MB 120MB 40%↓ Total number of DOM nodes 3120 / 3250 / 3230 3200 17800 / 18100 / 18100 18,000 82%↓

[0167] Scenario 2: Basic scenario with 100,000 nodes (low-configuration hardware + mixed node types)

[0168] Hardware: Intel Core i5-8400 + 8GB DDR4 + UHD 630;

[0169] Browser: Chrome 120.0.6099.224;

[0170] Dataset: 102,400 nodes, plain text 40%+ icons 30%+ buttons 20%+ custom 10%;

[0171] The test results are shown in Table 2.

[0172] Table 2. Performance Comparison of Hybrid Rendering Types on Low-Configuration Hardware with 100,000 Nodes

[0173] Test metrics Original values ​​for the example (3 times) Average value in this embodiment Original values ​​for comparison (3 times) Comparative average Performance improvement rate Rendering frame time (per frame) 7.7 / 8.1 / 8.2ms 8ms 34.2 / 35.1 / 35.7ms 35ms 77%↓ Memory usage 83 / 85 / 87MB 85MB 147 / 150 / 153MB 150MB 43%↓ Data loading delay - - - - - stuttering rate 8.0 / 8.3 / 8.6% 8.3% 50.8 / 52.1 / 53.1% 52% 84%↓ First screen rendering time - - - - - Node deployment response time - - - - - Hardware compatibility Normal / Normal / Normal 100% success rate Lag / Lag / Failure Success rate: 66.7% - JS heap memory usage 82.5 / 84.8 / 87.7MB 85MB 146.3 / 149.5 / 154.2MB 150MB 43%↓ Total number of DOM nodes - - - - -

[0174] Scenario 3: Complex scenario with 500,000 nodes (high-performance hardware + 30% custom templates)

[0175] Hardware: Intel Core i7-12700H + 16GB DDR5 + RTX 3060;

[0176] Browser: Chrome 120.0.6099.224;

[0177] Dataset: 512,000 nodes, plain text 30% + icons 25% + buttons 15% + custom elements 30%;

[0178] The test results are shown in Table 3.

[0179] Table 3. Performance Comparison of Complex Template Rendering with 500,000 Nodes on High-Performance Hardware

[0180] Test metrics Original values ​​for the example (3 times) Average value in this embodiment Original values ​​for comparison (3 times) Comparative average Performance improvement rate Rendering frame time (per frame) 6.3 / 6.6 / 6.6ms 6.5ms 27.5 / 28.2 / 28.3ms 28ms 77%↓ Memory usage (after stabilization) 176 / 182 / 182MB 180MB 315 / 322 / 323MB 320MB 44%↓ Data loading delay - - - - - stuttering rate 5.6 / 5.9 / 5.9% 5.8% 47.2 / 48.3 / 48.5% 48% 88%↓ First screen rendering time - - - - - Node deployment response time 87 / 91 / 92ms 90ms 542 / 551 / 557ms 550ms 84%↓ Crash scenarios with millions of nodes Not crashed / Not crashed / Not crashed Crash rate 0% Crash / Stutter / Crash Crash rate: 66.7% - JavaScript heap memory usage (after GC) 175.2 / 181.5 / 183.3MB 180MB 312.8 / 321.7 / 325.5MB 320MB 44%↓ Total number of DOM nodes - - - - -

[0181] (5) Experiment Summary

[0182] Systematic testing in this embodiment, conducted on node scales ranging from 100,000 to 500,000 and with varying hardware configurations, demonstrates that a leap in performance is achieved in loading and rendering tree-structured data through a closed-loop collaborative architecture between the predictive preloading engine and the progressive rendering engine.

[0183] Breakthrough in rendering performance: In all test scenarios, the rendering frame time of the example was stable at 4-8ms, which is more than 75% lower than the 16-35ms of the comparison example; the stuttering rate was controlled at 3.2%-8.3%, which is more than 80% better than the 35%-52% of the comparison example, and the main thread blocking problem was fundamentally alleviated.

[0184] Memory efficiency optimization: The example reduces memory usage by 40%-44% compared to the comparison, reduces the total number of DOM nodes by 82%, and the memory growth rate is only 5% after 30 minutes of continuous scrolling, which is significantly better than the 28% growth rate of the comparison, effectively curbing the risk of long-tail memory leaks in large-scale tree structures.

[0185] Accelerated Interaction Response: The node expansion response time is reduced from 320-550ms in the comparative example to 60-90ms in the example, the first screen rendering time is compressed to less than 80ms, and the data loading latency is reduced by 75%, enabling real-time visual feedback for user operations.

[0186] Stability in extreme scenarios: In a complex scenario with 500,000 nodes, the comparative example crashed at a rate of 66.7% due to memory overflow, while the example did not experience any crashes and maintained a deployment response time of less than 90ms, verifying the robustness of this example under ultra-large-scale datasets;

[0187] Hardware compatibility: In low-configuration hardware scenarios, the success rate of the implementation example reached 100%, while the comparative example suffered from a 52% stuttering rate and 33.3% adaptation failure due to resource contention. This proves that the implementation example achieves cross-hardware environment adaptability through the first rendering load balancing model.

[0188] Technical effect attribution: The performance improvement stems from the dual engines achieving real-time parameter feedback and reverse adjustment through a shared state interface. The predictive preloading engine dynamically adjusts the preloading strategy based on the rendering load, while the progressive rendering engine avoids single-frame overload through depth-aware frame-by-frame scheduling. Together, they resolve the core bottlenecks of traditional virtual scrolling in scenarios with hundreds of thousands of nodes, such as rendering blocking and loading latency.

[0189] Example 3

[0190] exist Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the electronic device may include a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the tree-structured big data loading method based on dual-engine collaboration as described in Embodiment 1.

[0191] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] Example 4

[0193] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, it implements the tree-structured big data loading method based on dual-engine collaboration as described in Embodiment 1.

[0194] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0195] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0196] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0197] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Ruby, and Go, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0198] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for loading large amounts of data in a tree structure based on dual-engine collaboration, characterized in that, The method, which includes the steps of configuring a parallel-running and resident predictive preloading engine and a progressive rendering engine, comprises the following steps: Receive the raw tree data, build a flat index, and load the configuration parameters; Create a rendering container, start status monitoring, instantiate dual engines and establish a shared parameter channel; By monitoring scrolling behavior and predicting the unfolding probability of each node based on the operation probability model of the preloading engine and the shared rendering duration per frame of the progressive rendering engine, it determines whether to trigger asynchronous loading. The operation probability model is as follows: , where P load The expansion probability for each node, σ and β are weighting coefficients, and α is a baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. γ is the total height of the scrolling container, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame. Time_1 is the rendering time per frame under standard conditions. In response to the expand operation command, the progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendered nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering. The first rendering load balancing model is as follows: Where Time_1 is the rendering time per frame under standard conditions. This is the estimated total rendering time. This represents the total number of rendered frames. The current node depth, The maximum depth of the tree, This represents the current available CPU utilization. The node complexity coefficient is . This is the rolling state coefficient; The predictive preloading engine and the progressive rendering engine feed back to each other through shared parameter channels, adjusting the unfolding probability threshold and the rendering load threshold.

2. The tree-structured big data loading method based on dual-engine collaboration according to claim 1, characterized in that, The steps of receiving the original tree data, constructing a flat index, and loading configuration parameters also include: A collaborative state pool is constructed in memory, and the collaborative state pool space is logically divided into a shared data access layer and a global configuration space. The system receives raw tree data, converts it into node data with a unified structure through a data standardization function, and then stores the unified node data in the shared data access layer for real-time operation and updating by the dual engines. Each node includes at least the following fields: node ID field, parent node ID field, display label field, node depth field, node type field, whether there are child nodes field, whether it is expanded field, and whether it is rendered field. Based on the original tree data, a flattened node index table is constructed. The flattened node index table includes a parent-child node mapping hash table and a node lookup hash table. The flattened node index table also records the unique identifier, depth, parent node reference, and visibility status of each node. The flattened node index table is stored in the shared data access layer so that the prediction preloading engine and the progressive rendering engine can perform node positioning, status query, and rendering mark update in O(1) time complexity. The system receives a set of configuration parameters from the business layer. The set of configuration parameters is loaded into the global configuration space once during system initialization for dual engines to perform read-only operations. The set of configuration parameters includes at least: the maximum depth threshold of the tree, the basic rendering time of a single node, and the node complexity mapping table.

3. The tree-structured big data loading method based on dual-engine collaboration according to claim 1 or 2, characterized in that, The steps of creating a rendering container, starting status monitoring, instantiating dual engines, and establishing a shared parameter channel also include: Create a tree-structured scrolling container with a fixed viewport height and a fixed scrolling style, render the root node to the container, and establish a mapping index table between node IDs and DOM elements in memory for the dual engines to read node position information; Instantiate a scroll history manager to listen for scroll events and record the displacement, timing, and direction to a circular buffer; Create a visual status monitor instance and bind its root node parameter to the tree scroll container, and configure multiple thresholds to monitor changes in the visual status of the node; By using an event delegation mechanism, node click events are bound to the tree-shaped scrolling container to capture user expand / collapse operations; The progressive rendering engine and the predictive preloading engine are launched in parallel, respectively reading the rendering baseline parameters and performance target parameters in the shared configuration, and initializing the rendering queue and the preloading task queue. A memory communication channel is established, and the two engines complete state synchronization and instruction transmission by reading and writing real-time load parameters, task queues and node state indexes in the shared state object.

4. The tree-structured big data loading method based on dual-engine collaboration according to claim 1 or 2, characterized in that, The process of monitoring scrolling behavior, predicting the preloading engine's operation probability model, and calculating and sharing the unfolding probability of each node based on the shared rendering duration of each frame in the progressive rendering engine, and then determining whether to trigger asynchronous loading, also includes: When a scroll event is triggered, the scroll event callback is executed. The scroll history manager records the scroll position and timestamp, calculates the current scroll speed and acceleration, and the visible node list is updated in real time by the visible status monitor. The predictive preloading engine is based on an operational probability model, which combines the rendering duration of each frame output by the progressive rendering engine to calculate the unfolding probability of each node. Based on the unfold probability result, determine whether the unfold probability exceeds the preset unfold probability threshold and whether the target node is within the preload distance. If so, trigger asynchronous loading of unloaded node data near the visible area; otherwise, release all logically invisible node data and resources. The logically invisible nodes include nodes that are outside the visible area and child nodes that are collapsed and invisible within the visible area.

5. The tree-structured big data loading method based on dual-engine collaboration according to claim 2, characterized in that, The response to the expanded operation command, the progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendering nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering, further includes: In response to the expand operation command, the progressive rendering engine reads the list of unrendered child nodes and the shared state parameters. Based on the first rendering load balancing model, the rendering duration per frame and the number of rendering nodes per frame corresponding to the current rendering batch are calculated. The rendering duration of each frame is written to the shared data access layer for the prediction preloading engine to read; The rendering task is arranged through the frame scheduling interface. The unrendered child nodes are traversed according to the number of rendering nodes per frame. The node rendering function is executed to create DOM elements, bind scroll events and add them to their parent node container, and update the node rendering status flag at the same time. The progressive rendering engine submits the node tasks to be rendered in batches to the request animation frame callback function. After each frame is rendered, it checks the number of remaining nodes. If the number is greater than zero, it continues to submit the next batch until all tasks are processed.

6. The tree-structured big data loading method based on dual-engine collaboration according to claim 5, characterized in that, The predictive preloading engine and the progressive rendering engine feed back to each other through a shared parameter channel to adjust the unfolding probability threshold and the rendering load threshold. The process also includes: After each frame of the progressive rendering engine is rendered, the rendering performance index is written to the shared data access layer. The closed-loop coupler reads the rendering performance index in real time and compares it with the preset rendering load threshold. If the rendering performance index is determined to be lower than the preset rendering load threshold, a negative adjustment signal is broadcast to the prediction preloading engine through the shared parameter channel to automatically lower the unfolding probability threshold. When the predictive preloading engine executes asynchronous loading tasks, it writes the preloading progress to the shared data access layer. When the closed-loop coupler detects that the rendering load threshold has a recovery margin and the preloading progress is lagging, it sends a positive adjustment signal to the progressive rendering engine, allowing the rendering engine to moderately increase the rendering load threshold.

7. The tree-structured big data loading method based on dual-engine collaboration according to claim 1, characterized in that, In the process of responding to the expanded operation command, the progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendering nodes per frame based on the first rendering load balancing model, and then performs frame-by-frame rendering, the process further includes: replacing the first rendering load balancing model with a second rendering load balancing model, calculating the rendering duration and number of rendering nodes per frame corresponding to the current rendering batch, and adding a calculation factor window density to the second rendering load balancing model based on the first rendering load balancing model. The second rendering load balancing model is as follows: Where Time_2 is the rendering time per frame at the current viewport density. , This represents the number of nodes rendered in the currently visible area. The height is the viewport height.

8. A tree-structured big data loading device based on dual-engine collaboration, characterized in that, The apparatus includes a parallel-running and resident predictive preloading engine and a progressive rendering engine. The data preprocessing module is used to receive raw tree data, build a flattened index, and load configuration parameters; The initialization and listening module is used to create the rendering container, start state monitoring, instantiate the dual engines, and establish a shared parameter channel. The interaction and prediction module is used to monitor scrolling behavior. The prediction preloading engine, based on an operation probability model and combined with the rendering duration shared by the progressive rendering engine per frame, calculates and shares the unfolding probability of each node, and then determines whether to trigger asynchronous loading. The operation probability model is as follows: Where P load The expansion probability for each node, σ and β are weighting coefficients, and α is a baseline constant term. For rolling acceleration, For the scrolling speed, This represents the minimum distance from the node to the edge of the visible area. γ is the total height of the scrolling container, and γ(Time_1) is the rendering capability adaptation coefficient based on the rendering time per frame. Time_1 is the rendering time per frame under standard conditions. The frame-by-frame rendering module responds to expand operation commands. The progressive rendering engine reads the list of unrendered child nodes, calculates and shares the rendering duration and number of rendered nodes per frame based on the first rendering load balancing model, and then executes frame-by-frame rendering. The first rendering load balancing model is as follows: Where Time_1 is the rendering time per frame under standard conditions. This is the estimated total rendering time. This represents the total number of rendered frames. The current node depth, The maximum depth of the tree, This represents the current available CPU utilization. The node complexity coefficient is . This is the rolling state coefficient; The feedback adjustment module is used to predict that the preloading engine and the progressive rendering engine will provide feedback to each other through shared parameter channels, and adjust the unfolding probability threshold and the rendering load threshold.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the tree-structured big data loading method based on dual-engine collaboration as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the tree-structured big data loading method based on dual-engine collaboration as described in any one of claims 1 to 7.

Images