Virtual scrolling method and system for processing big data list

Abstract

The invention provides a virtual scrolling method and system for processing a big data list. The method comprises the following steps: architecture design: dividing the system into a pre-rendering and caching module, a data change detection module, an intelligent scrolling control module and a dynamic adjustment scroll bar and rendering module; according to the method, the list items which are possibly accessed are pre-rendered and cached in advance according to the historical access data of the user and the prediction algorithm, so that the loading time and the resource consumption are reduced; a data change detection technology is utilized, only changed data items are updated and rendered, unnecessary page redrawing is reduced, and the rendering efficiency is improved; through combination of user interaction behaviors and page states, an intelligent scrolling control algorithm is realized, smooth scrolling experience is provided, and page flickering and jamming are avoided; the scroll bar and the rendering mode can be dynamically adjusted according to the height condition of the list item, and the requirements for fixed height and unfixed height are met at the same time.

Description

Technical Field

[0001] The present invention relates to a virtual scrolling method, in particular to a virtual scrolling method for processing a large data list, and belongs to the technical field of large data processing. Background Art

[0002] Big data lists are lists that contain a large number of data items, which usually come from databases, data warehouses, or other data sources. When dealing with big data lists, common challenges include how to effectively render, manage, and interact with these large numbers of data items.

[0003] Virtual Scroll is a front-end technology used to efficiently render and manage lists of large numbers of data items. The core idea of ​​this technology is to render only the data items visible to the user, rather than rendering the entire list at once. When the user scrolls the list, new data items are dynamically loaded and rendered into view.

[0004] With the advent of the big data era, the display of lists with huge amounts of data has become one of the common problems in many front-end developments. Traditional list display methods have problems such as low efficiency and high resource consumption when processing large amounts of data. Therefore, a virtual scrolling method and system for processing big data lists are proposed. Summary of the invention

[0005] In view of this, the present invention provides a virtual scrolling method and system for processing a large data list to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial choice.

[0006] The technical solution of the embodiment of the present invention is implemented as follows: A virtual scrolling method for processing a large data list comprises the following steps:

[0007] Architecture design - divide the system into pre-rendering and caching module, data change detection module, intelligent scroll control module and dynamic adjustment scroll bar and rendering module;

[0008] Collect user historical access data and conduct data analysis to explore user interest preferences and behavior patterns;

[0009] Extract key features for predictive analysis, determine list items for pre-rendering, and render and load page content;

[0010] Page content caching - caching page content after rendering and loading;

[0011] Use WebSocket to monitor data changes in real time and only update and render the changed data items;

[0012] Monitor the user's scrolling behavior, calculate the scroll bar position, determine the list items in the current visible area, and control the triggering frequency of the scrolling event;

[0013] Get the height information of the list items, distinguish between list items with fixed height and list items with variable height, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

[0014] Further preferably, the pre-rendering and caching module, the data change detection module, the intelligent scrolling control module and the dynamic scroll bar adjustment and rendering module are independent of each other and are loosely coupled through event monitoring and triggering mechanisms;

[0015] Among them, the event monitoring and triggering mechanism includes data change monitoring, data object change monitoring, scroll event monitoring, viewport size change and content height change monitoring.

[0016] Further preferably, the collection of user historical access data and data analysis is performed using a pre-rendering and caching module, which is used to pre-render and cache list items that may be accessed in the future in advance based on the user's historical access data and a prediction algorithm;

[0017] The historical access data includes but is not limited to page visits, browsing time, and interactive behaviors.

[0018] Further preferably, the key features include but are not limited to frequently visited pages, visit time, active user groups, and user click paths;

[0019] The predictive analysis is to use a predictive analysis model to predict the user's future access target, so as to subsequently determine the list items for pre-rendering according to the predicted access target.

[0020] Further preferably, the page content caching method is local storage or memory, which is used to reduce requests to the server and speed up page loading.

[0021] Further preferably, the use of WebSocket to monitor data changes in real time is performed using a data change detection module, which is used to use data change detection technology to update and render only the changed data items, thereby improving rendering efficiency.

[0022] Further preferably, the monitoring of the user's scrolling behavior and the calculation of the scroll bar position are performed by using an intelligent scrolling control module, which is used to combine the user's interactive behavior and the page state to implement an intelligent scrolling control algorithm;

[0023] The triggering frequency of the scrolling event is controlled by using throttling technology to limit the time so that it can only be triggered once within a specific time, and anti-shake technology is used to delay the execution of the event to achieve a smooth scrolling experience.

[0024] Further preferably, the obtaining of the height information of the list items and distinguishing between list items with fixed height and list items with variable height is performed by using a dynamic adjustment scroll bar and rendering module, which is used to dynamically adjust the scroll bar and rendering mode according to the height of the list items.

[0025] A virtual scrolling system for processing large data lists, comprising:

[0026] The pre-rendering and caching module is used to collect historical user access data, conduct data analysis to explore user interest preferences and behavior patterns, extract key features for predictive analysis, determine the list items for pre-rendering, render and load page content in advance, and cache the rendered page content;

[0027] The data change detection module is used to monitor data changes in real time using WebSocket and only update and render the changed data items;

[0028] The intelligent scrolling control module monitors the user's scrolling behavior, calculates the scroll bar position, determines the list items in the current visible area, and controls the triggering frequency of the scrolling event based on the calculation results to achieve a smooth scrolling experience.

[0029] Further preferably, it also includes:

[0030] The dynamic adjustment scroll bar and rendering module is used to obtain the height information of the list items, distinguish between list items with fixed height and list items with variable height, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

[0031] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:

[0032] 1. The present invention pre-renders and caches the list items that may be accessed in advance according to the user's historical access data and prediction algorithm, thereby reducing loading time and resource consumption.

[0033] 2. The present invention utilizes data change detection technology to update and render only the data items that have changed, thereby reducing unnecessary page redrawing and improving rendering efficiency.

[0034] 3. The present invention implements an intelligent scrolling control algorithm by combining user interaction behavior and page status, providing a smooth scrolling experience and avoiding page flickering and freezing.

[0035] Fourth, the present invention can dynamically adjust the scroll bar and rendering mode according to the height of the list items, and meet the needs of fixed height and non-fixed height at the same time.

[0036] The above summary is for illustrative purposes only and is not limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0039] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0040] It should be noted that the terms "first", "second", "symmetrical", "array", etc. are only used to distinguish between description and position description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the definition of "first", "symmetrical", etc. can explicitly or implicitly include one or more of these features; similarly, when the number of certain features is not limited in the form of words such as "two" or "three", it should be noted that this feature also explicitly or implicitly includes one or more feature quantities.

[0041] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] Embodiment 1

[0043] like Figure 1 As shown, an embodiment of the present invention provides a virtual scrolling method for processing a large data list, comprising the following steps:

[0044] S1. Architecture design - Divide the system into pre-rendering and caching module, data change detection module, intelligent scroll control module and dynamic scroll bar adjustment and rendering module;

[0045] S2. Collect user historical access data and conduct data analysis to explore user interest preferences and behavior patterns;

[0046] S3, extract key features for prediction analysis, determine the list items for pre-rendering, and render and load the page content;

[0047] S4, page content cache - cache the page content after rendering and loading;

[0048] S5. Use WebSocket to monitor data changes in real time and only update and render the changed data items;

[0049] S6, monitoring the user's scrolling behavior, calculating the scroll bar position, determining the list items in the current visible area, and controlling the triggering frequency of the scrolling event;

[0050] S7. Obtain height information of the list items, distinguish between list items with fixed heights and list items with variable heights, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

[0051] In one embodiment, the pre-rendering and caching module, the data change detection module, the intelligent scrolling control module, and the dynamic scroll bar adjustment and rendering module are independent of each other and are loosely coupled through event monitoring and triggering mechanisms;

[0052] Among them, the event monitoring and triggering mechanism includes data change monitoring, data object change monitoring, scroll event monitoring, viewport size change and content height change monitoring.

[0053] Data change monitoring is based on pre-rendering and cache modules monitoring data change events. When the data changes, the module can trigger a cache update event to update the cache.

[0054] Data object change monitoring is based on the data change detection module monitoring the change events of data objects. When the data object changes, it triggers events of other modules. For example, if the data change detection module detects that the data model has changed, it can notify other modules through the event bus.

[0055] Scroll event monitoring is based on the intelligent scroll control module to monitor scroll events and preload data according to the user's scrolling behavior. When preloading data, it can trigger events of the prerendering and caching modules;

[0056] Viewport size change and content height change monitoring are based on dynamically adjusting the scroll bar and the rendering module monitors viewport size change events and content height change events. When these events occur, it can calculate the new scroll position and trigger the scroll event.

[0057] The collection of user historical access data and data analysis is carried out using the pre-rendering and caching module, which is used to pre-render and cache the list items that may be accessed in the future in advance based on the user's historical access data and prediction algorithm;

[0058] Historical access data includes but is not limited to page visits, browsing time, and interactive behaviors.

[0059] Key features include, but are not limited to, frequently visited pages, visit times, active user groups, and user click paths;

[0060] Predictive analysis is to use a predictive analysis model to predict the user's future access goals, so as to subsequently determine the list items for pre-rendering processing based on the predicted access goals.

[0061] Page content is cached in local storage or memory to reduce requests to the server and speed up page loading.

[0062] By collecting historical user access data, including page visits, browsing time, interactive behavior, etc., data analysis explores user interest preferences and behavior patterns, and extracts features such as frequently visited pages, access time, etc. Based on the prediction results of feature extraction, the list items that need to be pre-rendered are determined, and the page content is rendered and loaded in advance. The rendered page content is cached in local storage or memory for subsequent fast loading.

[0063] In one embodiment, real-time monitoring of data changes using WebSocket is performed using a data change detection module, which is used to update and render only the changed data items using data change detection technology, thereby improving rendering efficiency.

[0064] Use WebSocket to monitor data changes in real time, and only update and render the changed data items, avoiding unnecessary page redrawing and improving page loading speed and performance;

[0065] A basic example of using WebSocket to listen for data changes and update the page when the data changes:

[0066]

[0067]

[0068]

[0069] In the example, a WebSocket connection is created and corresponding event handlers are defined when the connection is opened, a server message is received, the connection is closed, and a connection error occurs. When the server sends a message through WebSocket, the onmessage event handler is called, which parses the received data and calls the updateDataList function to update the data list on the page.

[0070] The updateDataList function clears the existing list items and creates new list items based on the newly received data, thus avoiding unnecessary page redrawing. This approach can significantly improve page performance, especially when processing large amounts of data.

[0071] In one embodiment, monitoring the user's scrolling behavior and calculating the scroll bar position is performed by using an intelligent scrolling control module, which is used to combine the user's interactive behavior and the page state to implement an intelligent scrolling control algorithm;

[0072] The triggering frequency of the scrolling event is controlled by using throttling technology to limit the time so that it can only be triggered once within a specific time, and anti-shake technology is used to delay the execution of the event to achieve a smooth scrolling experience.

[0073] Throttling technology can limit a function to be triggered only once within a specific time. Even if there are multiple triggers in a short period of time, it will not be executed multiple times. The specific example is as follows:

[0074]

[0075] Debouncing technology can delay the execution of a function until a certain period of time after a section of code has been executed. The specific example is as follows:

[0076]

[0077] In one embodiment, the height information of the list items is obtained, and the list items with fixed height and those with variable height are distinguished by using a dynamic scroll bar adjustment and rendering module, which is used to dynamically adjust the scroll bar and rendering mode according to the height of the list items.

[0078] For list items with fixed height, you can directly obtain the fixed height through the CSS style sheet;

[0079] For list items with variable heights, use JavaScript to dynamically measure and use window.getComputedStyle to get the height attribute of each list item;

[0080] To distinguish between list items with fixed and non-fixed heights, JavaScript is used to traverse the list items and determine whether each list item has a fixed height.

[0081] Based on the number of list items and height information in the visible area, we can calculate where the scroll bar should stay so that the user can see different content. This can be achieved by manipulating the scrollTop property of the scroll bar through JavaScript, as shown below:

[0082]

[0083]

[0084]

[0085]

[0086] In this example, a measureVariableHeightItems function is first defined, which measures the heights of all list items with variable heights after the page is loaded, and stores these heights in the dataset.height attribute of the element; then, a calculateScrollPosition function is defined, which calculates the position where the scroll bar should reach. Assume that the list items with fixed heights should occupy at least the height of the visible window. Use the reduce function to calculate the total height of all list items with variable heights. Then, check whether the visible window contains list items with variable heights. If so, adjust the position of the scroll bar according to the total height of these list items; and add a window size change event listener. When the window size changes, it will re-call the measureVariableHeightItems and calculateScrollPosition functions to ensure the correctness of the scroll bar position.

[0087] Embodiment 2

[0088] The embodiment of the present invention also

[0089] A virtual scrolling system for processing large data lists is provided, including:

[0090] The pre-rendering and caching module is used to collect historical user access data, conduct data analysis to explore user interest preferences and behavior patterns, extract key features for predictive analysis, determine the list items for pre-rendering, render and load page content in advance, and cache the rendered page content;

[0091] The data change detection module is used to monitor data changes in real time using WebSocket and only update and render the changed data items;

[0092] The intelligent scrolling control module monitors the user's scrolling behavior, calculates the scroll bar position, determines the list items in the current visible area, and controls the triggering frequency of the scrolling event based on the calculation results to achieve a smooth scrolling experience.

[0093] In one embodiment, it further includes:

[0094] The dynamic adjustment scroll bar and rendering module is used to obtain the height information of the list items, distinguish between list items with fixed height and list items with variable height, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

[0095] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A virtual scrolling method for processing a large data list, characterized in that: The following steps are involved: Architecture design - divide the system into pre-rendering and caching module, data change detection module, intelligent scroll control module and dynamic adjustment scroll bar and rendering module; Collect user historical access data and conduct data analysis to explore user interest preferences and behavior patterns; Extract key features for predictive analysis, determine list items for pre-rendering, and render and load page content; Page content caching - caching page content after rendering and loading; Use WebSocket to monitor data changes in real time and only update and render the changed data items; Monitor the user's scrolling behavior, calculate the scroll bar position, determine the list items in the current visible area, and control the triggering frequency of the scrolling event; Get the height information of the list items, distinguish between list items with fixed height and list items with variable height, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

2. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The pre-rendering and caching module, the data change detection module, the intelligent scrolling control module and the dynamic scroll bar adjustment and rendering module are independent of each other and are loosely coupled through event monitoring and triggering mechanisms; Among them, the event monitoring and triggering mechanism includes data change monitoring, data object change monitoring, scroll event monitoring, viewport size change and content height change monitoring.

3. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The collection of user historical access data and data analysis is performed using a pre-rendering and caching module, which is used to pre-render and cache list items that may be accessed in the future in advance based on the user's historical access data and prediction algorithms; The historical access data includes but is not limited to page visits, browsing time, and interactive behaviors.

4. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The key features include but are not limited to frequently visited pages, visit time, active user groups, and user click paths; The predictive analysis is to use a predictive analysis model to predict the user's future access target, so as to subsequently determine the list items for pre-rendering according to the predicted access target.

5. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The page content caching method is local storage or memory, which is used to reduce requests to the server and speed up page loading.

6. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The use of WebSocket to monitor data changes in real time is performed using a data change detection module, which is used to use data change detection technology to update and render only the data items that have changed, thereby improving rendering efficiency.

7. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The monitoring of the user's scrolling behavior and the calculation of the scroll bar position are performed by using an intelligent scrolling control module, which is used to combine the user's interactive behavior and the page status to implement an intelligent scrolling control algorithm; The triggering frequency of the scrolling event is controlled by using throttling technology to limit the time so that it can only be triggered once within a specific time, and anti-shake technology is used to delay the execution of the event to achieve a smooth scrolling experience.

8. The virtual scrolling method for processing a large data list according to claim 1, characterized in that: The acquisition of the height information of the list items and the differentiation of the list items with fixed height and those with variable height are performed by adopting a dynamic adjustment scroll bar and rendering module, which is used to dynamically adjust the scroll bar and rendering mode according to the height of the list items.

9. A virtual scrolling system for processing a large data list according to any one of claims 1 to 8, characterized in that: include: The pre-rendering and caching module is used to collect historical user access data, conduct data analysis to explore user interest preferences and behavior patterns, extract key features for predictive analysis, determine the list items for pre-rendering, render and load page content in advance, and cache the rendered page content; The data change detection module is used to monitor data changes in real time using WebSocket and only update and render the changed data items; The intelligent scrolling control module monitors the user's scrolling behavior, calculates the scroll bar position, determines the list items in the current visible area, and controls the triggering frequency of the scrolling event based on the calculation results to achieve a smooth scrolling experience.

10. A virtual scrolling system for processing a large data list according to claim 9, characterized in that: Also includes: The dynamic adjustment scroll bar and rendering module is used to obtain the height information of the list items, distinguish between list items with fixed height and list items with variable height, and dynamically adjust the scroll bar position according to the number of list items and height information in the visible area.

Images