Web worker-based data processing method and client

Abstract

The application provides a web worker-based data processing method and a client, the method comprising: receiving a to-be-processed data set sent by a server; splitting the to-be-processed data set into a plurality of first sub-data sets; creating a plurality of first sub-threads based on web workers, and performing parallel processing on data in each first sub-thread to obtain a plurality of processed data sets; putting all the processed data into a first main thread, and rendering all the processed data in the first main thread based on a virtual scrolling technology to control a display page to display first target content. The method uses web workers to open a plurality of sub-threads to perform segmented requests on data, processes the data at the same time, and cooperates with virtual scrolling to quickly render a node tree, realizes a second-opening effect, and solves the problem that in the case of a large amount of data, the time for the client to process data increases significantly, causing screen lag.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to a web worker-based data processing method, client, computer-readable storage medium, and electronic device. Background Technology

[0002] Most current large-scale node tree designs use asynchronous loading or virtual scrolling. Asynchronous loading can lead to significant performance degradation and browser lag when the amount of child node data is large or the number of expanded child nodes increases. Moreover, it requires backend searching, resulting in high costs for backend interface modifications. Virtual scrolling combined with full data processing can significantly increase interface response time when the data volume exceeds 100,000. In addition, the frontend's large-scale data conversion and calculation can lead to longer first-screen rendering time and a poor user experience. Summary of the Invention

[0003] The main objective of this application is to provide a web worker-based data processing method, client, computer-readable storage medium, and electronic device to at least solve the problem of screen lag caused by a significant increase in the time required for client data processing when the amount of data is large.

[0004] To achieve the above objectives, according to one aspect of this application, a web worker-based data processing method is provided, applied to a client. The data processing method includes: receiving a dataset to be processed sent by a server, the dataset including multiple pieces of data to be processed, the data being data supporting the client to run a target application; splitting the dataset into multiple first subsets, each first subset including at least one piece of data to be processed; creating multiple first threads based on web worker, placing each first subset into its corresponding first thread and processing the data in each first thread in parallel to obtain multiple processed datasets, each processed dataset including at least one piece of data, each piece of data being the processed data of the corresponding piece of data to be processed; placing all the processed data into a first main thread, and rendering all the processed data in the first main thread based on virtual scrolling technology, so that the display page displays the first target content.

[0005] Optionally, after rendering all the processed data in the first main thread based on virtual scrolling technology to control the display page to display target content, the method further includes: obtaining search information, the search information being information input by the user to the client; searching all the processed data according to the search information to determine target data, the target data being data that matches the search information; placing all the target data into a second main thread, and rendering all the target data in the second main thread based on virtual scrolling technology, so that the display page displays the second target content.

[0006] Optionally, based on the search information, searching all the processed data to determine the target data includes: splitting all the processed data into multiple search datasets, each search dataset including at least one of the processed data; creating multiple second sub-threads based on a web worker; placing each search dataset into its corresponding second sub-thread, and performing parallel searches on the data in each second sub-thread based on the search information to obtain a target dataset, the target dataset including at least one of the target data.

[0007] Optionally, rendering all the processed data in the first main thread based on virtual scrolling technology to display the first target content on the display page includes: receiving a scrolling operation on the display page; determining display data based on the scrolling operation, wherein the display data is the data of the display content of the next frame of the display page; and rendering the display data based on the virtual scrolling technology to display the first target content on the display page, wherein the first target content is the display content of the next frame of the display page.

[0008] Optionally, there are multiple display data. Rendering the display data based on the virtual scrolling technology to display the first target content on the display page includes: determining a DOM element corresponding to the display data, wherein the DOM element is a node in a target node tree, and the target node tree corresponds to the display content of the display page; and rendering each DOM element according to the cascading relationship of the nodes in the target node tree to display the first target content on the display page.

[0009] Optionally, multiple first sub-threads are created based on a web worker, including: determining the order of magnitude of the data to be processed in the dataset to be processed; determining the number of first sub-threads based on the order of magnitude of the data to be processed, wherein the number of first sub-threads is positively correlated with the order of magnitude of the data to be processed, the number of first sub-threads is less than or equal to a maximum threshold number, and the amount of data to be processed in one first sub-thread is less than or equal to the maximum threshold amount of data; and creating each first sub-thread based on the number of first sub-threads.

[0010] Optionally, splitting the dataset to be processed into multiple first sub-datasets includes: splitting the dataset to be processed into multiple first sub-datasets on an average basis, wherein the amount of data to be processed in each first sub-dataset is the same.

[0011] According to another aspect of this application, a client is provided, comprising: a receiving unit for receiving a dataset to be processed sent by a server, the dataset to be processed including multiple data to be processed, the data to be processed being data supporting the client to run a target application; a splitting unit for splitting the dataset to be processed into multiple first sub-data sets, each first sub-data set including at least one of the data to be processed; a processing unit for creating multiple first sub-threads based on a web worker, placing each first sub-data set into its corresponding first sub-thread and performing parallel processing on the data in each first sub-thread to obtain multiple processed datasets, each processed dataset including at least one processed data, each processed data being the processed data of the corresponding data to be processed; and a rendering unit for placing all the processed data into a first main thread and rendering all the processed data in the first main thread based on virtual scrolling technology, so that the display page displays the first target content.

[0012] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform any of the described web worker-based data processing methods.

[0013] According to another aspect of this application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing any of the webworker-based data processing methods described above.

[0014] Applying the technical solution of this application, the above-mentioned data processing method based on web worker is applied to a client. First, a dataset to be processed is received from the server. The dataset includes multiple pieces of data to be processed, which are data supporting the client to run the target application. Then, the dataset to be processed is split into multiple first subsets, each of which includes at least one piece of data to be processed. Next, multiple first sub-threads are created based on web worker, and each first subset is placed into its corresponding first sub-thread. The data in each first sub-thread is processed in parallel to obtain multiple processed datasets. Each processed dataset includes at least one piece of data to be processed, and each piece of data is the processed data of the corresponding piece of data to be processed. Finally, all the processed data is placed into the first main thread, and all the processed data in the first main thread is rendered based on virtual scrolling technology to control the display page to display the first target content. This method utilizes web workers to start multiple child threads to request data in segments, process the data simultaneously, and then push it to the main thread. It also uses virtual scrolling to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, it also uses web workers to perform segmented search combinations, which improves search efficiency and solves the problem of significantly increased client-side data processing time and screen lag when dealing with large amounts of data. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for executing a web worker-based data processing method is shown in an embodiment of this application.

[0017] Figure 2 A schematic flowchart of a web worker-based data processing method according to an embodiment of this application is shown.

[0018] Figure 3 A flowchart illustrating another web worker-based data processing method provided according to an embodiment of this application is shown.

[0019] Figure 4 A structural block diagram of a client provided according to an embodiment of this application is shown.

[0020] The above figures include the following reference numerals:

[0021] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0026] Web worker: JavaScript that runs in the background, independent of other scripts, and does not affect page performance;

[0027] Virtual scrolling: When a user scrolls to view content, only the relevant data is loaded in the visible area;

[0028] Node tree: Nodes have hierarchical relationships with each other. Parent, child, and sibling nodes are used to describe this relationship. A parent node has child nodes, and child nodes at the same level are called sibling nodes (sibling nodes).

[0029] As described in the background section, existing virtual scrolling technologies generally use full data loading. When the data volume is in the tens of thousands, the request interface takes a long time, and the web processing time also increases significantly, resulting in a long initial loading time for the node tree and a poor user experience. To solve the problem of screen lag caused by the significant increase in client data processing time when the data volume is large, embodiments of this application provide a web worker-based data processing method, client, computer-readable storage medium, and electronic device.

[0030] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0031] The methods and embodiments provided in this application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal based on a web worker-based data processing method according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0032] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the web worker-based data processing method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] This embodiment provides a web worker-based data processing method that runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] Figure 2 This is a flowchart of a web worker-based data processing method according to an embodiment of this application. For example... Figure 2 As shown, the method includes the following steps:

[0035] Step S201: Receive the dataset to be processed sent by the server. The dataset to be processed includes multiple datasets to be processed. The dataset to be processed is data that supports the client to run the target application.

[0036] Specifically, the dataset to be processed includes all the data that supports the client in running the target application. This allows the data to be used directly when the user uses the target application, saving the time of the client sending a request to the server and the server sending data back to the client, reducing screen lag and improving the user experience.

[0037] The client can be a mobile phone, computer, etc.

[0038] In addition, before receiving the dataset to be processed from the server, the client must first send a request to the server, and the server will send the dataset to be processed to the client in response to the client's request.

[0039] Step S202: The dataset to be processed is split into multiple first sub-datasets, and each of the first sub-datasets includes at least one of the above-mentioned data to be processed.

[0040] Specifically, the dataset to be processed typically contains hundreds of thousands of data points or more. At this level, browser lag occurs, and searches require backend queries, leading to high costs associated with backend API modifications. Dividing the dataset into multiple first subsets and then processing them in parallel using web worker sub-threads can significantly reduce processing time and improve efficiency. The number of first subsets should be the same as the number of first sub-threads created using web workers. When the dataset is large, the number of first subsets and first sub-threads can be increased accordingly; however, the number should not be excessive, otherwise it will also negatively impact processing efficiency.

[0041] The process of splitting the dataset to be processed into multiple first sub-datasets includes: equally splitting the dataset to be processed into multiple first sub-datasets, wherein the amount of data to be processed in each of the first sub-datasets is the same.

[0042] Specifically, dividing the dataset to be processed into multiple first subsets on an equal basis ensures that the amount of data in each first subset is the same and the processing time is the same, thereby minimizing the data processing time, improving the efficiency of data processing, and avoiding the situation where some first subsets have too little data and are processed quickly, while others have too much data and are processed slowly.

[0043] Step S203: Create multiple first sub-threads based on web worker, and put each of the above first sub-data sets into the corresponding first sub-threads and process the data in each of the above first sub-threads in parallel to obtain multiple processed datasets. Each of the above processed datasets includes at least one processed data, and each of the above processed data is the data after processing the corresponding data to be processed.

[0044] Specifically, the above steps utilize web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. At the same time, virtual scrolling is used to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, web workers are also used to perform segmented search combinations, which improves search efficiency.

[0045] In the first sub-thread, data processing involves assembling the data obtained from the server and converting the data format so that the converted format can be recognized and executed by the client.

[0046] Current large-scale node tree designs mostly employ asynchronous loading or virtual scrolling. Asynchronous loading, when dealing with large amounts of child node data or an increasing number of expanded child nodes, leads to significant performance degradation, browser lag, and requires backend searching, resulting in high backend API modification costs. Virtual scrolling combined with full data processing significantly increases API response time when dealing with data volumes exceeding 100,000 records. Furthermore, the frontend's large-scale data conversion and calculations result in longer initial screen rendering times and a poor user experience. The method described in the above embodiments, combining web workers with virtual scrolling, effectively solves these problems.

[0047] The specific implementation steps for creating multiple first child threads based on web worker are as follows:

[0048] Step S2031: Determine the order of magnitude of the data to be processed in the dataset to be processed.

[0049] Step S2032: Based on the order of magnitude of the data to be processed, determine the number of the first sub-threads. The number of the first sub-threads is positively correlated with the order of magnitude of the data to be processed. The number of the first sub-threads is less than or equal to the maximum threshold number. The amount of data to be processed in one of the first sub-threads is less than or equal to the maximum threshold amount of data.

[0050] Step S2033: Create each of the above-mentioned first child threads according to the number of the above-mentioned first child threads.

[0051] Specifically, dynamically adjusting the number of the first sub-threads based on the magnitude of the data to be processed can maximize thread resource conservation and improve data processing efficiency while meeting loading requirements. When the amount of data in the dataset to be processed is large, the number of the first sub-dataset and the first sub-thread can be increased accordingly. However, the number of the first sub-dataset and the first sub-thread should not be excessive, otherwise it will also affect the efficiency of data processing.

[0052] Step S204: Place all the processed data into the first main thread, and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0053] Specifically, by combining web worker and virtual scrolling technology, the problem of lag in searching and rendering large amounts of node trees is effectively solved.

[0054] With the development of computer hardware, enterprises are paying more and more attention to user experience, especially for loading and searching large amounts of data in node trees, such as the organizational tree of financial institutions, where the data volume can reach hundreds of thousands or even more. To avoid browser lag, most companies use lazy loading of child nodes or virtual scrolling technology. Lazy loading enables fuzzy search, but the cost of modifying the backend interface is huge, and it requires making requests to the server, resulting in a poor user experience. Virtual scrolling generally uses full data loading, and when the data volume is in the tens of thousands, the request interface takes a long time, and the web processing time also increases significantly, resulting in a long initial loading time for the node tree and a poor user experience.

[0055] Therefore, using a lazy loading scheme for child nodes to process data leads to a significant performance drop and browser lag when the amount of child node data is large or the number of expanded child nodes increases. Furthermore, searching requires the backend to perform a full search of the entire dataset to return a tree structure, resulting in low search efficiency and high backend interface modification costs. While using a virtual scrolling + lazy loading scheme solves the lag problem when the number of child nodes is large or when expanding many child nodes, the issues of low search efficiency and high backend interface modification costs still exist. Finally, using a virtual scrolling + full data processing scheme results in long API request times and long web data processing times, leading to a longer initial loading time and a poor user experience.

[0056] The above embodiment utilizes web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. At the same time, virtual scrolling enables the node tree to be rendered quickly, achieving a near-instantaneous loading effect. When searching, web workers are also used to perform segmented search combinations, which improves search efficiency.

[0057] The specific implementation steps for rendering all the processed data in the first main thread based on virtual scrolling technology to display the first target content on the display page include:

[0058] Step S301: Receive a scrolling operation performed on the above-mentioned display page;

[0059] Step S302: Based on the above scrolling operation, determine the display data, which is the data of the display content of the next frame of the display page;

[0060] Step S303: Render the display data based on the virtual scrolling technology described above, so that the display page displays the first target content, which is the display content of the next frame of the display page.

[0061] Specifically, this allows for precise positioning of the content to be displayed in the visualization area (i.e., the display page), and only the data required by the visualization area needs to be loaded, greatly reducing data loading time, reducing screen lag issues, and improving user experience.

[0062] The above-mentioned displayed data includes multiple items, and the specific implementation steps of step S303 include:

[0063] Step S3031: Determine the DOM element corresponding to the above-mentioned display data. The above-mentioned DOM element is a node in the target node tree, and the above-mentioned target node tree corresponds to the display content of the above-mentioned display page.

[0064] Step S3032: Render each of the above-mentioned DOM elements according to the cascading relationship of the nodes in the target node tree, so that the above-mentioned display page displays the above-mentioned first target content.

[0065] Specifically, this allows for precise and rapid data rendering, improving browser rendering performance by dynamically generating DOM data for the visible area.

[0066] After rendering all the processed data in the first main thread based on virtual scrolling technology to control the display page to show the target content, the method further includes the following steps:

[0067] Step S401: Obtain search information, which is the information input by the user into the client.

[0068] Step S402: Based on the above search information, search all the above-mentioned processed data to determine the target data, which is the data that conforms to the above search information;

[0069] The specific implementation steps of step S402 are as follows:

[0070] Step S4021: Split all the above-mentioned processed data into multiple search datasets, each of the above-mentioned search datasets including at least one of the above-mentioned processed data.

[0071] Step S4022: Create multiple second child threads based on the web worker;

[0072] Step S4023: Each of the above-mentioned search datasets is placed into the corresponding second sub-thread, and the data in each of the above-mentioned second sub-threads is searched in parallel according to the above-mentioned search information to obtain the target dataset, wherein the target dataset includes at least one of the above-mentioned target data.

[0073] Specifically, by starting multiple web workers to group large amounts of data, parallel and fast searching can be achieved through grouping, thereby improving search efficiency.

[0074] Step S403: Place all the aforementioned target data into the second main thread, and render all the aforementioned target data in the second main thread based on virtual scrolling technology, so that the display page displays the second target content.

[0075] Specifically, by enabling multiple web worker requests to load data, the transmission efficiency between the front-end and server-side is improved. The combination of web workers and virtual scrolling technology effectively solves the problem of stuttering during the search and rendering of large-scale node trees.

[0076] The web worker-based data processing method described in this application is applied to a client. First, it receives a dataset to be processed from a server. This dataset includes multiple pieces of data to be processed, which are data supporting the client's execution of the target application. Then, it splits the dataset into multiple first subsets, each containing at least one piece of data to be processed. Next, it creates multiple first sub-threads based on the web worker, placing each first subset into its corresponding first sub-thread and processing the data in each first sub-thread in parallel to obtain multiple processed datasets. Each processed dataset contains at least one piece of processed data, which is the processed version of the corresponding piece of data to be processed. Finally, it places all the processed data into a first main thread and renders all the processed data in the first main thread using virtual scrolling technology to control the display page to show the first target content. This method utilizes web workers to start multiple child threads to request data in segments, process the data simultaneously, and then push it to the main thread. It also uses virtual scrolling to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, it also uses web workers to perform segmented search combinations, which improves search efficiency and solves the problem of significantly increased client-side data processing time and screen lag when dealing with large amounts of data.

[0077] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the web worker-based data processing method of this application will be described in detail below with reference to specific embodiments.

[0078] This embodiment relates to a specific data processing method based on web workers, such as... Figure 3 As shown, it includes the following steps:

[0079] Step S1: The web client requests data from the backend in segments via a web worker;

[0080] Step S2: After receiving the interface response data, the web worker child thread processes and transforms the data according to the web display requirements;

[0081] Step S3: The main thread combines all the data from the web worker child threads and displays the node tree using virtual scrolling technology;

[0082] Step S4: When searching the node tree based on the user's input search content, dynamically start the number of webworker child threads according to the data volume to perform segmented search;

[0083] Step S5: The main thread combines the search results from the sub-threads and displays the node tree after the search using virtual scrolling technology.

[0084] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0085] This application also provides a client. It should be noted that the client in this application can be used to execute the web worker-based data processing method provided in this application. This client is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the client described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0086] The client provided in the embodiments of this application will be described below.

[0087] Figure 4 This is a schematic diagram of a client according to an embodiment of this application. For example... Figure 4As shown, the client includes a receiving unit 10, a splitting unit 20, a processing unit 30, and a rendering unit 40. The receiving unit 10 receives a dataset to be processed sent by the server. The dataset to be processed includes multiple pieces of data to be processed, which are data supporting the client to run the target application. The splitting unit 20 splits the dataset to be processed into multiple first sub-datasets, each of which includes at least one piece of data to be processed. The processing unit 30 creates multiple first sub-threads based on a web worker, puts each of the first sub-datasets into the corresponding first sub-thread, and processes the data in each first sub-thread in parallel to obtain multiple processed datasets. Each of the processed datasets includes at least one piece of data to be processed, and each piece of data to be processed is the data of the corresponding piece of data to be processed. The rendering unit 40 puts all the processed data into a first main thread and renders all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0088] The aforementioned client of this application includes a receiving unit, a splitting unit, a processing unit, and a rendering unit. The receiving unit is used to receive a dataset to be processed sent by the server. The dataset to be processed includes multiple pieces of data to be processed, which are data that support the client in running the target application. The splitting unit is used to split the dataset to be processed into multiple first subsets, each of which includes at least one piece of data to be processed. The processing unit is used to create multiple first sub-threads based on web workers, and to put each first subset into the corresponding first sub-thread and process the data in each first sub-thread in parallel to obtain multiple processed datasets. Each processed dataset includes at least one piece of data to be processed, and each piece of data to be processed is the data of the corresponding piece of data to be processed. The rendering unit is used to put all the processed data into the first main thread and to render all the processed data in the first main thread based on virtual scrolling technology to control the display page to display the first target content. This client utilizes web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. It also uses virtual scrolling to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, it also uses web workers to perform segmented search combinations, which improves search efficiency and solves the problem of significantly increased client data processing time and screen lag when dealing with large amounts of data.

[0089] In some optional solutions, the aforementioned client further includes a first acquisition module, a first determination module, and a first rendering module. The first acquisition module, after rendering all the processed data in the first main thread based on virtual scrolling technology to control the display page to show the target content, acquires search information, which is the information input by the user to the client. The first determination module, based on the search information, searches all the processed data to determine the target data, which is data that matches the search information. The first rendering module places all the target data into a second main thread and renders all the target data in the second main thread based on virtual scrolling technology, causing the display page to show the second target content. By starting multiple web worker requests to load data, the transmission efficiency between the front-end and server is improved. The combination of web worker and virtual scrolling technology effectively solves the problem of large-scale node tree search and rendering lag.

[0090] In some optional solutions, the first determining module includes a splitting submodule, a creation submodule, and a search submodule. The splitting submodule is used to split all the processed data into multiple search datasets, each of which includes at least one of the processed data. The creation submodule is used to create multiple second sub-threads based on web workers. The search submodule is used to place each of the search datasets into its corresponding second sub-thread and perform parallel searches on the data in each second sub-thread according to the search information to obtain the target dataset, which includes at least one of the target data. By starting multiple web workers to group a large amount of data, parallel and fast searching is achieved through grouping, thus improving search efficiency.

[0091] In some optional solutions, the rendering unit includes a first receiving module, a second determining module, and a second rendering module. The first receiving module receives scrolling operations applied to the display page. The second determining module determines display data based on the scrolling operations, whereby the display data is the content to be displayed in the next frame of the display page. The second rendering module renders the display data based on the virtual scrolling technology, causing the display page to display the first target content, which is the content to be displayed in the next frame of the display page. This allows for precise positioning of the content to be displayed in the visualization area (i.e., the display page), requiring only the data needed for the visualization area to be loaded, significantly reducing data loading time, minimizing screen lag, and improving user experience.

[0092] As an optional solution, there are multiple sets of display data. The second rendering module includes a determination submodule and a rendering submodule. The determination submodule is used to determine the DOM element corresponding to the display data. The DOM element is a node in the target node tree, and the target node tree corresponds to the display content of the display page. The rendering submodule is used to render each of the DOM elements according to the cascading relationship of the nodes in the target node tree, so that the display page displays the first target content. This allows for accurate and fast data rendering, and improves browser rendering performance by dynamically generating visible area DOM data.

[0093] An optional scheme includes a processing unit comprising a third determining module, a fourth determining module, and a creation module. The third determining module determines the order of magnitude of the data to be processed in the dataset. The fourth determining module determines the number of first sub-threads based on the order of magnitude of the data to be processed. The number of first sub-threads is positively correlated with the order of magnitude of the data to be processed, and the number of first sub-threads is less than or equal to a maximum threshold number. The amount of data to be processed in one first sub-thread is less than or equal to the maximum threshold amount of data. The creation module creates each first sub-thread based on the number of first sub-threads. Dynamically adjusting the number of first sub-threads based on the order of magnitude of the data to be processed can maximize the saving of thread resources and improve data processing efficiency while meeting loading requirements.

[0094] In this embodiment, the splitting unit includes a splitting module, which is used to equally split the dataset to be processed into multiple first subsets, wherein the amount of data to be processed in each of the first subsets is the same. Splitting the dataset to be processed into multiple first subsets ensures that the amount of data in each first sub-thread is the same, and the data processing time is the same, minimizing data processing time, improving data processing efficiency, and avoiding situations where some first sub-threads have too little data and process quickly, while others have too much data and process slowly.

[0095] The aforementioned client includes a processor and a memory. The aforementioned receiving units, etc., are all stored as program units in the memory, and the processor executes these program units stored in the memory to implement the corresponding functions. All of the aforementioned modules reside in the same processor; alternatively, the aforementioned modules may be located in different processors in any combination.

[0096] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured; adjusting kernel parameters can address the issue of significantly increased client-side data processing time and subsequent screen stuttering when dealing with large amounts of data.

[0097] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0098] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the web worker-based data processing method.

[0099] Specifically, data processing methods based on web workers include:

[0100] Step S201: Receive the dataset to be processed sent by the server. The dataset to be processed includes multiple datasets to be processed. The dataset to be processed is data that supports the client to run the target application.

[0101] Specifically, the dataset to be processed includes all the data that supports the client in running the target application. This allows the data to be used directly when the user uses the target application, saving the time of the client sending a request to the server and the server sending data back to the client, reducing screen lag and improving the user experience.

[0102] Step S202: The dataset to be processed is split into multiple first sub-datasets, and each of the first sub-datasets includes at least one of the above-mentioned data to be processed.

[0103] Specifically, the dataset to be processed typically contains hundreds of thousands of data points or more. At this level, browser lag occurs, and searches require backend queries, leading to high costs associated with backend API modifications. Dividing the dataset into multiple first subsets and then processing them in parallel using web worker sub-threads can significantly reduce processing time and improve efficiency. The number of first subsets should be the same as the number of first sub-threads created using web workers. When the dataset is large, the number of first subsets and first sub-threads can be increased accordingly; however, the number should not be excessive, otherwise it will also negatively impact processing efficiency.

[0104] Step S203: Create multiple first sub-threads based on web worker, and put each of the above first sub-data sets into the corresponding first sub-threads and process the data in each of the above first sub-threads in parallel to obtain multiple processed datasets. Each of the above processed datasets includes at least one processed data, and each of the above processed data is the data after processing the corresponding data to be processed.

[0105] Specifically, the above steps utilize web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. At the same time, virtual scrolling is used to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, web workers are also used to perform segmented search combinations, which improves search efficiency.

[0106] Step S204: Place all the processed data into the first main thread, and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0107] Specifically, by combining web worker and virtual scrolling technology, the problem of lag in searching and rendering large amounts of node trees is effectively solved.

[0108] This invention provides a processor for running a program, wherein the program executes the web worker-based data processing method.

[0109] Specifically, data processing methods based on web workers include:

[0110] Step S201: Receive the dataset to be processed sent by the server. The dataset to be processed includes multiple datasets to be processed. The dataset to be processed is data that supports the client to run the target application.

[0111] Specifically, the dataset to be processed includes all the data that supports the client in running the target application. This allows the data to be used directly when the user uses the target application, saving the time of the client sending a request to the server and the server sending data back to the client, reducing screen lag and improving the user experience.

[0112] Step S202: The dataset to be processed is split into multiple first sub-datasets, and each of the first sub-datasets includes at least one of the above-mentioned data to be processed.

[0113] Specifically, the dataset to be processed typically contains hundreds of thousands of data points or more. At this level, browser lag occurs, and searches require backend queries, leading to high costs associated with backend API modifications. Dividing the dataset into multiple first subsets and then processing them in parallel using web worker sub-threads can significantly reduce processing time and improve efficiency. The number of first subsets should be the same as the number of first sub-threads created using web workers. When the dataset is large, the number of first subsets and first sub-threads can be increased accordingly; however, the number should not be excessive, otherwise it will also negatively impact processing efficiency.

[0114] Step S203: Create multiple first sub-threads based on web worker, and put each of the above first sub-data sets into the corresponding first sub-threads and process the data in each of the above first sub-threads in parallel to obtain multiple processed datasets. Each of the above processed datasets includes at least one processed data, and each of the above processed data is the data after processing the corresponding data to be processed.

[0115] Specifically, the above steps utilize web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. At the same time, virtual scrolling is used to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, web workers are also used to perform segmented search combinations, which improves search efficiency.

[0116] Step S204: Place all the processed data into the first main thread, and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0117] Specifically, by combining web worker and virtual scrolling technology, the problem of lag in searching and rendering large amounts of node trees is effectively solved.

[0118] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0119] Step S201: Receive the dataset to be processed sent by the server. The dataset to be processed includes multiple datasets to be processed. The dataset to be processed is data that supports the client to run the target application.

[0120] Step S202: The dataset to be processed is split into multiple first sub-datasets, and each of the first sub-datasets includes at least one of the above-mentioned data to be processed.

[0121] Step S203: Create multiple first sub-threads based on web worker, and put each of the above first sub-data sets into the corresponding first sub-threads and process the data in each of the above first sub-threads in parallel to obtain multiple processed datasets. Each of the above processed datasets includes at least one processed data, and each of the above processed data is the data after processing the corresponding data to be processed.

[0122] Step S204: Place all the processed data into the first main thread, and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0123] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.

[0124] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0125] Step S201: Receive the dataset to be processed sent by the server. The dataset to be processed includes multiple datasets to be processed. The dataset to be processed is data that supports the client to run the target application.

[0126] Step S202: The dataset to be processed is split into multiple first sub-datasets, and each of the first sub-datasets includes at least one of the above-mentioned data to be processed.

[0127] Step S203: Create multiple first sub-threads based on web worker, and put each of the above first sub-data sets into the corresponding first sub-threads and process the data in each of the above first sub-threads in parallel to obtain multiple processed datasets. Each of the above processed datasets includes at least one processed data, and each of the above processed data is the data after processing the corresponding data to be processed.

[0128] Step S204: Place all the processed data into the first main thread, and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content.

[0129] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0135] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0136] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0137] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0138] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0139] 1) The above-described web worker-based data processing method of this application is applied to a client. First, it receives a dataset to be processed sent by the server. The dataset to be processed includes multiple pieces of data to be processed, which are data that support the client to run the target application. Then, it splits the dataset to be processed into multiple first subsets, each of which includes at least one piece of data to be processed. Next, it creates multiple first sub-threads based on web worker, and puts each first subset into its corresponding first sub-thread and processes the data in each first sub-thread in parallel to obtain multiple processed datasets. Each processed dataset includes at least one piece of processed data, which is the processed data of the corresponding piece of data to be processed. Finally, it puts all the processed data into the first main thread and renders all the processed data in the first main thread based on virtual scrolling technology to control the display page to display the first target content. This method utilizes web workers to start multiple child threads to request data in segments, process the data simultaneously, and then push it to the main thread. It also uses virtual scrolling to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, it also uses web workers to perform segmented search combinations, which improves search efficiency and solves the problem of significantly increased client-side data processing time and screen lag when dealing with large amounts of data.

[0140] 2) The client described above in this application includes a receiving unit, a splitting unit, a processing unit, and a rendering unit. The receiving unit is used to receive a dataset to be processed sent by the server. The dataset to be processed includes multiple pieces of data to be processed, which are data that support the client in running the target application. The splitting unit is used to split the dataset to be processed into multiple first subsets, each of which includes at least one piece of data to be processed. The processing unit is used to create multiple first sub-threads based on web workers, and to put each first subset into the corresponding first sub-thread and to process the data in each first sub-thread in parallel to obtain multiple processed datasets. Each processed dataset includes at least one piece of data to be processed, and each piece of data to be processed is the data of the corresponding data to be processed. The rendering unit is used to put all the processed data into the first main thread and to render all the processed data in the first main thread based on virtual scrolling technology to control the display page to display the first target content. This client utilizes web workers to start multiple child threads to request data in segments, process the data, and then push it to the main thread. It also uses virtual scrolling to render the node tree quickly, achieving a near-instantaneous loading effect. When searching, it also uses web workers to perform segmented search combinations, which improves search efficiency and solves the problem of significantly increased client data processing time and screen lag when dealing with large amounts of data.

[0141] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A data processing method based on web workers, characterized in that, Applied to the client side, the data processing method includes: The server receives a dataset to be processed, which includes multiple datasets to be processed, and the datasets to be processed are data that supports the client in running the target application. The dataset to be processed is split into multiple first sub-datasets, and each first sub-dataset includes at least one of the datasets to be processed. Multiple first child threads are created based on web worker, and each first sub-dataset is put into the corresponding first child thread and the data in each first child thread is processed in parallel to obtain multiple processed datasets. Each processed dataset includes at least one processed data, and each processed data is the data after processing the corresponding data to be processed. All the processed data is placed into the first main thread, and all the processed data in the first main thread is rendered based on virtual scrolling technology so that the display page displays the first target content; Obtain search information, which is information input by the user into the client; Based on the search information, all the processed data are searched to determine the target data, which is data that matches the search information. All the target data is placed in the second main thread, and all the target data in the second main thread is rendered based on virtual scrolling technology, so that the display page displays the second target content; The process of searching all the processed data according to the search information to determine the target data includes: splitting all the processed data into multiple search datasets, each search dataset including at least one of the processed data; creating multiple second sub-threads based on web worker; placing each search dataset into its corresponding second sub-thread, and performing parallel search on the data in each second sub-thread according to the search information to obtain the target dataset, which includes at least one of the target data.

2. The data processing method according to claim 1, characterized in that, Rendering all processed data in the first main thread using virtual scrolling technology, so that the display page shows the first target content, including: Receive scrolling operations performed on the displayed page; Based on the scrolling operation, display data is determined, whereby the display data is the data of the display content of the next frame of the display page; The display data is rendered based on the virtual scrolling technology, so that the display page displays the first target content, which is the display content of the next frame of the display page.

3. The data processing method according to claim 2, characterized in that, The displayed data includes multiple data sets. The displayed data is rendered based on the virtual scrolling technology, causing the display page to display the first target content, including: Determine the DOM element corresponding to the displayed data, wherein the DOM element is a node in the target node tree, and the target node tree corresponds to the displayed content of the display page; According to the cascading relationship of the nodes in the target node tree, each of the DOM elements is rendered so that the display page displays the first target content.

4. The data processing method according to claim 1, characterized in that, Multiple first child threads are created based on web worker, including: Determine the order of magnitude of the data to be processed in the dataset to be processed; The number of first sub-threads is determined based on the order of magnitude of the data to be processed. The number of first sub-threads is positively correlated with the order of magnitude of the data to be processed. The number of first sub-threads is less than or equal to the maximum threshold number. The amount of data to be processed in one first sub-thread is less than or equal to the maximum threshold amount of data. Create each of the first child threads based on the number of first child threads.

5. The data processing method according to claim 1, characterized in that, The dataset to be processed is split into multiple first subsets, including: The dataset to be processed is divided into multiple first subsets, wherein the amount of data to be processed in each first subset is the same.

6. A client application, characterized in that, include: The receiving unit is used to receive the dataset to be processed sent by the server. The dataset to be processed includes multiple data to be processed, and the data to be processed is data that supports the client to run the target application. A splitting unit is used to split the dataset to be processed into multiple first sub-datasets, each of which includes at least one of the datasets to be processed. The processing unit is configured to create multiple first sub-threads based on web worker, and put each first sub-dataset into the corresponding first sub-thread and process the data in each first sub-thread in parallel to obtain multiple processed datasets. Each processed dataset includes at least one processed data, and each processed data is the data after processing the corresponding data to be processed. The rendering unit is used to put all the processed data into the first main thread and render all the processed data in the first main thread based on virtual scrolling technology so that the display page displays the first target content. The first acquisition module is used to acquire search information, which is information input by the user into the client. The first determining module is used to search all the processed data according to the search information to determine the target data, wherein the target data is data that conforms to the search information; The first rendering module is used to put all the target data into the second main thread, and render all the target data in the second main thread based on virtual scrolling technology, so that the display page displays the second target content; The first determining module includes a splitting submodule, a creation submodule, and a search submodule. The splitting submodule is used to split all the processed data into multiple search datasets, and each search dataset includes at least one of the processed data. The creation submodule is used to create multiple second sub-threads based on a web worker. The search submodule is used to put each search dataset into its corresponding second sub-thread and perform parallel searches on the data in each second sub-thread according to the search information to obtain a target dataset, which includes at least one target data.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the web worker-based data processing method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including methods for performing the webworker-based data processing method according to any one of claims 1 to 5.

Images