A tree node creation and query method

Abstract

The present invention provides a tree node creation and query method, including obtaining data, and further comprising the following steps: using asynchronous parallel processing technology to perform multi-threaded batch processing on data; storing the processed data in a database; calling the database to load the tree data; using hierarchical loading, data grouping technology, and data asynchronous transmission technology to create and query the tree. The present invention proposes a tree node creation and query method, which uses data grouping technology, asynchronous parallel processing, data asynchronous transmission, and hierarchical technology loading to create and query the tree node, meeting the growing data processing needs and solving the efficiency problems brought about by the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of databases, in particular to a tree node creation and query method. Background Art

[0002] The tree structure is a common data structure used to represent data with a hierarchical relationship. In computer science, the tree structure is widely used in web page rendering, data storage and other fields.

[0003] Traditional tree node creation and query are usually implemented through recursive algorithms and single-threaded methods, which are extremely inefficient under large-scale data. For scenarios where business data reaches more than hundreds of thousands of tree data and needs to be processed simultaneously, traditional tree node creation and query methods can no longer meet the needs, and new methods and technologies need to be explored. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the present invention proposes a tree node creation and query method, which adopts data grouping technology, asynchronous parallel processing, data asynchronous transmission, and layered technology loading to realize the creation and query of tree nodes, meet the growing data processing needs, and solve the efficiency problems brought by the existing technology.

[0005] The object of the present invention is to provide a tree node creation and query method, comprising obtaining data, and further comprising the following steps:

[0006] Step 1: Use asynchronous parallel processing technology to perform multi-threaded batch processing on data;

[0007] Step 2: Store the processed data into the database;

[0008] Step 3: Call the database to load tree data;

[0009] Step 4: Use layered loading, data grouping technology, and data asynchronous transmission technology to create and query the tree.

[0010] Preferably, step 4 includes the following sub-steps:

[0011] Step 41: The front end requests data through layering, lazy loading, and asynchronous transmission methods;

[0012] Step 42: The front end uses a load balancing polling algorithm using data grouping technology to call a relatively idle server to read the corresponding single-layer data;

[0013] Step 43: The client receives the service request number from the server, and concurrently calls the load balancing polling algorithm service, and each server executes a query of the corresponding single-layer data and returns the query result to the browser;

[0014] Step 44: The front-end browser asynchronously loads the single-layer data;

[0015] Step 45: Complete the loading of all single-layer data in a loop.

[0016] In any of the above schemes, preferably, the layering method is to group and classify all acquired data to form data groups of different levels, and then manage the data according to the levels, and the tree reads, loads and displays the data according to the levels.

[0017] In any of the above solutions, preferably, the lazy loading method is to load and display only the relevant data of the clicked node and all child nodes for the tree node.

[0018] In any of the above schemes, preferably, the asynchronous transmission method is that when the front end clicks multiple nodes, the front end response of the later clicked node does not need to wait until the previous clicked node is loaded before background data loading.

[0019] In any of the above solutions, preferably, step 42 includes calculating each single layer of data using a load balancing polling algorithm.

[0020] In any of the above solutions, preferably, the load balancing polling algorithm includes the following sub-steps:

[0021] Step 421: Deployment n +1 server, one of which is the algorithm server for deploying the service request number algorithm, and the others n The station is a data query request server, and the name of the data query request server is ( N 1 , N 2 , N 3 , …, N n );

[0022] Step 422: Obtain performance parameters of each data query request server;

[0023] Step 423: Calculate the maximum data query volume under the optimal performance of a single server;

[0024] Step 424: The algorithm server calculates the service request number k using the service request number algorithm according to the single-layer data volume requested by the browser.

[0025] In any of the above solutions, preferably, the algorithm server is configured as 4-a core 4-a ， G memory; N 1 Server 4-1a Core 4-1a ， G memory;N 2 The server is configured as 4-2a core 4-2a ， G memory, ..., N n The server is configured with 4-na cores and 4-na ， G memory, of which 4-na core is the first n Number of cores in a server, 4-na ， G is the n The amount of memory on each server;

[0026] Maximum memory server N max =Max(4-a ， G, 4-2a ， G, ..., 4-na ， G)

[0027] Maximum number of cores server H max =Max(4-a core, 4-2a core, …, 4-na core).

[0028] In any of the above solutions, preferably, the optimal performance means that the memory occupancy rate and the CPU occupancy rate are not less than 50% and not more than 70%, and the data query time does not exceed 2 seconds.

[0029] In any of the above solutions, preferably, the data query request server N n The maximum amount of data to query under optimal performance is t n Ten thousand~ s n Ten thousand data, F for n The minimum value of the maximum data query amount range under the optimal performance among the data query request servers.

[0030] In any of the above solutions, preferably, step 423 includes setting the maximum data query amount under optimal performance ( N 1 , N 2 , N 3 , …, N n ) are arranged from large to small according to the minimum value of the maximum data query amount under the optimal performance, and the result is ( U 1 , U 2 , U 3 , …, U n ),in U1 The minimum value of the maximum data query volume under the optimal performance is greater than U 2 The minimum value of the maximum data query amount under the optimal performance, U 2 The minimum value of the maximum data query volume under the optimal performance is greater than U 3 The minimum value of the maximum data query amount under the optimal performance, …, U n-1 The minimum value of the maximum data query volume under the optimal performance is greater than U n The minimum value of the maximum data query amount under the optimal performance.

[0031] In any of the above schemes, it is preferred that U n The maximum data query volume range under the optimal performance is b n Ten thousand~ b ' n Ten thousand.

[0032] In any of the above solutions, preferably, the service request number algorithm includes

[0033] 1) When the query data volume is ≤ F ten thousand, use the H with the largest number of cores. max The server performs data query and forms a service request k;

[0034] 2) When the amount of query data is greater than F ten thousand, the data query task is distributed among query servers with different performances, and the server with the best performance is selected. n The query server uses a balanced processing and weighted load factor calculation method to perform data query.

[0035] In any of the above solutions, preferably, the optimal performance of the single server operation refers to U 1 ~ U num-1 The query volume of the last server reaches the minimum value of the optimal query volume range. U num The server handles the remaining queries, i.e.

[0036] U 1 Server Processing b 1 Ten thousand pieces of data form a service request k 1 ;

[0037] U 2Server Processing b 2 Ten thousand pieces of data form a service request k 2 ;

[0038] …

[0039] U num-1 Server Processing b num-1 Ten thousand pieces of data form a service request k num-1 ;

[0040] Final m -( b 1 + b 2 +…+ b num-1 ) 10,000 pieces of data U num The server processes and forms another service request k num ,in, m is the total data volume, num The number of query servers required for each query.

[0041] In any of the above schemes, it is preferred that n The balanced processing of query servers means calculating the query volume of each server according to the query volume factor of each query server as a percentage of the total factor, that is,

[0042] N 1 Data query volume factor Z 1 = Num (4-1a) / (( Num (4-1a)+( Num (4-2a)+…+(Num(4-na));

[0043] N 2 Data query volume factor Z 2 =( Num (4-2a) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0044] …

[0045] N n Data query volume factorZ n =( Num (4-na) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0046] in: Num (4-1a) is the number of cores of the first machine; Num (4-2a) is the number of cores of the second machine; ...; Num (4-na) is the n The value of the number of cores on the machine;

[0047] N 1 deal with F*Z 1 Ten thousand pieces of data form a service request k 1 ; N 2 deal with F*Z 2 Ten thousand pieces of data form another service request k 2 ;…; N n deal with F*Z n Ten thousand pieces of data form another service request k n .

[0048] In any of the above schemes, preferably, the weighted load factor calculation method is to calculate the server query volume for each server by multiplying the weight by the total number of data volume query volume factors, that is,

[0049] Assumptions N 1 The weight is p 1 , N 2 The weight is p 2 , …, N n The weight is p n , p 1 + p 2 +…+ p n =1),

[0050] N 1 The data processing volume is m*Z 1 * p 1 Ten thousand, forming a service request k 1 ; N 2 The data processing volume is m*Z 2 * p 2 Ten thousand, forming another service request k 2 ;…; N n The data processing volume is m*Z n *p n Ten thousand, forming another service request k n .

[0051] In any of the above solutions, preferably, step 43 includes the following sub-steps:

[0052] Step 431: The client receives the service request number from the server. k , each k The corresponding data service request content and the server identifier for the corresponding task execution;

[0053] Step 432: The browser concurrently calls the service request number algorithm service on the algorithm server to calculate the query of the corresponding single-layer data executed by the designated server;

[0054] Step 433: After the query is completed, the result is returned to the service request number algorithm service, and then the service request number algorithm service returns the result to the browser.

[0055] The present invention proposes a tree node creation and query method, which is more efficient than traditional synchronous creation, data synchronous transmission, and data one-time loading technologies, and can be better applied to processing complex big data tree creation and query scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 The present invention is a flowchart of a preferred embodiment of the tree node creation and query method according to the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. Embodiment 1

[0058] like Figure 1 As shown, execute step 1000 to obtain data.

[0059] Execute step 1100 to perform multi-threaded batch processing on the data using asynchronous parallel processing technology.

[0060] Execute step 1200 to store the processed data in a database.

[0061] Execute step 1300 to call the database to load the tree data.

[0062] Execute step 1400, using layered loading, data grouping technology, and data asynchronous transmission technology to create and query the tree, including the following sub-steps:

[0063] Execute step 1410, the front end requests data through a layered, lazy loading, and asynchronous transmission method, wherein the layered method is to group and classify all acquired data into data groups of different levels, and then manage the data according to the levels, and the tree reads, loads, and displays the data according to the levels;

[0064] The lazy loading method is to load and display only the relevant data of the clicked node and all its child nodes for the tree node;

[0065] The asynchronous transmission method is that when the front end clicks multiple nodes, the front end response of the later clicked node does not need to wait until the front clicked node is loaded before the background data is loaded.

[0066] Execute step 1420, the front end uses the load balancing polling algorithm of data grouping technology to call a relatively idle server to read the corresponding single-layer data, and each single-layer data is calculated using the load balancing polling algorithm.

[0067] The load balancing polling algorithm includes the following sub-steps:

[0068] Execute step 1421, deployment n +1 server, one of which is the algorithm server for deploying the service request number algorithm, and the others n The station is a data query request server, and the name of the data query request server is ( N 1 , N 2 , N 3 , …, N n ); The algorithm server is configured as 4-a core 4-a ， G memory; N 1 Server 4-1a Core 4-1a ， G memory; N 2 The server is configured as 4-2a core 4-2a ， G memory, ..., Nn The server is configured with 4-na cores and 4-na ， G memory, of which 4-na core is the first n Number of cores in a server, 4-na ， G is the n The amount of memory on each server;

[0069] Maximum memory server N max =Max(4-a ， G, 4-2a ， G, ..., 4-na ， G)

[0070] Maximum number of cores server H max =Max(4-a core, 4-2a core, …, 4-na core).

[0071] Execute step 1422 to obtain performance parameters of each data query request server;

[0072] Execute step 1423 to calculate the maximum data query volume under the optimal performance of a single server, where the optimal performance means that the memory occupancy rate and the CPU occupancy rate are not less than 50% and not more than 70%, and the data query time does not exceed 2 seconds. N n The maximum amount of data to query under optimal performance is t n Ten thousand~ s n Ten thousand data, F for n The minimum value of the maximum data query amount range under the optimal performance among the data query request servers.

[0073] The maximum data query volume under optimal performance ( N 1 , N 2 , N 3 , …, N n ) are arranged from large to small according to the minimum value of the maximum data query amount under the optimal performance, and the result is ( U 1 , U 2 , U 3 , …, U n ),in U 1 The minimum value of the maximum data query volume under the optimal performance is greater than U 2The minimum value of the maximum data query amount under the optimal performance, U 2 The minimum value of the maximum data query volume under the optimal performance is greater than U 3 The minimum value of the maximum data query amount under the optimal performance, …, U n-1 The minimum value of the maximum data query volume under the optimal performance is greater than U n The minimum value of the maximum data query amount under the optimal performance. U n The maximum data query volume range under the optimal performance is b n Ten thousand~ b ' n Ten thousand.

[0074] Execute step 1424, the algorithm server calculates the service request number using the service request number algorithm according to the single-layer data volume requested by the browser k , the service request number algorithm includes

[0075] 1) When the query data volume is ≤ F ten thousand, any data query server can perform the query operation under the required performance. max The server performs data query and forms a service request k ;

[0076] 2) When the amount of query data is greater than F ten thousand, the data query task is distributed among query servers with different performances, and the server with the best performance is selected. n The query server uses a balanced processing and weighted load factor calculation method to perform data query.

[0077] In the actual query process, the amount of data does not necessarily require all servers to run to achieve query performance. In this solution, the number of query servers required for each query is defined as num .

[0078] The optimal performance of a single server is U 1 ~ U num-1 The query volume of the last server reaches the minimum value of the optimal query volume range. U num The server handles the remaining queries, i.e.

[0079] U 1 Server Processing b 1 Ten thousand pieces of data form a service requestk 1 ;

[0080] U 2 Server Processing b 2 Ten thousand pieces of data form a service request k 2 ;

[0081] …

[0082] U num-1 Server Processing b num-1 Ten thousand pieces of data form a service request k num-1 ;

[0083] Final m -( b 1 + b 2 +…+ b num-1 ) 10,000 pieces of data U num The server processes and forms another service request k num ,in, m is the total data volume, num The number of query servers required for each query.

[0084] Said n The balanced processing of query servers means calculating the query volume of each server according to the query volume factor of each query server as a percentage of the total factor, that is,

[0085] N 1 Data query volume factor Z 1 = Num (4-1a) / (( Num (4-1a)+( Num (4-2a)+…+(Num(4-na));

[0086] N 2 Data query volume factor Z 2 =( Num (4-2a) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0087] …

[0088] N n Data query volume factor Z n =( Num (4-na) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0089] in: Num (4-1a) is the number of cores of the first machine; Num (4-2a) is the number of cores of the second machine; ...; Num (4-na) is the n The value of the number of cores on the machine;

[0090] N 1 deal with F*Z 1 Ten thousand pieces of data form a service request k 1 ; N 2 deal with F*Z 2 Ten thousand pieces of data form another service request k 2 ;…; N n deal with F*Z n Ten thousand pieces of data form another service request k n .

[0091] The weighted load factor calculation method is to calculate the server query volume by multiplying the weight of each server by the total number of data volume query volume factors, that is,

[0092] Assumptions N 1 The weight is p 1 , N 2 The weight is p 2 , …, N n The weight is p n , p 1 + p 2 +…+ p n =1),

[0093] N 1 The data processing volume is m*Z 1 * p 1 Ten thousand, forming a service request k 1 ; N 2 The data processing volume is m*Z 2 * p 2 Ten thousand, forming another service request k 2 ;…; N n The data processing volume is m*Z n *p n Ten thousand, forming another service request k n .

[0094] Execute step 1430, the client receives the service request number of the server, and concurrently calls the load balancing polling algorithm service, each server executes the query of the corresponding single-layer data and returns the query result to the browser, including the following sub-steps:

[0095] Execute step 1431, the client receives the service request number from the server k , each k The corresponding data service request content and the server identifier for the corresponding task execution;

[0096] Step 1432 is executed, the browser concurrently calls the service request number algorithm service on the algorithm server, and the designated server is calculated to execute the query of the corresponding single-layer data;

[0097] Execute step 1433, and after the query is completed, return the result to the service request number algorithm service, and then the service request number algorithm service returns the result to the browser.

[0098] Execute step 1440, the front-end browser asynchronously loads the single-layer data.

[0099] Execute step 1450 to loop through the loading of all single-layer data. Embodiment 2

[0100] The purpose of the present invention is to provide a more advanced, efficient and flexible tree node creation and query method to meet the growing data processing needs and solve the efficiency problems brought by the prior art. The data query efficiency is improved and the data request process is optimized by using grouping technology for the data required by the business system.

[0101] The present invention adopts data grouping technology, asynchronous parallel processing, data asynchronous transmission, and hierarchical technology loading to realize the creation and query of tree nodes. Compared with traditional synchronous creation, data synchronous transmission, and one-time data loading technology, this solution is more efficient and can be better applied to complex large data tree creation and query scenarios.

[0102] The main steps are as follows:

[0103] Step 1: Get data;

[0104] Step 2: Use asynchronous parallel processing technology to perform multi-threaded batch processing on data;

[0105] Step 3: Store the processed data into the database;

[0106] Step 4: Call the database to load tree data;

[0107] Step 5: Use layered loading, data grouping technology, and data asynchronous transmission technology to create and query the tree.

[0108] Among them, step 5 is the main protection content.

[0109] Step 5 includes the following sub-steps:

[0110] Step 51: The front end requests data through layering, lazy loading, and asynchronous transmission technology (assuming that the total data request volume is m million);

[0111] The hierarchical method is: group and classify all the acquired data to form data groups of different levels, and then manage the data according to the levels. The tree reads, loads and displays the data according to the levels.

[0112] The lazy loading method is: for tree nodes, only load and display the relevant data of the clicked node and all child nodes.

[0113] The asynchronous transmission method is: when the front end clicks multiple nodes, the front end response of the later clicked node does not need to wait until the previous clicked node is loaded before loading the background data.

[0114] Step 52: The front end uses a load balancing polling algorithm using data grouping technology to call a relatively idle server to read the corresponding single-layer data;

[0115] Each single layer of data is calculated using a load balancing polling algorithm (called the "service request number algorithm"), the specific algorithm is as follows:

[0116] (1) Deployment n +1 server, one of which is used to deploy the "Service Request Number Algorithm" (hereinafter referred to as the "algorithm server") server, and the othern The server for data query request (the server name is ( N 1 , N 2 … N n ); data is saved on storage.

[0117] (2) Obtain performance parameters of each machine

[0118] Assumption: Algorithm server is configured with 4-a cores ， G memory; N 1 Server 4-1a Core 4-1a ， G memory; N 2 The server is configured as 4-2a core 4-2a ， G memory, ..., N n The server is configured with 4-na cores and 4-na ， G memory.

[0119] Maximum memory server N max =Max(4-a ， G, 4-2a ， G, ..., 4-na ， G)

[0120] Maximum number of cores server H max =Max(4-a core, 4-2a core, …, 4-na core)

[0121] (3) Calculate the maximum data query volume under the optimal performance of a single server

[0122] Optimal performance means: memory occupancy and CPU occupancy are not less than 50% and not more than 70%, and data query time does not exceed 2 seconds;

[0123] Assumptions:

[0124] N 1 The maximum amount of data query under the optimal performance of the server is t 1 Ten thousand~ t 1 Ten thousand data;

[0125] N 2 The maximum amount of data query under the optimal performance of the server is t 2 Ten thousand~ t 2 Ten thousand data;

[0126] …

[0127] N n The maximum amount of data query under the optimal performance of the server is t n Ten thousand~ s n Ten thousand data;

[0128] have:

[0129] F for n The minimum value of the "maximum data query volume range under optimal performance" among the data query servers.

[0130] The maximum data query volume under optimal performance ( N 1 , N 2 , N 3 , …, N n ) are arranged from large to small according to the minimum value of the maximum data query amount under the optimal performance, and the result is ( U 1 , U 2 , U 3 , …, U n ),in U 1 The minimum value of the maximum data query volume under the optimal performance is greater than U 2 The minimum value of the maximum data query amount under the optimal performance, U 2 The minimum value of the maximum data query volume under the optimal performance is greater than U 3 The minimum value of the maximum data query amount under the optimal performance, …, and so on.

[0131] Assumptions:

[0132] U 1 The maximum data query volume range under the optimal performance is b 1 Ten thousand~ b’ 1 Ten thousand

[0133] U 2 The maximum data query volume range under the optimal performance is b 2 Ten thousand~ b’ 2 Ten thousand

[0134] …

[0135] U n The maximum data query volume range under the optimal performance is b n Ten thousand~ b’ n Ten thousand.

[0136] (4) The algorithm server uses the "service request number algorithm" to calculate the "service request number" based on the amount of single-layer data requested by the browser. k .

[0137] The "Service Request Number Algorithm" is as follows:

[0138] 1) When the query data volume is ≤ F ten thousand records

[0139] Using H with a larger number of cores max The server performs data query and forms a service request k;

[0140] 2) When the query data volume > F ten thousand records

[0141] 2.1) Optimal performance when running on a single server

[0142] Algorithm description: U 1 ~ U num-1 The query volume of the last server reaches the minimum value of the optimal query volume range. U num The server handles the remaining query volume.

[0143] U 1 Server Processing b 1 Ten thousand pieces of data form a service request k 1 ;

[0144] U 2 Server Processing b 2 Ten thousand pieces of data form a service request k 2 ;

[0145] …

[0146] U num-1 Server Processing b num-1 Ten thousand pieces of data form a service request k num-1 ;

[0147] Finalm -( b 1 + b 2 +…+ b num-1 ) 10,000 pieces of data U num The server processes and forms another service request k num .

[0148] 2.2) Balanced processing of N query servers

[0149] Algorithm description: Calculate the query volume of each server according to the proportion of the query volume factor of each query server to the total factor.

[0150] N 1 Data query volume factor Z 1 = Num (4-1a) / (( Num (4-1a)+( Num (4-2a)+…+(Num(4-na));

[0151] N 2 Data query volume factor Z 2 =( Num (4-2a) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0152] …

[0153] N n Data query volume factor Z n =( Num (4-na) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na));

[0154] in: Num (4-1a) is the number of cores of the first machine; Num (4-2a) is the number of cores of the second machine; ...; Num (4-na) is the n The value of the number of cores on the machine;

[0155] In this case: N1 deal with F*Z 1 Ten thousand pieces of data form a service request k 1 ; N 2 deal with F*Z 2 Ten thousand pieces of data form another service request k 2 ;…; N n deal with F*Z n Ten thousand pieces of data form another service request k n .

[0156] 2.3) Weighted load factor calculation method

[0157] Algorithm description: Each server calculates the server query volume by multiplying its weight by the total number of data volume query volume factors.

[0158] Assumptions N 1 The weight is p 1 , N 2 The weight is p 2 , …, N n The weight is p n ( p 1 , p 2 … p n Dynamically adjusted according to the age of the machine hardware and actual conditions, ranging from 0 to 1. p 1 + p 2 +…+ p n =1)

[0159] N 1 The data processing volume is: m*Z 1 * p 1 Ten thousand, forming a service request k 1 ; N 2 The data processing volume is: m*Z 2 * p2 Ten thousand, forming another service request k 2 ;…; N n The data processing volume is: m*Z n *p n Ten thousand, forming another service request k n .

[0160] Step 53: The client receives the "service request number" from the server k , each k The corresponding data service request content and the server identifier of the corresponding task execution are then called by the browser concurrently on the "Service Request Number Algorithm" service on the algorithm server. The "Service Request Number Algorithm" calculates the query of the corresponding single-layer data on the designated server. After the query is completed, the result is returned to the "Service Request Number Algorithm" service, and then the "Service Request Number Algorithm" service returns the result to the browser.

[0161] Step 54: The front-end browser asynchronously loads the single-layer data;

[0162] Step 55: Loop through all single-layer data loading. Embodiment 3

[0163] Each single layer of data is calculated using a load balancing polling algorithm (called the "service request number algorithm"), the specific algorithm is as follows:

[0164] (1) Deploy three servers. Server A deploys the "service request number algorithm", and servers C and D are data query request servers. The data is stored in the storage.

[0165] (2) Obtain performance parameters of each machine

[0166] Here are: A, 4 cores and 8G memory; C, 16 cores and 16G memory; D, 32 cores and 16G memory.

[0167] (3) Calculate the maximum data query volume under the optimal performance of a single server

[0168] Optimal performance means: memory occupancy and CPU occupancy are not less than 50% and not more than 70%, and data query time does not exceed 2 seconds;

[0169] After testing, it was found that:

[0170] The maximum data query volume under the optimal performance of server C is 100,000 to 270,000 data items;

[0171] The maximum data query volume under the optimal performance of D server is 230,000 to 400,000 data;

[0172] (4) Server A calculates the “service request number” k based on the amount of single-layer data requested by the browser using the “service request number algorithm”.

[0173] The "Service Request Number Algorithm" is as follows:

[0174] 1) When m≤100,000

[0175] Use the D server with a larger number of cores to query the data, forming a service request k;

[0176] 2) When m>100,000

[0177] 2.1) Optimize the performance of server C

[0178] Server C processes 200,000 pieces of data, forming a service request k1; the remaining (m-200,000) pieces are all processed by server D, forming another service request k2.

[0179] 2.2) Make the D server run with optimal performance

[0180] Server D processes 400,000 pieces of data, forming a service request k1; the remaining (m-400,000) pieces of data are all processed by server C, forming another service request k2.

[0181] 2.3) C and D balanced processing

[0182] C data query factor = C core number / (C core number + D core number) = 1 / 3; D data query factor = D core number / (C core number + D core number) = 2 / 3

[0183] C processes 1 / 3 million data and forms a service request k1; D processes 2 / 3 million data and forms another service request k2.

[0184] 2.4) Weighted C and D load factor calculation method, assuming that the weight of C is p and the weight of D is q (p and q are dynamically adjusted according to the age of the machine hardware and the actual situation, ranging from 0 to 1, p+q=1)

[0185] C data query volume factor = p / (p+q); D data query volume factor = q / (p+q)

[0186] The data processing volume of C is: m*p / (p+q) ten thousand, forming a service request k1; the data processing volume of D is: m*q / (p+q) ten thousand, forming another service request k2. Embodiment 4

[0187] In this embodiment, the configuration of the server is shown in Table 1.

[0188] Table 1 Server configuration table

[0189] Serial number Server Type Number of cores (unit: core) Memory (Unit: G) 1 Algorithm Server 4 8 2 N1 Server 16 16 3 N2 Server 32 16

[0190] After testing, it was found that:

[0191] The maximum data query volume under the optimal performance of N1 server is 100,000 to 270,000 data items;

[0192] The maximum data query volume under the optimal performance of N2 server is 230,000 to 400,000 data;

[0193] 2. The beneficial effects of the present invention are as follows:

[0194] As shown in Table 2, the efficiency of data loading and display is improved: the traditional method takes 120.76 seconds to load and display 600,000 data, but only 1.8 seconds after using the combination of layering technology, data grouping technology, and data asynchronous transmission technology.

[0195] Table 1 Data loading and display efficiency improvement effect

[0196] Serial number Data volume Traditional algorithm data transmission speed Data transmission speed of the present invention 1 5000 records 3.48s 0.18s 2 20,000 records 35.31s 0.3s 3 100,000 records 60s 0.4s 4 600,000 records 120.76s 1.8s Embodiment 5

[0197] In inventory management, large logistics companies use tree list data to display the hierarchical relationship of inventory items. The inventory items can be represented as tree nodes, and the relationship between nodes represents the classification and storage location of the items. Through tree list data, companies can better manage the location and quantity of inventory items and improve inventory turnover and accuracy.

[0198] In order management on large e-commerce platforms, tree list data can be used to represent the hierarchical relationship of orders. An order can contain multiple sub-orders, and the relationship between sub-orders represents the dependency between orders. Through tree list data, enterprises can better manage the status and execution process of orders and improve the efficiency and accuracy of order processing.

[0199] In project management of large projects, the tree list can well display the relationship between planning, milestones, and tasks, helping project managers to better analyze project progress and task dependencies in order to better allocate resources and adjust plans.

[0200] In supply chain management, the tree list of large supply chain platforms well displays the relationship between suppliers, manufacturers, and distributors, helping enterprises to better analyze each link in the supply chain, optimize logistics and inventory management, and improve the efficiency and stability of the entire supply chain.

[0201] In order to better understand the present invention, the above is described in detail in conjunction with the specific embodiments of the present invention, but it is not intended to limit the present invention. Any simple modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention. Each embodiment in this specification focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referenced to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A tree node creation and query method, comprising obtaining data, and further comprising the following steps: Step 1: Use asynchronous parallel processing technology to perform multi-threaded batch processing on data; Step 2: Store the processed data into the database; Step 3: Call the database to load tree data; Step 4: Use layered loading, data grouping technology, and data asynchronous transmission technology to create and query the tree, including the following sub-steps: Step 41: The front end requests data through layering, lazy loading, and asynchronous transmission methods; Step 42: The front end uses a load balancing polling algorithm of data grouping technology to call a relatively idle server to read the corresponding single-layer data. The step 42 includes calculating each single-layer data using a load balancing polling algorithm. The load balancing polling algorithm includes the following sub-steps: Step 421: Deployment n +1 server, one of which is the algorithm server for deploying the service request number algorithm, and the others n The station is a data query request server, and the name of the data query request server is ( N 1, N 2, N 3, ..., N n ); Step 422: Obtain performance parameters of each data query request server; Step 423: Calculate the maximum data query volume under the optimal performance of a single server, and convert the maximum data query volume under the optimal performance ( N 1, N 2, N 3, ..., N n ) are arranged from large to small according to the minimum value of the maximum data query amount under the optimal performance, and the result is ( U 1, U 2, U 3, ..., U n ),in U The minimum value of the maximum data query volume under the optimal performance of 1 is greater than U The minimum value of the maximum data query volume under the optimal performance of 2, U The minimum value of the maximum data query volume under the optimal performance of 2 is greater than U The minimum value of the maximum data query amount under the optimal performance of 3,…, U n-1 The minimum value of the maximum data query volume under the optimal performance is greater than U n The minimum value of the maximum data query amount under the optimal performance, U n The maximum data query volume range under the optimal performance is b n Ten thousand~ b ' n Ten thousand; Step 424: the algorithm server calculates the service request number k using the service request number algorithm according to the single-layer data volume requested by the browser; The service request number algorithm includes: 1) When the query data volume is ≤ F ten thousand, use the H with the largest number of cores. max The server performs data query and forms a service request k; 2) When the amount of query data is greater than F ten thousand, the data query task is distributed among query servers with different performances, and the server with the best performance is selected. n A method of balancing processing and weighted load factor calculation of query servers is used to query data; Said n The balanced processing of query servers means calculating the query volume of each server according to the query volume factor of each query server as a percentage of the total factor, that is, N 1Data query volume factor Z 1= Num (4-1a) / (( Num (4-1a)+( Num (4-2a)+…+(Num(4-na)); N 2Data query volume factor Z 2=( Num (4-2a) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na)); … N n Data query volume factor Z n =( Num (4-na) / ( Num (4-1a)+( Num (4-2a)+…+( Num (4-na)); in: Num (4-1a) is the number of cores of the first machine; Num (4-2a) is the number of cores of the second machine; ...; Num (4-na) is the n The value of the number of cores on the machine; N 1. Processing F*Z 10,000 pieces of data form a service request k 1; N 2. Processing F*Z 20,000 pieces of data, forming another service request k 2;…; N n deal with F*Z n Ten thousand pieces of data form another service request k n ; Step 43: The client receives the service request number from the server, and concurrently calls the load balancing polling algorithm service, and each server executes a query of the corresponding single-layer data and returns the query result to the browser; Step 44: The front-end browser asynchronously loads the single-layer data; Step 45: Complete the loading of all single-layer data in a loop.

2. The tree node creation and query method according to claim 1, characterized in that: The algorithm server is configured as 4-a core 4-a ， G memory; N1 server 4-1a core 4-1a ， G memory; N2 server configuration is 4-2a core 4-2a ， G memory, ..., N n The server is configured with 4-na cores and 4-na ， G memory, of which 4-na core is the first n Number of cores in a server, 4-na ， G is the n The amount of memory on each server; Maximum memory server N max =Max(4-a ， G, 4-2a ， G, ..., 4-na ， G) Maximum number of cores server H max =Max(4-a core, 4-2a core, …, 4-na core).

3. The tree node creation and query method according to claim 2, characterized in that: The data query request server N n The maximum amount of data to query under optimal performance is t n Ten thousand~ s n Ten thousand data, F for n The minimum value of the maximum data query amount range under the optimal performance among the data query request servers.

4. The tree node creation and query method according to claim 3, characterized in that: The optimal performance of a single server is U 1~ U num-1 The query volume of the last server reaches the minimum value of the optimal query volume range. U num The server handles the remaining queries, i.e. U 1 Server Processing b 10,000 pieces of data form a service request k 1; U 2 Server Processing b 20,000 pieces of data form a service request k 2; … U num-1 Server Processing b num-1 Ten thousand pieces of data form a service request k num-1 ; Final m -( b 1+ b 2+…+ b num-1 ) 10,000 pieces of data U num The server processes and forms another service request k num ,in, m is the total data volume, num The number of query servers required for each query.

Images