A Visualization Big Screen Data Optimization Loading Method Based on a Two-Level Caching Mechanism

Abstract

The present invention proposes a method for optimizing the loading of visual large-screen data based on a two-level cache mechanism, and particularly relates to a method for smooth data visualization in a multi-service concurrent request environment. The present invention is committed to solving the problem of how to load data and render charts on the interface fastest and improve the user experience of the data visualization large screen under the conditions of limited browser thread concurrency, multiple front-end display components, and high back-end data latency. By designing a mechanism of two-level data cache, default data display, and polling request cycle, when the component fails to request service data within a specified time period, it reads the "more" real default data and displays the service data requested with delay on the upper layer at a certain cycle.

Description

Technical Field

[0001] The present invention relates to a data optimization loading method, in particular to a visualization large screen data optimization loading method based on a secondary cache mechanism. Background Art

[0002] A single data visualization large screen interface usually contains dozens or hundreds of display components. Usually, the data between components is isolated, and each component obtains data from the background through an independent Ajax request. However, the browser can support at most 8 Ajax requests online simultaneously. Therefore, the data request congestion of the data large screen interface is serious, and the data request-refresh time of some components is extremely long, resulting in poor user experience. Summary of the Invention

[0003] Object of the Invention: The technical problem to be solved by the present invention is to provide a visualization large screen data optimization loading method based on a secondary cache mechanism in view of the deficiencies of the prior art.

[0004] To solve the above technical problem, the present invention discloses a visualization large screen data optimization loading method based on a secondary cache mechanism, including the following steps:

[0005] Step 1, initialization, reading the data in the second-level cache into the first-level cache;

[0006] The first-level cache has a format of <Key, Value>; where the value Value = {timeStamp, data}, timeStamp represents the timestamp, and data represents the data displayed by the chart component; the key Key is layerId, and layerId represents the chart layer id;

[0007] The second-level cache has a metadata model, that is, a data structure including: layerid, data, and timestamp, specifically as follows:

[0008] The type of layerid is Integer, representing the layer identifier; the type of data is Blob, representing the JSON format character of the layer data, stored in binary format; the type of timestamp is date, representing the calculated timestamp.

[0009] Step 2, for the component data of the visualization large screen, use the method of combined request and combined return to request and process the data service, that is, in the visualization large screen system, the front-end page sends an Ajax request, and the Controller layer of the back-end controller splits the request into multi-threaded tasks according to the number of components in the page for data processing;

[0010] The front end is the page display part, including the page and the components in the page; the back end is the application service, that is, the logic processing layer.

[0011] Step 3, set the memory data space as the first-level cache. After the Controller layer of the back end receives a request, it first searches for data within the time limit of the first-level cache. If found, it reads the cached data of the first-level cache and feeds the data back to the front-end page. If not found, it executes the asynchronous thread executeRequest.

[0012] Step 4, execute the asynchronous thread executeRequest: set the timeout duration, that is, the timeout duration for the asynchronous thread executeRequest to wait for data response. Use the asynchronous thread to call the data service, that is, depending on whether the data service called according to the request times out or not, execute step 4-1 or step 4-2 respectively.

[0013] Step 4-1, when the data service called by the request does not time out, the asynchronous thread returns the data service result to the Controller layer and finally returns it to the page.

[0014] Step 4-2, when the data service called by the request times out, read the second-level cache data and return the data to the Controller layer and finally return it to the page; the second-level cache is the database-level cache, which stores the default data of each layer in the page.

[0015] Step 5, after the data service request in step 3 is successful, store the requested data in the first-level cache and the second-level cache.

[0016] The request process for visualizing the large-screen data includes: the back-end request process and the front-end request process; among them, the front-end request process means that the front-end page requests all component data in the page at one time; the back-end request process means that the back end splits according to different components into different threads for requesting index data to obtain component index data.

[0017] The specific steps of the back-end request process are as follows:

[0018] Step a1, initialize the data request processing thread pool.

[0019] Step a2, split the request received by the Controller layer into multiple threads and put them into the thread pool.

[0020] Step a2-1, each sub-thread, that is, each thread split out, that is, each component has a thread to request its own index data, and this sub-thread requests the data service of the component itself.

[0021] Step a2-2, the sub-thread returns the corresponding result.

[0022] Step a2-3: The child thread synchronizes the response result data to the first-level cache and the second-level cache.

[0023] Step a3: Determine whether all threads have fed back data. According to the judgment result, execute step a3-1 or step a3-2 respectively.

[0024] Step a3-1: If the judgment is yes, then jump to execute step a4.

[0025] Step a3-2: If the judgment is no, then determine whether the request times out, and execute step a3-2-1 or step a3-2-2 respectively according to whether it times out.

[0026] Step a3-2-1: If the judgment is timeout, then obtain the identifier of the component that has not fed back, and jump to execute step a4.

[0027] Step a3-2-2: If the judgment is not timeout, then delay for 0.1 s and jump to execute step a3.

[0028] Step a4: Read the first-level cache and return the read data to the Controller layer of the controller.

[0029] The described front-end request process specifically includes the following steps:

[0030] Step b1: The front-end page requests data from the Controller layer of the back-end for the first time.

[0031] Step b2: Obtain the first data response from the Controller layer of the back-end and load the component data.

[0032] Step b3: Determine whether all component data on the page has been fed back. According to the judgment result, execute step b3-1 or step b3-1 respectively.

[0033] Step b3-1: If yes, then jump to execute step b4.

[0034] Step b3-2: If no, then jump to execute step b3-2-1.

[0035] Step b3-2-1: Delay for the first duration T1.

[0036] Step b3-2-2: The front-end page requests data for the second time.

[0037] Step b3-2-3: Obtain the second data response and load the component data.

[0038] Step b3-2-4: Determine whether all component data on the page have been fed back, and execute Step b3-2-4-1 or Step b3-2-4-2 respectively according to the judgment result;

[0039] Step b3-2-4-1: If so, jump to Step b4;

[0040] Step b3-2-4-2: If not, jump to execute Step b3-2-4-2-1;

[0041] Step b3-2-4-2-1: Delay for the second duration T2;

[0042] Step b3-2-4-2-2: The front-end page requests data for the second time;

[0043] Step b3-2-4-2-3: Obtain the second data response and load the component data;

[0044] Step b4: End.

[0045] The first duration T1 and the second duration T2 are calculated according to the duration prediction model.

[0046] The duration prediction model specifically includes:

[0047] Definition 1: The time series sampling period is n, and the component-level service response time is t com (n), represents the component service response duration of the i-th component in the n-th adoption cycle, and the average component-level service response duration is The maximum component-level service response duration is

[0048] The page-level service response time is t page (n), represents the page service response duration of the k-th page in the n-th sampling cycle; the average page-level service response duration is The maximum page-level service response duration is

[0049] The application-level service response time is t app (n), represents the application service response duration of the j-th application in the n-th sampling cycle; the average application-level service response duration is The maximum application-level service response duration is

[0050] Definition 2: Input vector Observation vector Expressed as a linear expression as Y(n) = β T X(n), that is, the parameter vector to be estimated Estimate the parameter vector β by the least squares method T , the method is as follows:

[0051]

[0052] Among them, is the estimated value of the parameter vector; adopt the recursive least squares method:

[0053]

[0054]

[0055] P(n + 1) = (I - K(n + 1)x T (n + 1))P(n)

[0056] Among them, the intermediate variable P(n) = X T (n)X(n); from the value of, estimate X(n + 1), that is:

[0057]

[0058]

[0059] Definition 3: Define the relaxation index γ = {γ1, γ2, γ3}, and γ1 + γ2 + γ3 = 1; define the relaxation index δ = {δ1, δ2}, and δ1 + δ2 = 1;

[0060] The duration prediction model is expressed as The specific formula is expanded as follows:

[0061] .

[0062] The first duration T1 mentioned above, the specific calculation method is as follows:

[0063] When the front-end data is requested for the first time, calculate T according to the calculation steps of Definition 3, that is, obtain the first duration T1.

[0064] The second duration T2 mentioned above, the specific calculation method is as follows:

[0065] After the timeout of waiting for the duration of T1, calculate T according to the calculation steps of Definition 3, that is, obtain the second duration T2.

[0066] Beneficial effects:

[0067] 1. It solves the problem that under the conditions of limited browser thread concurrency, multiple components are displayed on the front end, and the backend data has high latency, enabling the interface to load data and render charts as quickly as possible, thus enhancing the user experience of the data visualization large screen.

[0068] 2. By designing mechanisms such as secondary data caching, default data display, and polling request cycles, when the component fails to request service data within a specified time period, it reads "more" realistic default data and displays the service data obtained from the delayed request at regular intervals. Brief Description of the Drawings

[0069] The following further specifically describes the present invention in conjunction with the drawings and specific embodiments, and the above and / or other advantages of the present invention will become clearer.

[0070] Figure 1 It is a schematic diagram of the data loading timing relationship based on the secondary caching mechanism in the present invention.

[0071] Figure 2 It is a schematic diagram of the page backend request process in the present invention.

[0072] Figure 3 It is a schematic diagram of the page front-end request process in the present invention. Specific Embodiments

[0073] The present invention is committed to solving the problem of how to load data and render charts as quickly as possible on the interface under the conditions of limited browser thread concurrency, multiple components are displayed on the front end, and the backend data has high latency, so as to enhance the user experience of the data visualization large screen.

[0074] The following are some noun explanations in the present invention, and their scope of use is limited to the present invention:

[0075] 1. Data service: The source service for obtaining data requested by the backend of the visualization large screen, which realizes obtaining display data from the backend system of the visualization large screen to the source. The data service is an RPC remote call service, mainly based on the Restful service, and mainly supports two methods: GET / POST.

[0076] 2. First-level cache: The data cache first accessed by the component, which stores the "freshest" data of all components of the visualization large screen. At the same time, the capacity of the first-level cache is small, not suitable for storing and querying a large amount of data, and even less suitable for data persistence. In the present invention, the first-level cache is the service memory of the visualization large screen.

[0077] 3. Second-level cache: In the present invention, it refers to the requested data persisted by each component, which corresponds one-to-one with the components in the built-in database of the visualization large screen. The second-level cache mainly serves as a "guarantee" for the large-screen components. When the visualization large-screen service is closed, the second-level cache can ensure that at least the most recent requested data of all components is saved. When the visualization large-screen service is restarted next time, the data is loaded again.

[0078] 4. Request cycle: The time interval for making a data service request again in the case of a data service request timeout. In the present invention, the data service can be re-requested at most 2 times, and the interval periods are T1 and T2 respectively.

[0079] The present invention discloses a visualization large-screen data optimized loading method based on a two-level cache mechanism, including the following steps:

[0080] Step 1, initialization, read the data of the second-level cache into the first-level cache;

[0081] The first-level cache has a format of <Key, Value>; where the value Value = {timeStamp, data}, timeStamp represents the time stamp, and data represents the data displayed by the chart component; the key Key is layerId, and layerId represents the chart layer id;

[0082] The second-level cache, the metadata model, that is, the data structure includes: layerid, data, and timestamp, specifically as follows:

[0083] The type of layerid is Integer, representing the layer identifier; the type of data is Blob, representing the JSON format character of the layer data, stored in binary format; the type of timestamp is date, representing the calculated time stamp.

[0084] Step 2, the component data of the visualization large screen uses the method of combined request and combined return to request and process the data service, that is, in the visualization large-screen system, the front-end page sends an Ajax request, and the back-end controller Controller layer splits the request into multi-threaded tasks according to the number of components in the page for data processing;

[0085] The front-end is the page display part, including the page and the components in the page; the back-end is the application service, that is, the logic processing layer.

[0086] Step 3: Set the memory data space as the first-level cache. After the backend Controller layer receives a request, it first looks for data within the time limit of the first-level cache. If found, it reads the cached data from the first-level cache and returns the data to the front-end page. If not found, it executes the asynchronous thread executeRequest.

[0087] Step 4: Execute the asynchronous thread executeRequest: Set the timeout duration, that is, the timeout duration for the asynchronous thread executeRequest to wait for data response. Use the asynchronous thread to call the data service. That is, depending on whether the data service called according to the request times out or not, execute step 4-1 or step 4-2 respectively.

[0088] Step 4-1: When the data service called by the request does not time out, the asynchronous thread returns the data service result to the Controller layer of the controller and finally returns it to the page.

[0089] Step 4-2: When the data service called by the request times out, read the second-level cache data and return the data to the Controller layer of the controller and finally return it to the page. The second-level cache is a database-level cache that stores the default data of each layer in the page.

[0090] Step 5: After the data service request in step 3 is successful, store the requested data in the first-level cache and the second-level cache.

[0091] The request process for visualizing the large-screen data includes: the backend request process and the front-end request process. Among them, the front-end request process is that the front-end page requests all component data in the page at one time. The backend request process is that the backend splits it into different threads for requesting index data according to different components to obtain component index data.

[0092] The specific steps of the backend request process are as follows:

[0093] Step a1: Initialize the data request processing thread pool.

[0094] Step a2: Split the request received by the Controller layer into multiple threads and put them into the thread pool.

[0095] Step a2-1: Each child thread, that is, each thread split out, that is, each component has a thread to request its own index data, and this child thread requests the data service of the component itself.

[0096] Step a2-2: The child thread returns the corresponding result.

[0097] Step a2-3: The child thread synchronizes the response result data to the first-level cache and the second-level cache.

[0098] Step a3: Determine whether all threads have fed back data. According to the judgment result, execute step a3-1 or step a3-2 respectively;

[0099] Step a3-1: If the judgment result is yes, then jump to execute step a4;

[0100] Step a3-2: If the judgment result is no, then determine whether the request times out, and execute step a3-2-1 or step a3-2-2 respectively according to whether it times out;

[0101] Step a3-2-1: If the judgment result is timeout, then obtain the identifier of the component that has not fed back, and jump to execute step a4;

[0102] Step a3-2-2: If the judgment result is not timeout, then delay for 0.1 s and jump to execute step a3;

[0103] Step a4: Read the first-level cache and return the obtained data to the Controller layer of the controller.

[0104] The described front-end request process specifically includes the following steps:

[0105] Step b1: The front-end page requests data from the Controller layer of the back-end for the first time;

[0106] Step b2: Obtain the first data response from the Controller layer of the back-end and load the component data;

[0107] Step b3: Determine whether all component data on the page has been fed back. According to the judgment result, execute step b3-1 or step b3-1 respectively;

[0108] Step b3-1: If so, then jump to execute step b4;

[0109] Step b3-2: If not, then jump to execute step b3-2-1;

[0110] Step b3-2-1: Delay for the first duration T1;

[0111] Step b3-2-2: The front-end page requests data for the second time;

[0112] Step b3-2-3: Obtain the second data response and load the component data;

[0113] Step b3-2-4: Determine whether all component data on the page has been fed back. According to the judgment result, execute step b3-2-4-1 or step b3-2-4-2 respectively;

[0114] Step b3-2-4-1: If so, then jump to step b4;

[0115] Step b3-2-4-2-2, if not, jump to execute Step b3-2-4-2-1;

[0116] Step b3-2-4-2-1, delay for the second duration T2;

[0117] Step b3-2-4-2-2, the front-end page requests data for the second time;

[0118] Step b3-2-4-2-3, obtain the second data response and load the component data;

[0119] Step b4, end.

[0120] The first duration T1 and the second duration T2 are calculated according to the duration prediction model.

[0121] The duration prediction model specifically includes:

[0122] Definition 1: The time series sampling period is n, and the component-level service response time is t com (n), represents the component service response duration of the i-th component in the n-th adoption cycle, and the average component-level service response duration is The maximum component-level service response duration is

[0123] The page-level service response time is represents the page service response duration of the k-th page in the n-th sampling cycle; the average page-level service response duration is The maximum page-level service response duration is

[0124] The application-level service response time is t app (n), represents the application service response duration of the j-th application in the n-th sampling cycle; the average application-level service response duration is The maximum application-level service response duration is

[0125] Definition 2: Input vector Observation vector It is expressed by the linear expression as Y(n) = β T X(n), that is, the parameter vector Estimate the parameter vector β by the least squares method T , the method is as follows:

[0126]

[0127] Among them, is the estimated value of the parameter vector; the recursive least squares method is adopted:

[0128]

[0129]

[0130] P(n + 1) = (I - K(n + 1)X T (n + 1))P(n)

[0131] Among them, the intermediate variable P(n) = X T (n)x(n); from the value of, estimate X(n + 1), that is:

[0132]

[0133]

[0134] Definition 3: Define the relaxation index γ = {γ1, γ2, γ3}, and γ1 + γ2 + γ3 = 1; define the relaxation index δ = {δ1, δ2}, and δ1 + δ2 = 1;

[0135] The duration prediction model is expressed as The specific formula is expanded as follows:

[0136]

[0137] For the described first duration T1, the specific calculation method is as follows:

[0138] When the first request for front - end data is made, calculate T according to the calculation steps of Definition 3, that is, obtain the first duration T1.

[0139] For the described second duration T2, the specific calculation method is as follows:

[0140] After the timeout of waiting for the duration T1, calculate T according to the calculation steps of Definition 3, that is, obtain the second duration T2.

[0141] Embodiment:

[0142] A method for optimizing the loading of visualization large - screen data based on a two - level cache mechanism aims to optimize the data request, processing, and response processes and architectures of the visualization large - screen. Adopt the strategy of data two - level cache, and finally achieve the smooth display of the data of the visualization large - screen components, enhancing the friendliness of the user experience.

[0143] The present invention at least includes the following contents:

[0144] 1. The present invention is dedicated to solving problems such as slow response to data service requests on a visualization large screen and long waiting time for interface data display. The present invention designs a data request method for component secondary caching, as Figure 1 shown, and its basic process and implementation steps are as follows:

[0145] Step 1: Read the data in the second-level cache into the first-level cache during initialization.

[0146] Step 2: For component data, use the method of combined request and combined return. The browser makes an Ajax request and splits it into multi-threaded tasks according to the number of components in the Controller layer of the browser to request and process the data service;

[0147] Step 3: Set the memory data space as the first-level cache. The first-level cache is in the <Key,Value> format, where the value Value = {timeStamp, data}, timeStamp represents the time stamp, data represents the data displayed by the chart component, and the key Key is layerId, where layerId represents the chart layer id. After the backend Controller layer receives the request, it first searches for the data within the validity period of the first-level cache. If found, it reads the cached data in the first-level cache and feeds back the data. If not found, it executes the asynchronous thread executeRequest;

[0148] The frontend mentioned in the text is page display, which is static files such as html and js in the code. The backend is an application service, usually a logic processing layer, and its form is a runnable jar package.

[0149] Step 4: executeRequest is an asynchronous thread, and set the timeout duration (the timeout duration for the asynchronous thread executeRequest to wait for data response); use the asynchronous thread to call the data service;

[0150] Step 4.1: When the data service called by the request does not time out, return the data service result (return to the Controller layer and finally return to the page (browser));

[0151] Step 4.2: When the data service times out, read the data in the second-level cache (the second-level cache is a database-level cache that stores the default data of each layer);

[0152] Step 5: After the data service request is successful in Step 3, insert the requested data into the first-level cache and the second-level cache.

[0153] 2. The second-level cache model of the present invention stores the data request results of components in the database. The metadata models of the second-level cache are shown in Table 1 and Table 2. Among them, Table 1 is the metadata model of the second-level cache, which is used to describe the data structure of the second-level cache. The specific form is as follows:

[0154] The type of Layerid is Integer, representing the layer identifier; the type of data is Blob, representing the JSON format characters of the layer data, stored in binary format; the type of timestamp is date, representing the calculation timestamp;

[0155] Table 1 Second-level cache metadata model table

[0156]

[0157] Table 2 is the information flow relationship table, which is used to represent the information interaction relationship between the first-level and second-level caches. Specifically, it includes the information flow direction and form of the data synchronization between the first-level and second-level caches in two cases: the initialization of the visualization large screen and the data service response.

[0158] Table 2 Information flow relationship

[0159]

[0160] 3. The back-end request process of the present invention (the back-end request process is the process in which the back-end master control service requests the index data of each chart component. In the present invention, the front-end page requests all the chart component data of the page at one time. The back-end splits it into multiple threads for requesting index data of components to obtain the component index data), as Figure 2 shown, the specific steps are as follows:

[0161] Step 1: Initialize the data request processing thread pool;

[0162] Step 2: Split the request received by the Controller into multiple threads and put them into the thread pool;

[0163] Step 2.1: The sub-thread (the sub-thread is each thread split out, that is, each chart component has a thread for requesting its own index data) requests (requests the data from the data providing service of the chart component itself) data;

[0164] Step 2.2: The result of the sub-thread is returned;

[0165] Step 2.3: Synchronize the response result data to the first-level and second-level caches;

[0166] Step 3: Determine whether all threads have fed back data?

[0167] Step 3.1: Yes, jump to Step 4;

[0168] Step 3.2: No. Determine whether the request times out?

[0169] Step 3.2.1: Yes. Obtain the identifier of the un - feedback component and jump to Step 4;

[0170] Step 3.2.2: No. Wait for 0.1 s and jump to Step 3;

[0171] Step 4: Read the first - level cache and return the Controller data.

[0172] 4. The request process of the front - end page (front - end page, the front - end page requests data of all components at one time) of the present invention is as Figure 3 shown, and the specific steps are as follows:

[0173] Step 1: The Web page makes the first request (to the Controller layer of the back - end) for data;

[0174] Step 2: Obtain the first data response (from the Controller layer of the back - end) and load the component data;

[0175] Step 3: Determine whether all component data on the page has been fed back?

[0176] Step 3.1: Yes. Jump to Step 4;

[0177] Step 3.2: No. Jump to Step 3.2.1;

[0178] Step 3.2.1: Delay for T1;

[0179] Step 3.2.2: The Web page makes the second request for data;

[0180] Step 3.2.3: Obtain the second data response and load the component data;

[0181] Step 3.2.4: Determine whether all component data on the page has been fed back?

[0182] Step 3.2.4.1: Yes. Jump to Step 4;

[0183] Step 3.2.4.2: No. Jump to Step 3.2.4.2.1;

[0184] Step 3.2.4.2.1: Delay for T2;

[0185] Step 3.2.4.2.2: The Web page makes the second request for data;

[0186] Step 3.2.4.2.3: Obtain the second data response and load the component data;

[0187] Step 4: End.

[0188] 4. The selection of T1 and T2 in the present invention involves predicting the response duration. Therefore, a response duration prediction model is designed, and the specific algorithm is as follows:

[0189] Definition 1: The time series sampling period is n, and the component-level service response time is t com (n), The average duration is The maximum duration is The page-level service response time is The average duration is The maximum duration is The application-level service response time is t app (n), The average duration is The maximum duration is

[0190] Definition 2: The input vector The observation vector Can be linearly expressed as Y(n) = β T X(n), that is, the parameter vector to be estimated β can be estimated by the least squares method T :

[0191]

[0192] The general least squares method has too high a dimensionality with cycle iteration and a large matrix calculation complexity. Therefore, the recursive least squares method is adopted:

[0193]

[0194]

[0195] P(n + 1) = (I - K(n + 1)X T (n + 1))P(n)

[0196] Where P(n) = X T (n)X(n).

[0197] Therefore, from the value of , we can estimate X(n + 1), that is

[0198] Definition 3: Define the relaxation index γ = {γ1, γ2, γ3}, and γ1 + γ2 + γ3 = 1; define the relaxation index δ = {δ1, δ2}, and δ1 + δ2 = 1. The calculation model of T can be expressed as The specific formula expansion is in the following form:

[0199]

[0200] When the front-end data is requested for the first time, T (i.e., T1) is calculated according to the calculation steps of Definition 3. After the timeout of the waiting duration of T1, T (i.e., T2) is calculated according to the calculation steps of Definition 3.

[0201] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit. Among them, the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, it can run the invention content of a visualization large-screen data optimization loading method based on a secondary cache mechanism and some or all of the steps in each embodiment of the present invention. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), or the like.

[0202] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of a computer program and its corresponding general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a computer program, that is, a software product. The computer program software product can be stored in the storage medium, including several instructions for causing a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of the present invention.

[0203] The present invention provides an idea and method for a visualization large-screen data optimization loading method based on a secondary cache mechanism. There are many methods and ways to specifically implement this technical solution. The above is only the preferred implementation manner of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and retouches can be made, and these improvements and retouches should also be regarded as the protection scope of the present invention. Each component not clearly defined in this embodiment can be implemented by the prior art.

Claims

1. A visualization large screen data optimized loading method based on a secondary cache mechanism, characterized in that, It includes the following steps: Step 1, initialization: Read the data in the second-level cache into the first-level cache. Step 2, for the component data of the visualization dashboard, use the method of merge request and merge return to request and process the data service. That is, in the visualization dashboard system, the front-end page sends an Ajax request, and the Controller layer of the back-end divides the request into multi-threaded tasks according to the number of components in the page for data processing. Step 3, set the memory data space as the first-level cache. After receiving the request, the Controller layer of the back-end first searches for the data within the time limit of the first-level cache. If found, it reads the cached data in the first-level cache and feeds the data back to the front-end page. If not found, it executes the asynchronous thread executeRequest. Step 4, execute the asynchronous thread executeRequest: Set the timeout duration, that is, the timeout duration for the asynchronous thread executeRequest to wait for data response. Use the asynchronous thread to call the data service. That is, according to whether the data service called by the request times out or not, execute step 4-1 or step 4-2 respectively. Step 4-1, when the data service called by the request does not time out, the asynchronous thread returns the data service result to the Controller layer and finally returns it to the page. Step 4-2, when the data service called by the request times out, read the data in the second-level cache and return the data to the Controller layer and finally return it to the page. The second-level cache is a database-level cache that stores the default data of each layer in the page. Step 5, after the data service request in step 3 is successful, store the requested data in the first-level cache and the second-level cache.

2. A method for optimizing the loading of visual big screen data based on a secondary cache mechanism according to claim 1, characterized in that, The first-level cache described in step 1 is in the format of <Key,Value>. Among them, the value Value = {timeStamp, data}, where timeStamp represents the timestamp and data represents the data displayed by the chart component. The key Key is layerId, and layerId represents the chart layer id. For the second-level cache, the metadata model, that is, the data structure, includes: layerid, data, and timestamp, specifically as follows: The type of layerid is Integer, representing the layer identifier; the type of data is Blob, representing the JSON format character of the layer data, stored in binary format; the type of timestamp is date, representing the calculated timestamp.

3. A method for optimizing the loading of visualization large screen data based on a secondary cache mechanism according to claim 2, characterized in that The front-end described in step 2 is the page display part, including the page and the components in the page; the back-end is the application service, that is, the logic processing layer.

4. A method for optimizing the loading of visualization large screen data based on a secondary cache mechanism according to claim 3, characterized in that, The request process for optimizing the loading of the visualization dashboard data includes: the back-end request process and the front-end request process. Among them, the front-end request process is that the front-end page requests all the component data in the page at one time; the back-end request process is that the back-end divides it into different threads for requesting index data of components to obtain the component index data.

5. A visualization large screen data optimized loading method based on a secondary cache mechanism according to claim 4, characterized in that The specific steps of the back-end request process are as follows: Step a1, initialize the data request processing thread pool. Step a2: Split the requests received by the Controller layer into multiple threads and put them into the thread pool. Step a2-1: Each child thread, that is, each thread split out, and each component has a thread that requests its own metric data. This child thread requests the data service of the component itself. Step a2-2: The child thread returns the corresponding result. Step a2-3: The child thread synchronizes the response result data to the first-level cache and the second-level cache. Step a3: Determine whether all threads have fed back data. According to the judgment result, execute Step a3-1 or Step a3-2 respectively. Step a3-1: If the judgment is yes, then jump to execute Step a4. Step a3-2: If the judgment is no, then determine whether the request times out, and execute Step a3-2-1 or Step a3-2-2 respectively according to whether it times out. Step a3-2-1: If the judgment is timeout, then obtain the identifier of the component that has not fed back, and jump to execute Step a4. Step a3-2-2: If the judgment is not timeout, then delay for 0.1 s and jump to execute Step a3. Step a4: Read the first-level cache and return the read data to the Controller layer.

6. A visualization large screen data optimized loading method based on a secondary cache mechanism according to claim 5, characterized in that, The specific front-end request process includes the following steps: Step b1: The front-end page requests data from the Controller layer of the back-end for the first time. Step b2: Obtain the first data response from the Controller layer of the back-end and load the component data. Step b3: Determine whether all component data on the page has been fed back. According to the judgment result, execute Step b3-1 or Step b3-1 respectively. Step b3-1: If yes, then jump to execute Step b4. Step b3-2: If no, then jump to execute Step b3-2-1. Step b3-2-1: Delay for the first duration T1. Step b3-2-2: The front-end page requests data for the second time. Step b3-2-3: Obtain the second data response and load the component data. Step b3-2-4: Determine whether all component data on the page has been fed back. According to the judgment result, execute Step b3-2-4-1 or Step b3-2-4-2 respectively. Step b3-2-4-1: If yes, then jump to Step b4. Step b3-2-4-2: If no, then jump to execute Step b3-2-4-2-1. Step b3-2-4-2-1: Delay for the second duration T2. Step b3-2-4-2-2: The front-end page requests data for the second time. Step b3-2-4-2-3: Obtain the second data response and load the component data. Step b4: End.

7. A method for optimizing the loading of visualized large-screen data based on a secondary cache mechanism according to claim 6, characterized in that, The first duration T1 and the second duration T2 are calculated according to the duration prediction model.

8. A method for optimizing the loading of visualization big screen data based on a secondary cache mechanism according to claim 7, characterized in that, The specific duration prediction model includes: Definition 1: The timing sampling period is n, and the component-level service response time is t com (n), represents the component service response duration of the i-th component in the n-th adoption cycle, and the average component-level service response duration is The maximum component-level service response duration is The page-level service response time is t page (n), indicating the page service response duration of the k-th page in the n-th sampling period; the average page-level service response duration is The maximum page-level service response duration is The application-level service response time is t app (n), denotes the application service response duration of the j-th application in the n-th sampling period; the average application-level service response duration is The maximum application-level service response duration is Definition 2: Input vector Observation vector It is expressed by the linear expression as Y(n) = β T X(n), that is, the parameter vector to be estimated Estimate the parameter vector γ by the least squares method T , and the method is as follows: wherein, is the estimated value of the parameter vector; the recursive least squares method is adopted: P(n + 1) = (I - K(n + 1)X T (n + 1))P(n) Among them, the intermediate variable P(n) = X T (n)X(n); from the value of, estimate X(n + 1), that is: Definition 3: Define the relaxation index γ = {γ1, γ2, γ3}, and γ1 + γ2 + γ3 = 1; define the relaxation index δ = {δ1, δ2}, and δ1 + δ2 = 1. The duration prediction model is expressed as The specific formula expansion is in the following form:

9. A method for optimizing the loading of visualization large screen data based on a secondary cache mechanism according to claim 8, characterized in that, The specific calculation method of the first duration T1 is as follows: When the front-end data is requested for the first time, calculate T according to the calculation steps of Definition 3, that is, obtain the first duration T1.

10. A method for optimizing the loading of visual big screen data based on a secondary cache mechanism according to claim 9, characterized in that, The specific calculation method of the second duration T2 is as follows: After the timeout of the waiting duration T1, calculate T according to the calculation steps of Definition 3, and thus obtain the second duration T2.

Images