Periodic data service refresh method and system for complex data visualization

Abstract

The present disclosure provides a periodic data service refresh method and system for complex data visualization, including: based on the refresh interval of each chart in the data visualization interface, obtaining a list of data services required for the chart, the execution time of each data service and the periodic refresh time interval; based on the execution time and periodic refresh time interval of each data service, performing a refresh scenario judgment on the current data visualization interface to obtain a current refresh scenario judgment result; wherein, the refresh scenarios include a first scenario, a second scenario and a third scenario; based on the obtained scenario judgment result, executing a corresponding parallel strategy, wherein, for the third scenario, by clustering data services according to two dimensions of periodic refresh time interval and data service execution time, conflicting data services are obtained, and the conflicting data services are concurrently executed to achieve periodic data service refresh for complex data visualization.

Description

Technical Field

[0001] The present disclosure belongs to the field of data visualization technology, and in particular relates to a periodic data service refresh method and system for complex data visualization. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Data visualization technology has been widely used in various industries. It mainly displays data through images, charts, animations, etc., and interprets the relationships and trends between data. Common data visualization applications are relatively simple. The general process is: obtain data, process data, and display results. The display results are represented by a small number of charts. Among them, some data visualization applications are more complex. Their complexity is reflected in:

[0004] (1) A display result page contains multiple different charts, and the data, data sources, data refresh intervals, etc. corresponding to different charts may not be the same.

[0005] (2) The party providing data (also known as: data source, data service provider) has certain restrictions on concurrent requests to the party acquiring data (also known as: data service requester, data requester, data user) (for example, no more than 5 concurrent accesses within 1 second).

[0006] In summary, this complex data visualization application needs to meet the following requirements in its data service refresh strategy:

[0007] (1) The refresh of a single data service must basically meet the periodic refresh time limit requirements (e.g., refresh every 10 seconds).

[0008] (2) The number of concurrent access service requests for data services should be kept as small as possible to prevent excessive concurrent access from causing a significant impact on the data provider, or from causing adverse effects on the application itself due to the provider's refusal of service if the number exceeds a certain limit.

[0009] (3) Since the Web system is not a real-time system, the real-time requirements are not strict, and the refresh time timeout within a certain range can be allowed.

[0010] The above requirements can be summarized as "minimizing the number of concurrent data service requests while meeting periodic refresh time limits." Existing research and application results in data visualization generally focus on data presentation or the construction and application of targeted data visualization applications in a specific field, lacking consideration of the above-mentioned issues. Summary of the Invention

[0011] In order to solve the above problems, the present disclosure provides a method and system for periodic data service refresh for complex data visualization. The scheme divides the data service refresh of complex data visualization applications into scenarios, and based on different scenario divisions, combines a clustering analysis method to obtain conflicting data services, and concurrently executes the conflicting data services to achieve periodic data service refresh for complex data visualization. The scheme can effectively meet the needs of minimizing the number of concurrent service requests, effectively preventing the problem of excessive concurrent access numbers causing a significant impact on data providers and the problem of data providers being refused services when exceeding a certain limit, which has adverse effects on their own applications.

[0012] According to a first aspect of an embodiment of the present disclosure, a periodic data service refresh method for complex data visualization is provided, comprising:

[0013] Based on the refresh interval of each chart in the data visualization interface, obtain the list of data services required by the chart, the execution time of each data service, and the periodic refresh interval of each data service;

[0014] Based on the execution time and periodic refresh time interval of each data service, a refresh scenario judgment is performed on the current data visualization interface to obtain a current refresh scenario judgment result; wherein the refresh scenario includes a first scenario, a second scenario, and a third scenario;

[0015] Based on the obtained scenario judgment results, the corresponding parallel strategy is executed. For the third scenario, by clustering the data services according to the two dimensions of periodic refresh time interval and data service execution time, conflicting data services are obtained, and the conflicting data services are executed concurrently to achieve periodic data service refresh for complex data visualization.

[0016] Furthermore, based on the execution time and periodic refresh time interval of each data service, the refresh scenario of the current data visualization interface is judged, specifically:

[0017] If the sum of the execution time of all data services is not greater than the minimum value of the periodic refresh time interval of all data services, it is determined to be the first scenario;

[0018] If the difference between the periodic refresh time interval and the execution time of each service in all data services is less than the minimum execution time of all services, it is determined to be the second scenario;

[0019] Situations other than the first and second scenarios are determined to be the third scenario.

[0020] Furthermore, for the first scenario, all data services can be executed sequentially in any order without concurrent execution; for the second scenario, all data services are executed in parallel.

[0021] Furthermore, the third scenario includes two situations: all data services have the same execution time and some data services have different execution times. For the situation where the data services have the same execution time, all data services are clustered according to the data service refresh time interval; for the situation where the data services have different execution times, all data services are clustered from two data dimensions: periodic refresh time interval and data service execution time.

[0022] Furthermore, the conflicting data services are obtained by clustering the data services according to the two dimensions of periodic refresh time interval and data service execution time, specifically:

[0023] After clustering analysis, data service refresh intervals are short and services with similar execution times are clustered together to form several clusters. Among them, for isolated points that appear in the clustering process, an isolated point set is constructed.

[0024] According to a preset rule, a representative refresh time interval is selected from each cluster, and all clusters are sorted in ascending order according to the representative refresh time interval;

[0025] By comparing the data service refresh time intervals between adjacent clusters, inter-cluster merging is performed and conflicting data services are placed in the same cluster.

[0026] Furthermore, during the clustering process, it is necessary to ensure that the refresh time intervals of each cluster do not overlap.

[0027] Furthermore, the concurrent execution of conflicting data services is specifically as follows:

[0028] The cluster with the largest amount of data among all clusters is defined as the largest cluster, and several queue sets with the largest amount of data in the cluster are constructed;

[0029] For each queue, take a data service from different clusters and add it to the queue until all data services in all clusters are taken out; the elements in each queue come from different clusters;

[0030] Create a new queue and add the data services in the isolated point set to the queue;

[0031] For the obtained queues, data services are executed sequentially within the queues, and data services are executed concurrently between the queues.

[0032] According to a second aspect of an embodiment of the present disclosure, a periodic data service refresh system for complex data visualization is provided, including:

[0033] A data acquisition unit, which is used to acquire a list of data services required by the chart, the execution time of each data service, and the periodic refresh time interval of each data service based on the refresh interval of each chart in the data visualization interface;

[0034] A scene determination unit, configured to determine a refresh scene for the current data visualization interface based on the execution time and periodic refresh time interval of each data service, and obtain a current refresh scene determination result; wherein the refresh scene includes a first scene, a second scene, and a third scene;

[0035] A policy execution unit is used to execute the corresponding parallel strategy based on the obtained scenario judgment results. For the third scenario, data services are clustered according to the two dimensions of periodic refresh time interval and data service execution time to obtain conflicting data services, and the conflicting data services are executed concurrently to achieve periodic data service refresh for complex data visualization.

[0036] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the method for refreshing periodic data services for complex data visualization is implemented.

[0037] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method for periodic data service refresh for complex data visualization is implemented.

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] The present disclosure provides a method and system for periodic data service refresh for complex data visualization. The solution is designed for a data service refresh strategy for complex data visualization applications, focusing on solving the problem of "minimizing the number of concurrent data service requests that meet the periodic refresh time limit requirements". The solution divides the data service refresh of complex data visualization applications into scenarios, and based on different scenario divisions, combines a clustering analysis method to obtain conflicting data services, and concurrently executes the conflicting data services to achieve periodic data service refresh for complex data visualization. The data service refresh strategy described in the present disclosure can effectively meet the minimization of the number of concurrent service requests, effectively preventing the problem of excessive concurrent access numbers causing a significant impact on data providers and the problem of data providers being refused service when exceeding a certain limit, which has adverse effects on their own applications.

[0040] Advantages of additional aspects of the present disclosure will be given in part in the following description and in part will become apparent from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure.

[0042] Figure 1 A schematic diagram of a layered architecture of a complex data visualization application described in an embodiment of the present disclosure;

[0043] Figure 2 This is a flow chart of a periodic data service refresh method for complex data visualization described in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] In the absence of conflict, the embodiments of the present disclosure and the features thereof may be combined with each other.

[0048] Explanation of terms:

[0049] Data visualization: refers to the use of graphical means to clearly and effectively express data, data relationships and the deep information contained in the data.

[0050] Data visualization applications: computer software systems that provide data visualization capabilities, including web pages, websites, mobile apps, etc.

[0051] Front-end and back-end: Generally, the data visualization application interface directly faces users and is called the front-end. The underlying software module or independent software system that provides data for the upper-level data visualization application interface is called the back-end.

[0052] Periodic Data Services: Data services are back-end software modules that provide data to front-end interfaces. These services typically provide data through software interfaces, such as functions, RESTful APIs (also known as RESTful services), and Web Services. Periodic data services refresh at predefined intervals.

[0053] Service Request: A service request is a service user's (caller's) request for a service. A data service request emphasizes that the primary responsibility of the called service is to provide data.

[0054] Refresh time limit: also known as "refresh interval", "refresh time interval", etc., refers to the minimum time interval between two service requests for periodic data services to obtain data, usually in seconds or minutes.

[0055] Example 1:

[0056] The purpose of this embodiment is to provide a periodic data service refresh method for complex data visualization.

[0057] First of all, it should be noted that Figure 1 As shown, complex data visualization applications are generally built using a layered architecture with no less than three layers. The "data visualization interface" displays the final results of the data (graphs, tables, etc.). The "data service provider" is the "data source" and is used to provide raw data to the "data visualization interface". On the one hand, the "data service user" receives the raw data provided by the data source, and on the other hand, processes the obtained data. The processed result data is the data required by the "data visualization interface" and is sent to the "data visualization interface" for use. The "service request scheduler" and the "data service user" both belong to the "middle layer", between the "interface" and the "data". The "service request scheduler" plans and schedules the behavior of the "data service user" requesting the "data source" according to the scheduling method proposed in this solution.

[0058] In a "data visualization interface", generally, one interface includes multiple chart elements. Each chart element requires a different data refresh time interval, and the refresh time interval can also be set without setting it.

[0059] The "data service user" calls the data service provided by the "data service provider" to obtain the original data according to the refresh interval requirements of the "data visualization interface".

[0060] A "data service provider" typically provides more than one data service to a "data service user." If only one data service is provided, service request data must be differentiated using the data service's entry parameters, and different result data provided based on the parameters. In the case of only one data service and service requests differentiated by different entry parameters, this solution treats them as separate data services.

[0061] "Service request scheduling" is the core component of this solution, responsible for the planning and execution scheduling of data service request strategies. It can exist as a separate module or system, or as part of the "data service user".

[0062] like Figure 2 As shown, a periodic data service refresh method for complex data visualization includes:

[0063] Based on the refresh interval of each chart in the data visualization interface, obtain the list of data services required by the chart, the execution time of each data service, and the periodic refresh interval of each data service;

[0064] Based on the execution time and periodic refresh time interval of each data service, a refresh scenario judgment is performed on the current data visualization interface to obtain a current refresh scenario judgment result; wherein the refresh scenario includes a first scenario, a second scenario, and a third scenario;

[0065] Based on the obtained scenario judgment results, the corresponding parallel strategy is executed. For the third scenario, by clustering the data services according to the two dimensions of periodic refresh time interval and data service execution time, conflicting data services are obtained, and the conflicting data services are executed concurrently to achieve periodic data service refresh for complex data visualization.

[0066] Furthermore, based on the execution time and periodic refresh time interval of each data service, the refresh scenario of the current data visualization interface is judged, specifically:

[0067] If the sum of the execution time of all data services is not greater than the minimum value of the periodic refresh time interval of all data services, it is determined to be the first scenario;

[0068] If the difference between the periodic refresh time interval and the execution time of each service in all data services is less than the minimum execution time of all services, it is determined to be the second scenario;

[0069] Situations other than the first and second scenarios are determined to be the third scenario.

[0070] Furthermore, for the first scenario, all data services can be executed sequentially in any order without concurrent execution; for the second scenario, all data services are executed in parallel.

[0071] Furthermore, the third scenario includes two situations: all data services have the same execution time and some data services have different execution times. For the situation where the data services have the same execution time, all data services are clustered according to the data service refresh time interval; for the situation where the data services have different execution times, all data services are clustered from two data dimensions: periodic refresh time interval and data service execution time.

[0072] Furthermore, the conflicting data services are obtained by clustering the data services according to the two dimensions of periodic refresh time interval and data service execution time, specifically:

[0073] After clustering analysis, data service refresh intervals are short and services with similar execution times are clustered together to form several clusters. Among them, for isolated points that appear in the clustering process, an isolated point set is constructed.

[0074] According to a preset rule, a representative refresh time interval is selected from each cluster, and all clusters are sorted in ascending order according to the representative refresh time interval;

[0075] By comparing the data service refresh time intervals between adjacent clusters, inter-cluster merging is performed and conflicting data services are placed in the same cluster.

[0076] Furthermore, during the clustering process, it is necessary to ensure that the refresh time intervals of each cluster do not overlap.

[0077] Furthermore, the concurrent execution of conflicting data services is specifically as follows:

[0078] The cluster with the largest amount of data among all clusters is defined as the largest cluster, and several queue sets with the largest amount of data in the cluster are constructed;

[0079] For each queue, take a data service from different clusters and add it to the queue until all data services in all clusters are taken out; the elements in each queue come from different clusters;

[0080] Create a new queue and add the data services in the isolated point set to the queue;

[0081] For the obtained queues, data services are executed sequentially within the queues, and data services are executed concurrently between the queues.

[0082] Specifically, for ease of understanding, the solution described in this embodiment is described in detail below with reference to specific examples:

[0083] (1) Scheduling strategy generation.

[0084] According to the refresh interval of each chart in the "data visualization interface", the data service refresh interval data of the data source needs to be requested, and the scheduling strategy is generated using the strategy generation algorithm of this method.

[0085] Specifically, the scheduling strategy generation includes the following steps:

[0086] Step 1: Data refresh cycle data collection

[0087] This stage is primarily manual processing. Based on the data required by each chart in the "Data Visualization Interface," a list of data services required by the corresponding "Data Service Provider" is compiled, along with the refresh interval for each data service (typically in seconds, minutes, hours, etc.). This compiled data serves as the input dataset S for the clustering algorithm in the next step.

[0088] Step 2: Cluster the dataset S from the previous step.

[0089] The key problem solved by this method can be defined as the following model.

[0090] Data source S provides m data services, which are represented as a set S: S = {s1, s2, ..., s m}, each service s i The execution time is e i , the periodic refresh time interval is ts i For any data service s i Belongs to S, data service user c needs every interval ts i To request (call) this service, there is obviously: ts i -e i >0, generally in practice ts i will be much larger than e i .

[0091] Let the sum of the execution time of all services be Es, ts min The minimum refresh interval for all service cycles, e min The minimum execution time of all services.

[0092] The following situations exist in actual scenarios.

[0093] Scene 1: E s <=ts min , that is, within the minimum interval, all services can be executed sequentially, then all services can be executed sequentially in any order without concurrent execution.

[0094] Scenario 2: For any service s in Si , there are ts i -e i <e min That is, for any service, the remaining idle time after its execution is completed is not enough to complete any other service. At this time, all services must be executed in parallel, and the number of parallel tasks is the number of services m.

[0095] The third scenario: other situations (i.e. situations other than the first and second scenarios).

[0096] For the third scenario, this method addresses the key issue of how to minimize the number of concurrent tasks while meeting service refresh deadlines. The basic idea is to identify the set of data services that must be requested in parallel. There may be multiple sets of data services that must be requested in parallel. The number of services in the set with the most data services is the number of concurrent request tasks.

[0097] According to whether the service execution time is the same, there are two cases. In one case, assuming that the execution time of all services is e i , i∈[1,m] are the same, set to t e ; In another case, e i In either case, in practice, the service execution time is generally much shorter than the service refresh cycle interval.

[0098] First consider the case where the service execution time is the same. Specifically, any two data services s x 、s y , if the period interval between two services |ts x -ts y |<ε and service execution time t e >=ε (ε is a given smaller number, which can be set according to actual needs), then s x and s y Otherwise, during the sequential execution of periodic tasks, at least one of the requests will inevitably exceed the specified interval due to repeated execution. e <ε, if t e Closer to ε, after many execution cycles (error, delay, etc.) accumulation, or there are other services and s x 、s y Executed in the same request task, even if t e <ε, it is also very likely that a request will exceed the time interval in one (or more) cycles. For such services that may conflict x 、s y , we call it tasks that must be executed concurrently.

[0099] For all service execution times e i , i∈[1,m] are not necessarily the same, we only need to let t e The maximum execution time of all services can simplify the problem to the case where the service time is the same as before.

[0100] This method clusters services based on two dimensions: periodic refresh interval and service execution time, to identify potentially conflicting services. The specific algorithm is as follows:

[0101] Step 1: (1) For the case where the service execution time is the same, all data services in the data set S are clustered according to the service refresh time interval; (2) For the case where the service execution time is different, clustering is performed based on the two data dimensions of refresh time interval and service execution time. The clustering method is not limited (density clustering, k-means, etc. can be used for clustering. For case (1), since it is one-dimensional data, natural breakpoint method (Natural Breaks) can also be used for division). After clustering, services with small service refresh intervals and similar execution times are clustered together to form several clusters, and some isolated points remain. All isolated points that do not belong to a cluster constitute an isolated point set.

[0102] Step 2: For the clusters clustered in step 1, select a representative refresh time interval for each cluster (which can be the minimum, maximum, average or center point value, etc., all clusters can be based on the same standard), sort all clusters from small to large according to the representative refresh time interval, and arrange them into a queue. Let t i max is the maximum execution time of serving cluster i (t e Note that you should adjust the clustering parameters to ensure that the refresh intervals of each cluster do not overlap. That is, for two clusters A and B, the refresh intervals of A are either all greater than or all less than those of B.

[0103] For two adjacent clusters A and B, A is in front (A's refresh interval is smaller) and B is in the back, if the difference between the refresh interval of the largest data in A and the refresh interval of the smallest data in B is less than or equal to n*t max (n is an integer greater than 0, n is an adjustable parameter used to adjust the cluster distance merging parameter, t max is the maximum service execution time of A and B), then A and B are too close and may conflict, so A and B are merged into one cluster.

[0104] Step 3: After processing in Step 2, all cluster centers are far apart, while the data within a cluster is close together. Theoretically, each data service within a cluster cannot be executed as a single task due to its close refresh intervals, and must be executed in parallel. However, tasks from any two different clusters can be placed in a single task and executed sequentially, minimizing the probability of conflict. Let the cluster with the largest amount of data among all clusters be the largest cluster, with the number of data items P, where P is the calculated initial maximum number of concurrent tasks.

[0105] Step 4: According to the maximum number of cluster data P in Step 3, define P queue sets: SP = {q1, q2, .., q P}, each queue q in SP i (i∈[1,P]) is initially empty. For each queue q in SP i (i∈[1, P]), take a data from the clusters clustered in step 3 and add it (in any way, it can be sequential or random), and delete the data from the cluster. If the cluster is empty, no more data will be taken. After step 4, except for the isolated point data set, all data in the same cluster have been added to different queues. Since each data in the queue comes from a different cluster, the distance between the data (refresh interval, execution time) is far, so each queue q i The probability of conflict during the service sequential execution cycle refresh is low and can be executed sequentially; while different queues must be executed concurrently.

[0106] Step 5: Create a new queue q P+1 , add all isolated point data. From the above steps, we can see that all isolated data do not belong to existing clusters and are far away from each other. The possibility of conflict when executing a single task sequentially is low.

[0107] Step 3: Scheduling strategy generation

[0108] The P+1 queues formed in step 2 above are P+1 parallel tasks. The "service request scheduling" generates P+1 concurrent request tasks. Each task corresponds to a queue, and the data in the queue can be served sequentially.

[0109] (2) Strategy execution.

[0110] Executes data service refresh requests according to the scheduling policy.

[0111] (3)Strategy optimization.

[0112] If the data service refresh fails to meet the time interval (that is, individual services exceed the execution interval), the clustering algorithm parameters can be adjusted (adjusted according to actual needs) to re-form a new scheduling strategy and execute it.

[0113] Example 2:

[0114] The purpose of this embodiment is to provide a periodic data service refresh system for complex data visualization.

[0115] A periodic data service refresh system for complex data visualization, including:

[0116] A data acquisition unit, which is used to acquire a list of data services required by the chart, the execution time of each data service, and the periodic refresh time interval of each data service based on the refresh interval of each chart in the data visualization interface;

[0117] A scene determination unit, configured to determine a refresh scene for the current data visualization interface based on the execution time and periodic refresh time interval of each data service, and obtain a current refresh scene determination result; wherein the refresh scene includes a first scene, a second scene, and a third scene;

[0118] A policy execution unit is used to execute the corresponding parallel strategy based on the obtained scenario judgment results. For the third scenario, data services are clustered according to the two dimensions of periodic refresh time interval and data service execution time to obtain conflicting data services, and the conflicting data services are executed concurrently to achieve periodic data service refresh for complex data visualization.

[0119] Furthermore, the system described in this embodiment corresponds to the method described in Example 1, and its technical details are described in detail in Example 1, so they will not be repeated here.

[0120] In further embodiments, there is also provided:

[0121] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor. When the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0122] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0123] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0124] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in embodiment 1 is performed.

[0125] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0126] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0127] The above-mentioned embodiment provides a method and system for periodic data service refresh for complex data visualization, which can be implemented and has broad application prospects.

[0128] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A periodic data service refresh method for complex data visualization, characterized in that: include: Based on the refresh interval of each chart in the data visualization interface, obtain the list of data services required by the chart, the execution time of each data service, and the periodic refresh interval of each data service; Based on the execution time and periodic refresh time interval of each data service, a refresh scenario judgment is performed on the current data visualization interface to obtain a current refresh scenario judgment result; wherein the refresh scenario includes a first scenario, a second scenario, and a third scenario; Based on the obtained scenario judgment results, the corresponding parallel strategy is executed. The third scenario includes two scenarios: all data services have the same execution time and some data services have different execution times. For the scenario with the same execution time, all data services are clustered according to the data service refresh time interval; for the scenario with different execution times, all data services are clustered from the two data dimensions of periodic refresh time interval and data service execution time to obtain conflicting data services. The conflicting data services are then executed concurrently to achieve periodic data service refresh for complex data visualization. The concurrent execution of conflicting data services is specifically as follows: The cluster with the largest amount of data among all clusters is defined as the largest cluster, and several queue sets with the largest amount of data in the cluster are constructed; For each queue, take a data service from different clusters and add it to the queue until all data services in all clusters are taken out; the elements in each queue come from different clusters; Create a new queue and add the data services in the isolated point set to the queue; For the obtained queues, data services are executed sequentially within the queues, and data services are executed concurrently between the queues.

2. A periodic data service refresh method for complex data visualization according to claim 1, characterized in that: The refresh scenario judgment of the current data visualization interface is performed based on the execution time and periodic refresh time interval of each data service, specifically: If the sum of the execution time of all data services is not greater than the minimum value of the periodic refresh time interval of all data services, it is determined to be the first scenario; If the difference between the periodic refresh time interval and the execution time of each service in all data services is less than the minimum execution time of all services, it is determined to be the second scenario; Situations other than the first and second scenarios are determined to be the third scenario.

3. The method for periodic data service refresh for complex data visualization according to claim 1, characterized in that: For the first scenario, all data services can be executed sequentially in any order without concurrent execution; for the second scenario, all data services are executed in parallel.

4. The method for periodic data service refresh for complex data visualization according to claim 1, characterized in that: By clustering data services according to the two dimensions of periodic refresh time interval and data service execution time, the conflicting data services are obtained, specifically: After clustering analysis, data service refresh intervals are short and services with similar execution times are clustered together to form several clusters. Among them, for isolated points that appear in the clustering process, an isolated point set is constructed. According to a preset rule, a representative refresh time interval is selected from each cluster, and all clusters are sorted in ascending order according to the representative refresh time interval; By comparing the data service refresh time intervals between adjacent clusters, inter-cluster merging is performed and conflicting data services are placed in the same cluster.

5. The method for periodic data service refresh for complex data visualization according to claim 1, characterized in that: During the clustering process, it is necessary to ensure that the refresh time intervals of each cluster do not overlap.

6. A periodic data service refresh system for complex data visualization, characterized by: include: A data acquisition unit, which is used to acquire a list of data services required by the chart, the execution time of each data service, and the periodic refresh time interval of each data service based on the refresh interval of each chart in the data visualization interface; A scene determination unit, configured to determine a refresh scene for the current data visualization interface based on the execution time and periodic refresh time interval of each data service, and obtain a current refresh scene determination result; wherein the refresh scene includes a first scene, a second scene, and a third scene; A policy execution unit is configured to execute a corresponding parallel policy based on the obtained scenario judgment result. The third scenario includes two scenarios: all data services have the same execution time and some data services have different execution times. For the scenario where the data services have the same execution time, all data services are clustered according to the data service refresh time interval; for the scenario where the data services have different execution times, all data services are clustered based on the two data dimensions of periodic refresh time interval and data service execution time to obtain conflicting data services. The conflicting data services are then executed concurrently to achieve periodic data service refresh for complex data visualization. The concurrent execution of conflicting data services is specifically as follows: The cluster with the largest amount of data among all clusters is defined as the largest cluster, and several queue sets with the largest amount of data in the cluster are constructed; For each queue, take a data service from different clusters and add it to the queue until all data services in all clusters are taken out; the elements in each queue come from different clusters; Create a new queue and add the data services in the isolated point set to the queue; For the obtained queues, data services are executed sequentially within the queues, and data services are executed concurrently between the queues.

7. An electronic device comprising a memory, a processor, and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the method for periodic data service refresh for complex data visualization according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the program implements a periodic data service refresh method for complex data visualization as described in any one of claims 1 to 5.

Images