Method, device, equipment and storage medium for generating kanban data source

Abstract

The present application relates to the field of artificial intelligence technology, and discloses a method, device, equipment and storage medium for generating a kanban data source, wherein the method includes: obtaining the initial point-of-sale summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server and storing it in the Redis database; obtaining the data update timing task and the point-of-sale statistical configuration data according to the second preset time interval; executing the data update timing task, and performing point-of-sale summary data statistics for each point-of-sale keyword in the point-of-sale statistical configuration data according to the Redis database, obtaining the target point-of-sale summary data corresponding to each point-of-sale keyword, and updating each target point-of-sale summary data to the target kanban data source. It reduces the resource requirements for back-end processing, avoids the use of Oracle database for data aggregation and query, reduces the CPU resources and memory consumption of the business database, and improves the generation efficiency of the data kanban.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and storage medium for generating a kanban data source. Background Art

[0002] Data dashboards can visualize progress information, and thus are increasingly widely used. In a system with a distributed architecture, the application consists of a server cluster, and the business functions are processed in a sharded, multi-threaded manner, and data is stored in multiple tables. The traditional method of using Oracle database (relational database management system) for data aggregation and query results in excessive CPU (central processing unit) resources and memory consumption of the business database. The distributed architecture requires the collection of data from multiple tables for each node, which reduces the efficiency of generating data dashboards. Summary of the invention

[0003] The main purpose of the present application is to provide a method, device, equipment and storage medium for generating a dashboard data source, aiming to solve the technical problem that, when generating a data dashboard for a distributed architecture system, an Oracle database is used for data aggregation and query, resulting in excessive CPU resource and memory consumption of the business database, and the distributed architecture needs to collect data from multiple tables for each node, thereby reducing the efficiency of generating the data dashboard.

[0004] In order to achieve the above-mentioned invention object, the present application proposes a method for generating a kanban data source, the method comprising:

[0005] Obtaining a synchronization request for tracking point summary data sent by the target application server according to a first preset time interval;

[0006] In response to the tracking point summary data synchronization request, an initial tracking point summary data set is obtained, which is sent by the target application server according to the one-way linked list blocking queue of the target application server;

[0007] Storing the initial tracking point summary data set in a Redis database;

[0008] Acquire data to update scheduled tasks and tracking point statistics configuration data according to a second preset time interval;

[0009] Execute the data update timing task, and perform the buried point summary data statistics for each buried point keyword in the buried point statistics configuration data according to the Redis database, obtain the target buried point summary data corresponding to each buried point keyword, and update each target buried point summary data to the target kanban data source.

[0010] Furthermore, the step of executing the data update timing task, performing point-of-sale summary data statistics for each point-of-sale keyword in the point-of-sale statistics configuration data according to the Redis database, and obtaining target point-of-sale summary data corresponding to each point-of-sale keyword includes:

[0011] By executing the data update timing task, the buried point summary data corresponding to each buried point keyword in the buried point statistical configuration data is obtained from the Redis database to obtain the buried point summary data set to be processed;

[0012] By executing the data update timing task, according to the embedding point statistics configuration data, embedding point summary data statistics are performed on each embedding point summary data set to be processed, so as to obtain the target embedding point summary data corresponding to each embedding point keyword.

[0013] Furthermore, after the step of updating the target tracking point summary data to the target dashboard data source, the step further includes:

[0014] Acquire a kanban data acquisition request according to a third preset time interval, wherein the kanban data acquisition request carries kanban configuration data;

[0015] According to the dashboard configuration data, target tracking summary data is obtained from the target dashboard data source as the tracking summary data set to be displayed;

[0016] The summary data set of tracking points to be displayed is sent to a kanban application, wherein the kanban application generates kanban data to be updated according to the summary data set of tracking points to be displayed, and updates a target kanban according to the kanban data to be updated.

[0017] Furthermore, the step of obtaining the tracking point summary data synchronization request sent by the target application server according to the first preset time interval includes:

[0018] Obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein the target application server obtains the business service application startup request;

[0019] Acquire preset queue configuration data according to the business service application startup request;

[0020] According to the preset queue configuration data, a one-way linked list blocking queue is loaded into the Java virtual machine memory;

[0021] The tracking point summary data synchronization request is sent according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory.

[0022] Furthermore, the step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval includes:

[0023] The tracking point summary data synchronization request is obtained according to the first preset time interval, and is sent by the target application server based on the one-way linked list blocking queue and the asynchronous polling thread. The target application server responds to the business service application start request, starts the asynchronous polling thread according to the first preset time interval, executes the asynchronous polling thread, and sends the tracking point summary data synchronization request according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory.

[0024] Furthermore, the step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval also includes:

[0025] Obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein, after the step of obtaining the business service application startup request, the target application server further includes: obtaining a tracking point keyword set corresponding to the application identification set carried by the business service application startup request;

[0026] Declare multiple string variables according to the embedded keyword set, and store each of the string variables in the stack area of ​​the Java virtual machine;

[0027] Initializing the values ​​of each of the string variables in the stack area of ​​the Java virtual machine according to the embedding keyword set;

[0028] Acquire buried point data based on each of the string variables in the stack area of ​​the Java virtual machine;

[0029] Update the one-way linked list blocking queue according to the buried point data.

[0030] Further, the step of obtaining the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server includes:

[0031] Obtaining a Redis server allocation rule according to the tracking point summary data synchronization request;

[0032] Determine the target Redis server identifier according to the Redis server allocation rule and the server identifier carried by the tracking point summary data synchronization request;

[0033] Sending the target Redis server identifier to the target application server;

[0034] According to the target Redis server identifier, an initial buried point summary data set is obtained, which is sent by the target application server according to the one-way linked list blocking queue. The target application server executes an asynchronous polling thread to obtain buried point data of a quantity equal to a preset quantity from the one-way linked list blocking queue as the buried point data set to be counted. According to the buried point data set to be counted, buried point summary data statistics are performed for each buried point keyword to obtain the initial buried point summary data set, and the initial buried point summary data set is sent to the server corresponding to the target Redis server identifier.

[0035] The present application also proposes a device for generating a kanban data source, the device comprising:

[0036] A request acquisition module, used to acquire a synchronization request for tracking point summary data sent by a target application server according to a first preset time interval;

[0037] An initial tracking point summary data set acquisition module is used to respond to the tracking point summary data synchronization request and acquire the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server;

[0038] A Redis database update module, used to store the initial tracking point summary data set in a Redis database;

[0039] A data acquisition module, used to acquire data update timing tasks and tracking point statistics configuration data according to a second preset time interval;

[0040] The target kanban data source update module is used to execute the data update timing task. According to the Redis database, the burying point summary data statistics are performed for each burying point keyword in the burying point statistics configuration data, and the target burying point summary data corresponding to each burying point keyword is obtained, and each target burying point summary data is updated to the target kanban data source.

[0041] The present application also proposes a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0042] The present application also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0043] The method, device, equipment and storage medium for generating a kanban data source of the present application, wherein the method first preliminarily summarizes the tracking point data through each application server and uploads it to the Redis database, then obtains the data from the Redis database and finally summarizes it to update the target kanban data source, and distributes the preliminary summary to the application server, thereby reducing the resource requirements for back-end processing, avoiding the use of Oracle database for data aggregation and query, reducing the CPU resources and memory consumption of the business database, and improving the generation efficiency of the data kanban. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram of a process of generating a kanban data source according to an embodiment of the present application;

[0045] Figure 2 This is a schematic block diagram of the structure of a device for generating a kanban data source according to an embodiment of the present application;

[0046] Figure 3 A schematic block diagram of the structure of a computer device according to an embodiment of the present application.

[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0050] Reference Figure 1 In an embodiment of the present application, a method for generating a kanban data source is provided, the method comprising:

[0051] S1: Obtaining a tracking point summary data synchronization request sent by a target application server according to a first preset time interval;

[0052] S2: responding to the tracking point summary data synchronization request, obtaining the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server;

[0053] S3: storing the initial tracking point summary data set in a Redis database;

[0054] S4: Acquire data to update the scheduled task and the tracking point statistics configuration data according to the second preset time interval;

[0055] S5: Execute the data update timing task, and perform the buried point summary data statistics for each buried point keyword in the buried point statistics configuration data according to the Redis database, obtain the target buried point summary data corresponding to each buried point keyword, and update each target buried point summary data to the target kanban data source.

[0056] In this embodiment, the tracking point data is first preliminarily summarized by each application server and then uploaded to the Redis database. The data is then obtained from the Redis database and finally summarized to update the target dashboard data source. The preliminary summary is distributed to the application server, which reduces the resource requirements for back-end processing, avoids the use of Oracle database for data aggregation and query, reduces the CPU resources and memory consumption of the business database, and improves the generation efficiency of the data dashboard.

[0057] For S1, the first preset time interval is a specific value, for example, the first preset time interval is 5 seconds.

[0058] The target application server is the application server that needs to synchronize the tracking point summary data. The application server is loaded with a business service application, and each business service function of the business service application is set with a tracking point, and each tracking point generates tracking point data according to the tracking point configuration.

[0059] The target application server's tracking point adopts the aspect tracking method, calling an asynchronous thread that does not affect the business service at each business service function of the business service application to synchronously implement the function of assisting in the statistics of business data and performance, thereby achieving the decoupling of the business service function and the generation of tracking point data.

[0060] The tracking point can be configured through the configuration platform of the target application server, and it will take effect once the configuration is completed, without the need to re-release the business service application.

[0061] The triggering methods of tracking points include: event triggering or database interaction triggering.

[0062] The tracking keyword of the tracking point is a field obtained by combining indicators from multiple dimensions of the business. The combined dimensions of the tracking keyword include but are not limited to: "time + project name", "time + personnel", "personnel + list type", "project name + list type", "list type + channel". For example, if the business indicator is "number of lists received on the same day", the tracking keyword is "2021-01-10-pojectid-recive", which is not specifically limited in this example.

[0063] For example, the combination dimension of the embedding keyword is "year-month-day-project name-employee number", and no specific limitation is given in this example.

[0064] For another example, the combination dimension of the embedding keywords is "year-month-day-white list-customer name-pushed to downstream system", which is not specifically limited in this example.

[0065] The tracking point summary data synchronization request is a request to perform a preliminary summary of the tracking point data and send the preliminary summarized tracking point summary data to the Redis server in the Redis cluster.

[0066] Redis (Remote Dictionary Server), also known as remote dictionary service, is an open source log-type, Key-Value database written in ANSI C language, supporting the network, memory-based and persistent, and providing APIs in multiple languages.

[0067] A Redis cluster includes multiple Redis servers.

[0068] The tracking points are recorded in the form of KV. The Key in KV is the tracking keyword, and the Value in KV is the collected tracking point data.

[0069] Optional. The value of Value is a numeric value used to provide counter statistics.

[0070] The tracking point summary data synchronization request carries an application server identifier, which can be data that uniquely identifies an application server, such as a server name or server ID.

[0071] The tracking point summary data is the data obtained by summarizing and counting the tracking point data collected by the target application server in the same period (that is, the collection period, each period corresponding to the first preset time interval is a collection period) for each tracking point keyword, thereby completing the tracking point data aggregation of a single application server from the perspective of the tracking point keyword. In other words, the tracking point keyword in the tracking point summary data sent by the target application server to the Redis cluster each time is unique.

[0072] Among them, the target application server sends a synchronization request for the tracking point summary data to the Redis cluster at a first preset time interval.

[0073] For S2, the target Redis server identifier is determined according to the tracking point summary data synchronization request, and the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server is obtained through the target Redis server identifier.

[0074] One-way linked list blocking queue, that is, a blocking queue based on a linked list.

[0075] The one-way linked list blocking queue includes multiple node nodes. The one-way linked list blocking queue forms a linked list through multiple node nodes, and takes data from the head and inserts data from the tail.

[0076] The node stores the value of the node object, which is composed of Key (that is, the embedding keyword) and Value (the embedding data corresponding to the embedding keyword).

[0077] Node is the endpoint of a network connection, or the connection point between two (or more) lines.

[0078] The initial burial point summary data set includes one or more initial burial point summary data.

[0079] For S3, the initial tracking point summary data set is stored in the database physical table of the Redis database, forming a quasi-real-time statistical data snapshot of the target application server within a certain period of time, which provides a basis for the subsequent comprehensive aggregation of the tracking point summary data to generate a kanban data source.

[0080] It can be understood that a Redis cluster includes one or more Redis databases.

[0081] For S4, the second preset time interval is a specific value, for example, the second preset time interval is 10 seconds.

[0082] A second preset time interval is used to obtain a data update timing task and a tracking point statistics configuration data from a database, and the data update timing task is started.

[0083] The tracking point statistics configuration data includes tracking point keywords and statistical rules. The statistical rules include but are not limited to: summing, counting, taking the maximum value, taking the minimum value, and calculating the average value.

[0084] For S5, by executing the data update timing task, the initial burying point summary data corresponding to each burying point keyword is obtained from the Redis database, and then the initial burying point summary data corresponding to each burying point keyword is statistically analyzed according to the statistical rules in the burying point statistical configuration data, and the statistical data is used as the target burying point summary data corresponding to the burying point keyword.

[0085] Among them, the data update timing task connects to the Redis database through Jdbc (Java Database Connection) to obtain each initial burying point summary data corresponding to each burying point keyword from the Redis database.

[0086] Among them, by executing the data update timing task, each target tracking point summary data in the target tracking point summary data is inserted into the target kanban data source of the relational database.

[0087] The relational database uses an Oracle database, which only needs to provide a query service of a target kanban data source to the kanban application, and does not need to aggregate data, thereby reducing the CPU resources and memory consumption of the Oracle database.

[0088] In one embodiment, the above-mentioned step of executing the data update timing task, performing point-of-sale summary data statistics for each point-of-sale keyword in the point-of-sale statistics configuration data according to the Redis database, and obtaining target point-of-sale summary data corresponding to each point-of-sale keyword includes:

[0089] S51: acquiring the tracking point summary data corresponding to each tracking point keyword in the tracking point statistical configuration data from the Redis database by executing the data update timing task, and obtaining the tracking point summary data set to be processed;

[0090] S52: By executing the data update timing task, according to the embedding point statistics configuration data, the embedding point summary data statistics are performed on each embedding point summary data set to be processed, so as to obtain the target embedding point summary data corresponding to each embedding point keyword.

[0091] This embodiment further summarizes the point summary data in the Redis database by adopting a data update timing task, thereby obtaining a quasi-real-time statistical data snapshot of each application server within a certain period of time, providing a basis for the target dashboard data source to provide simple queries to the dashboard application, and improving the generation efficiency of the data dashboard.

[0092] For S51, by executing the data update timing task, the buried point summary data corresponding to each buried point keyword in the buried point statistical configuration data in the most recent time period is obtained from the Redis database, and each buried point summary data (that is, the initial buried point summary data) obtained is used as the buried point summary data set to be processed. In other words, the buried point summary data set to be processed corresponds to the initial buried point summary data of each application server for the same buried point keyword in the same time period.

[0093] For S52, by executing the data update timing task, according to the embedding point statistics configuration data, embedding point summary data statistics are performed on each embedding point summary data set to be processed, and the data obtained from the statistics of each embedding point summary data set to be processed is used as a target embedding point summary data.

[0094] The target embedding point summary data is summary data for the same embedding point keyword based on the embedding point data of each application server in the same period of time.

[0095] In one embodiment, after the above step of updating the target tracking point summary data to the target dashboard data source, the following step is further included:

[0096] S71: acquiring a kanban data acquisition request according to a third preset time interval, wherein the kanban data acquisition request carries kanban configuration data;

[0097] S72: According to the dashboard configuration data, target tracking summary data is obtained from the target dashboard data source as the tracking summary data set to be displayed;

[0098] S73: Sending the summary data set of buried points to be displayed to the kanban application, wherein the kanban application generates kanban data to be updated according to the summary data set of buried points to be displayed, and updates the target kanban according to the kanban data to be updated.

[0099] This embodiment realizes the rapid acquisition of data from the target kanban data source to update the target kanban. The target kanban data source only needs to provide the kanban application with query services of the target kanban data source and does not need to aggregate data. This avoids the use of Oracle database for data aggregation and query, thereby improving the efficiency of data kanban generation.

[0100] For S71, the kanban data acquisition request sent by the kanban application may be acquired according to the third preset time interval, or the kanban data acquisition request may be triggered by the program implementing the present application according to the third preset time interval.

[0101] The kanban data acquisition request is a request to acquire data from the target kanban data source to update the target kanban.

[0102] The dashboard configuration data includes: indicator identification and indicator algorithm rules. The indicator identification can be data that uniquely identifies an indicator, such as indicator name, indicator ID, etc. The indicator algorithm rules include the algorithm and the algorithm value range.

[0103] For S72, according to the kanban configuration data, a SQL (Structured Query Language) query statement is used to obtain target tracking summary data from the target kanban data source, and each target tracking summary data obtained is used as the tracking summary data set to be displayed.

[0104] For S73, the summary data set of buried points to be displayed is sent to the kanban application, so that the kanban application calculates the indicator value for each preset indicator algorithm rule according to the summary data set of buried points to be displayed, and uses the calculated indicator values ​​as the kanban data to be updated.

[0105] The kanban application updates the target kanban data according to the kanban data to be updated, thereby obtaining an updated kanban.

[0106] In one embodiment, the step of obtaining the tracking point summary data synchronization request sent by the target application server according to the first preset time interval includes:

[0107] S11: acquiring the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein the target application server acquires the business service application startup request;

[0108] S12: Acquire preset queue configuration data according to the business service application startup request;

[0109] S13: Loading a one-way linked list blocking queue into the Java virtual machine memory according to the preset queue configuration data;

[0110] S14: Send the tracking point summary data synchronization request according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory.

[0111] This embodiment realizes that when the business service application is started, the one-way linked list blocking queue is loaded into the Java virtual machine memory through the target application server, providing a basis for sending a tracking point summary data synchronization request and sending an initial tracking point summary data set through the target application server.

[0112] For S11, the target application server may obtain a business service application start request input by a user, or may obtain a business service application start request sent by a third-party application system.

[0113] The business service application start request is a request to start a business service application.

[0114] The business service application startup request carries an application identification set. The application identification set includes one or more application identifications. The application identification can be data that uniquely identifies a business service application, such as an application name and an application ID.

[0115] For S12, the target application server responds to the business service application start request, and the preset queue configuration data can be obtained from the storage space of the target application server, or the preset queue configuration data can be obtained from a third-party application system.

[0116] The preset queue configuration data includes: queue length. The queue length is a specific value expressed as Integer.MAX_VALUE.

[0117] For S13, the one-way linked list blocking queue is loaded in the Java (object-oriented programming language) virtual machine memory, and the length of the one-way linked list blocking queue is the same as the queue length in the preset queue configuration data, thereby initializing reserved space in the memory to store the buried point data.

[0118] For S14, through the target application server, at each time point corresponding to the first preset time interval, the tracking point summary data synchronization request is generated according to the link address of the one-way linked list blocking queue, and the tracking point summary data synchronization request is sent to the Redis cluster.

[0119] In one embodiment, the step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval includes:

[0120] S111: Obtain the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue and the asynchronous polling thread according to the first preset time interval, wherein the target application server responds to the business service application start request, starts the asynchronous polling thread according to the first preset time interval, executes the asynchronous polling thread, and sends the tracking point summary data synchronization request according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory.

[0121] This embodiment sends the tracking point summary data synchronization request through an asynchronous polling thread, thereby reducing the computing resources of the Redis cluster, and the asynchronous polling thread can be decoupled from the business service application, avoiding the impact of the synchronization of the tracking point summary data on the business service application providing business services.

[0122] For S111, the asynchronous polling thread is a scheduled task that runs independently and is not associated with other processes. The asynchronous polling thread is a scheduled task that is executed at the first preset time interval.

[0123] For example, an asynchronous polling thread is automatically executed every 5 seconds.

[0124] Among them, through the target application server, the asynchronous polling thread is executed at each time point corresponding to the first preset time interval, and the tracking point summary data synchronization request is generated and sent to the Redis cluster according to the link address of the one-way linked list blocking queue.

[0125] In one embodiment, the step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval further includes:

[0126] S111: acquiring the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein, after the step of acquiring the business service application startup request, the target application server further includes: acquiring a tracking point keyword set corresponding to the application identification set carried by the business service application startup request;

[0127] S112: Declare multiple string variables according to the embedding keyword set, and store each of the string variables in a stack area of ​​a Java virtual machine;

[0128] S113: Initializing the values ​​of each of the string variables in the stack area of ​​the Java virtual machine according to the embedding keyword set;

[0129] S114: Acquire embedded point data based on each of the string variables in the stack area of ​​the Java virtual machine;

[0130] S115: Update the one-way linked list blocking queue according to the buried point data.

[0131] After the step of obtaining the business service application startup request, this embodiment first declares and initializes a string variable in the stack area of ​​the Java virtual machine of the target application server according to the tracking keyword set corresponding to the application identification set carried by the business service application startup request, obtains tracking data based on each initialized string variable, and updates the one-way linked list blocking queue with the obtained tracking data, thereby realizing the target application server automatically collecting the tracking data and storing it in the one-way linked list blocking queue, which provides a basis for subsequent preliminary tracking summary data statistics based on the one-way linked list blocking queue.

[0132] For S111, the application identification set and the embedding keyword library carried by the business service application startup request are obtained through the target application server, and the embedding keyword is searched in the embedding keyword library for each application identification in the application identification set carried by the business service application startup request to obtain the embedding keyword set.

[0133] The tracking keyword library includes: application identifier and tracking keyword.

[0134] For S112, a string variable is declared for each burying keyword in the burying keyword set through the target application server, and each string variable is stored in the stack area of ​​the Java virtual machine of the target application server, thereby facilitating the rapid calling of the burying keyword for burying data collection.

[0135] For S113, through the target application server, the embedding keyword set is used to assign values ​​to each of the string variables in the stack area of ​​the Java virtual machine, thereby storing the embedding keyword set as the value of each of the string variables in the stack area of ​​the Java virtual machine, so that the value of the string variable in the stack area of ​​the Java virtual machine can be retrieved by calling the string name, which is conducive to quickly calling the embedding keyword for embedding data collection.

[0136] For S114, through the target application server, for each of the string variables in the stack area of ​​the Java virtual machine, the embedded point data is obtained from each business service application in the target application server.

[0137] For S115, through the target application server, the embedded point data and the embedded point keyword corresponding to the embedded point data are used as associated data, and the associated data is updated to the node at the tail of the one-way linked list blocking queue. In other words, the embedded point keyword corresponding to the embedded point data is updated to the Key of the node, and the embedded point data is updated to the Value of the node.

[0138] In one embodiment, the step of obtaining the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server includes:

[0139] S21: Obtaining a Redis server allocation rule according to the tracking point summary data synchronization request;

[0140] S22: Determine the target Redis server identifier according to the Redis server allocation rule and the server identifier carried in the tracking point summary data synchronization request;

[0141] S23: Sending the target Redis server identifier to the target application server;

[0142] S24: According to the target Redis server identifier, an initial buried point summary data set is obtained, which is sent by the target application server according to the one-way linked list blocking queue, wherein the target application server executes an asynchronous polling thread, obtains a number of buried point data equal to a preset number from the one-way linked list blocking queue as the buried point data set to be counted, and according to the buried point data set to be counted, buried point summary data statistics are performed for each buried point keyword to obtain the initial buried point summary data set, and the initial buried point summary data set is sent to the server corresponding to the target Redis server identifier.

[0143] This embodiment first determines the target Redis server identifier according to the Redis server allocation rule, then executes an asynchronous polling thread through the target application server, performs point-by-point summary data statistics on the data in the one-way linked list blocking queue, sends the data obtained by the point-by-point summary data statistics, and distributes the preliminary summary to the application server, thereby reducing the resource requirements for back-end processing and avoiding the use of Oracle database for data aggregation and query.

[0144] For S21, when the Redis cluster receives the tracking point summary data synchronization request, it obtains the Redis server allocation rule from the database.

[0145] Redis server allocation rules include but are not limited to: load balancing rules and fixed Redis server rules. Load balancing rules mean that the tasks processed by each Redis server are evenly distributed. Fixed Redis server rules mean that each application server uses the same Redis server.

[0146] For S22, when the Redis server allocation rule is a balanced load rule, the balanced load rule is adopted to determine the target Redis server identifier according to the acquired task amount of each Redis server in the Redis cluster; when the Redis server allocation rule is a fixed Redis server rule, the server identifier corresponding to the target application server is searched for the Redis server identifier in the Redis server allocation rule, and the Redis server corresponding to the found Redis server identifier is used as the target Redis server identifier. The data synchronization start request is generated according to the target Redis server identifier.

[0147] The target application server parses the request to start data synchronization and obtains a target Redis server identifier.

[0148] The target Redis server identifier is also the server identifier of the Redis server that receives the data synchronized by the target application server.

[0149] The server identifier can be data that uniquely identifies a server, such as a server name or server ID.

[0150] For S23, the target Redis server identifier is encapsulated into the response value corresponding to the tracking point summary data synchronization request, and the response value is sent to the target application server.

[0151] For S24, the target application server parses the target Redis server identifier from the response value corresponding to the tracking point summary data synchronization request.

[0152] The target application server executes an asynchronous polling thread at the first preset time interval, obtains the same number of buried point data as the preset number from the one-way linked list blocking queue of the target application server, and uses each of the obtained buried point data as the buried point data set to be counted.

[0153] It can be understood that when the number of buried point data in the one-way linked list blocking queue is less than a preset number, all buried point data in the one-way linked list blocking queue are taken out as the buried point data set to be counted.

[0154] Through the target application server, the asynchronous polling thread is executed at each time point corresponding to the first preset time interval, and the data flow method of List.stream().collect() is adopted to batch collect the buried point data with the same buried point keyword from the buried point data set to be counted, so as to obtain multiple buried point data sets to be calculated; the value of each value in each buried point data set to be calculated is summarized and counted to obtain the initial buried point summary data corresponding to each buried point data set to be calculated, and each of the initial buried point summary data obtained is used as the initial buried point summary data set.

[0155] List.stream().collect() is a method developed in Java language to classify and summarize data streams.

[0156] The asynchronous polling thread is executed at each time point corresponding to the first preset time interval through the target application server, and the initial buried point summary data set is sent to the server corresponding to the target Redis server identifier.

[0157] It can be understood that after receiving the initial tracking point summary data set, the server corresponding to the target Redis server identifier stores the received initial tracking point summary data set in the Redis database.

[0158] Reference Figure 2 The present application also proposes a device for generating a kanban data source, the device comprising:

[0159] The request acquisition module 100 is used to acquire the tracking point summary data synchronization request sent by the target application server according to the first preset time interval;

[0160] The initial tracking point summary data set acquisition module 200 is used to respond to the tracking point summary data synchronization request and acquire the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server;

[0161] A Redis database update module 300 is used to store the initial tracking point summary data set in a Redis database;

[0162] The data acquisition module 400 is used to acquire data update timing tasks and tracking point statistics configuration data according to a second preset time interval;

[0163] The target kanban data source update module 500 is used to execute the data update timing task, and according to the Redis database, perform the burying point summary data statistics for each burying point keyword in the burying point statistics configuration data, obtain the target burying point summary data corresponding to each burying point keyword, and update each target burying point summary data to the target kanban data source.

[0164] In this embodiment, the tracking point data is first preliminarily summarized by each application server and then uploaded to the Redis database. The data is then obtained from the Redis database and finally summarized to update the target dashboard data source. The preliminary summary is distributed to the application server, which reduces the resource requirements for back-end processing, avoids the use of Oracle database for data aggregation and query, reduces the CPU resources and memory consumption of the business database, and improves the generation efficiency of the data dashboard.

[0165] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a method for generating a kanban data source. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for generating a kanban data source is implemented. The method for generating a kanban data source includes: obtaining a tracking point summary data synchronization request sent by a target application server according to a first preset time interval; responding to the tracking point summary data synchronization request, obtaining an initial tracking point summary data set sent by the target application server according to a one-way linked list blocking queue of the target application server; storing the initial tracking point summary data set in a Redis database; obtaining a data update timing task and tracking point statistics configuration data according to a second preset time interval; executing the data update timing task, and performing tracking point summary data statistics for each tracking point keyword in the tracking point statistics configuration data according to the Redis database, obtaining target tracking point summary data corresponding to each tracking point keyword, and updating each target tracking point summary data to the target kanban data source.

[0166] In this embodiment, the tracking point data is first preliminarily summarized by each application server and then uploaded to the Redis database. The data is then obtained from the Redis database and finally summarized to update the target dashboard data source. The preliminary summary is distributed to the application server, which reduces the resource requirements for back-end processing, avoids the use of Oracle database for data aggregation and query, reduces the CPU resources and memory consumption of the business database, and improves the generation efficiency of the data dashboard.

[0167] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for generating a kanban data source is implemented, including the following steps: obtaining a tracking point summary data synchronization request sent by a target application server according to a first preset time interval; responding to the tracking point summary data synchronization request, obtaining an initial tracking point summary data set sent by the target application server according to a one-way linked list blocking queue of the target application server; storing the initial tracking point summary data set in a Redis database; obtaining a data update timing task and tracking point statistics configuration data according to a second preset time interval; executing the data update timing task, and performing tracking point summary data statistics for each tracking point keyword in the tracking point statistics configuration data according to the Redis database, obtaining target tracking point summary data corresponding to each tracking point keyword, and updating each of the target tracking point summary data to the target kanban data source.

[0168] The method for generating the kanban data source executed above first summarizes the tracking point data through each application server and uploads it to the Redis database, then obtains the data from the Redis database and finally summarizes it to update the target kanban data source, and distributes the preliminary summary to the application server, which reduces the resource requirements of the back-end processing, avoids the use of Oracle database for data aggregation and query, reduces the CPU resources and memory consumption of the business database, and improves the generation efficiency of the data kanban.

[0169] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in this application and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0170] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.

[0171] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for generating a kanban data source, characterized in that: The method comprises: Obtaining a tracking point summary data synchronization request sent by the target application server according to a first preset time interval; In response to the tracking point summary data synchronization request, an initial tracking point summary data set is obtained, which is sent by the target application server according to the one-way linked list blocking queue of the target application server; Storing the initial tracking point summary data set in a Redis database; Acquire data to update scheduled tasks and tracking point statistics configuration data according to a second preset time interval; Execute the data update timing task, perform point-of-sale summary data statistics for each point-of-sale keyword in the point-of-sale statistics configuration data according to the Redis database, obtain the target point-of-sale summary data corresponding to each point-of-sale keyword, and update each target point-of-sale summary data to the target dashboard data source; The step of executing the data update timing task, performing point-of-sale summary data statistics for each point-of-sale keyword in the point-of-sale statistics configuration data according to the Redis database, and obtaining target point-of-sale summary data corresponding to each point-of-sale keyword includes: By executing the data update timing task, the buried point summary data corresponding to each buried point keyword in the buried point statistical configuration data is obtained from the Redis database to obtain the buried point summary data set to be processed, wherein the buried point summary data set to be processed corresponds to the initial buried point summary data of each application server for the same buried point keyword in the same period of time; By executing the data update timing task, according to the embedding point statistics configuration data, embedding point summary data statistics are performed on each embedding point summary data set to be processed, so as to obtain the target embedding point summary data corresponding to each embedding point keyword; The step of obtaining the tracking point summary data synchronization request sent by the target application server according to the first preset time interval includes: Obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein the target application server obtains the business service application startup request; Acquire preset queue configuration data according to the business service application startup request, wherein the preset queue configuration data includes a queue length; According to the preset queue configuration data, a one-way linked list blocking queue is loaded into the Java virtual machine memory, wherein the length of the one-way linked list blocking queue is the same as the queue length in the preset queue configuration data; The tracking point summary data synchronization request is sent according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory, wherein, through the target application server, at each time point corresponding to the first preset time interval, the tracking point summary data synchronization request is generated according to the link address of the one-way linked list blocking queue, and the tracking point summary data synchronization request is sent to the Redis cluster.

2. The method for generating a kanban data source according to claim 1, characterized in that: After the step of updating the target tracking point summary data to the target dashboard data source, the method further includes: Acquire a kanban data acquisition request according to a third preset time interval, wherein the kanban data acquisition request carries kanban configuration data; According to the dashboard configuration data, target tracking summary data is obtained from the target dashboard data source as the tracking summary data set to be displayed; The summary data set of tracking points to be displayed is sent to a kanban application, wherein the kanban application generates kanban data to be updated according to the summary data set of tracking points to be displayed, and updates a target kanban according to the kanban data to be updated.

3. The method for generating a kanban data source according to claim 1, characterized in that: The step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval includes: The tracking point summary data synchronization request is obtained according to the first preset time interval, and is sent by the target application server based on the one-way linked list blocking queue and the asynchronous polling thread. The target application server responds to the business service application start request, starts the asynchronous polling thread according to the first preset time interval, executes the asynchronous polling thread, and sends the tracking point summary data synchronization request according to the first preset time interval and the one-way linked list blocking queue in the Java virtual machine memory.

4. The method for generating a kanban data source according to claim 1, characterized in that: The step of obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval also includes: Obtaining the tracking point summary data synchronization request sent by the target application server based on the one-way linked list blocking queue according to the first preset time interval, wherein, after the step of obtaining the business service application startup request, the target application server further includes: obtaining a tracking point keyword set corresponding to the application identification set carried by the business service application startup request; Declare multiple string variables according to the embedded keyword set, and store each of the string variables in the stack area of ​​the Java virtual machine; Initializing the values ​​of each of the string variables in the stack area of ​​the Java virtual machine according to the embedding keyword set; Acquire buried point data based on each of the string variables in the stack area of ​​the Java virtual machine; Update the one-way linked list blocking queue according to the buried point data.

5. The method for generating a kanban data source according to claim 1, characterized in that: The step of obtaining the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server includes: Obtaining a Redis server allocation rule according to the tracking point summary data synchronization request; Determine the target Redis server identifier according to the Redis server allocation rule and the server identifier carried by the tracking point summary data synchronization request; Sending the target Redis server identifier to the target application server; According to the target Redis server identifier, an initial buried point summary data set is obtained, which is sent by the target application server according to the one-way linked list blocking queue. The target application server executes an asynchronous polling thread to obtain buried point data of a quantity equal to a preset quantity from the one-way linked list blocking queue as the buried point data set to be counted. According to the buried point data set to be counted, buried point summary data statistics are performed for each buried point keyword to obtain the initial buried point summary data set, and the initial buried point summary data set is sent to the server corresponding to the target Redis server identifier.

6. A device for generating a kanban data source, used to implement the method according to any one of claims 1 to 5, characterized in that: The device comprises: A request acquisition module, used to acquire a synchronization request for tracking point summary data sent by a target application server according to a first preset time interval; An initial tracking point summary data set acquisition module is used to respond to the tracking point summary data synchronization request and acquire the initial tracking point summary data set sent by the target application server according to the one-way linked list blocking queue of the target application server; A Redis database update module, used to store the initial tracking point summary data set in a Redis database; A data acquisition module, used to acquire data update timing tasks and tracking point statistics configuration data according to a second preset time interval; The target kanban data source update module is used to execute the data update timing task. According to the Redis database, the burying point summary data statistics are performed for each burying point keyword in the burying point statistics configuration data, and the target burying point summary data corresponding to each burying point keyword is obtained, and each target burying point summary data is updated to the target kanban data source.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images