Data indicator comprehensive management and visual evaluation method and system

Abstract

The present invention proposes a method and system for comprehensive management and visualization evaluation of data indicators applied to a visualization terminal. The method includes a data acquisition step, a data decomposition step, a data collection step, a data association step, and a data visualization step. The visualization terminal includes a first CPU and a second CPU; the data collection step corresponds to a first stack process and a second stack process; the first stack process runs on the first CPU, and the second stack process runs on the second CPU; the first stack process and the second stack process are connected by a data pipeline. The system includes a data acquisition subsystem, a data decomposition subsystem, a data collection subsystem, a data association subsystem, and a data visualization subsystem. The present invention realizes the decomposition and combination of time series data streams and the rapid visualization operation of the invariant part and the associated part after association, reducing the data processing volume and data processing cost.

Description

Technical Field

[0001] The present invention belongs to the field of data indicator processing and visualization technology, and in particular relates to a data indicator comprehensive management and visualization evaluation system, a data indicator comprehensive management and visualization evaluation method, a computer program instruction medium and an image processing terminal for implementing the method. Background Art

[0002] Data visualization can, to a certain extent, better convey information and monitor data, thereby promoting faster discovery and resolution of problems. The selection of appropriate data indicators can help to better build a data visualization system and avoid potential risks.

[0003] In product and operation work, we come into contact with different data and different indicators. Many times, the data we work on is done at the level of a single point, and the data that is finally displayed is often scattered and cannot be connected in series to discover global problems. The systematization and visualization of indicators connect scattered data in series, allowing you to see the overall situation through a single point, and solve single-point problems through the overall situation, so as to discover the value level and classification level of the data, so that the data gradually exerts its value, from a series of unrelated digital measurements to a certain business value, which can play a role in monitoring, early warning, and control. The digital measurements with business value referred to here are data indicators.

[0004] Chinese invention patent application CN201910298385.7 proposes a report generation method including: configuring data filling logic for filling in data in a unit's budget report and final account report; determining whether the data filled in the report satisfies the logic review formula, if so, saving the filled data to a preset database, if not, outputting a prompt message to prompt the filler to modify the filled data; receiving a report export request, and obtaining a report template corresponding to the ID of the target report according to the ID of the target report; obtaining the target report data from the preset database according to the data indicators; generating the target report based on the template framework and the target report data, which can reduce tedious report statistics work, improve report generation efficiency, and liberate manpower.

[0005] Chinese invention patent application CN202010106706.1 proposes a visualization method for financial big data analysis, including: S01, metadata collection and management; S02, data quality management; S03, data standardization; S04, data warehouse management; S05, data visualization; S06, data analysis; The present invention aggregates massive data to realize the mining of weak data correlation, generate more data value, better perform relevant computational analysis on the collected data, and upgrade algorithms and models, and through data governance, enable multi-dimensional analysis and dynamic analysis of the system and then visualize it.

[0006] However, for large-scale continuous spatiotemporal changing data, existing technologies are based on the data generation time sequence itself, and perform indicator visualization analysis according to a preset unchanging scale or sequence, which brings huge data pressure to the processing system itself, and the processing time is long, and the response to the subsequent visualization model output effect is poor. Summary of the invention

[0007] In order to solve the above technical problems, the present invention proposes a data indicator comprehensive management and visual evaluation system and method, a computer program instruction medium and an image processing terminal for implementing the method.

[0008] In terms of system technical solutions, the present invention proposes a data indicator comprehensive management and visualization evaluation system including a data acquisition subsystem, a data decomposition subsystem, a data collection subsystem, a data association subsystem and a data visualization subsystem.

[0009] Functionally, the above subsystems are implemented as follows:

[0010] Data acquisition subsystem: acquires time data streams, the time data streams coming from multiple heterogeneous terminals, the heterogeneous terminals including mobile terminals and desktop terminals;

[0011] Data decomposition subsystem: decomposing the time data stream into a constant part and a variable part in each time series data block according to the time series data block;

[0012] Data collection subsystem: connected to the data decomposition subsystem, based on the size of each time series data block, establishes a first data stack of a first predetermined size in the first process, establishes a second data stack of a second predetermined size in the second process, and stores the unchanged part of each time series data block in the second data stack, and stores the variable part of each time series data block in the first data stack;

[0013] Data association subsystem: taking out variable part data from the first data stack, and taking out constant part data from the second data stack, wherein the variable part data and the constant part data are associated with each other through time sequence;

[0014] Data visualization subsystem: Visualize the time series data blocks associated by time series according to the time series, and the visualization includes the time prediction trend of the variable part data corresponding to each unchanged part of the time series data block.

[0015] As one of the further advantages of the present invention, the first process is connected to the second process via a unidirectional data pipe; the second predetermined size is greater than the first predetermined size;

[0016] Furthermore, the data collection subsystem updates the first predetermined size based on the prediction scale of the time prediction trend displayed by the data visualization subsystem.

[0017] In terms of method and technical solution, the present invention proposes a data indicator comprehensive management and visual evaluation method, which is applied to a visual terminal and includes the following steps:

[0018] Data acquisition step: acquiring a time data stream to be analyzed, wherein the time data stream includes a plurality of time series data blocks, and each time series data block includes a variable part and a constant part;

[0019] Data decomposition step: for each time series data block in the time data stream, decomposing the variable part and the invariant part thereof;

[0020] Data collection step: collecting the variable parts into a first data stack according to their corresponding time sequence; collecting the invariant parts into a second data stack;

[0021] Data association step: taking out variable part data from the first data stack, and taking out constant part data from the second data stack, wherein the variable part data and the constant part data are associated with each other through time series;

[0022] Data visualization step: Visually display the time series data blocks associated by time series according to the time series, and the visual display includes the time prediction trend of the variable part data corresponding to each unchanged part of the time series data block.

[0023] Wherein, the data association step further comprises:

[0024] Determine whether the first data stack is full. If so, take out all variable portion data from the first data stack, and search the second data stack for the constant portion data corresponding to all the variable portion data taken out, and take out the corresponding constant portion data from the second data stack.

[0025] In the third aspect of the present invention, an electronic device is also disclosed, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for executing the steps in the method technical solution.

[0026] As a further implementation, the electronic device can be a terminal device including a processor and a memory, especially an image processing terminal device, including a mobile terminal, a desktop terminal, a server and a server cluster, etc., which contains a storage medium and automatically executes program instructions through program instructions to implement all the step instructions of the method.

[0027] It should be pointed out that the data object targeted by the visualization evaluation method of the present invention is a time series data stream, that is, a data stream with time series changes, rather than general universal data.

[0028] As a more specific example, in the technical solution of the present invention, the unchanged part of each time series data stream includes data static attributes, and the data static attributes include data indicator names;

[0029] The variable portion of each time series data stream includes dynamic data attributes, and the data dynamic attributes include data indicator values.

[0030] The present invention applies a data acquisition step, a data decomposition step, a data collection step, a data association step and a data visualization step to a visualization terminal, wherein the visualization terminal includes a first CPU and a second CPU; the data collection step corresponds to a first stack process and a second stack process; the first stack process runs on the first CPU, and the second stack process runs on the second CPU; the first stack process and the second stack process are connected by a data pipeline, thereby realizing the decomposition and combination of time series data flows and the rapid visualization operation of the associated unchanged part and the associated part, thereby reducing the data processing amount and the data processing cost.

[0031] Further advantages of the present invention will be further reflected in detail in the specific embodiments section in conjunction with the drawings of the specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 This is a schematic diagram of the main steps of a data indicator comprehensive management and visual evaluation method according to an embodiment of the present invention.

[0034] Figure 2 is realized Figure 1 Schematic diagram of the image processing device of the method

[0035] Figure 3 is realized Figure 1 A schematic diagram of the structure of a visualization terminal of the data indicator comprehensive management and visualization evaluation method

[0036] Figure 4 yes Figure 1Schematic diagram of data processing of some steps of the method for comprehensive management and visual evaluation of data indicators

[0037] Figure 5 This is a schematic diagram of the subsystem structure of a data indicator comprehensive management and visual evaluation system according to an embodiment of the present invention.

[0038] Figure 6 is realized Figure 1 Schematic diagram of a computer-readable storage medium and a terminal device of the method process DETAILED DESCRIPTION

[0039] The invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0040] Reference Figure 1 , is a schematic diagram of the main steps of a data indicator comprehensive management and visual evaluation method according to an embodiment of the present invention.

[0041] like Figure 1 The method mainly includes a data acquisition step, a data decomposition step, a data collection step, a data association step and a data visualization step, and each step is specifically implemented as follows:

[0042] Data acquisition step: acquiring a time data stream to be analyzed, wherein the time data stream includes a plurality of time series data blocks, and each time series data block includes a variable part and a constant part;

[0043] Data decomposition step: for each time series data block in the time data stream, decomposing the variable part and the invariant part thereof;

[0044] Data collection step: collecting the variable parts into a first data stack according to their corresponding time sequence; collecting the invariant parts into a second data stack;

[0045] Data association step: taking out variable part data from the first data stack, and taking out constant part data from the second data stack, wherein the variable part data and the constant part data are associated with each other through time series;

[0046] Data visualization step: Visually display the time series data blocks associated by time series according to the time series, and the visual display includes the time prediction trend of the variable part data corresponding to each unchanged part of the time series data block.

[0047] In this embodiment, the time data stream is also called time series data stream or spatiotemporal data.

[0048] As its typical feature, the data stream usually contains a constant part and a variable part. The variable part evolves over time and space, thus providing conditions for trend prediction and generation of visual evolution diagrams.

[0049] In various practical scenarios, spatiotemporal data streams continuously generate large amounts of data every moment.

[0050] Unlike traditional data sets, these data are massive, temporally ordered, rapidly changing, and potentially infinite. If the traditional analysis is done based on the order in which the data is generated, it will bring huge data pressure.

[0051] This embodiment realizes the decomposition and combination of the time series data stream and the rapid visualization operation of the associated unchanged part and the associated part, thereby reducing the data processing amount and data processing cost.

[0052] As a more specific example, the invariant portion of each time series data stream includes data static attributes, and the data static attributes include data indicator names;

[0053] The variable portion of each time series data stream includes dynamic data attributes, and the data dynamic attributes include data indicator values.

[0054] Figure 1 The method can be automatically executed through program instructions by a terminal device including a processor and a memory, especially an image processing terminal device, including a mobile terminal, a desktop terminal, a server, and a server cluster.

[0055] See also Figure 2 , an image processing device is provided, the image processing device includes a memory and a processor, the processor and the memory communicate through a bus, the image processing device includes a human-computer interaction interface, such as a touch screen, through human-computer interaction operation, the execution progress of each step of the aforementioned data indicator comprehensive management and visual evaluation method can be dynamically and visually viewed.

[0056] Among them, the processor includes an application processor and a baseband processor. The processor is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory, and calling data stored in the memory, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Among them, the application processor mainly processes the operating system, user interface and application program, etc., and the baseband processor mainly processes wireless communication. It can be understood that the above-mentioned baseband processor may not be integrated into the processor. The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store the operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the electronic device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0057] As a more specific example, Figure 1 The method can be applied to a visualization terminal, for example, see Figure 3 , some or all of the steps of the method may be implemented using a mobile terminal, the mobile terminal comprising a first CPU and a second CPU;

[0058] The data collection step corresponds to the first stack process and the second stack process;

[0059] The first stack process runs on the first CPU, and the second stack process runs on the second CPU; the first stack process and the second stack process are connected by a data pipe.

[0060] Preferably, the data pipeline is a unidirectional data pipeline.

[0061] Data pipeline technology was originally used for data transfer between different databases (data sources), such as data backup and data restoration. Using data pipeline technology can avoid process blocking or using third-party agents for data transmission.

[0062] In the present invention, data pipeline technology is applied to data transmission between different processes for the first time, which can avoid interference between different processes. In particular, the use of unidirectional data pipelines makes the transmission of feedback and control signals stable and reliable.

[0063] See next Figure 4 , Figure 4 yes Figure 1A schematic diagram of data processing for some steps of the method for comprehensive management and visual evaluation of data indicators.

[0064] Reference Figure 4 The suitability diagram, the corresponding steps include:

[0065] S1: Acquire a time data stream to be analyzed, wherein the time data stream includes a plurality of time series data blocks;

[0066] S2: for each time series data block in the time data stream, decompose the variable part and the invariant part thereof;

[0067] S3: grouping the variable parts into a first data stack according to their corresponding time sequences; and grouping the invariant parts into a second data stack.

[0068] Specifically, in this embodiment, the first data stack has a first predetermined size, and the second data stack has a second predetermined size; the first predetermined size is smaller than the second predetermined size.

[0069] S4: taking out variable part data from the first data stack, and taking out constant part data from the second data stack, wherein the variable part data is associated with the constant part data through a time sequence;

[0070] Preferably, the time series association here refers to establishing a mapping relationship between all indicator values ​​(variable part) for the same indicator name (unchanged part) generated in the same time period or time point.

[0071] More specifically, performing the association specifically includes:

[0072] Determine whether the first data stack is full. If so, take out all variable portion data from the first data stack, and search the second data stack for the constant portion data corresponding to all the variable portion data taken out, and take out the corresponding constant portion data from the second data stack.

[0073] Obviously, different from the periodic processing or volume-based processing of the prior art, this embodiment adopts stack storage, and only takes out data when the stack is full to execute the subsequent visualization process. It can reduce the inter-process scheduling as much as possible while meeting the current device processing capacity, and maximize the use of the stack processing capacity.

[0074] Although, Figure 4 Not shown, but the corresponding steps may also include:

[0075] S5: Visually display the time series data blocks associated by time series according to the time series, wherein the visual display includes the time prediction trend of the variable part data corresponding to each constant part of the time series data block.

[0076] It should be noted that, as to how to realize the visualization evolution of time series data and spatiotemporal data and realize time series trend analysis, etc., according to different data types, there are different visualization analysis methods in this field, including time trend analysis models, etc. The present invention does not elaborate on this, and this is not the focus of improvement of the present invention. The focus of the present invention is on data grouping and scheduling before entering the visualization processing. Therefore, the methods adopted in the specific visualization stage can refer to the prior art (such as the public literature mentioned in the background technology).

[0077] Preferably, in this example, the first predetermined size of the first data stack is adjusted according to the prediction scale of the time prediction trend displayed in the data visualization step, so as to realize dynamic closed-loop feedback and achieve mutual matching and dynamic updating of the visualized software processing process and the hardware resource scheduling process.

[0078] based on Figure 1-Figure 4 For an introduction to the principle of Figure 5 , provides a data index comprehensive management and visual evaluation system, the system includes a data acquisition subsystem, a data decomposition subsystem, a data collection subsystem, a data association subsystem and a data visualization subsystem, which can be used to achieve Figure 1-Figure 4 The embodiment described.

[0079] Specifically, in Figure 5 In the data acquisition subsystem, the time data stream is used to acquire the time data stream from multiple heterogeneous terminals, and the heterogeneous terminals include mobile terminals and desktop terminals;

[0080] The data decomposition subsystem is used to decompose the time data stream into a constant part and a variable part in each time series data block according to the time series data block;

[0081] The data collection subsystem is connected to the data decomposition subsystem, and is used to establish a first data stack of a first predetermined size in the first process, and establish a second data stack of a second predetermined size in the second process based on the size of each time series data block, and store the unchanged part of each time series data block in the second data stack, and store the variable part of each time series data block in the first data stack;

[0082] The data association subsystem is used to take out the variable part data from the first data stack and take out the constant part data from the second data stack, and the variable part data is associated with the constant part data through time sequence;

[0083] The data visualization subsystem is used to visualize the time series data blocks associated by time series according to the time series, and the visualization display includes the time prediction trend of the variable part data corresponding to each unchanged part of the time series data block.

[0084] As a further preference, the data association subsystem is connected to the data collection subsystem;

[0085] When the first data stack is full, the data collection subsystem sends a full stack signal to the data association subsystem; the data association subsystem takes out all variable part data from the first data stack, and searches for the constant part data corresponding to all the variable part data taken out from the second data stack, and takes out the corresponding constant part data from the second data stack.

[0086] The first process is connected to the second process via a unidirectional data pipe.

[0087] The second predetermined size is larger than the first predetermined size;

[0088] Furthermore, the data collection subsystem updates the first predetermined size based on the prediction scale of the time prediction trend displayed by the data visualization subsystem.

[0089] Finally, see Figure 6 This embodiment also provides a computer-readable storage medium on which computer program instructions are stored; the program instructions are executed by an image terminal processing device including a processor and a memory to implement all or part of the steps of the method. The processor and the memory are connected via a bus to form internal communication of the terminal device.

[0090] The technical solution of the present application may essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or all or part of the technical solution. The software product is stored in a memory and includes a number of instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application.

[0091] The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0092] Aiming at massive, temporally ordered, rapidly changing and potentially infinite empty data streams, the present invention changes the traditional method of analyzing data in the order in which it is generated, thus avoiding huge data pressure. The present invention realizes the decomposition and combination of temporal data streams and the rapid visualization of the associated parts and the associated parts after association, and adopts a dynamically finer data stack and inter-process pipeline communication, thus reducing the amount of data processing and the cost of data processing, and ensuring stable data transmission.

[0093] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data index comprehensive management and visual evaluation method, the method is applied to a visual terminal, It is characterized in that The method comprises the following steps: S1, data acquisition step: acquiring a time data stream to be analyzed, wherein the time data stream includes a plurality of time series data blocks, and each time series data block includes a variable part and a constant part; S2, data decomposition step: for each time series data block in the time data stream, decomposing the variable part and the invariant part thereof; S3, data collection step: collecting the variable parts into a first data stack according to their corresponding time sequence; Collecting the unchanged part into a second data stack; The first data stack has a first predetermined size, the second data stack has a second predetermined size, the first predetermined size being lower than the second predetermined size; S4, data association step: taking out variable part data from the first data stack, and taking out constant part data from the second data stack, wherein the variable part data and the constant part data are associated with each other through time series; S5, data visualization step: visually displaying the time series data blocks associated by time series according to the time series, wherein the visual display includes a time prediction trend of the variable part data corresponding to each constant part of the time series data block; The temporal association refers to establishing a mapping relationship between all variable parts generated in the same time period or time point for the same constant part; The data association step further comprises: Determine whether the first data stack is full, and if so, take out all variable partial data from the first data stack, and search the second data stack for constant partial data corresponding to all the taken out variable partial data, and take out the corresponding constant partial data from the second data stack; The unchanged part of each time series data stream includes data static attributes, and the data static attributes include data indicator names; The variable portion of each time series data stream includes data dynamic attributes, wherein the data dynamic attributes include data indicator values; The data collection step further includes: adjusting a first predetermined size of the first data stack according to a prediction scale of the time prediction trend displayed in the data visualization step; The method is automatically executed through program instructions by terminal devices including processors and memories, including mobile terminals, desktop terminals, servers, and server clusters; Provided is an image processing device, the image processing device comprising a memory and a processor, the processor and the memory communicating via a bus, the image processing device comprising a human-computer interaction interface, through which the execution progress of each step of the aforementioned data indicator comprehensive management and visual evaluation method can be dynamically and visually viewed; The processor includes an application processor and a baseband processor. The processor is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory and calling data stored in the memory, thereby monitoring the electronic device as a whole. The application processor mainly processes the operating system, user interface and application program, and the baseband processor mainly processes wireless communication. The memory is used to store software programs and modules. The processor executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory. The memory includes a program storage area and a data storage area. The program storage area stores the operating system and at least one application program required for a function. The data storage area can store data created according to the use of the electronic device. The method is applied to a visualization terminal, and some or all steps of the method are implemented by a mobile terminal, the mobile terminal includes a first CPU and a second CPU; the data collection step corresponds to a first stack process and a second stack process; The first stack process runs on the first CPU, and the second stack process runs on the second CPU; The first stack process and the second stack process are connected by a data pipe, and the data pipe is a unidirectional data pipe.

2. A data index comprehensive management and visual evaluation system, which implements the method according to claim 1, and comprises a data acquisition subsystem, a data decomposition subsystem, a data collection subsystem, a data association subsystem and a data visualization subsystem; Features: The data acquisition subsystem is used to acquire a time data stream, where the time data stream comes from a plurality of heterogeneous terminals, including a mobile terminal and a desktop terminal; The data decomposition subsystem is used to decompose the time data stream into a constant part and a variable part in each time series data block according to the time series data block; The data collection subsystem is connected to the data decomposition subsystem, and is used to establish a first data stack of a first predetermined size in the first process, and establish a second data stack of a second predetermined size in the second process based on the size of each time series data block, and store the unchanged part of each time series data block in the second data stack, and store the variable part of each time series data block in the first data stack; The data association subsystem is used to take out the variable part data from the first data stack and take out the constant part data from the second data stack, and the variable part data is associated with the constant part data through time sequence; The data visualization subsystem is used to visualize the time series data blocks associated by time series according to the time series, and the visualization display includes the time prediction trend of the variable part data corresponding to each unchanged part of the time series data block.

3. A data indicator comprehensive management and visual evaluation system as claimed in claim 2, Features: The data association subsystem is connected to the data collection subsystem; When the first data stack is full, the data collection subsystem sends a full stack signal to the data association subsystem; The data association subsystem takes out all variable partial data from the first data stack, searches the second data stack for the invariant partial data corresponding to all the taken out variable partial data, and takes out the corresponding invariant partial data from the second data stack.

4. A data index comprehensive management and visual evaluation system as claimed in claim 3, Features: The first process is connected to the second process via a unidirectional data pipe.

5. A data index comprehensive management and visual evaluation system as claimed in claim 4, Features: The second predetermined size is larger than the first predetermined size; Furthermore, the data collection subsystem updates the first predetermined size based on the prediction scale of the time prediction trend displayed by the data visualization subsystem.

Images