Data display method and device

Abstract

The invention discloses a data display method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to a received data display request, analyzing a service identifier from the display request, and searching a data query interface corresponding to the service identifier; according to the data query interface, obtaining target service data from a local storage, the service data in the local storage being obtained by performing distributed real-time streaming processing according to a node topological relation corresponding to the data display request; and visually displaying the target business data. According to the embodiment, distributed real-time stream processing is carried out based on the node topological relation, the data and the data source are decoupled, high-performance real-time data visual display is achieved, good expandability and flexibility are achieved, and the data display efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for data display. Background Art

[0002] With the development of information technology and the improvement of digitalization, data, as a carrier of information, has gradually become an important production factor. Data dashboards visualize data to help users keep abreast of business status, data changes, and manage business goals. Currently, data dashboards are mainly implemented using database clusters or search engine ES clusters for data support.

[0003] In the process of implementing the present invention, the inventors found that the prior art has the following problems:

[0004] Data dashboards based on search engine ES clusters generally have certain data delays and poor real-time performance, which brings a bad experience to users; data dashboards based on database clusters or other file media will have a strong dependence on data sources, especially for database clusters, which will occupy their precious resources, bring certain pressure to the database cluster, affect other businesses, and cannot meet the actual application needs well. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and device for data display, which obtains target business data obtained by distributed real-time stream processing calculation based on node topology relationship according to the data query interface obtained by the data display request, and performs visual display. The embodiment of the present invention performs distributed real-time stream processing based on node topology relationship, which not only decouples data from data source and realizes high-performance real-time data visualization, but also has good scalability and flexibility, and improves the efficiency of data display.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for data presentation is provided, including:

[0007] In response to receiving a data display request, parsing a service identifier from the display request, and searching for a data query interface corresponding to the service identifier;

[0008] According to the data query interface, target business data is obtained from local storage, where the business data in the local storage is obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request;

[0009] The target business data is visually displayed.

[0010] Optionally, before parsing the service identifier from the display request, the method includes: parsing the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node from the display request; and searching for a corresponding node topology relationship from a node topology relationship library according to the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node.

[0011] Optionally, when the node topology relationship is not found in the node topology relationship library, the corresponding basic node topology relationship is searched from the node topology relationship library according to the topological connection relationship between the logical node to be processed and the logical node to be processed; according to the topological connection relationship between the logical node to be processed and the logical node to be processed, the logical nodes and / or topological connection relationships in the basic node topology relationship are updated to obtain the node topology relationship corresponding to the data display request.

[0012] Optionally, before obtaining the target business data, the method also includes: pulling data from a data source, storing the data in a message queue according to topics, and the data exists in the form of a collection; according to the node topology relationship corresponding to the data display request, using distributed stream processing to process the messages in each topic to obtain business data, so that the data query interface can obtain the target business data from the business data.

[0013] Optionally, data is pulled from a data source and stored in a message queue according to topics, including: based on a preset data identifier, filtering out the data to be pulled from the data source, and matching the data to be pulled with data rules to obtain matching data; based on category information in the data identifier of the matching data, classifying the matching data, and sorting the matching data in each category; storing the sorted matching data of each category in the corresponding message queue according to the topic corresponding to the category.

[0014] Optionally, the node topology relationship includes a source logical node, an intermediate logical node and an output logical node; according to the node topology relationship corresponding to the data display request, the messages in each topic are processed using a distributed stream processing method, including: filtering out the to-be-processed data corresponding to the source logical node from the messages of each topic, and performing a logical operation on the to-be-processed data according to the logical processing rules in the source logical node to obtain a source logical node result; according to the source logical node result, combined with the topological connection relationship between the source logical node and the intermediate logical node in the node topology relationship, performing a logical operation on the data in the intermediate logical node step by step to obtain an intermediate logical node result; according to the intermediate logical node result, combined with the node topology relationship, performing a logical operation on the data in the output logical node to obtain business data.

[0015] Optionally, the business data is stored in a cache; in response to receiving a historical data display request, target historical business data corresponding to the historical data display request is acquired through the cache.

[0016] Optionally, visually displaying the target business data includes: visually displaying the target business data in real time through a window mode of distributed real-time stream processing.

[0017] According to a second aspect of an embodiment of the present invention, a data display device is provided, including:

[0018] A query interface confirmation module, configured to, in response to receiving a data display request, parse a service identifier from the display request and search for a data query interface corresponding to the service identifier;

[0019] A target business data acquisition module, used to acquire target business data from local storage according to the data query interface, wherein the business data in the local storage is obtained by performing distributed real-time stream processing according to the node topology relationship corresponding to the data display request;

[0020] The display module is used to visually display the target business data.

[0021] According to a third aspect of an embodiment of the present invention, there is provided an electronic device for displaying data, including:

[0022] one or more processors;

[0023] a storage device for storing one or more programs,

[0024] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0025] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0026] An embodiment of the invention has the following advantages or beneficial effects: by responding to receiving a data display request, parsing the service identifier from the display request, and finding the data query interface corresponding to the service identifier; according to the data query interface, obtaining the target service data from the local storage, the service data in the local storage is obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request; the technical scheme for visually displaying the target service data realizes the data query interface obtained according to the data display request, obtains the target service data obtained by distributed real-time stream processing calculation based on the node topology relationship, and performs visual display. The embodiment of the present invention performs distributed real-time stream processing based on the node topology relationship, which not only decouples the data from the data source and realizes the visual display of high-performance real-time data, but also has good scalability and flexibility, and improves the efficiency of data display. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0028] Figure 1 is a schematic diagram of the main process of the method for data display according to an embodiment of the present invention;

[0029] Figure 2 is a logical schematic diagram of distributed parallel stream threads according to an embodiment of the present invention;

[0030] Figure 3 It is a schematic diagram of the Kafka Streams node topology relationship of an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the processing flow of business data of the data dashboard of an embodiment of the present invention;

[0032] Figure 5 is a schematic diagram of main modules of a device for data display according to an embodiment of the present invention;

[0033] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0034] Figure 7 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that in the technical solution of the present invention, the collection / collection, update, analysis, use, transmission, storage and other aspects of user personal information involved are in compliance with the provisions of relevant laws and regulations, are used for legal and reasonable purposes, are not shared, leaked or sold outside of these legal uses, and are subject to supervision and management by national regulatory authorities. Necessary measures should be taken for user personal information to selectively block the use or access to personal information data to prevent illegal access to such personal information data, ensure that persons who have access to personal information data comply with the provisions of relevant laws and regulations, and ensure the security of user personal information. In addition, once these user personal information data are no longer needed, the risks should be minimized by limiting or even prohibiting data collection and / or deleting data.

[0036] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0037] Currently, in the implementation of data dashboards, data dashboards based on search engine ES clusters generally have certain data delays and poor real-time performance, which brings a bad experience to users; data dashboards based on database clusters or other file media will have a strong dependence on data sources, especially for database clusters, which will occupy their precious resources, bring certain pressure to the database cluster, affect other businesses, and cannot meet the actual application needs well.

[0038] In order to solve the above problems existing in the prior art, the present invention proposes a method for data display, which obtains the target business data obtained by distributed real-time stream processing calculation based on the node topology relationship according to the data query interface obtained by the data display request, and performs visual display. The embodiment of the present invention performs distributed real-time stream processing based on the node topology relationship, which not only decouples the data from the data source and realizes the visual display of high-performance real-time data, but also has good scalability and flexibility, and improves the efficiency of data display.

[0039] In the introduction to the embodiments of the present invention, the terms and their meanings are as follows:

[0040] Kafka Streams: is a class library that implements stream computing capabilities, high performance, no external dependencies other than Kafka, and fully utilizes Kafka's horizontal expansion and fault tolerance capabilities;

[0041] HashMap: Implementation of the Map interface based on hash table;

[0042] Kafka Connect: is a tool for scalable and reliable streaming of data between Kafka and other systems.

[0043] Figure 1 is a schematic diagram of the main process of the method for data display according to an embodiment of the present invention, such as Figure 1 As shown, the data display method of the embodiment of the present invention includes the following steps S101 to S103.

[0044] Step S101: In response to receiving a data display request, a service identifier is parsed from the display request, and a data query interface corresponding to the service identifier is searched.

[0045] Specifically, as a data visualization tool, the data dashboard supports dynamic and real-time display of data according to the user's query needs. Especially in the field of medical Internet, with the rapid development of health products, the continuous expansion of platform business, terminal pharmacies, and medical institutions, users need to view business data, revenue data, and operation data of various dimensions through the data dashboard. After receiving the data display request sent by the user, the display request is parsed to obtain the business identifier corresponding to the display request, and then the corresponding data query interface is found according to the business identifier. For example, if the transaction sales, profit, and year-on-year growth rate of a certain drug in a specified area need to be displayed, the identification of the drug in the order system is obtained by parsing the display request, and the corresponding order data query interface is found according to the identification.

[0046] According to one embodiment of the present invention, before parsing the service identifier from the display request, the method includes: parsing the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node from the display request; and searching for the corresponding node topology relationship from a node topology relationship library according to the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node.

[0047] Specifically, before the service identifier is parsed out according to the received display request, the topological connection relationship between the logical nodes required for data display and each logical node can be determined according to the code program for data display, such as the order sales amount logical node, the sales statistics logical node, etc., and the order sales amount logical node requires the sales statistics logical node as the upstream logical node. This relationship between the upstream and downstream logical nodes is the topological connection relationship, and the logical nodes here need to create and configure corresponding data processing logic rules. The required topological connection relationship between the logical node to be processed and the logical node to be processed is encapsulated into the data display request, so that before the service identifier is parsed out, according to the topological connection relationship between the logical node to be processed and the logical node to be processed, the corresponding node topological relationship is searched from the node topological relationship library that has been created and maintained by the system, and the node topological relationship found can be directly used in the subsequent distributed real-time stream processing, avoiding the repeated creation of existing standard logical nodes and topological connection relationships, and improving the efficiency of distributed real-time stream processing.

[0048] According to another embodiment of the present invention, the method includes: when the node topology relationship is not found in the node topology relationship library, searching for the corresponding basic node topology relationship from the node topology relationship library according to the topological connection relationship between the logical node to be processed and the logical node to be processed; according to the topological connection relationship between the logical node to be processed and the logical node to be processed, updating the logical nodes and / or topological connection relationships in the basic node topology relationship to obtain the node topology relationship corresponding to the data display request.

[0049] Specifically, when a node topology relationship that completely matches the topological connection relationship between the logical node to be processed and the logical node to be processed is not found in the node topology relationship library created from the system, the topological connection relationship between the logical node to be processed and the logical node to be processed is input into the system node topology relationship library for matching and screening, and a basic node topology relationship with a higher matching degree is screened out; then, according to the topological connection relationship between the logical node to be processed and the logical node to be processed, the logical nodes and / or topological connection relationships in the basic node topology relationship are updated to obtain a node topology relationship. For example, the basic node topology relationship includes an order sales amount logical node, a sales volume statistics logical node and a cost logical node, and the logical node to be processed also includes a profit logical node. In this case, it is only necessary to add a profit logical node to the basic node topology relationship, and use the existing order sales amount logical node and cost logical node as the upstream logical nodes of the newly added profit logical node, so as to quickly obtain the node topology relationship corresponding to the data display request.

[0050] Through the construction of the above-mentioned scalable and flexible node topology relationship, it is easy to reuse the existing logical nodes and node topology relationships, avoid building similar or repeated node topology relationships, save development resources, improve development efficiency, and facilitate the standardization of node topology relationships. Of course, the new node topology relationship obtained based on the expansion of the basic node topology relationship can also be synchronized to the system node topology relationship library to enrich the node topology relationship library so that subsequent developers can directly call it.

[0051] Furthermore, the service identifier corresponding to the data display request, such as the order service, is parsed from the display request, and then the data query interface corresponding to the service identifier pre-assigned by the system can be used to obtain the data query interface corresponding to the order service.

[0052] Step S102: acquiring target business data from local storage according to the data query interface, wherein the business data in the local storage is obtained by performing distributed real-time stream processing according to the node topology relationship corresponding to the data display request.

[0053] According to one embodiment of the present invention, before obtaining the target business data, the method also includes: pulling data from a data source, storing the data in a message queue according to topics, and the data exists in the form of a collection; according to the node topology relationship corresponding to the data display request, using distributed stream processing to process the messages in each topic to obtain business data, so that the data query interface can obtain the target business data from the business data.

[0054] Specifically, the embodiment of the present invention uses the distributed real-time stream processing engine Kafka Streams to process data for the query and display tasks of massive real-time data based on the existing Kafka message queue system architecture, so that real-time stream processing does not require external dependencies on tasks, nor does it involve the construction of other R&D ecosystems. It can fully utilize Kafka's horizontal expansion and fault tolerance capabilities to achieve lightweight, low-latency, and high-performance distributed real-time stream processing while ensuring accuracy.

[0055] Before obtaining the target business data, it is necessary to integrate the data source of the external database or other file media with Kafka Streams. You can use Kafka Connect to read data from the data source by monitoring, or by using Kafka Connect to read data from the data source by statement query, and then write the read data into the Kafka topic to generate a message. If the data is pulled from the database table, the corresponding topic rule can be: business topic + database table name where the data is located. Usually in massive data business scenarios, the pulled data is a data set of a certain scale. The data in the embodiment of the present invention is the concept of a data set.

[0056] Furthermore, for business scenarios with massive data, by drawing on the logic of distributed parallel stream threads and integrating the data to be pulled through KafkaConnect's unique distributed mode, multiple Kafka Connects can be used in parallel to filter out the data fragments to be pulled from the messages of each topic, aggregate the data fragments to be processed, and enable multiple stream threads to obtain the data to be pulled in parallel.

[0057] Figure 2 This is a logical diagram of the distributed parallel stream threads of the embodiment of the present invention. In the figure, two stream threads are taken as an example, and each stream thread corresponds to a consumer and a producer. The consumer of each stream thread obtains data messages from the topic partition of the corresponding stream task, and saves the data messages in the record buffer, while each stream task obtains data messages from the record buffer of the stream thread for processing, and then outputs the information to the output topic through the producer.

[0058] According to another embodiment of the present invention, data is pulled from a data source and stored in a message queue according to topics, including: filtering out the data to be pulled from the data source according to a preset data identifier, and matching the data to be pulled with data rules to obtain matching data; classifying the matching data according to category information in the data identifier of the matching data, and sorting the matching data in each category; and storing the sorted matching data of each category in a corresponding message queue according to the topic corresponding to the category.

[0059] Specifically, according to the business scenarios involved in the data dashboard, preset data identifiers to be pulled are set. When the data source contains data with preset data identifiers or the data with preset data identifiers is updated, the data to be pulled is filtered out according to the preset data identifiers. Then, according to the preset data rules, the data to be pulled is matched with rules. The data rules are usually implemented in a flexible and scalable configuration method. The data to be pulled is filtered according to the data rules, and the data structure of the data to be pulled that meets the preset data rules is matched to adapt to the subsequent stream processing. For example, in the order business scenario, it is agreed that only the specified type of order data can be used as the source data of the data dashboard. Then, the specified type can be used as a data rule to perform a secondary screening on the data to be pulled. Accordingly, the data structure of the specified type of order data is matched to unify the data structure in the system: the fields ORDER_ID, BUSINESS_TYPE, AMOUNT, and DATE_TIME of the original data column can be converted into the system-general field names id, type, amount, and date.

[0060] Furthermore, based on the matching data obtained after the above processing, the matching data is classified according to the category information in the data identifier of the matching data, such as matching data of order type, matching data of after-sales type, matching data of user account type, etc., and then the matching data in each category is sorted according to the preset dimension. Considering that the matching data pulled may be sorted in disorder due to communication abnormalities such as system delay and network congestion, in order to ensure the real-time accuracy of the data dashboard, the matching data in each category is sorted by time or version number through the thread pool queue in the Kafka message queue system. In addition, in special scenarios where the amount of data increases suddenly, classification and sorting can also play a role in buffering matching data, avoiding the risk of system downtime caused by the transmission of instantaneous massive matching data. Finally, the sorted matching data is stored in the corresponding message queue according to the topic corresponding to the category, which is convenient for subsequent data search.

[0061] According to another embodiment of the present invention, the node topology relationship includes a source logical node, an intermediate logical node and an output logical node; according to the node topology relationship corresponding to the data display request, the messages in each topic are processed using a distributed stream processing method, including: filtering out the to-be-processed data corresponding to the source logical node from the messages of the each topic, and performing a logical operation on the to-be-processed data according to a logical processing rule in the source logical node to obtain a source logical node result; according to the source logical node result, in combination with a topological connection relationship between the source logical node and the intermediate logical node in the node topology relationship, performing a logical operation on the data in the intermediate logical node step by step to obtain an intermediate logical node result; according to the intermediate logical node result, in combination with the node topology relationship, performing a logical operation on the data in the output logical node to obtain business data.

[0062] Specifically, based on the matching data stored in Kafka, the stream processing engine Kafka Streams is used to process the message data. Kafka Streams defines the data processing logic through the node topology relationship. The logical nodes in the node topology relationship include logical processing rules, and the logical processing rules will clearly define the data to be processed. The general node topology relationship includes source logical nodes without upstream relationships, output logical nodes without downstream relationships, and intermediate logical nodes that are not source logical nodes and non-output logical nodes. According to the node topology relationship determined above, according to the topological connection direction, according to the data identifier of the to-be-processed data in the logical processing rule in the source logical node, determine the subject corresponding to the category, filter out the to-be-processed data from the message of the corresponding subject, and then perform logical operations on the to-be-processed data according to the logical processing rule to obtain the source logical node result; perform logical operations on the data in each intermediate logical node step by step according to the topological connection relationship between the source logical node and the intermediate logical node, and obtain the results of each intermediate logical node in the node topology relationship; finally, according to each intermediate logical node result, in combination with the associated output logical node in the node topology relationship, perform logical operations on the data in the output logical node according to the logical processing rule of the output logical node, and obtain the business data obtained by distributed stream processing.

[0063] Furthermore, for the business data obtained through distributed stream processing, a local "state warehouse" provided by Kafka Streams is used to synchronize business data. This "state warehouse" is also called state storage. The local state storage of Kafka Streams can be used to store and query state data. Each stream processing task in Kafka Streams embeds one or more state storages that can be accessed through the API. The state storage can be a persistent Key-Value or memory HashMap, or other data structures, providing real-time business data for interactive queries of the data dashboard. After the synchronization of business data is completed, external storage such as data sources can be removed to achieve decoupling of data from data sources. KafkaStreams accesses real-time business data in the local state storage through the data read-only query interface, and obtains real-time target business data for interactive queries of the data dashboard from the business data.

[0064] Figure 3This is a schematic diagram of the Kafka Streams node topology relationship of an embodiment of the present invention. In this figure, each logical node is represented by a processor, and the data source is obtained by inputting a topic, and the to-be-processed data corresponding to the source logical node (corresponding to processor 1 and processor 2 in the figure) is obtained. According to the logical processing rules in the source logical node, the to-be-processed data is logically operated to obtain the source logical node result; then, combined with the topological connection relationship between processor 1 and processor 2 in the node topology relationship, the result of processor 1 is applied to processor 5 (intermediate logical node), and the data in processor 5 is logically operated. Similarly, the results of each intermediate logical node are calculated step by step, for example, the result of processor 2 is applied to processor 3 (intermediate logical node), and the data in processor 3 is logically operated. Finally, the results of processor i (intermediate logical node) and processor j (intermediate logical node) are aggregated or connected to obtain the output logical node (corresponding to the data reception in the figure), and the business data is output through the output topic. In addition, when the business scenario changes based on the current business scenario, the corresponding basic node topology relationship is found through the system node topology relationship library, including processor 2 and processor 3 in the figure. Processor 2 and processor 3 can be updated according to the logical node to be processed to obtain processor 7 and processor 8 required by the current business scenario.

[0065] Through the above-mentioned stream processing method based on node topology relationship, not only the business data is obtained, but also the reuse of node topology relationship is realized, the scalability of stream processing is improved, the stream processing efficiency is improved, and the practical application is better met.

[0066] According to yet another embodiment of the present invention, the method further includes: storing the business data in a cache; and in response to receiving a historical data display request, acquiring target historical business data corresponding to the historical data display request through the cache.

[0067] Specifically, for the business data obtained through distributed stream processing, on the one hand, it can provide real-time target business data for the upper-level data dashboard, and on the other hand, it can also be stored in the Redis cache as historical business data through message publishing and subscription. When the data dashboard needs to display historical data, it can directly read the business data in the Redis cache without having to pull the historical data from the data source again for processing, which saves resources and improves the response speed and efficiency of data display.

[0068] Step S103: Visually display the target business data.

[0069] According to an embodiment of the present invention, visually displaying the target business data includes: visually displaying the target business data in real time through a window mode of distributed real-time stream processing.

[0070] Specifically, the real-time target business data acquired as described above is processed in a windowed manner to realize the aggregated display of the target business data within the specified time window, such as displaying the sales order quantity curve within every 5 minutes, the number of user logins within every 1 hour, etc. In the embodiment of the present invention, the real-time dynamic display of the target business data of the data dashboard within the specified time period is realized through the session window, flip window, and sliding / jumping window of Kafka Streams. Among them, the session window is based on user activities, and the session to which the data belongs is usually determined by the timestamp of the target business data; the flip window is usually used to count the target business data within the specified time period, for example, the transaction of business orders is counted every 10 seconds, and after the 10 seconds are over, the window will roll over to the next 10-second window; the sliding / jumping window is somewhat similar to the flip window, except that it has a sliding / jumping interval, which is usually less than the duration of the window, and the window slides / jumps according to the preset sliding / jumping interval.

[0071] Figure 4It is a schematic diagram of the processing flow of business data of the data dashboard of an embodiment of the present invention. Taking order data, after-sales data, procurement data (data for establishing procurement relationships) and user data as examples, Kafka uses Kafka Connect to filter out the data to be pulled from these four business scenario databases, and performs data rule matching, category division and sorting on the data to be pulled, and stores them in the corresponding message queues 1, queues 2, queues 3 and queues 4 according to the topics corresponding to the categories. Then map the data in the message queue to a preset format, and perform partition mapping according to the preset partitioning rules (hash storage based on data identifiers in the embodiment of the present invention). According to the node topology relationship corresponding to the data display request, the required order, after-sales, construction and procurement and user data are obtained from the corresponding partitions respectively. The business processor 1, business processor 2, ..., business processor 5 in the figure are source logical nodes, wherein the data to be processed involved in business processor 1 include amount data, address data, item data and promotion data in order data, and the data to be processed involved in business processor 2 include membership data, first order data in user data, and enterprise data in construction and procurement data, etc. According to the logical processing rules in each business processor (source logical node) in the figure, logical operations are performed on the data to be processed to obtain the source logical node result; operations are performed step by step according to the node topology relationship to obtain output logical nodes aggregation processor 1, aggregation processor 2..., aggregation processor m. The embodiment of the present invention expands the local state storage, and synchronizes the business data of aggregation processor 1, ..., aggregation processor m to local state storage 1, local state storage 2, ..., local state storage n as needed. According to the business identifier parsed from the received display request, the data query interface is found, and the local state storage of Kafka Streams is connected through the data query interface. According to the preset time window configuration, the real-time target business data is obtained for large-screen display of the data dashboard. In addition, the business data obtained through distributed stream processing can also be stored in the Redis cache as historical business data through message publishing and subscription. When the data dashboard needs to display historical data, the business data in the Redis cache is directly read for display.

[0072] Figure 5 Schematic diagram of the main modules of the device for data display according to an embodiment of the present invention. Figure 5 As shown, the data display device 500 mainly includes a query interface confirmation module 501 , a target business data acquisition module 502 and a display module 503 .

[0073] The query interface confirmation module 501 is used to parse the service identifier from the display request in response to receiving the data display request, and search for the data query interface corresponding to the service identifier;

[0074] The target business data acquisition module 502 is used to acquire the target business data from the local storage according to the data query interface, where the business data in the local storage is obtained by performing distributed real-time stream processing according to the node topology relationship corresponding to the data display request;

[0075] The display module 503 is used to visually display the target business data.

[0076] According to one embodiment of the present invention, the data display device 500 also includes a node topology relationship determination module (not shown in the figure), which is used to: before parsing the service identifier from the display request, the method includes: parsing the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node from the display request; according to the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node, searching for the corresponding node topology relationship from the node topology relationship library.

[0077] According to another embodiment of the present invention, the node topology relationship determination module (not shown in the figure) is also used to: when the node topology relationship is not found in the node topology relationship library, search for the corresponding basic node topology relationship from the node topology relationship library according to the topological connection relationship between the logical node to be processed and the logical node to be processed; update the logical nodes and / or topological connection relationships in the basic node topology relationship according to the topological connection relationship between the logical node to be processed and the logical node to be processed, and obtain the node topology relationship corresponding to the data display request.

[0078] According to another embodiment of the present invention, the data display device 500 also includes a business data acquisition module (not shown in the figure), which is used to: before acquiring the target business data, pull data from the data source, store the data in the message queue according to the topic, and the data exists in the form of a collection; according to the node topology relationship corresponding to the data display request, use distributed stream processing to process the messages in each topic to obtain business data, so that the data query interface can obtain the target business data from the business data.

[0079] According to another embodiment of the present invention, the business data acquisition module (not shown in the figure) is also used to: filter out the data to be pulled from the data source according to a preset data identifier, and match the data to be pulled with data rules to obtain matching data; classify the matching data according to the category information in the data identifier of the matching data, and sort the matching data in each category; store the matching data of each category after sorting into the corresponding message queue according to the subject corresponding to the category.

[0080] According to an embodiment of the present invention, the node topology relationship includes a source logical node, an intermediate logical node and an output logical node; the business data acquisition module (not shown in the figure) is used to: filter out the to-be-processed data corresponding to the source logical node from the messages of the various topics, and perform logical operations on the to-be-processed data according to the logical processing rules in the source logical node to obtain a source logical node result; according to the source logical node result, combined with the topological connection relationship between the source logical node and the intermediate logical node in the node topology relationship, perform logical operations on the data in the intermediate logical node step by step to obtain an intermediate logical node result; according to the intermediate logical node result, combined with the node topology relationship, perform logical operations on the data in the output logical node to obtain business data

[0081] According to another embodiment of the present invention, the data display device 500 also includes a cache module (not shown in the figure), which is used to: store the business data in a cache; and in response to receiving a historical data display request, obtain the target historical business data corresponding to the historical data display request through the cache.

[0082] According to yet another embodiment of the present invention, the display module 503 is further configured to: display the target business data in real-time and visual manner through a window mode of distributed real-time stream processing.

[0083] Figure 6 is an exemplary system architecture diagram to which embodiments of the present invention can be applied.

[0084] like Figure 6 As shown, system architecture 600 may include terminal devices 601, 602, 603, network 604 and server 605. Network 604 is used to provide a medium for communication links between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0085] Users can use terminal devices 601, 602, 603 to interact with server 605 through network 604 to receive or send messages, etc. Various communication client applications, such as data display applications, etc., can be installed on terminal devices 601, 602, 603 (only as an example).

[0086] The terminal devices 601 , 602 , and 603 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers, etc.

[0087] Server 605 may be a server that provides various services, such as a background management server (only an example) that provides support for data display by users using terminal devices 601, 602, and 603. In response to receiving a data display request, the background management server may parse a service identifier from the display request and search for a data query interface corresponding to the service identifier; obtain target service data from local storage according to the data query interface, and the service data in the local storage is obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request; visualize the target service data, etc., and feed back the processing results (such as display data, etc. - only an example) to the terminal device.

[0088] It should be noted that the data display method provided in the embodiment of the present invention is generally executed by the server 605 , and accordingly, the data display device is generally set in the server 605 .

[0089] It should be understood that Figure 6 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0090] Reference below Figure 7 , which shows a schematic diagram of the structure of a computer system 700 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 7 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0091] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the system 700 are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0092] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage section 708 as needed.

[0093] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the above-mentioned functions defined in the system of the present invention are executed.

[0094] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0095] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0096] The units involved in the embodiments of the present invention may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes: a query interface confirmation module, a target business data acquisition module and a display module.

[0097] The names of these modules do not, in some cases, constitute a limitation on the modules themselves. For example, a display module may also be described as a “module for visually displaying the target business data”.

[0098] On the other hand, the present invention also provides a computer-readable medium, which may be included in the device described in the embodiment; or it may exist independently without being assembled into the device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by a device, the device includes: in response to receiving a data display request, parsing a service identifier from the display request, and searching for a data query interface corresponding to the service identifier; according to the data query interface, obtaining target service data from local storage, the service data in the local storage being obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request; and visually displaying the target service data.

[0099] The technical solution according to the embodiment of the present invention has the following advantages or beneficial effects: by responding to receiving a data display request, parsing the business identifier from the display request, and finding the data query interface corresponding to the business identifier; according to the data query interface, obtaining the target business data from the local storage, the business data in the local storage is obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request; the technical solution for visually displaying the target business data realizes the data query interface obtained according to the data display request, obtains the target business data obtained by distributed real-time stream processing calculation based on the node topology relationship, and performs visual display. The embodiment of the present invention performs distributed real-time stream processing based on the node topology relationship, which not only decouples the data from the data source and realizes the visual display of high-performance real-time data, but also has good scalability and flexibility, and improves the efficiency of data display.

[0100] The specific implementation methods described herein do not constitute limitations on the scope of protection of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A method for data presentation, characterized in that: include: In response to receiving a data display request, parsing a service identifier from the display request, and searching for a data query interface corresponding to the service identifier; According to the data query interface, target business data is obtained from local storage, where the business data in the local storage is obtained by distributed real-time stream processing according to the node topology relationship corresponding to the data display request; The target business data is visually displayed.

2. The method according to claim 1, characterized in that Before parsing the service identifier from the display request, the method includes: Parsing the to-be-processed logical nodes and the topological connection relationship between the to-be-processed logical nodes from the display request; According to the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node, a corresponding node topological relationship is searched from a node topological relationship library.

3. The method according to claim 2, characterized in that The method comprises: In the case that the node topology relationship is not found in the node topology relationship library, searching the corresponding basic node topology relationship from the node topology relationship library according to the topological connection relationship between the to-be-processed logical node and the to-be-processed logical node; According to the to-be-processed logical node and the topological connection relationship between the to-be-processed logical nodes, the logical nodes and / or the topological connection relationship in the basic node topological relationship are updated to obtain a node topological relationship corresponding to the data display request.

4. The method according to claim 1, characterized in that Before acquiring the target business data, the method further includes: Pull data from a data source and store the data in a message queue according to a topic. The data exists in the form of a collection. According to the node topology relationship corresponding to the data display request, the messages in each topic are processed using a distributed stream processing method to obtain business data, so that the data query interface can obtain target business data from the business data.

5. The method according to claim 4, characterized in that Pull data from the data source and store the data in the message queue according to the topic, including: According to the preset data identifier, the data to be pulled is screened out from the data source, and the data rule matching is performed on the data to be pulled to obtain matching data; Classify the matching data according to the category information in the data identifier of the matching data, and sort the matching data in each category; The sorted matching data of each category is stored in the corresponding message queue according to the topic corresponding to the category.

6. The method according to claim 5, characterized in that The node topology relationship includes a source logical node, an intermediate logical node and an output logical node; According to the node topology corresponding to the data display request, the messages in each topic are processed using a distributed stream processing method, including: Filtering the data to be processed corresponding to the source logical node from the messages of each topic, performing logical operations on the data to be processed according to the logical processing rules in the source logical node, and obtaining a source logical node result; According to the source logical node result, combined with the topological connection relationship between the source logical node and the intermediate logical node in the node topological relationship, logical operations are performed on the data in the intermediate logical nodes step by step to obtain the intermediate logical node result; According to the intermediate logic node result and in combination with the node topology relationship, a logical operation is performed on the data in the output logic node to obtain business data.

7. The method according to claim 4, characterized in that The method further comprises: Storing the business data in a cache; In response to receiving a historical data display request, target historical business data corresponding to the historical data display request is acquired through the cache.

8. The method according to claim 1, characterized in that: Visually display the target business data, including: The target business data is displayed visually in real time through the window mode of distributed real-time stream processing.

9. A device for displaying data, characterized in that: include: A query interface confirmation module, configured to, in response to receiving a data display request, parse a service identifier from the display request and search for a data query interface corresponding to the service identifier; A target business data acquisition module, used to acquire target business data from local storage according to the data query interface, wherein the business data in the local storage is obtained by performing distributed real-time stream processing according to the node topology relationship corresponding to the data display request; The display module is used to visually display the target business data.

10. A mobile electronic device terminal, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images