Method for automatically pushing index data to index platform and related device

Abstract

The invention discloses a method for automatically pushing index data to an index platform and a related device. The method comprises the following steps: reading data from an index routing table through a timed task; constructing a query statement according to the read data; establishing a connection with a database where the index data is located by using a database connection pool technology; the query statement is executed, the index data is queried from the database where the index data is located, the method and the related device can improve the flexibility and the real-time performance of index data pushing, and the cost is low.

Description

Technical Field

[0001] The present invention belongs to the field of data processing and information technology, and relates to a method and related devices for automatically pushing indicator data to an indicator platform. Background Art

[0002] In today's digital age, businesses and organizations are accumulating vast amounts of data. To effectively utilize this data, specific metrics need to be extracted and pushed to a metrics platform for analysis, display, and decision support. While several data push methods exist, most suffer from the following issues:

[0003] Lack of flexibility: Traditional data push methods are typically custom-developed for specific business scenarios or systems, making them difficult to adapt to changing business needs and the addition of new metrics. Changes to business logic or data structures require redevelopment and redeployment of related programs.

[0004] High maintenance costs: Manually configuring query statements, database connections, and other information during data push is error-prone and difficult to manage. When multiple data sources and metrics are involved, maintenance becomes tedious and prone to confusion.

[0005] Insufficient real-time performance: Some existing data push mechanisms cannot meet the needs of real-time or quasi-real-time indicator data updates, and cannot provide the latest data support for decision-making in a timely manner. Summary of the Invention

[0006] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a method and related devices for automatically pushing indicator data to an indicator platform. This method and related devices can improve the flexibility and real-time performance of indicator data pushing and are relatively low in cost.

[0007] To achieve the above object, the present invention discloses a method for automatically pushing indicator data to an indicator platform, comprising:

[0008] Read data from the indicator routing table through a scheduled task;

[0009] Build query statements based on the read data;

[0010] Use database connection pool technology to create a connection to the database where the indicator data is located;

[0011] Execute the query statement to query the indicator data from the database where the indicator data is located.

[0012] The method of automatically pushing indicator data to the indicator platform according to the present invention is further improved in that:

[0013] Furthermore, before reading data from the indicator routing table through the scheduled task, the process also includes:

[0014] Construct an indicator routing table, in which the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition.

[0015] Furthermore, the scheduled task is a cron expression of Linux or a scheduled task of Windows.

[0016] Furthermore, after executing the query statement and searching the database where the indicator data is located, the method further includes:

[0017] According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform through the communication protocol, and the communication protocol is HTTP, HTTPS or message queue.

[0018] The present invention discloses a system for automatically pushing indicator data to an indicator platform, comprising:

[0019] The reading module is used to read data from the indicator routing table through a scheduled task;

[0020] Query module, used to construct query statements based on the read data;

[0021] Create a module for establishing a connection to the database where the indicator data is located using database connection pool technology;

[0022] The acquisition module is used to execute a query statement to query the indicator data from the database where the indicator data is located.

[0023] Further improvements to the system for automatically pushing indicator data to the indicator platform of the present invention are:

[0024] Furthermore, before reading data from the indicator routing table through the scheduled task, the process also includes:

[0025] Construct an indicator routing table, in which the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition.

[0026] Furthermore, the scheduled task is a cron expression of Linux or a scheduled task of Windows.

[0027] Furthermore, after executing the query statement and searching the database where the indicator data is located, the method further includes:

[0028] According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform through the communication protocol, and the communication protocol is HTTP, HTTPS or message queue.

[0029] The present invention discloses a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for automatically pushing indicator data to an indicator platform are implemented.

[0030] The present invention discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for automatically pushing indicator data to an indicator platform are implemented.

[0031] The present invention has the following beneficial effects:

[0032] During specific operations, the method and related devices for automatically pushing indicator data to the indicator platform described in the present invention can add or modify data indicators through the indicator routing table to improve the flexibility of indicator data acquisition. At the same time, by reading data from the indicator routing table through scheduled tasks, the workload and error probability of manual configuration are reduced, and the cost is low. In addition, real-time or quasi-real-time push of indicator data can be achieved, providing timely data support for decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0034] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] In the description of the present invention, it is to be understood that the terms “include” and “comprise” indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0037] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0038] It should be further understood that the term "and / or" as used in the present specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0039] It should be understood that although the terms "first," "second," and "third" may be used to describe preset ranges in embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are merely used to distinguish one preset range from another. For example, without departing from the scope of embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0040] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] The accompanying drawings illustrate various schematic diagrams of structures according to embodiments disclosed herein. These figures are not drawn to scale; for clarity, some details are exaggerated and some details may be omitted. The shapes of the various regions and layers shown in the figures, as well as their relative sizes and positional relationships, are merely exemplary and may deviate in practice due to manufacturing tolerances or technical limitations. Those skilled in the art may design regions / layers with different shapes, sizes, and relative positions as needed.

[0043] The method for automatically pushing indicator data to the indicator platform of the present invention comprises the following steps:

[0044] 1) Constructing an indicator routing table; the indicator routing table stores the indicator code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition;

[0045] Indicator shortcode: used to uniquely identify each indicator, making it easier to identify and manage throughout the push process.

[0046] Database where indicator data is stored: Specify the name of the database where indicator data is stored. Multiple types of databases are supported, such as relational databases (MySQL, Oracle, etc.) and non-relational databases (MongoDB, etc.).

[0047] Data table: The name of the data table that specifically stores indicator data, accurate to the table level.

[0048] Data field: The specific field corresponding to the indicator data, which can support combined query of single or multiple fields.

[0049] Query conditions: Set the conditions for filtering and extracting indicator data, such as time range, specific attribute values, etc., which can be expressed using SQL statements or other suitable query syntax.

[0050] 2) Scheduled task mechanism;

[0051] Utilize timed task scheduling tools, such as Linux's cron expression and Windows's scheduled tasks, to execute timed tasks at preset time intervals. The function of the timed tasks is to read the data in the indicator routing table at the specified time.

[0052] 3) Query statement assembly;

[0053] The scheduled task reads data from the indicator routing table and automatically assembles the read data into a corresponding query statement based on the database type, data table, data field, and query conditions stored in the indicator routing table. For example, for a MySQL database, it assembles a standard SELECT statement, including the specified fields, the data table in the FROM clause, and the query conditions in the WHERE clause.

[0054] 4) Database connection and data query;

[0055] Use database connection pool technology (such as C3P0, Druid, etc.) to create a connection with the database where the indicator data is located, execute the assembled query statement through the established connection, and query the indicator data that meets the conditions from the database.

[0056] 5) Data push;

[0057] According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform. The push process uses a reliable communication protocol, such as HTTP, HTTPS or message queue (RabbitMQ, Kafka, etc.) to ensure that the data reaches the indicator platform accurately.

[0058] It should be noted that the present invention has the following characteristics:

[0059] Improved flexibility: Through the design of the indicator routing table, the present invention allows users to add or modify corresponding records in the routing indicator routing table when business requirements change or new indicators are added, without having to modify a large amount of code, which greatly improves the system's adaptability to business changes.

[0060] Reduce maintenance costs: Automatically assemble query statements and manage database connections, reducing manual configuration workload and the likelihood of errors. Centrally manage indicator routing tables, making data push information readily available and facilitating maintenance and troubleshooting.

[0061] Enhanced real-time performance: With the help of the scheduled task mechanism, the time interval for data push can be flexibly set according to business needs, realizing real-time or quasi-real-time push of indicator data, providing timely data support for decision-making.

[0062] Example 2

[0063] The system for automatically pushing indicator data to the indicator platform of the present invention includes:

[0064] The reading module is used to read data from the indicator routing table through a scheduled task;

[0065] Query module, used to construct query statements based on the read data;

[0066] Create a module for establishing a connection to the database where the indicator data is located using database connection pool technology;

[0067] The acquisition module is used to execute a query statement to query the indicator data from the database where the indicator data is located.

[0068] In this embodiment, before reading data from the indicator routing table through the scheduled task, the following steps are also included:

[0069] Construct an indicator routing table, in which the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition.

[0070] In this embodiment, the scheduled task is a cron expression of Linux or a scheduled task of Windows.

[0071] In this embodiment, after executing the query statement to query the indicator data from the database where the indicator data is located, the following steps are further performed:

[0072] According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform through the communication protocol, and the communication protocol is HTTP, HTTPS or message queue.

[0073] The division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0074] Example 3

[0075] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for automatically pushing the indicator data to the indicator platform are implemented. For example, the steps include: reading data from the indicator routing table through a scheduled task; constructing a query statement based on the read data; creating a connection to the database where the indicator data is located using database connection pool technology; and executing the query statement to query the indicator data from the database where the indicator data is located. Before the reading of data from the indicator routing table through the scheduled task, the method also includes: constructing an indicator routing table, wherein the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field, and the query condition. After the execution of the query statement to query the indicator data from the database where the indicator data is located, the method also includes: pushing the indicator data obtained by the query to the corresponding indicator platform through a communication protocol according to the indicator short code in the indicator routing table, wherein the communication protocol is HTTP, HTTPS, or a message queue. The memory may include internal memory, such as high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus. This internal bus may be an Industry Standard Architecture bus, a Peripheral Component Interconnect Standard bus, an Extended Industry Standard Architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0076] Example 4

[0077] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a method for automatically pushing indicator data to an indicator platform. The steps include: reading data from an indicator routing table via a scheduled task; constructing a query statement based on the read data; establishing a connection to the database where the indicator data resides using database connection pool technology; and executing the query statement to query the indicator data from the database where the indicator data resides. Prior to reading data from the indicator routing table via the scheduled task, the method further includes: constructing an indicator routing table that stores an indicator short code, the name of the database where the indicator data resides, the name of the data table, data fields, and query conditions. After executing the query statement to query the indicator data from the database where the indicator data resides, the method further includes: pushing the obtained indicator data to the corresponding indicator platform via a communication protocol, such as HTTP, HTTPS, or a message queue, based on the indicator short code in the indicator routing table. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include a read-only memory (ROM), a hard disk, a flash memory, an optical disk, a magnetic disk, and the like.

[0078] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0079] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0082] Those skilled in the art will readily identify other embodiments of the present invention after considering the specification and disclosure of the invention. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

[0083] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

[0084] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any simple modification, change and equivalent structural change made to the above embodiment based on the technical essence of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for automatically pushing indicator data to an indicator platform, characterized in that: include: Read data from the indicator routing table through a scheduled task; Build query statements based on the read data; Use database connection pool technology to create a connection to the database where the indicator data is located; Execute the query statement to query the indicator data from the database where the indicator data is located.

2. The method for automatically pushing indicator data to the indicator platform according to claim 1, characterized in that: Before reading data from the indicator routing table through the scheduled task, the following steps are also included: Construct an indicator routing table, in which the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition.

3. The method for automatically pushing indicator data to the indicator platform according to claim 1, characterized in that: The scheduled task is a Linux cron expression or a Windows scheduled task.

4. The method for automatically pushing indicator data to an indicator platform according to claim 1, characterized in that: After executing the query statement and searching the database where the indicator data is located, the method further includes: According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform through the communication protocol, and the communication protocol is HTTP, HTTPS or message queue.

5. A system for automatically pushing indicator data to an indicator platform, characterized in that: include: The reading module is used to read data from the indicator routing table through a scheduled task; Query module, used to construct query statements based on the read data; Create a module for establishing a connection to the database where the indicator data is located using database connection pool technology; The acquisition module is used to execute a query statement to query the indicator data from the database where the indicator data is located.

6. The system for automatically pushing indicator data to an indicator platform according to claim 5, characterized in that: Before reading data from the indicator routing table through the scheduled task, the following steps are also included: Construct an indicator routing table, in which the indicator routing table stores the indicator short code, the name of the database where the indicator data is located, the name of the data table, the data field and the query condition.

7. The system for automatically pushing indicator data to an indicator platform according to claim 5, characterized in that: The scheduled task is a Linux cron expression or a Windows scheduled task.

8. The system for automatically pushing indicator data to an indicator platform according to claim 5, characterized in that: After executing the query statement and searching the database where the indicator data is located, the method further includes: According to the indicator short code in the indicator routing table, the indicator data obtained by the query is pushed to the corresponding indicator platform through the communication protocol, and the communication protocol is HTTP, HTTPS or message queue.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for automatically pushing indicator data to the indicator platform as described in any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically pushing indicator data to an indicator platform as described in any one of claims 1 to 4 are implemented.

Images