An expression-based data processing system

Abstract

The present invention provides a data processing system based on expressions, which includes: a data connection management unit connected to a database, a data source template unit, a data source management unit, a time granularity management unit, and an expression parsing unit respectively. The expression parsing unit obtains expression data through a data access interface component, so that the expression parsing unit can parse the expression according to a preset expression specification, and find corresponding configuration item tables and data tables from the data source template unit and the data source management unit according to the time granularity set by the time granularity management unit, and perform at least one operation including reading, writing back, and aggregation, so as to realize unified access to heterogeneous data sources and flexible data definition functions.

Description

Technical Field

[0001] The present invention relates to data processing technologies, and particularly to a technology for processing data sources based on expressions. Background Art

[0002] In enterprise applications, there are a large number of discrete data calculation and statistical scenarios. For example, in enterprise energy management statistics, it is necessary to flexibly increase or decrease the types of energy media, and perform deviation analysis and verification based on various scenarios such as plans, year-on-year comparisons, month-on-month comparisons, and custom periods to generate statistical reports.

[0003] Traditional reports and data applications usually rely on predefined data themes. It is very difficult to perform calculations and presentations between heterogeneous data, and different time granularity parameters, data backfilling, and other scenarios also need to be considered separately. Moreover, these data calculation logics are scattered in software business logics, SQL statements, and report logics, so it is difficult to flexibly apply to various complex data scenarios. At the same time, it is even more difficult to perform mathematical calculations between heterogeneous data and batch backfill the calculation results into the database. Therefore, there is an urgent need for a solution in this technical field that can solve such data processing problems. Summary of the Invention

[0004] The main objective of the present invention is to provide an expression-based data processing system to achieve unified access to heterogeneous data sources and flexible data definition functions.

[0005] To achieve the above objective, the present invention provides an expression-based data processing system, which includes: a data connection management unit connected to a database, a data source template unit, a data source management unit, a time granularity management unit, and an expression parsing unit respectively. The expression parsing unit obtains expression data through a data access interface component, so that the expression parsing unit can parse the expression according to a preset expression specification, and according to the time granularity set by the time granularity management unit, look up corresponding configuration item tables and data tables in the data source template unit and the data source management unit, and perform at least one operation including reading, writing back, and aggregation;

[0006] The preset expression specification includes: project code, data source code, field name, time offset, and aggregation function parameters. The project code represents the project code of the configuration item table, the data source code stipulates the corresponding configuration item table and data table of the data, the field name corresponds to the column name actually existing in the data source mapping data table, the time offset identifies the time granularity offset within the system, and the aggregation function represents an aggregation function recognizable by the database, which is used to perform an aggregation operation on the data that meets the conditions;

[0007] The processing of the expression parsing includes:

[0008] Read / write discrete data: find the field of the corresponding data table according to the expression, and then return the data or update the field;

[0009] Batch data reading: Combine the data sources, time granularity, and time expressions with the same combination into the same query statement and read them at one time;

[0010] Data statistics: parse the corresponding fields, aggregation keywords, etc. according to the expression to generate statistical query statements;

[0011] Expression calculation: When multiple expressions are connected through the four arithmetic operators, the expression is extracted, the value is replaced after querying the data, and then the operation is performed in combination with the operator.

[0012] Time offset processing: According to the time granularity and the time offset expression, the switching, forward shifting and backward shifting transformation processing on the time granularity are performed.

[0013] Preferably, the time granularity management unit is used to define different settlement cycles, including at least one of annual, monthly, daily, shift, ten-day, and weekly, wherein all data managed by the data source management unit are linked to the time granularity and time.

[0014] Preferably, the data source template unit stores a data source template for mapping the configuration item table corresponding to the data based on the data connection, wherein the management fields of the data source template include: data source template code, data source template name, data connection, configuration table, whether it is a writable data source, and all necessary field information for registering the corresponding data table, including field name, Chinese name, type, length, precision, etc., and at least one of the primary key generation methods.

[0015] Preferably, the data source management unit stores data sources used to map the relationship with the configuration project table. Under the same data source template, multiple sets of data sources can be defined to realize different sets of data accounts. At the same time, the items corresponding to the configuration table can be included in the corresponding data source. The main fields of the data source management include: data source code and data source name.

[0016] Preferably, the data source management unit is associated with the time granularity management unit to define multiple time granularity data under the same data source, and each association needs to specify the data table, time granularity, primary key generation method, and primary key supplementary fields.

[0017] Preferably, the read operation steps include: forming a query condition based on the item code of the expression, in combination with the time granularity and time field, to generate an SQL query statement, obtaining a database connection from the data connection management unit, executing the SQL to perform data search, and performing data value extraction according to the field names in the returned result set, substituting the numerical values into the original input string for replacement, and performing mathematical operations on the numerical string after all replacements are completed to obtain the final calculation result.

[0018] Preferably, the write-back operation steps include: forming an UPDATE or INSERT statement based on the relationship between the field name and the written numerical value, in combination with the query condition, to perform field update of the data row. When writing back, it is also necessary to rely on the field definition information on the data source template to perform type processing and verification on the written field values.

[0019] Preferably, the aggregation operation steps include: calculating the data according to the time granularity, in combination with the time offset given in the expression and the aggregation function.

[0020] Through the data processing system based on expressions provided by the present invention, it is possible to conveniently implement a data source management solution for building a database to complete unified data reading, calculation, and storage. It is particularly suitable for scenarios such as discrete, multi-time granularity, heterogeneous data calculation, and data backfilling, greatly ignoring the complexity of calculation and statistics, enabling developers to handle complex data application scenarios without writing SQL or business logic and in a code-free and configuration-based manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings forming a part of this application are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0022] Figure 1 is a schematic diagram of the functional modules of the data processing system based on expressions of the present invention;

[0023] Figure 2 is a configuration diagram of the data processing system based on expressions of the present invention;

[0024] Figure 3 is a flowchart of the expression parsing of the data processing system based on expressions of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] To enable those skilled in the art to better understand the technical solution of the present invention, the following will, in conjunction with embodiments, clearly and completely describe the specific technical solution of the present invention to assist those skilled in the art in further understanding the present invention. Obviously, the embodiments described in this case are only a part of the embodiments of the present invention, rather than all the embodiments. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention and without conflict with each other, the embodiments and the features in the embodiments in this application can be combined with each other. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of disclosure and protection of the present invention.

[0026] In addition, the terms "first", "second", "S1", "S2", etc. in the specification, claims and drawings of the present invention are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order other than those described here. At the same time, the terms "including" and "having" in the present invention and any variations thereof are intended to cover non-exclusive inclusion. For those of ordinary skill in the art, the specific meanings of the above terms in this case can be understood in combination with the prior art according to specific circumstances.

[0027] In view of the large number of data utilization scenarios with complex design, such as time granularity conversion, multiple accounting sets, complex comparison, and frequent changes in caliber, etc. that occur in enterprise business operations, the inventive concept of this case proposes the concept of data source management oriented to time granularity to meet the needs of enterprises for data calculation, filling, statistics, comparison, etc. between heterogeneous data sources in business execution.

[0028] Among them, the data source management of this time granularity proposed in this case is characterized in that all the operation data of the enterprise is divided into countable time granularities such as year, quarter, ten-day period, month, week, day, shift, etc., and according to the data requirements of multiple accounting sets such as the actual production management, plan, balance, tracking, etc. of the enterprise, the field definition and declaration of the data table structure are realized.

[0029] For this reason, as Figures 1 to 3 shown, the data processing system based on expressions of the present invention mainly includes: a data connection management unit, a data source template unit, a data source management unit, a time granularity management unit, and components such as an expression parsing unit and a data access interface. The basic principle is to realize data mapping (fields) by expanding one or more sets of data accounting sets (data tables) for monitoring items (configuration tables) with the same configuration based on the benchmark time granularity and time scenarios.

[0030] Specifically, the time granularity management unit is used to define different settlement cycles in an enterprise, such as annual, monthly, daily, shift, dekad, weekly, etc. All data managed by the data source must be associated with the time granularity and time. That is, the data table must include two fields: TIMEGRANID and CLOCK. By managing the time granularity, the process of continuously summarizing enterprise data from small cycles (such as shifts, days, etc.) to larger cycles (such as months, years, etc.) can be achieved, solving the problem of automated data flow in enterprise data accounts.

[0031] The data connection management unit is used to manage different database connections, mainly including database driver cases, connection string maintenance, etc., for connecting different database table information. The data connection can be based on specific development languages and operating environments, targeting mainstream relational databases that can meet the ANSI SQL-2003 standard.

[0032] The data source template unit, whose data source template is used to map the data configuration item table on the basis of the data connection. The management fields of the data source template include: data source template code, data source template name, data connection, configuration table (or view), whether it is a writable data source, and register all necessary field information of the corresponding data table (including field name, Chinese name, type, length, precision, etc., primary key generation method).

[0033] Among them, the data source template is a declaration of various heterogeneous data sources for stored data items, manages database connection information, data table information, data table field information, stored project information, etc., belongs to the metadata of the mapping relationship, and solves the problem of different sources, various formats, complex calculation methods but yet to be presented or calculated simultaneously in data management.

[0034] The data source management unit, whose data source is used to map the relationship with the configuration item table (data source template). Under the same data source template, multiple sets of data sources can be defined to achieve different accounting sets of data, and at the same time, the items corresponding to the configuration table can be incorporated into the corresponding data source. The main fields of data source management include: data source code (data source template code + 2-digit data source category code), data source name, etc. Typical scenarios include the actual performance and planned data of an enterprise. Although the management scope (number of items) is different, they are comparable.

[0035] Among them, the data source mentioned in this case is associated with the time granularity, which means that under the same data source, multiple time granularity data can be defined. For each association, it is necessary to specify the data table, time granularity, primary key generation method, primary key supplementary field, etc. The primary key generation methods include: GUID, SEQUENCE, UUID, no generation required. Combining the primary key supplementary field corresponds to different generation logics, and can also be extended through programming. The application of the association is commonly used to meet the enterprise's need to define data in multiple calibers, such as different scenarios like issuing annual reports, monthly reports, weekly reports, etc.

[0036] An expression parsing unit, which has a set of specifications for expressions, such as self-describing data fields, time offsets, aggregations, etc. in the expression, and can ignore the complexity of the underlying data storage. The main composition of the expression is a variable-length segmented string, text separated by the '.' symbol, and the paradigm is as follows:

[0037] [{ItemId}.{SourceId}.{FieldName}.{Timeoffset}.{Aggragation}]

[0038] In the example, its components are as follows:

[0039] Parameter Name Parameter Description Parameter Type Is Mandatory Remarks ItemId Project Code String Yes Represents the project code of each configuration item SourceId Data Source Code String Yes Uppercase string. From the data source code. The data source specifies the corresponding configuration table and data table for the data FieldName Field Name String Yes Uppercase string. The field name should be the actual column name in the data table mapped by the data source Timeoffset Time Offset String Optional Identifies the time granularity offset within the system. The time granularity offset is achieved through the method of "X±N", the specified time granularity can be jumped to through the method of "@TimegranId", and the forward or backward movement based on the current time can be achieved through the method of "#±N" (regardless of whether the current is day or month, slide forward and backward according to the current granularity) Aggragation Aggregation Function String Optional Uppercase string. The value range includes MIN, MAX, SUM, AVG, etc., representing the aggregation functions recognizable by the database, used to perform aggregation operations on the data that meets the conditions

[0040] Among them, the expression requires that the time granularity and time must be provided as parameters in the context, and based on this, SQL access to the data source table is performed. The expression needs to use the "[", "]" symbols as prefixes and suffixes, and multiple expressions can be connected by four arithmetic symbols to form a calculation expression.

[0041] The first three segments of the expression are mandatory. Combined with the expression parsing system, it is finally used to locate the specified row and column of the data source table. The fourth segment of the expression is the time offset, which can jump from the current period to other periods, such as querying monthly data in a daily report. If there is a fifth segment, the fourth segment can also cooperate with the basic granularity provided for aggregation operations, such as querying the daily average value in a monthly report.

[0042] For example: The main expression methods for time granularity offset are in the form of "X±N@TimeGranId" or "#±N".

[0043] Among them, the value range of X is {yy, MM, dd, HH, mm, ss}, representing year, month, day, hour, minute, and second respectively (note case sensitivity). N is an integer and generally not equal to 0. It means to calculate N periods forward or backward from the current time granularity and time. When offsetting time, the middle time of the time series range corresponding to the current time granularity is used as the starting quantity. TimeGranId is used to jump to the specified period.

[0044] Among them, the offsets supported by the example in this embodiment mainly include:

[0045] Time granularity switching: In the way of @TimeGranId, the time granularity can be switched to another one. For example, when the current context time granularity is day and the time is 2021-09-21, this expression can obtain the current month 2021-09.

[0046] Time offset: There are two expressions for time offset:

[0047] 1) The expression "X±N" can be used to represent N similar periods forward / backward from the current period;

[0048] 2) You can use the expression "#±N" to represent the translation according to the current time granularity. For example, when the current context granularity is day and the time is 2021-09-21, the time after parsing of the expression "MM-1" is 2021-08-21; the time after parsing of the expression "#-12" is 2020-08-21. Here, you need to handle the validity of the time, especially the date, and when an unreasonable time occurs, you need to catch the exception.

[0049] The aggregation part of the expression can include common aggregation methods of common databases, such as MIN, MAX, SUM, COUNT, etc., or it can be a packaged unique aggregation method, which can be expanded here.

[0050] In addition, in the preferred implementation, the pre-declared data source category, time granularity offset, aggregation operation and other expressions can be segmented into text, combined with data source management, and converted into executable SQL statements. This mainly includes expression parsing and SQL parsing and execution of data reading, calculation, and writing back.

[0051] The processing of expression parsing includes:

[0052] Discrete data read / write: Find the field of the corresponding data table according to the expression, and then return the data or update the field. If it is a report or page field, the same expression can realize both read and write functions; the write expression can only be a simple expression, that is, it only includes the first three sections and cannot contain aggregation, time offset, etc.

[0053] Batch data reading: Combine data sources, time granularity, and time expressions with the same combination into the same query statement and read them all at once to speed up processing.

[0054] Data statistics: Generate statistical query statements based on expression parsing of corresponding fields, aggregation keywords, etc.

[0055] Expression calculation: When multiple expressions are connected through the four arithmetic operators, you can extract the expression, replace the value after querying the data, and then perform calculations with the operators to get the final result.

[0056] Time offset processing: In the processing of time granularity, if it is necessary to switch, move forward, move backward, etc. on the time granularity, it is necessary to combine the time offset expression for processing.

[0057] The main steps of expression parsing are:

[0058] a. Parse the input string and perform regular matching according to the expression rules.

[0059] b. Find all expressions in the string that meet the requirements and perform expression analysis.

[0060] c. Group and process concurrently through the SourceId of the expression set, and find the data table corresponding to the data source and the configuration item table corresponding to the data source template respectively.

[0061] d. Process in multiple cases:

[0062] Read: According to the ItemId of the expression, combine the time granularity TimeGranId and the time Clock field in the context to form a query condition to generate an SQL query statement, obtain a database connection from the connection management, execute the SQL to perform data search. Take data values according to the FieldName in the returned result set, substitute the values into the original input string for replacement, and perform mathematical operations on the numerical string after all replacements are completed to obtain the final calculation result.

[0063] Write back: The write expression only supports three-section expressions. It is necessary to form an UPDATE or INSERT statement according to the relationship between the FieldName and the written value, and combine the query conditions to perform field updates on the data rows. When writing back, it is also necessary to rely on the field definition information on the data source template to perform type processing and verification on the written field values.

[0064] Aggregate: According to the time granularity in the context (such as monthly), combine the time offset given in the expression (such as @DAY) and the aggregation field Aggregation to perform aggregation and cumulative processing on the data (such as SUM, MIN, MAX, etc.).

[0065] For example, in the scenario of parsing example formula expressions:

[0066] Taking the current time granularity as MONTH and the time as 2021-08, the process of parsing the expression:

[0067] [FA0024002.EIPL.ITEMVALUE.@YEAR.SUM] is as follows:

[0068] Expression parsing: The strings in the expression item are separated by the delimiter., and are recognized as different sections.

[0069] Read data source configuration: Find the data source configuration according to Section 2 (EIPL), and append the configuration information to the expression project information. Among them, the first two characters "EI" represent the data source template code, and "EIPL" represents the data source code. Combining this configuration, the data connection, storage table, and other database description information of this expression project can be found. For write requests, it is necessary to perform verification here by combining the configuration information of the data source template (such as field length, precision, etc.), filter out invalid requests, and record errors.

[0070] Process time offset: According to Section 4 (@YEAR), process the time offset. @YEAR is specified to take the year (2021) of the current time (MONTH, 2021-08). The time conversion is based on the relationship between sequences in the time granularity management.

[0071] Batch process by data source: If there are multiple expressions, group and process them according to the data source information parsed above.

[0072] Generate read / write SQL: For the same data source:

[0073] 1. If there is no aggregation expression, group and batch process according to the time granularity. Set the query conditions of each expression to the time granularity (TIMEGRANID), time (CLOCK), data source code (SOURCEID), and project ID list (ITEMID). The query field is the value of Section 3 of each project (such as ITEMVALUE), and the data table is the data table in the data source configuration, then the read SQL can be generated.

[0074] 2. When there is an aggregation expression, first combine the value of Section 5 to form a query field set (MAX(ITEMVALUE)). The data table rule remains unchanged. Then, it is necessary to add a grouping SQL (GROUP BY ITEMID,TIMEGRANID,SOURCEID), and it is necessary to determine the time granularity grouping of the expression to determine the writing method of the CLOCK condition in the query condition. If the time granularity of the expression is the same as the context time granularity, no time condition is added; if the time granularity weight of the expression is higher (such as year relative to month), it is necessary to find the start (2021-01) and current (2021-08) of the context time granularity it contains as the time condition (BETWEEN :START AND :END), and the time granularity is the context time granularity (MONTH); if the time granularity weight of the expression is smaller (such as day relative to month), it is necessary to find the start (2021-08-01) and end (2021-08-31) of the current granularity in the context time granularity it belongs to, and the time granularity is the time granularity of the expression (DAY).

[0075] 3. For writing SQL, it is necessary to combine the data source information to determine whether a record exists through the unique key (TIMEGRANID, CLOCK, SOURCEID, ITEMID). If the record exists, an UPDATE statement is generated. Otherwise, an INSERT statement is generated.

[0076] Obtain the data source connection configuration: Through the data source grouping of the expression, obtain the connection information of the data source. Combine the data connection management function to connect to the database.

[0077] Execute SQL: According to the database connection established above, execute SQL, map the returned result set to the expression items, and finally output to the requester.

[0078] On the other hand, the present invention also includes the basic definition of the data access interface of the expression, which is used to implement the data acquisition and saving of the expression. The typical data access interface definition is as follows.

[0079] IDictionary<string,object> QueryData(string timeGranId, string clock,List <string>formulaIds);

[0080] The data fetching interface makes a call to the expression system for data access by passing in the time granularity and time parameters and combining them with expressions, and finally returns a key-value pair composed of the expressions. No other parameters need to be passed in during this process.

[0081] void WriteData(string timeGranId, string clock,IDictionary<string,object> values);

[0082] The data write-back interface writes the values into the tables and fields set by the data source management corresponding to each expression according to the key-value pair composed of the time granularity, time parameters, expressions, and write-back values. No other parameters are required during the write-back process, and users can completely ignore the complexity brought by heterogeneous data sources, etc.

[0083] Since expressions can have rich combination methods, this interface can solve various complex data query scenario problems without adjusting the backend. Through data source management, seamless business expansion can be achieved, which is very suitable for the data statistics and analysis business scenarios of enterprises.

[0084] An example of the comprehensive application scenario is as follows:

[0085] Energy is measured by energy items, such as the oxygen consumption of blast furnaces and the power generation of power plants. When enterprises manage, they formulate energy plans according to the annual and monthly granularities for the main processes to guide daily production. The production performance includes data at shifts, days, months, and years, covering data such as the factory department, processes, units, and equipment of the enterprise.

[0086] The basic configuration part includes:

[0087] 1. Data tables to be connected: Energy item configuration table ENERGYITEMCONFIG, Energy data table (the same data table exists for each granularity and accounting set) ENERGYITEMDATA.

[0088] The main fields and example data of the energy configuration table include:

[0089] ITEMID ITEMNAME UNITNAME GL0011001 Oxygen Consumption of Blast Furnace Cubic Meter GL0111001 Oxygen Consumption of No. 1 Blast Furnace Cubic Meter GL0111002 Oxygen Consumption of No. 2 Blast Furnace Cubic Meter FA0024002 Power Generation of Power Plant Kilowatt-hour

[0090] The main fields and example data of the energy data table include:

[0091] RECID ITEMID ITEMNAME UNITNAME SOURCEID TIMEGRANID CLOCK ITEMVALUE STDVALUE 2392042 GL0011001 Oxygen Consumption of Blast Furnace Cubic Meter EIRL MONTH 2021-08 22243.235 201232.578 2392043 GL0111001 Oxygen Consumption of No. 1 Blast Furnace Cubic Meter EIRL MONTH 2021-08 22243.235 201232.578 2192043 FA0024002 Power Generation of Power Plant Kilowatt-hour EIPL DAY 2021-01-23 786542 12386542

[0092] 2. Definition of time granularity: Define the shift, day, month, and year time granularities for the enterprise, with the corresponding codes being SHIFT, DAY, MONTH, and YEAR respectively. Time offset can also be achieved through expressions in the time granularity.

[0093] 3. Data source template configuration: Configure the data source template for energy items, and define the data source template code as EI.

[0094] 4. Data source configuration: Configure two sets of data sources for plans (EIPL) and actuals (EIRL).

[0095] 5. Association between data source and time granularity: The planned energy item data corresponds to the time granularities of month and year; the actual energy item data is for shift, day, month, and year.

[0096] Typical application scenarios:

[0097] When there are multiple requirements on the same report / form, such as reading, writing, calculating, aggregating, time offset, etc. as described above, the platform can seamlessly support them. Only correct expressions need to be written, and the same data access entry can be used without any additional front-end or back-end programming.

[0098]

[0099] In summary, the data processing system based on expressions provided by the present invention realizes the mapping with database tables through expressions, enabling code-free data access for data that meets the specifications. All data sources that meet the expression specifications can achieve undifferentiated reading, writing back, and operation between expressions. At the same time, after the discrete expressions are processed by the parsing program and then grouped and aggregated, efficient batch reading and operation of data can be achieved.

[0100] Secondly, the expressions of the present invention can fully integrate various data processing operations. The main data processing operations supported include: data reading, data calculation, data aggregation, multi-accounting set support, time series and offset processing, etc. Through the time offset convention of expressions, data access or statistics at different time granularities can be achieved, such as cumulative and period aggregation; through the data source code convention of expressions, seamless expansion of multiple accounting set data such as enterprise actuals, plans, and final accounts can be achieved; through the aggregation convention of the present expressions, data statistics and aggregation can be achieved, realizing data aggregation operations such as cumulative, average, maximum value, minimum value, and other extensible data aggregation operations.

[0101] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can well understand and utilize the present invention. The present invention is only limited by the claims and their full scope and equivalents. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

[0102] Those skilled in the art can understand that in addition to implementing the systems, devices, and their respective modules provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the systems, devices, and their respective modules provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same program. Therefore, the systems, devices, and their respective modules provided by the present invention can be regarded as a kind of hardware component, and the modules included therein for implementing various programs can also be regarded as the structures within the hardware component; the modules for implementing various functions can also be regarded as either software programs for implementing the methods or the structures within the hardware component.

[0103] In addition, all or part of the steps in the methods of the above embodiments can be completed by a program instructing relevant hardware. The program is stored in a storage medium, including several instructions for enabling a single-chip microcomputer, a chip, or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical disks, etc., which can store program codes.

[0104] In addition, any combination can be made between various different embodiments of the embodiments of the present invention, as long as it does not violate the idea of the embodiments of the present invention, and it should also be regarded as the content disclosed in the embodiments of the present invention.< / string>

Claims

1. An expression-based data processing system, Features include: The data connection management unit is respectively connected with the database, the data source template unit, the data source management unit, the time granularity management unit, and the expression parsing unit, wherein the expression parsing unit obtains expression data through the data access interface component, so that the expression parsing unit can parse the expression according to the preset expression specification, and search the corresponding configuration project table and data table from the data source template unit and the data source management unit according to the time granularity set by the time granularity management unit, and perform at least one operation including reading, writing back, and aggregation; The preset expression specification includes: project code, data source code, field name, time offset, and aggregation function parameters, wherein the project code represents the project code of the configuration project table, the data source code stipulates the configuration project table and data table corresponding to the data, the field name corresponds to the actual column name in the data source mapping data table, the time offset identifies the time granularity offset in the system, and the aggregation function represents the aggregation function recognizable by the database, which is used to perform aggregation operations on data that meets the conditions; The expression parsing process includes: Read / write discrete data: find the field of the corresponding data table according to the expression, and then return the data or update the field; Batch data reading: Combine the data sources, time granularity, and time expressions with the same combination into the same query statement and read them at one time; Data statistics: parse the corresponding fields and aggregate keywords based on the expression to generate statistical query statements; Expression calculation: When multiple expressions are connected through the four arithmetic operators, the expression is extracted, the value is replaced after querying the data, and then the operation is performed in combination with the operator. Time offset processing: According to the time granularity and the time offset expression, the switching, forward shifting and backward shifting transformation processing on the time granularity are performed.

2. The expression-based data processing system according to claim 1, It is characterized in that The time granularity management unit is used to define different settlement cycles, including at least one of annual, monthly, daily, shift, ten-day, and weekly, wherein all data managed by the data source management unit are linked to the time granularity and time.

3. The expression-based data processing system according to claim 2, It is characterized in that The data source template unit stores a data source template for mapping the configuration item table corresponding to the data based on the data connection, wherein the management fields of the data source template include: data source template code, data source template name, data connection, configuration table, whether it is a writable data source, and all necessary field information for registering the corresponding data table, including at least one of the field name, Chinese name, type, length, precision, and primary key generation method.

4. The expression-based data processing system according to claim 3, It is characterized in that The data source management unit stores data sources that are used to map the relationships with the configuration item tables. Under the same data source template, multiple sets of data sources are defined to implement different accounting sets of data. At the same time, the items corresponding to the configuration tables are incorporated into the corresponding data sources. The main fields of the data source management include: data source code and data source name.

5. The expression-based data processing system according to claim 4, wherein, the data source management unit is associated with the time granularity management unit to define multiple time granularity data under the same data source. For each association, it is necessary to specify the data table, time granularity, primary key generation method, and primary key supplementary field.

6. The expression-based data processing system according to claim 1, wherein, the read operation steps include: forming a query condition based on the item code of the expression, combining the time granularity and time field to generate an SQL query statement, obtaining a database connection from the data connection management unit, executing the SQL to perform data search, and performing data value extraction according to the field names in the returned result set, substituting the values into the original input string for replacement, and performing mathematical operations on the value string after all replacements are completed to obtain the final calculation result.

7. The expression-based data processing system according to claim 1, wherein, the write-back operation steps include: forming an UPDATE or INSERT statement based on the relationship between the field name and the written value, combining the query condition to perform field update of the data row. When writing back, it is also necessary to rely on the field definition information on the data source template to perform type processing and verification on the written field values.

8. The expression-based data processing system according to claim 1, wherein, the aggregation operation steps include: calculating the data according to the time granularity, combining the time offset given in the expression and the aggregation function.

Images