Index data synchronization method and device and target server

Abstract

The invention relates to the technical field of index synchronization, and discloses an index data synchronization method and device and a target server, the method is applied to the target server with a target database, the target server is further connected with a to-be-synchronized server storing to-be-synchronized index data, and the method comprises the steps that the to-be-synchronized index data in the to-be-synchronized server is acquired, determining a current organization type corresponding to the to-be-synchronized index data; a preset mapping rule corresponding to the to-be-synchronized index data is determined based on the current organization type, the preset mapping rule is a rule for conversion between the index data of the current organization type and index data of a target organization type of a target database, and the preset mapping rule is constructed based on the index dimension and the index name of the to-be-synchronized index data; performing type conversion on the to-be-synchronized index data based on a preset mapping rule to obtain target index data of a target organization type; and synchronizing the target index data to the target database.

Description

Technical Field

[0001] This application relates to the field of indicator synchronization technology, and in particular to an indicator data synchronization method, apparatus and target server. Background Technology

[0002] With the development of big data technology, enterprises have accumulated a large amount of data, which is scattered across different data sources, such as Hive data warehouses and relational databases. In order to achieve unified management and analysis of the data, it is necessary to synchronize the metric data of the Hive data warehouse on the server to be synchronized to the relational database on the target server.

[0003] When synchronizing metric data from a server to be synchronized on a target server with a relational database, the database typically has a relatively fixed storage structure and requirements. Existing metric data can generally be divided into wide tables and narrow tables according to organizational type. The dimensions of the metric data and the representation methods of metric values ​​differ between wide and narrow tables. Therefore, existing technologies generally set up corresponding adapted databases based on the metric's organizational type (narrow or wide tables) to store the metric data to be synchronized from the server to be synchronized, representing the corresponding organizational type. This necessitates setting up two separate databases for synchronization storage, leading to high costs and wasted resources. Summary of the Invention

[0004] The main purpose of this application is to provide a method for synchronizing indicator data, which aims to solve the technical problem that the existing technology requires setting up two databases for synchronous storage, resulting in high costs and wasted resources.

[0005] To achieve the above objectives, this application proposes a method for synchronizing indicator data. The method is applied to a target server with a target database, and the target server is also connected to a server storing indicator data to be synchronized. The method includes:

[0006] Obtain the metric data to be synchronized from the server to be synchronized, and determine the current organization type corresponding to the metric data to be synchronized;

[0007] Based on the current organization type, a preset mapping rule is determined for the indicator data to be synchronized. The preset mapping rule is a rule for converting the indicator data of the current organization type to the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized.

[0008] Based on the preset mapping rules, the data to be synchronized is converted to a different type to obtain the target data of the target organization type.

[0009] The target indicator data is synchronized to the target database.

[0010] In one embodiment, the step of performing type conversion on the target indicator data to be synchronized based on the preset mapping rule to obtain target indicator data of the target organization type includes:

[0011] Obtain the indicator name, indicator dimension, and indicator value corresponding to the indicator data to be synchronized, and generate an indicator table for the target organization type based on the preset mapping rule, indicator dimension, and indicator name;

[0012] Map the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data for the target organization type.

[0013] In one embodiment, the step of mapping the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data of the target organization type includes:

[0014] Determine the name data type corresponding to the indicator name and the value data type corresponding to the indicator value;

[0015] Based on the preset data type conversion rules and the name data type, the indicator name is converted to a new data type to obtain the converted indicator name. Based on the preset mapping rules and the name data type, the indicator value is converted to a new data type to obtain the converted indicator value.

[0016] The converted indicator values ​​are mapped to the indicator table based on the preset mapping rules to obtain the target indicator data to be synchronized for the target organization type.

[0017] In one embodiment, the step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type includes:

[0018] When the current organization type is the first indicator organization type, the preset self-splicing mapping rule will be used as the preset mapping rule;

[0019] The step of generating an indicator table for the target organization type based on the preset mapping rule, the indicator dimension, and the indicator name includes:

[0020] Sort the indicator names and indicator dimensions according to the preset self-splicing mapping rules;

[0021] The sorting results are concatenated to obtain an indicator table for the target organization type.

[0022] In one embodiment, the step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type further includes:

[0023] When the current organization type is the second indicator organization type, the preset single table synchronization mapping rule will be used as the preset mapping rule;

[0024] The step of generating an indicator table for the target organization type based on the preset mapping rule, the indicator dimension, and the indicator name includes:

[0025] The order of the indicator names and indicator dimensions is adjusted according to the preset single-table synchronization mapping rules, and an indicator table for the target organization type is generated based on the order adjustment results.

[0026] In one embodiment, the target server connects to the server to be synchronized via a task scheduling tool. Before the step of obtaining the synchronization index data from the server to be synchronized, the method further includes:

[0027] Obtain a preset synchronization task, a preset synchronization frequency, and a preset synchronization time; and determine the resource group and dependency conditions required to execute the preset synchronization task based on the preset synchronization task, the preset synchronization frequency, and the preset synchronization time.

[0028] The polling interval is determined based on the resource group and the dependency conditions, and the condition dependency signal generated by the task scheduling tool is obtained based on the polling interval. The condition dependency signal is a signal generated by the task scheduling tool when it completes data scheduling and data processing of the synchronization index data in the synchronization server.

[0029] When the conditional dependency signal satisfies the dependency condition, the synchronization index data obtained by the task scheduling tool after data processing is acquired.

[0030] In one embodiment, the server connects to the server to be synchronized via a query engine, and before the step of obtaining the synchronization index data from the server to be synchronized, the method further includes:

[0031] Set the connection parameters corresponding to the query engine so that the query engine can obtain the business type corresponding to the metric data to be synchronized in the server to be synchronized through the connection parameters;

[0032] Upon receiving a pull instruction, the business requirements corresponding to the pull instruction are obtained. The pull instruction is an instruction generated when a user triggers a pull operation.

[0033] Based on the business requirements and the business type, the indicator data to be synchronized is determined so that the query engine can pull the indicator data to be synchronized to the target server.

[0034] In one embodiment, the number of indicator data to be synchronized is at least two, and the step of determining the indicator data to be synchronized based on the business requirements and the business type includes:

[0035] Based on the business requirements and the business type, the data of each of the indicators to be synchronized are filtered to determine the data of indicators to be retrieved.

[0036] The timeliness corresponding to the business requirement is obtained, and the timeliness is used to prioritize each of the target metrics data to be pulled, so that the query engine pulls the target metrics data to be synchronized to the target server according to the priority ranking result.

[0037] Furthermore, to achieve the above objectives, this application also proposes an indicator data synchronization device, the device comprising:

[0038] The data acquisition module is used to acquire the metric data to be synchronized from the server to be synchronized, and to determine the current organization type corresponding to the metric data to be synchronized.

[0039] The rule acquisition module is used to determine the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type. The preset mapping rule is a rule for converting the indicator data of the current organization type with the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized.

[0040] The data conversion module is used to perform type conversion on the indicator data to be synchronized based on the preset mapping rules to obtain target indicator data of the target organization type;

[0041] The data synchronization module is used to synchronize the target indicator data to the target database.

[0042] In addition, to achieve the above objectives, this application also proposes a target server, which includes: a target database, a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the indicator data synchronization method described above.

[0043] This application proposes a method, apparatus, and target server for synchronizing indicator data. The method is applied to a target server with a target database, and the target server is also connected to a server storing indicator data to be synchronized. The method includes: acquiring the indicator data to be synchronized from the server to be synchronized and determining the current organization type corresponding to the indicator data; determining a preset mapping rule based on the current organization type, whereby the preset mapping rule is a rule for converting indicator data of the current organization type to indicator data of the target organization type in the target database, and the preset mapping rule is constructed based on the indicator dimensions and indicator names of the indicator data to be synchronized; performing type conversion on the indicator data to be synchronized based on the preset mapping rule to obtain target indicator data of the target organization type; and synchronizing the target indicator data to the target database. Because this application automatically determines the organization type of the indicator data from the server to be synchronized, further determines the corresponding preset mapping rule based on the organization type, performs type conversion on the indicator data to be synchronized based on the preset mapping rule, converts it to target indicator data of the target organization type, and synchronizes the converted target indicator data to the target database, it eliminates the need to set up separate synchronization storage for the two databases, reducing the workload and cost of hardware equipment, storage resources, and maintenance, and reducing resource waste. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the first embodiment of the indicator data synchronization method proposed in this application;

[0047] Figure 2 This is an overall architecture diagram of the indicator data synchronization method proposed in the embodiments of this application;

[0048] Figure 3 This is an example of the narrow representation of Hive in the indicator data synchronization method proposed in this application embodiment;

[0049] Figure 4 This is the Hive width representation intent in the indicator data synchronization method proposed in the embodiments of this application;

[0050] Figure 5 This is an illustration of the Gaussian width representation in the index data synchronization method proposed in the embodiments of this application;

[0051] Figure 6 This is a flowchart of the third embodiment of the indicator data synchronization method proposed in this application;

[0052] Figure 7 A diagram of an indicator data synchronization device provided in an embodiment of this application;

[0053] Figure 8 This is a schematic diagram of the structure of a target server suitable for implementing the embodiments of this application.

[0054] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0058] Understandably, with the development of big data technology, enterprises have accumulated massive amounts of data, which are scattered across different data sources, such as Hive data warehouses and relational databases. To achieve unified data management and analysis, it is necessary to synchronize the metric data from the Hive data warehouse on the server to be synchronized to the relational database on the target server.

[0059] When synchronizing metric data between a target server and a server using a relational database, the database typically has a relatively fixed storage structure and requirements. Existing metric data can generally be divided into wide tables and narrow tables based on organizational type. The dimensions and value representation methods of metric data differ between wide and narrow tables. Therefore, current technologies generally use corresponding adapted databases based on the metric's organizational type (narrow or wide tables) to store the corresponding metric data from the server to be synchronized. This necessitates setting up two separate databases for synchronization storage, leading to high costs and wasted resources.

[0060] Therefore, to address the technical problem of existing technologies requiring the separate setup of two databases for synchronous storage, leading to high costs and resource waste, this embodiment proposes an indicator data synchronization method. This method is applied to a target server with a target database, and the target server is also connected to a server storing indicator data to be synchronized. The method includes: acquiring the indicator data to be synchronized from the server to be synchronized and determining the current organization type corresponding to the indicator data; determining a preset mapping rule based on the current organization type, whereby the preset mapping rule is a rule for converting indicator data of the current organization type to indicator data of the target organization type in the target database, and the preset mapping rule is constructed based on the indicator dimensions and indicator names of the indicator data to be synchronized; performing type conversion on the indicator data to be synchronized based on the preset mapping rule to obtain target indicator data of the target organization type; and synchronizing the target indicator data to the target database. Because this embodiment automatically determines the organization type of the indicator data to be synchronized from the server to be synchronized, further determines the corresponding preset mapping rule based on the organization type, performs type conversion on the indicator data to be synchronized based on the preset mapping rule, converts it into target indicator data of the target organization type, and synchronizes the converted target indicator data to the target database. By automatically identifying data organization types and applying preset mapping rules, there is no need to set up separate synchronous storage for the two databases, which reduces the workload and cost of hardware equipment, storage resources, and operation and maintenance, and reduces resource waste.

[0061] For ease of understanding, the following is combined with Figures 1 to 8 The method for synchronizing indicator data provided in the embodiments of this application, as well as the method, apparatus, and target server for synchronizing indicator data provided in the following embodiments, will be described in detail.

[0062] This application provides a method for synchronizing indicator data, referring to... Figure 1 , Figure 1 This is a flowchart of the first embodiment of the indicator data synchronization method proposed in this application.

[0063] like Figure 1 As shown, the method includes:

[0064] Step S10: Obtain the synchronization indicator data from the server to be synchronized, and determine the current organization type corresponding to the synchronization indicator data.

[0065] It should be noted that the executing entity in this embodiment can be a server device with index data synchronization capabilities, such as a target server, or a device capable of performing the aforementioned functions. This embodiment uses a target server (hereinafter referred to as the target server) for illustration.

[0066] It should also be noted that the aforementioned server to be synchronized can be a server storing the metric data to be synchronized. This metric data can be the company's business data, such as sales data, inventory data, and user behavior data. In specific implementations, the target server needs to establish a connection with the server to be synchronized in order to obtain the metric data. For example, the server to be synchronized could be a MySQL database storing sales data, while the target server could be an Amazon Redshift data warehouse used for generating data reports. The target server retrieves the sales data from the MySQL database via a network connection and synchronizes it to Amazon Redshift for further analysis and processing. In this embodiment, the server housing the Hive data warehouse is used as the server to be synchronized, and the server housing the relational database is used as the target server for explanation and illustration, but this does not impose specific limitations on this embodiment.

[0067] Furthermore, it should be noted that the aforementioned current organization type can be the organization type corresponding to the aforementioned metric data to be synchronized. This organization type can also refer to the storage and structuring method of the metric data in the data table, including wide table types and narrow table types. A wide table is a data table structure that uses multiple metrics as independent columns. Each metric corresponds to one column, and dimension information exists as rows. For example, a sales wide table might contain columns such as date, region, product category, sales amount, and sales volume. Each metric (such as sales amount or sales volume) corresponds to an independent field. A narrow table is a data table structure that sets the metric name and metric value as independent columns. It typically contains dimension columns, metric name columns, and metric value columns. Each combination of dimension and metric name corresponds to a metric value, using multiple rows of data to represent different metrics. For example, a sales narrow table might contain columns such as date, region, product category, metric name, and metric value. The metric name field's value is either "sales amount" or "sales volume," and the corresponding metric value field represents the specific sales amount or quantity.

[0068] In practice, when synchronizing metric data, the target server first retrieves the metric data to be synchronized from the server to be synchronized and determines the current organizational type of this data. For example, the sales data stored on the server to be synchronized may exist in the form of a narrow table, containing fields such as "time," "region," "product category," and "sales amount." After retrieving this data through a connection with the server to be synchronized, the target server analyzes its organizational type.

[0069] Step S20: Determine the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type. The preset mapping rule is a rule for converting the indicator data of the current organization type to the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized.

[0070] It should be noted that the aforementioned preset mapping rules can be rules for converting the indicator data of the current organization type to the indicator data of the target organization type in the target database. These preset mapping rules are constructed based on the indicator dimensions and indicator names of the indicator data to be synchronized. The aforementioned target organization type can be the organization type corresponding to the indicator data in the target database.

[0071] It is understandable that when storing metric data from a Hive data warehouse into a relational database, if a narrow table type is used as the target organization type, subsequent metric applications (such as online reports, AI Q&A, etc.) often involve concatenating or processing multiple metrics for display. Multiple metrics in the same narrow table often coexist during application, resulting in multiple associations with the same narrow table during underlying data retrieval, leading to excessive association levels. Therefore, this embodiment explains the solution of uniformly converting the metric data organization type in the Hive data warehouse to a wide table type in a relational database and storing it in the relational database, but does not impose specific limitations on this embodiment.

[0072] Furthermore, the aforementioned metric dimensions can be fields used to describe and differentiate different characteristics or attributes of metric data, such as time, region, product category, sales channel, customer type, etc. The metric name can be a name used to identify a specific metric, clearly defining its meaning, such as sales revenue, sales volume, inventory level, click-through rate, and conversion rate. In the specific implementation, when synchronizing metric data, the target server will first identify the current organizational type (wide table or narrow table) of the metric data to be synchronized, and determine the corresponding preset mapping rules accordingly.

[0073] Step S30: Based on the preset mapping rules, perform type conversion on the indicator data to be synchronized to obtain the target indicator data of the target organization type.

[0074] It should be noted that the target indicator data mentioned above can be indicator data whose organizational type has been converted to the target organizational type.

[0075] Step S40: Synchronize the target indicator data to the target database.

[0076] In practice, the target server will match the corresponding preset mapping rules according to the current organization type (wide table or narrow table) of the indicator data to be synchronized, and perform data transformation operations, such as field mapping and data type conversion, to generate target indicator data that conforms to the target database structure, and then synchronize it to the target database.

[0077] To facilitate understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume a company has multiple data sources, including a Hive data warehouse and a relational database. The Hive data warehouse stores a large amount of sales data, organized using both wide and narrow tables. Wide tables contain columns such as date, region, product category, sales amount, and sales volume; narrow tables contain columns such as date, region, product category, metric name, and metric value. The relational database stores business data that needs to be integrated with the Hive data.

[0078] The target server needs to synchronize sales data from the Hive data warehouse to a relational database for more efficient querying and analysis. During the synchronization process, the target server first retrieves the metric data to be synchronized from the Hive data warehouse. By parsing the data's structural characteristics, it determines whether the current organizational type is a wide table or a narrow table. For example, for wide table data, the target server will identify that each metric (such as sales revenue or sales volume) corresponds to an independent field; for narrow table data, it will identify that there are separate metric name and metric value fields, and different metric values ​​share the same dimension fields.

[0079] After determining the current organization type, the target server performs type conversion on the metric data to be synchronized based on preset mapping rules. These preset mapping rules are constructed based on the metric dimensions and metric names of the data to be synchronized, and are used to convert wide or narrow table data into the target organization type in the target database. For example, wide table data is converted into a wide table structure in a relational database, or narrow table data is converted into a wide table structure. Finally, the target server synchronizes the converted target metric data to the relational database.

[0080] This embodiment acquires the metric data to be synchronized from the server to be synchronized, automatically determines its organization type, further determines the corresponding preset mapping rule based on the organization type, performs type conversion on the metric data to be synchronized according to the preset mapping rule, converts it into target metric data of the target organization type, and synchronizes the converted target metric data to the target database. By automatically identifying the data organization type and applying the preset mapping rule, there is no need to set up separate synchronization storage for the two databases, reducing the workload and cost of hardware equipment, storage resources, and operation and maintenance, and reducing resource waste.

[0081] Based on the first embodiment, in the second embodiment, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 , Figure 2 This is a flowchart of a second embodiment of the indicator data synchronization method proposed in this application. Further, to achieve more accurate indicator data synchronization, the step of converting the type of the indicator data to be synchronized based on the preset mapping rule to obtain the target indicator data of the target organization type includes:

[0082] Step S31: Obtain the indicator name, indicator dimension, and indicator value corresponding to the indicator data to be synchronized, and generate an indicator table for the target organization type based on the preset mapping rule, the indicator dimension, and the indicator name.

[0083] It should be noted that the aforementioned indicator table can be a table structure used to store and display indicator data. It typically contains multiple fields to record different dimensions and specific indicator values. The indicator values ​​can be specific numerical values, representing the actual values ​​corresponding to the indicator name. For example, in sales data, the sales amount is 10,000, and the sales volume is 100.

[0084] In practice, the target server parses the metric data to be synchronized, extracting key information such as metric name, metric dimension, and metric value. Then, according to preset mapping rules, it processes this information, such as sorting and concatenating metric names, to generate a metric table that meets the requirements of the target organization type.

[0085] Further, the step of generating an indicator table for the target organization type based on the preset mapping rule, the indicator dimension, and the indicator name includes:

[0086] Obtain the field requirements corresponding to the target organization type, and determine the indicator dimension, the current field corresponding to the indicator name, and the field length corresponding to the current field;

[0087] The current field is evaluated based on its length and requirements.

[0088] Generate an indicator table for the target organization type based on the field classification assessment results.

[0089] It should be noted that the aforementioned field requirements can be the target database's definitions and constraints on fields, including field names, data types, maximum lengths, etc. The aforementioned field grading assessment can be an operation that grades fields based on their length and requirements to determine their importance and storage strategies. In specific implementations, when generating the indicator table for the target organization type, the target server first obtains the field requirements corresponding to the target organization type and determines the current fields and their lengths corresponding to the indicator dimensions and names. Then, it performs a field grading assessment on the current fields based on their length and requirements. Finally, it generates the indicator table for the target organization type based on the field grading assessment results.

[0090] For ease of understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. Assume the target server needs to synchronize sales data to the target database, and the target database requires the "metric name" field to be no more than 50 characters long. After obtaining this field requirement, the target server checks the length of the current "metric name" field. If it finds that some "metric names" exceed 50 characters, it will perform a field-level evaluation to determine their importance and priority. For example, the "sales amount" and "sales volume" fields are evaluated as high priority and must retain their full names; while the "detailed sales amount of a product in a certain region" field is evaluated as low priority and can be abbreviated or truncated. Based on the evaluation results, the target server generates a metric table for the target organization type, ensuring that the field lengths meet the requirements of the target database while retaining key information.

[0091] Step S32: Map the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data of the target organization type.

[0092] In practice, the target server first parses the metric data to be synchronized, extracting key information such as metric names and values. Then, according to preset mapping rules, it maps these metric values ​​to the metric table of the target organization type.

[0093] Furthermore, to improve synchronization flexibility, the step of mapping the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data for the target organization type includes:

[0094] Step S321: Determine the name data type corresponding to the indicator name and the value data type corresponding to the indicator value.

[0095] It should be noted that the data type of the name field mentioned above can be the data type of the indicator name field in the source system, such as STRING, VARCHAR, etc., and the data type of the value field mentioned above can be the data type of the indicator value field in the source system, such as BIGINT, DECIMAL, FLOAT, etc.

[0096] Step S322: Based on the preset data type conversion rules and the name data type, perform data type conversion on the indicator name to obtain the converted indicator name, and based on the preset mapping rules and the name data type, perform type conversion on the indicator value to obtain the converted indicator value.

[0097] It should be noted that the above-mentioned preset data type conversion rules can be pre-configured mapping tables that specify the correspondence between source and target types, such as Hive STRING → MySQL VARCHAR(255), Hive DOUBLE → MySQLDECIMAL(18, 2). The converted metric name can be the field name used in the target table after type conversion. The converted metric value can be the numerical value after type conversion, ensuring it matches the target field type.

[0098] Step S323: Map the converted indicator values ​​to the indicator table based on the preset mapping rules to obtain the target indicator data to be synchronized for the target organization type.

[0099] It should be noted that the above-mentioned preset mapping rules can be rules that define field positions, names, type conversions, and concatenation methods. The target metric data to be synchronized can be the complete set of records ultimately written to the target database.

[0100] In the specific implementation, after the target server reads the data, it first parses the name data type and value data type of each record; then it calls the preset data type conversion rules to convert the name data type to VARCHAR(64) as the column name and convert the value data type to DECIMAL(18,2) as the value; then it fills the converted index value into the corresponding column according to the preset mapping rules to form the target table.

[0101] refer to Figure 3 as well as Figure 4 , Figure 3 This is an example of the narrow representation of Hive in the indicator data synchronization method proposed in this application. Figure 4This diagram illustrates the Hive wide table representation in the indicator data synchronization method proposed in this application. As shown, the Hive narrow table includes column headers such as indicator KEY, date, region, product category, and indicator value. The specific indicator value is stored as a field, with each indicator value corresponding to one row. The Hive wide table includes column headers such as date, region, product category, sales amount, and sales volume. The indicator value is directly stored as a column under sales amount and sales volume. In this embodiment, both Hive narrow and Hive wide tables are used as the current organization type corresponding to the indicator data to be synchronized. Each indicator data to be synchronized corresponds to one of the Hive narrow or Hive wide tables. This is explained for illustrative purposes only and does not impose specific limitations on this embodiment.

[0102] Furthermore, considering that different rules are required when converting the indicator data to be synchronized for different organizational types, the step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organizational type includes:

[0103] When the current organization type is the first indicator organization type, the preset self-splicing mapping rule is used as the preset mapping rule.

[0104] It should be noted that the first indicator organization type mentioned above can be a narrow table type among indicator organization types. In this embodiment, Hive narrow table is used as the first indicator organization type for explanation, but this does not impose specific limitations on this embodiment. The above-mentioned preset self-concatenation mapping rule can be a rule constructed based on the indicator dimensions corresponding to each column header of the narrow table and the indicator names in the table. It is used to concatenate the indicator dimensions and the indicator names in the table to obtain the indicator table corresponding to the target organization type.

[0105] The step of generating an indicator table for the target organization type based on the preset mapping rules, the indicator dimensions, and the indicator names includes:

[0106] Sort the indicator names and indicator dimensions according to the preset self-splicing mapping rules;

[0107] The sorting results are concatenated to obtain an indicator table for the target organization type.

[0108] In its implementation, the target server first parses the metric data to be synchronized, extracting key information such as metric names and metric dimensions. Then, it sorts the metric names and metric dimensions according to a preset self-concatenation mapping rule. After sorting, the target server concatenates these metric names and metric dimensions to generate a target metric table in wide table format.

[0109] The step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type further includes:

[0110] When the current organization type is the second indicator organization type, the preset single table synchronization mapping rule will be used as the preset mapping rule;

[0111] It should be noted that the second indicator organization type mentioned above can be a wide table type among indicator organization types. In this embodiment, Hive wide tables are used as the second indicator organization type for explanation, but this does not impose specific limitations on this embodiment. The aforementioned preset self-concatenation mapping rule can be a rule constructed based on the indicator dimensions and indicator names corresponding to each column header of the wide table. It is used to sort the indicator tables corresponding to the target organization type based on the indicator dimensions and indicator names in the table.

[0112] The step of generating an indicator table for the target organization type based on the preset mapping rules, the indicator dimensions, and the indicator names includes:

[0113] The order of the indicator names and indicator dimensions is adjusted according to the preset single-table synchronization mapping rules, and an indicator table for the target organization type is generated based on the order adjustment results.

[0114] In the specific implementation, the target server first parses the metric data to be synchronized, extracting key information such as metric names and metric dimensions. Then, it adjusts the order of the metric names and metric dimensions according to a preset single-table synchronization mapping rule to ensure that the field order conforms to the requirements of the target database. Finally, it generates a metric table for the target organization type based on the order adjustment result.

[0115] refer to Figure 3 , Figure 4 as well as Figure 5 , Figure 5 This document illustrates the Gaussian wide table representation in the indicator data synchronization method proposed in this application. In this embodiment, a Gaussian wide table is used as the target organization type for explanation, but this does not impose specific limitations on this embodiment. When converting from a Hive narrow table to a Gaussian wide table, for each record in the Hive narrow table, based on the indicator KEY (i.e.... Figure 3 The indicator KEY determines the corresponding column in the Gauss wide table. For example, if the indicator KEY is "Sales Revenue" (i.e., Figure 3 The sales figures (in the data) are recorded, and their index values ​​are written into column col1 of the Gauss wide table (i.e., Figure 5 In the context of col1 (sales revenue); the key indicator is "sales volume" (i.e., sales volume). Figure 3 The sales volume (in the data) is recorded, and its index value is written into column col2 of the Gauss wide table (i.e., Figure 5 In the text, col2 (sales volume) and dimension information (such as region, product category, date) are used. Figure 3The region, product category, and date in the table are directly mapped to the corresponding dimension columns of the Gauss wide table (i.e., Figure 5 The data includes Dimension 1 (Region), Dimension 2 (Product Category), and Time Dimension (Date). When converting from a Hive wide table to a Gauss wide table, the col1 column of the Gauss wide table is directly entered with the sales amount from the Hive wide table (i.e.,...). Figure 4 The sales amount is entered in column col2, and the sales volume is entered directly in column col2. Figure 4 The sales volume value, dimension information (such as region, product category, date) (i.e.) Figure 4 The region, product category, and date in the table are also directly mapped to the corresponding dimension columns of the Gauss wide table (i.e., Figure 5 The dimensions are: Dimension 1 (Region), Dimension 2 (Product Category), and Time Dimension (Date).

[0116] Based on the first and second embodiments, in the third embodiment, the content that is the same as or similar to that in Embodiments 1 and 2 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart of the third embodiment of the indicator data synchronization method proposed in this application. Further, the target server connects to the server to be synchronized via a task scheduling tool. Before the step of obtaining the indicator data to be synchronized from the server to be synchronized, the method further includes:

[0117] Step S01: Obtain the preset synchronization task, preset synchronization frequency, and preset synchronization time; based on the preset synchronization task, preset synchronization frequency, and preset synchronization time, determine the resource group and dependency conditions required to execute the preset synchronization task.

[0118] It should be noted that the aforementioned preset synchronization tasks can be predefined data synchronization tasks, specifying the indicator data and target database to be synchronized. The aforementioned preset synchronization frequency can be the time interval for executing the synchronization task, such as hourly, daily, or weekly. The aforementioned preset synchronization time can be the specific time when the synchronization task starts execution, such as 2 AM every day. The aforementioned resource group can be the set of computing resources used to execute the synchronization task, such as the server's CPU, memory, and network bandwidth. The aforementioned dependency conditions can be conditions that need to be met before the synchronization task executes, such as whether the source data is ready and whether the preceding tasks have been completed.

[0119] In its implementation, the target server determines the task's execution cycle and start time based on a preset synchronization frequency and time. Simultaneously, it assesses the required computing resources, such as CPU, memory, and network bandwidth, based on the complexity and data volume of the preset synchronization task, and configures resource groups accordingly. Furthermore, the target server checks pre-execution dependencies, such as whether the source data is ready and whether preceding tasks have been completed. Only when all dependencies are met will the data synchronization task officially begin execution.

[0120] Step S02: Determine the polling interval time based on the resource group and the dependency conditions, and obtain the condition dependency signal generated by the task scheduling tool based on the polling interval time. The condition dependency signal is a signal generated by the task scheduling tool when it completes data scheduling and data processing of the synchronization index data in the synchronization server.

[0121] It should be noted that the aforementioned conditional dependency signals can be signals generated by the task scheduling tool after it has completed data scheduling and processing of the synchronizing index data in the server to be synchronized. These signals indicate that the data is ready and synchronization can proceed. The task scheduling tool can be used to manage and schedule data synchronization tasks, responsible for coordinating resource allocation, task execution order, and dependency management. In specific implementations, the target server determines the polling interval based on the resource group and the dependency conditions, and periodically retrieves the conditional dependency signals generated by the task scheduling tool at the polling interval. When the target server detects that the conditional dependency signals satisfy all dependency conditions, it formally begins executing the data synchronization task.

[0122] Furthermore, after the step of determining the polling interval based on the resource group and the dependency conditions, the method further includes:

[0123] Obtain the current resource usage status corresponding to the resource group and the task priority corresponding to the preset synchronization task;

[0124] The polling interval is adjusted based on the resource usage and task priority to obtain a new polling interval.

[0125] The conditional dependency signal generated by the task scheduling tool is obtained based on the new polling interval.

[0126] It should be noted that the aforementioned current resource usage can refer to the usage status of system resources (such as CPU, memory, network bandwidth, etc.) at the current point in time, typically expressed as a utilization percentage or absolute value. In practical implementations, the target server obtains the real-time utilization rate of various resources (such as CPU, memory, network bandwidth, etc.) within the resource group through a built-in monitoring module or integration with an external monitoring system. For example, the server might find that the current CPU utilization rate is 70%, the memory utilization rate is 60%, and the network bandwidth utilization rate is 40%.

[0127] The target server assesses current resource usage based on preset rules and dynamically adjusts the polling interval according to task priority. For example, if resource utilization is high and there are high-priority tasks to be executed, the polling interval may be shortened from 60 seconds to 30 seconds to speed up task response; conversely, if resources are sufficient and task priority is low, the polling interval may be extended to 120 seconds to reduce resource consumption.

[0128] According to the new polling interval, the target server periodically checks the conditional dependency signals generated by the task scheduling tool. Once the conditional dependency signal is detected to meet the dependency conditions, the server immediately obtains the processed synchronization indicator data and begins to execute the synchronization task.

[0129] Step S03: When the conditional dependency signal satisfies the dependency condition, obtain the synchronization index data obtained by the task scheduling tool after data processing.

[0130] It should be noted that the target server continuously monitors the conditional dependency signals generated by the task scheduling tool. When the signal indicates that the data to be synchronized has been scheduled and processed, and all dependency conditions have been met, the target server retrieves these processed synchronization metrics data from the task scheduling tool.

[0131] For ease of understanding, the following example is used for illustration, but it does not impose specific limitations on this embodiment. Assume that the target server obtains synchronized sales data, a preset synchronization frequency (once a day), and a preset synchronization time (2 AM) from a configuration file or management interface. Based on this information, the resource group required to execute the synchronization task (e.g., allocating 2 CPU cores, 4GB of memory, and 100Mbps network bandwidth) and dependencies (the business system completes daily data entry, and the data format is validated) are determined.

[0132] The target server determines the polling interval based on resource groups and dependencies (e.g., polling every 10 minutes). Then, according to this interval, it retrieves the conditional dependency signal generated by the task scheduler. This signal is generated when the task scheduler completes data scheduling (e.g., aggregating data from various business subsystems to the server to be synchronized) and data processing (e.g., format conversion, deduplication, etc.) of the sales data to be synchronized on the server to be synchronized. It indicates that the data is ready for synchronization.

[0133] When the target server detects that the conditional dependency signal meets the dependency condition (i.e., the business system has completed the data entry for the day and the data format has passed the verification), it obtains the sales indicator data to be synchronized (such as the sales amount, sales volume, sales area, etc. of each product) obtained by the task scheduling tool after data processing, and synchronizes it to the data warehouse to provide data support for the subsequent generation of sales reports.

[0134] Furthermore, the server connects to the server to be synchronized via a query engine. Before the step of obtaining the synchronization indicator data from the server to be synchronized, the method further includes:

[0135] Step S04: Set the connection parameters corresponding to the query engine so that the query engine can obtain the business type corresponding to the metric data to be synchronized in the server to be synchronized through the connection parameters.

[0136] It should be noted that the aforementioned query engine can be a tool or module used to retrieve and process data from a data source, capable of parsing query statements and returning results. The aforementioned connection parameters can be configuration information used to establish a connection with the data source, such as the database's IP address, port number, username, and password. The aforementioned business type can be the specific business area or functional module to which the metric data to be synchronized belongs, such as sales data, inventory data, or user behavior data.

[0137] In practice, when synchronizing data, the target server first sets the connection parameters for the query engine. These parameters allow the query engine to establish a connection with the server to be synchronized and obtain the business type corresponding to the metric data to be synchronized. For example, if the target server needs to synchronize sales data and user behavior data, it will set different connection parameters for these two business types. Once the query engine connects to the server using these parameters, it can identify and obtain metric data related to specific business types, thus ensuring the accuracy and relevance of data synchronization.

[0138] Step S05: Upon receiving a pull instruction, obtain the business requirements corresponding to the pull instruction. The pull instruction is an instruction generated when a user triggers a pull operation.

[0139] It should be noted that the aforementioned fetch command can be triggered by a user or the system to request specific metric data from the server to be synchronized. The aforementioned business requirements can be the data content and scope needed by the user or system based on the actual business scenario, such as querying sales data or user behavior data for a certain period.

[0140] In practice, when the target server receives a fetch command, it identifies the command to determine its corresponding business requirements. For example, when a business department needs to query sales data for the most recent week, it sends a fetch command via the interface or API. Upon receiving this command, the target server parses it to determine that the business requirement is to retrieve sales data within a specified time range. Based on this requirement, the target server further determines the required data range and type, and then accurately retrieves the relevant metric data from the server to be synchronized to meet the business department's query needs.

[0141] Step S06: Based on the business requirements and the business type, determine the indicator data to be synchronized, so that the query engine pulls the indicator data to be synchronized to the target server.

[0142] In practice, upon receiving a fetch command, the target server retrieves the corresponding business requirements based on the information carried in the command and determines the specific metrics data to be synchronized based on the business type. This process ensures that the query engine can accurately retrieve the required data from the server to be synchronized and transmit it to the target server to meet the user's query needs.

[0143] Furthermore, the number of indicator data to be synchronized is at least two, and the step of determining the indicator data to be synchronized based on the business requirements and the business type includes:

[0144] Step S061: Based on the business requirements and the business type, filter the data of each indicator to be synchronized to determine the data of indicators to be retrieved.

[0145] It should be noted that the aforementioned metrics data to be pulled can be data stored on the server to be synchronized and needs to be pulled to the target server according to business requirements.

[0146] Step S062: Obtain the timeliness corresponding to the business requirement, and sort the data of each indicator to be pulled according to the timeliness, so that the query engine pulls the data of the indicator to be synchronized to the target server according to the priority sorting result.

[0147] It's important to note that the aforementioned timeliness refers to the value and availability of data to the business within a specific timeframe. For example, some real-time monitoring services require near real-time data, while some report generation services may have lower timeliness requirements. In the specific implementation, when the target server executes the data retrieval task, it first obtains the timeliness requirements corresponding to the business needs and prioritizes each metric data to be retrieved based on these requirements. The target server then guides the query engine to retrieve the metric data to be synchronized to the target server in order of priority: the query engine retrieves high-priority data first, followed by low-priority data, ensuring that time-sensitive data is synchronized to the target server promptly, meeting the timeliness needs of different businesses. For instance, in a financial transaction monitoring scenario, stock price data has extremely high timeliness requirements; the target server will set its priority to the highest level to ensure that this type of data is retrieved and processed immediately, enabling traders to make decisions based on the latest market data.

[0148] This embodiment also provides a first embodiment of an indicator data synchronization device, please refer to... Figure 7 , Figure 7 This is a diagram of an indicator data synchronization device provided in an embodiment of this application. The indicator data synchronization device includes:

[0149] The data acquisition module is used to acquire the metric data to be synchronized from the server to be synchronized, and to determine the current organization type corresponding to the metric data to be synchronized.

[0150] The rule acquisition module is used to determine the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type. The preset mapping rule is a rule for converting the indicator data of the current organization type with the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized.

[0151] The data conversion module is used to perform type conversion on the indicator data to be synchronized based on the preset mapping rules to obtain the target indicator data of the target organization type;

[0152] The data synchronization module is used to synchronize the target indicator data to the target database.

[0153] Referring to the first embodiment of the indicator data synchronization device, this embodiment also proposes a second embodiment of the indicator data synchronization device. The contents that are the same as or similar to those in the first embodiment of the indicator data synchronization device can be referred to the above description, and will not be repeated hereafter.

[0154] The data conversion module is further configured to obtain the indicator name, indicator dimension, and indicator value corresponding to the indicator data to be synchronized, and generate an indicator table for the target organization type based on the preset mapping rule, the indicator dimension, and the indicator name; and map the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data for the target organization type.

[0155] The data conversion module is further configured to determine the name data type corresponding to the indicator name and the value data type corresponding to the indicator value; perform data type conversion on the indicator name based on the preset data type conversion rule and the name data type to obtain the converted indicator name; and perform type conversion on the indicator value based on the preset mapping rule and the name data type to obtain the converted indicator value; and map the converted indicator value to the indicator table based on the preset mapping rule to obtain the target indicator data to be synchronized for the target organization type.

[0156] The data conversion module is further configured to, when the current organization type is the first indicator organization type, use a preset self-concatenation mapping rule as the preset mapping rule; sort the indicator name and the indicator dimension according to the preset self-concatenation mapping rule; and concatenate the sorting results to obtain an indicator table for the target organization type.

[0157] The data conversion module is further configured to, when the current organization type is the second indicator organization type, use the preset single-table synchronous mapping rule as the preset mapping rule; adjust the order of the indicator name and the indicator dimension according to the preset single-table synchronous mapping rule, and generate an indicator table for the target organization type based on the order adjustment result.

[0158] Referring to the first embodiment and the second embodiment of the indicator data synchronization device, this embodiment also proposes a third embodiment of the indicator data synchronization device. The contents that are the same as or similar to the first embodiment and the second embodiment of the indicator data synchronization device can be referred to the above description, and will not be repeated hereafter.

[0159] The data acquisition module is further configured to acquire a preset synchronization task, a preset synchronization frequency, and a preset synchronization time; determine the resource group and dependency conditions required to execute the preset synchronization task based on the preset synchronization task, the preset synchronization frequency, and the preset synchronization time; determine a polling interval time based on the resource group and the dependency conditions; and acquire a condition dependency signal generated by the task scheduling tool based on the polling interval time. The condition dependency signal is a signal generated by the task scheduling tool when it completes data scheduling and data processing of the synchronization index data in the synchronization server. If the condition dependency signal satisfies the dependency conditions, acquire the synchronization index data obtained by the task scheduling tool after data processing.

[0160] The data acquisition module is further configured to set connection parameters corresponding to the query engine, so that the query engine can obtain the business type corresponding to the metric data to be synchronized in the server to be synchronized through the connection parameters; when a pull instruction is received, the module obtains the business requirements corresponding to the pull instruction, wherein the pull instruction is an instruction generated when a user triggers a pull operation; and based on the business requirements and the business type, determines the metric data to be synchronized, so that the query engine pulls the metric data to be synchronized to the target server.

[0161] The data acquisition module is further configured to filter each of the indicator data to be synchronized based on the business requirements and the business type, and determine the indicator data to be pulled; obtain the timeliness corresponding to the business requirements, and prioritize each of the indicator data to be pulled based on the timeliness, so that the query engine pulls the indicator data to be synchronized to the target server according to the priority ranking result.

[0162] The indicator data synchronization device provided in this embodiment, employing the indicator data synchronization method described in the above embodiments, can solve the technical problem of existing technologies requiring the separate setup of two databases for synchronized storage, leading to high costs and wasted resources. Compared with the prior art, the beneficial effects of the indicator data synchronization device provided in this embodiment are the same as those of the indicator data synchronization method provided in the above embodiments, and other technical features in the indicator data synchronization device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0163] This embodiment provides a target server, which includes: a target database, at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the index data synchronization method in Embodiment 1 above.

[0164] The following is for reference. Figure 8 , Figure 8 This is a schematic diagram of the structure of a target server suitable for implementing the embodiments of this application. The target server in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The target server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0165] like Figure 8 As shown, the target server may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the target server. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the target server to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows target servers with various systems, it should be understood that implementing or having all of the systems shown is not required. More or fewer systems may be implemented alternatively.

[0166] Specifically, according to this embodiment, the process described above with reference to the flowchart can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the disclosed embodiments of this embodiment.

[0167] The target server provided in this embodiment employs the indicator data synchronization method described in the above embodiments, which solves the technical problem that existing technologies require setting up two separate databases for synchronous storage, leading to high costs and wasted resources. Compared with the prior art, the beneficial effects of the target server provided in this embodiment are the same as those of the indicator data synchronization method provided in the above embodiments, and other technical features of the target server are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0168] It should be understood that the various parts disclosed in this embodiment can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0169] The above description is merely a specific implementation of this embodiment, but the protection scope of this embodiment is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this embodiment should be included within the protection scope of this embodiment. Therefore, the protection scope of this embodiment should be determined by the protection scope of the claims.

[0170] This embodiment provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the index data synchronization method in the above embodiment.

[0171] The computer-readable storage medium provided in this embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0172] The aforementioned computer-readable storage medium may be included in the target server; or it may exist independently and not assembled into the target server.

[0173] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the target server, cause the target server to synchronize indicator data.

[0174] Computer program code for performing the operations of this embodiment can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this embodiment. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0176] The modules described in this embodiment can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0177] The readable storage medium provided in this embodiment is a computer-readable storage medium. This medium stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned indicator data synchronization method. This solves the technical problem in the prior art where separate databases are required for synchronization, leading to higher costs and wasted resources. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this embodiment are the same as those of the indicator data synchronization method provided in the above embodiments, and will not be elaborated upon here.

[0178] The above descriptions are only some embodiments and do not limit the patent scope of this embodiment. All equivalent structural transformations made based on the technical concept of this application and the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the patent protection scope of this application.

Claims

1. A method for synchronizing indicator data, characterized in that, The method is applied to a target server with a target database, the target server also being connected to a server storing data of metrics to be synchronized, the method comprising: Obtain the metric data to be synchronized from the server to be synchronized, and determine the current organization type corresponding to the metric data to be synchronized; Based on the current organization type, a preset mapping rule is determined for the indicator data to be synchronized. The preset mapping rule is a rule for converting the indicator data of the current organization type to the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized. Based on the preset mapping rules, the data to be synchronized is converted to a different type to obtain the target data of the target organization type. The target indicator data is synchronized to the target database.

2. The method as described in claim 1, characterized in that, The step of performing type conversion on the target indicator data to be synchronized based on the preset mapping rule to obtain the target indicator data of the target organization type includes: Obtain the indicator name, indicator dimension, and indicator value corresponding to the indicator data to be synchronized, and generate an indicator table for the target organization type based on the preset mapping rule, indicator dimension, and indicator name; Map the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data for the target organization type.

3. The method as described in claim 2, characterized in that, The step of mapping the corresponding indicator value to the indicator table according to the indicator name to obtain the target indicator data of the target organization type includes: Determine the name data type corresponding to the indicator name and the value data type corresponding to the indicator value; Based on the preset data type conversion rules and the name data type, the indicator name is converted to a new data type to obtain the converted indicator name. Based on the preset mapping rules and the name data type, the indicator value is converted to a new data type to obtain the converted indicator value. The converted indicator values ​​are mapped to the indicator table based on the preset mapping rules to obtain the target indicator data to be synchronized for the target organization type.

4. The method as described in claim 2, characterized in that, The step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type includes: When the current organization type is the first indicator organization type, the preset self-splicing mapping rule will be used as the preset mapping rule; The step of generating an indicator table for the target organization type based on the preset mapping rules, the indicator dimensions, and the indicator names includes: Sort the indicator names and indicator dimensions according to the preset self-splicing mapping rules; The sorting results are concatenated to obtain an indicator table for the target organization type.

5. The method as described in claim 2, characterized in that, The step of determining the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type further includes: When the current organization type is the second indicator organization type, the preset single table synchronization mapping rule will be used as the preset mapping rule; The step of generating an indicator table for the target organization type based on the preset mapping rules, the indicator dimensions, and the indicator names includes: The order of the indicator names and indicator dimensions is adjusted according to the preset single-table synchronization mapping rules, and an indicator table for the target organization type is generated based on the order adjustment results.

6. The method as described in claim 1, characterized in that, The target server connects to the server to be synchronized via a task scheduling tool. Before the step of obtaining the synchronization indicator data from the server to be synchronized, the method further includes: Obtain a preset synchronization task, a preset synchronization frequency, and a preset synchronization time; and determine the resource group and dependency conditions required to execute the preset synchronization task based on the preset synchronization task, the preset synchronization frequency, and the preset synchronization time. The polling interval is determined based on the resource group and the dependency conditions, and the condition dependency signal generated by the task scheduling tool is obtained based on the polling interval. The condition dependency signal is a signal generated by the task scheduling tool when it completes data scheduling and data processing of the synchronization index data in the synchronization server. When the conditional dependency signal satisfies the dependency condition, the synchronization index data obtained by the task scheduling tool after data processing is acquired.

7. The method as described in claim 1, characterized in that, The server connects to the server to be synchronized via a query engine. Before the step of obtaining the metric data to be synchronized from the server to be synchronized, the method further includes: Set the connection parameters corresponding to the query engine so that the query engine can obtain the business type corresponding to the metric data to be synchronized in the server to be synchronized through the connection parameters; Upon receiving a pull instruction, the business requirements corresponding to the pull instruction are obtained. The pull instruction is an instruction generated when a user triggers a pull operation. Based on the business requirements and the business type, the indicator data to be synchronized is determined so that the query engine can pull the indicator data to be synchronized to the target server.

8. The method as described in claim 7, characterized in that, The number of indicator data to be synchronized is at least two. The step of determining the indicator data to be synchronized based on the business requirements and the business type includes: Based on the business requirements and the business type, the data of each indicator to be synchronized is filtered to determine the indicator data to be retrieved. The timeliness corresponding to the business requirement is obtained, and the timeliness is used to prioritize each of the target metrics data to be pulled, so that the query engine pulls the target metrics data to be synchronized to the target server according to the priority ranking result.

9. A device for synchronizing indicator data, characterized in that, The device includes: The data acquisition module is used to acquire the metric data to be synchronized from the server to be synchronized, and to determine the current organization type corresponding to the metric data to be synchronized. The rule acquisition module is used to determine the preset mapping rule corresponding to the indicator data to be synchronized based on the current organization type. The preset mapping rule is a rule for converting the indicator data of the current organization type with the indicator data of the target organization type in the target database. The preset mapping rule is constructed based on the indicator dimension and indicator name of the indicator data to be synchronized. The data conversion module is used to perform type conversion on the indicator data to be synchronized based on the preset mapping rules to obtain the target indicator data of the target organization type; The data synchronization module is used to synchronize the target indicator data to the target database.

10. A target server, characterized in that, The server includes: a target database, a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the index data synchronization method as described in any one of claims 1 to 8.

Images