Data processing method and device, electronic equipment, storage medium and product

Abstract

The embodiment of the invention discloses a data processing method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: responding to a data change event, and determining a target change type corresponding to the data change event, the target change type comprises a data source level change type, a data source table level change type, a data source field level change type and / or a field enumeration value change type; calling a target dependency relationship table corresponding to the target change type to determine an affected target downstream service based on the target dependency relationship table and change content in the data change event, the dependency relationship represents at least one of a data table on which at least one downstream service depends, source link information of the data table, a field in the data table and a lower enumeration value of the field. According to the technical scheme, the capture range and real-time performance of the data source change content are improved, the downstream business influenced by the data change event can be determined, and then the adaptability and accuracy of the data source change under the global data are improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of information processing technology, and in particular, to a data processing method, device, electronic device, storage medium and product. Background Art

[0002] With the development of Internet technology, the types and number of data sources that each platform relies on are increasing. In order to meet the digital needs of each platform, data sources can be collected and aggregated into data warehouses to support the subsequent data management and application of the platform.

[0003] For each platform, the data source or data table content extracted by the data warehouse will change with the adjustment of the business. In order to determine the changed content in a timely and effective manner, an adaptation rule can be configured for it to perceive the data change in a timely manner based on the adaptation rule.

[0004] When implementing the technical solution based on the above method, the inventor found the following problems:

[0005] For data that is not covered by the adaptation rules, changes in its content cannot be perceived in a timely manner; further, only users who configure the adaptation rules can be aware of changes in data content in a timely manner, but cannot determine the impact of the changed data content on downstream systems. Summary of the invention

[0006] The present invention provides a data processing method, device, electronic device, storage medium and product to improve the capture scope and real-time performance of data source change content, determine downstream businesses affected by data source data changes, and thereby improve the adaptability and accuracy of data source changes under global data.

[0007] In a first aspect, an embodiment of the present invention provides a data processing method, the method comprising:

[0008] In response to a data change event, determining a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type;

[0009] Retrieving a target dependency table corresponding to the target change type, so as to determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event;

[0010] The dependency relationship is used to characterize at least one data table on which at least one downstream service depends, source link information of the at least one data table, a field in the data table, and at least one item of an enumeration value under the field.

[0011] Furthermore, the method further includes: determining the target change type corresponding to the data change event includes:

[0012] When the data change event is a change in data source information, determining that the target change type is a data source level change type;

[0013] When the data change event is a change in table information of a data table, determining that the target change type is a data source table-level change type;

[0014] When the data change event is a change in the field type in the data table, determining that the target change type is a data source field-level change type;

[0015] When the data change event is a change in an enumeration value under at least one field in the data table, the target change type is determined to be a field enumeration value change type.

[0016] Furthermore, the method also includes: the data source information includes at least one of a data source name, a data source user name, and a data source password; the table information includes a table name and a table field; and the field enumeration value is a content value that can be selected and displayed under the field.

[0017] Furthermore, the method further includes: determining that the data change event is a change in an enumeration value under at least one field in the data table, including:

[0018] Acquire at least one data table, and collect enumeration values ​​under at least one field in the at least one data table to obtain actual global enumeration information of each field;

[0019] When there is a difference between the theoretical global enumeration information and the actual global enumeration information of each field created in advance, it is determined that the enumeration value under the field is changed.

[0020] Furthermore, the method further comprises:

[0021] For the at least one downstream business, obtaining at least one data table on which the downstream business depends, source link information of the at least one data table, table information of the at least one data table, field type of each field in the at least one data table, and enumeration value of at least one field in the at least one data table; wherein the source link information is used to characterize information on which the data table used by the downstream business depends;

[0022] Determine a dependency table corresponding to the data source level change type according to the service identifier of the at least one downstream service, the at least one data table corresponding to the service identifier, and source link information of the at least one data table;

[0023] Determine, according to the service identifier of the at least one downstream service and the table information corresponding to the service identifier, a dependency table corresponding to the data source table-level change type;

[0024] Determine a dependency table corresponding to the field-level change type of the data source according to the business identifier of the at least one downstream business and the field type of each field in the at least one data table corresponding to the business identifier;

[0025] According to the service identifier of the at least one downstream service and the enumeration value of at least one field in the at least one data table corresponding to the service identifier, a dependency table corresponding to the change type of the field enumeration value is determined.

[0026] Further, the method further includes: the change content includes at least one of a data source change, a table information change, a field type change, and an enumeration value change; and determining a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event includes:

[0027] A target downstream business corresponding to the data change event is determined based on the target dependency table and at least one of a data source change, a table information change, a field type change, and an enumeration value change in the change content.

[0028] Furthermore, the method further comprises:

[0029] Based on the target downstream business and the change content in the data change event, a change tracking event is generated, and the change tracking event is displayed or sent to a terminal device corresponding to the target downstream business.

[0030] Furthermore, the method further includes: the change tracking event also includes an adaptation solution corresponding to the change content;

[0031] The change content includes the changed table name, and the adaptation solution is the data source dependent table name change;

[0032] The changed content is a newly added field, and the adaptation solution is to add the newly added field to the data table or not to add the newly added field;

[0033] The changed content is a deleted field, and the adaptation scheme includes abnormal deletion and notifying the upstream or invalid field deletion to set the content of the deleted field to empty;

[0034] The change content is a change in the field type of the target field, and the adaptation solution includes copying the original data table to which the target field belongs, updating the field type of the target field to the target type, and deleting the original data table.

[0035] Furthermore, the method further comprises:

[0036] In response to a triggering operation on the change tracking event, receiving feedback operation data;

[0037] When the feedback operation data is preset operation data, an event state of the change tracking event is adjusted to determine an execution result of the change tracking event based on the event state.

[0038] In a second aspect, an embodiment of the present invention further provides a data processing device, the device comprising:

[0039] a change type determination module, configured to determine, in response to a data change event, a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type;

[0040] A downstream business determination module, used to retrieve a target dependency table corresponding to the target change type, so as to determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event;

[0041] The dependency relationship is used to characterize at least one data table on which at least one downstream service depends, source link information of the at least one data table, a field in the data table, and at least one item of an enumeration value under the field.

[0042] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0043] one or more processors;

[0044] a storage device for storing one or more programs,

[0045] When one or more programs are executed by one or more processors, the one or more processors implement any data processing method as described in any embodiment of the present invention.

[0046] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute any data processing method as described in any of the embodiments of the present invention.

[0047] According to the technical solution of the embodiment of the present invention, when a data change event occurs in a data source, a target change type corresponding to the data change event can be determined in response to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type. By responding to the data change event, data changes at different granularities of the source level, table level, field level, and enumeration value level of the data source can be captured in a timely manner, thereby improving the capture scope and real-time performance of the data source change content; further, a target dependency table corresponding to the target change type can be retrieved to determine the target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event, wherein the dependency relationship is used to characterize at least one data table, source link information of at least one data table, a field in a data table, and at least one of the enumeration values ​​under the field on which at least one downstream business depends. According to the technical solution of the present embodiment, the downstream business affected by the data change event can be determined through the dependency table, thereby prompting the downstream business to perform data adaptation operations, thereby improving the adaptability and accuracy of the data source change under the global data. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A flowchart of a data processing method provided by an embodiment of the present invention;

[0050] Figure 2 A flowchart of another data processing method provided by an embodiment of the present invention;

[0051] Figure 3 A flowchart of another data processing method provided by an embodiment of the present invention;

[0052] Figure 4 A flowchart of another data processing method provided by an embodiment of the present invention;

[0053] Figure 5 A flowchart of another data processing method provided by an embodiment of the present invention;

[0054] Figure 6 A flowchart of another data processing method provided by an embodiment of the present invention;

[0055] Figure 7 A schematic diagram of a specific implementation process of a data processing method provided by an embodiment of the present invention;

[0056] Figure 8 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;

[0057] Fig. 9 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0059] Before introducing the technical solution, the application scenario of the embodiment of the present invention can be described first. The technical solution provided by the embodiment of the present invention can be applied to any scenario where it is necessary to determine the change impact scope of the data source change event and to perform global data adaptation in a timely manner.

[0060] Here, we can first summarize the concepts involved in this embodiment and the relationships between the various concepts.

[0061] The data source is a database that stores raw data. The data warehouse is a centralized repository that collects and integrates data from multiple data sources. The downstream business system is a downstream business service system for the data warehouse. The downstream business system can perform a variety of different downstream businesses. Different downstream businesses need to rely on some data content in the data warehouse to perform corresponding business processing operations. In summary, the data warehouse can receive raw data from multiple data sources and process the raw data from different sources. These processed data contents can provide data support for the task execution process of the downstream business system.

[0062] For example, the data in a data warehouse depends on a large number of data sources. If the data warehouse does not effectively adapt to changes in the data source, it will affect the data in the data warehouse, thereby causing problems in the processing of downstream businesses. For example, if the newly added fields of the data source are not adapted, the downstream business corresponding to the changed fields cannot obtain the corresponding data support, resulting in inaccurate data processing logic. Therefore, it is necessary to implement effective adaptation when the data source is changed to reduce the data impact caused by the data change. The technical solution of the embodiment of the present invention aims to make appropriate data impact assessments in a timely manner according to the type of data source change, and promptly inform the downstream businesses that will be affected, so as to ensure the reliability and availability of global data.

[0063] Figure 1 A flow chart of a data processing method provided in an embodiment of the present invention is applicable to any situation where it is necessary to determine the change impact scope of a data source change event and to perform global data adaptation in a timely manner. The method can be executed by a data processing device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC or a server.

[0064] like Figure 1 As shown, the data processing method includes:

[0065] S110 . In response to a data change event, determine a target change type corresponding to the data change event.

[0066] Among them, data change events refer to any events that affect subsequent data processing. For example, the data source itself has changed, the data table in the data source has changed (adding, deleting, modifying, etc. operations on the data table), the specific content in the data table has changed, etc. Data change events can be used to monitor and respond to changes in data sources in real time. When a data change event is detected, adaptive changes are made to the data in the links related to the data source to ensure the accuracy and consistency of the data in the link "data source-data warehouse-downstream business system".

[0067] Among them, the target change type includes at least one of the data source level change type, data source table level change type, data source field level change type and field enumeration value change type. The data source level change type refers to the change type at the data source connection level and library level, for example, the change type corresponding to the change of the data source name, data source user name, data source password, etc. The data source table level change type refers to the change type corresponding to the data table stored in the data source, for example, the change type corresponding to the change of the table name of some data tables in the data source, the addition of new fields to the data table, the deletion of fields in the data table, the change of fields in the data table, etc. The data source field level change type refers to the change type corresponding to some specific fields contained in the data table in the data source, for example, the change type corresponding to the change of the data type corresponding to some specific fields in a data table. The field enumeration value change type refers to the change type corresponding to the change of the enumeration value under some fields in the data table.

[0068] It should be noted that the data in the data source and the data warehouse are stored in the form of data tables. For any data table, the data table may include multiple fields and specific content (ie, enumeration values) under each field.

[0069] Exemplarily, the data sources associated with the data warehouse may include three, namely, data source A, data source B, and data source C, and each data source may include a large number of data tables. Here, data source A is taken as an example for explanation. Data source A may include multiple data tables, for example, data source A includes data table G, data table H, and data table R. Table 1 is a partial example table of data table G. As shown in Table 1, "data table G" is the name of the data table. Data table G includes five fields, namely, "first field", "second field", "third field", "fourth field", and "fifth field". The enumeration values ​​under the first field include "NN", "YY", and "PP". The enumeration values ​​under the second field include "D", "F", "D", ..., and the enumeration values ​​corresponding to other fields can be known in the same way.

[0070] Table 1 Data Table G

[0071] First Field Second field The third field The fourth field Fifth field NN D O 2 3 YY F P 5 6 PP D L 8 9

[0072] In this embodiment, for any data source associated with the data warehouse, when any of the following situations occurs in the data source, such as a data source connection level change, a data table change in the data source, a field change in the data table, or a change in the enumeration value under the field, a data change event is generated. At this time, the target change type can be determined according to the specific change content corresponding to the data change event.

[0073] Specifically, if the specific change content corresponding to the data change event is a change at the data source connection level (for example, the data source name changes), the target change type is a data source level change type; if the specific change content corresponding to the data change event is a change in the data table in the data source (taking Table 1 as an example, the name of data table G changes, the sixth field is added, the third field is deleted, etc.), the target change type is a data source table level change type; if the specific change content corresponding to the data change event is a change in the field in the data table (continuing to take Table 1 as an example, the field type corresponding to the first field in data table G changes from numeric type to character type), the target change type is a data source field level change type; if the specific change content corresponding to the data change event is a change in the enumeration value under the field (continuing to take Table 1 as an example, the enumeration value under at least one field in data table G changes), the target change type is a field enumeration value change type.

[0074] S120: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0075] In this embodiment, for different data change types, dependency tables corresponding to various data change types can be predetermined, and corresponding relationships between various data change types and corresponding dependency tables can be established. Here, the dependency table adapted to the target change type can be used as the target dependency table.

[0076] Next, we will explain in detail what a dependency table is. Dependencies can be represented in the form of a data table, and such a table can be called a dependency table. Dependencies are used to represent at least one data table on which at least one downstream business depends, the source link information of at least one data table, the fields in the data table, and at least one of the enumeration values ​​under the field. The source link information of the data table is used to represent the information on which the data table used by the downstream business depends. It should be noted that the dependency table also includes the dependent data source processing logic.

[0077] For example, see 2 for some examples of dependency tables. As shown in Table 2, the first row in Table 2 represents the "Col1" field of the downstream business "Tb1" (Tb1 is the business identifier), which depends on the data content of the "Col3" field in the "Tb2" data table in the data warehouse (i.e., it depends on the upstream data table), and the processing logic of the dependent field is the "Sum" operation. The upstream data table depends on the "Col1" field in the "Tt1" data table of the "Mysql_order" data source, and the processing logic of the dependent data source is the "Sum" operation. Similarly, the meaning of the second row can be understood.

[0078] Table 2. Partial example of dependency table

[0079]

[0080] It should be noted that the data tables that downstream businesses rely on refer to the data tables in the data warehouse. Since the data warehouse can receive a large number of data tables from different data sources and classify and summarize these data, different data layers are generated. Each data layer can include multiple data tables, and there are certain dependencies between data tables in different layers. Therefore, the source link information of the data table not only refers to which data source it comes from, but also includes the data table dependency link information inside the data warehouse.

[0081] The change content in the data change event may include at least one of a data source change, a table information change, a field type change, and an enumeration value change.

[0082] In this embodiment, multiple downstream businesses can be carried out in the downstream business system, and the processing process for each downstream business is consistent. Any one of the downstream businesses is used as an example for explanation. For a certain downstream business, the dependent data corresponding to the downstream business can be obtained by parsing the downstream business. Among them, the dependent data may include some data tables that the downstream business depends on, the source link information of these data tables, the table information of these data tables (for example, table name, fields in the table, etc.), the field type of each field in these data tables, and / or the enumeration value of at least one field in these data tables. And the processing logic of these dependent data can also be parsed out. Based on the same method, the dependent data corresponding to each downstream business and the processing logic of the dependent data can be obtained. By summarizing the dependent data tables involved in the downstream business and the source link information of these data tables, the dependency table corresponding to the data source level change type can be obtained; by summarizing the table information of the dependent data tables involved in the downstream business, the dependency table corresponding to the data source table level change type can be obtained; by summarizing the field types of certain fields in the dependent data tables involved in the downstream business, the dependency table corresponding to the data source field level change type can be obtained; by summarizing the enumeration values ​​under certain fields in the dependent data tables involved in the downstream business, the dependency table corresponding to the field enumeration value change type can be obtained. Thus, the dependency tables corresponding to various data change types can be obtained, and these dependency tables can be stored in the preset storage space. When a data change event occurs in a certain data source, the corresponding dependency table can be retrieved from the preset storage space.

[0083] Specifically, when a data change event occurs in a data source, the content of the change is easy to determine (for example, the content of the change can be at least one of a data source change, a table information change, a field type change, and an enumeration value change), so that the target change type corresponding to the data change event can be determined, and further, the target dependency table corresponding to the target change type is retrieved from the preset storage space. Furthermore, based on the target dependency table and the content of the change, it can be determined which downstream businesses will be affected by the data change event, and these downstream businesses are the target downstream businesses.

[0084] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations. For example, network requests are intercepted and processed with the authorization of the user.

[0085] According to the technical solution of the embodiment of the present invention, when a data change event occurs in a data source, a target change type corresponding to the data change event can be determined in response to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type. By responding to the data change event, data changes at different granularities of the source level, table level, field level, and enumeration value level of the data source can be captured in a timely manner, thereby improving the capture scope and real-time performance of the data source change content; further, a target dependency table corresponding to the target change type can be retrieved to determine the target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event, wherein the dependency relationship is used to characterize at least one data table, source link information of at least one data table, a field in a data table, and at least one of the enumeration values ​​under the field on which at least one downstream business depends. According to the technical solution of the present embodiment, the downstream business affected by the data change event can be determined through the dependency table, thereby prompting the downstream business to perform data adaptation operations, thereby improving the adaptability and accuracy of the data source change under the global data.

[0086] Figure 2 This is a schematic diagram of a data processing method provided by an embodiment of the present invention. Based on the above embodiment, the data source level change type is described in detail. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0087] like Figure 2 As shown, the method specifically comprises the following steps:

[0088] S210. For at least one downstream service, obtain at least one data table on which the downstream service depends and source link information of at least one data table.

[0089] The source link information is used to characterize the information that the data table used by the downstream business depends on. The data table that the downstream business depends on refers to one or more data tables in the data warehouse. These data tables can be original data tables in the data source, or data tables after the data from multiple different data sources are aggregated and integrated. For the aggregated and integrated data table, the data table also includes the source link information of the data table.

[0090] In this embodiment, the data table that each downstream service depends on and the source link information of certain data tables can be obtained here.

[0091] S220: Determine a dependency table corresponding to a data source level change type according to a service identifier of at least one downstream service, at least one data table corresponding to the service identifier, and source link information of at least one data table.

[0092] The service identifier is unique identification information corresponding to each downstream service. For example, the service identifier can be represented as "000251".

[0093] In this embodiment, for each downstream business, a mapping relationship between a business identifier, at least one data table, source link information of certain data tables, and processing logic can be constructed, that is, each downstream business can obtain a corresponding data strip. Thus, by summarizing the data strips corresponding to all downstream businesses, a dependency table corresponding to the data source level change type can be obtained. This dependency table can be stored in a preset storage space for subsequent steps. The advantage of this setting is that by pre-constructing a dependency table at the data source level, when a data change event at the data source level occurs, the target downstream business that will be affected by the data change event can be quickly and conveniently determined by data query.

[0094] S230: In response to a data change event, when the data change event is a change in data source information, determine that the target change type is a data source level change type.

[0095] The data source information includes at least one of a data source name, a data source user name, and a data source password. The data source name refers to an information name used to identify a database connection, which can be a symbolic name. The data source user can be the login account information corresponding to a database. The data source password can be a password corresponding to the login account information.

[0096] In this embodiment, when a trigger event of a data source name, data source user name and / or data source password change occurs in a data source associated with a data warehouse, the target change type is determined to be a data source level change type. In particular, this step can be monitored in real time through data source connectivity detection to continuously verify data source connectivity. Through multiple types of data source information monitoring dimensions, data change events at the data source level can be accurately captured.

[0097] S240: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0098] The change content may include a data source change, which may include a change in the data source name, data source user name, and data source password.

[0099] In this embodiment, after obtaining the target change type of the data source level change type, the pre-created dependency table of the data source level change type can be retrieved from the preset storage space. Then, based on the target dependency table and the data source change in the change content, the target downstream business corresponding to the data change event is determined. In this way, when a data source level coarse-grained data change event occurs in a certain data source, the global data impact brought about by the data change can be analyzed in a timely manner, providing effective reference information for ensuring the consistency and accuracy of global data.

[0100] For example, if the data source name, data source user and / or data source password of data source A is changed (the change content is the data source change), a data change event is generated, and the target change type corresponding to the data change event is the data source level change type. At this time, the target dependency table of the data source level change type can be retrieved from the preset storage space. According to the recorded content in the target dependency table and the relevant information corresponding to the data source A in the change content, for example, the data source identification information corresponding to the data source A, it can be determined which downstream businesses will be affected by the data change event, and these downstream businesses are the target downstream businesses.

[0101] The technical solution of the embodiment of the present invention obtains at least one data table and the source link information of at least one data table on which the downstream business depends for at least one downstream business associated with the data warehouse, and determines the dependency table corresponding to the data source level change type according to the business identifier of at least one downstream business, at least one data table corresponding to the business identifier, and the source link information of at least one data table. Thus, when a data change event occurs in a data source associated with the data warehouse in which the data source information changes, the target change type can be determined as the data source level change type in response to the data change event. Then, the dependency table corresponding to the data source level change type is retrieved, and the target downstream business corresponding to the data change event is determined based on the target dependency table and the data source change in the change content. The technical solution provided by the embodiment of the present invention can timely analyze the global data impact brought about by the data change when a data source level coarse-grained data change event occurs in a data source, providing effective reference information for ensuring the consistency and accuracy of global data.

[0102] Figure 3 This is a schematic diagram of a data processing method provided by an embodiment of the present invention. Based on the above embodiment, the case of changing the type of data source table level is detailed. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0103] like Figure 3 As shown, the method specifically comprises the following steps:

[0104] S310: For at least one downstream business, obtain table information of at least one data table on which the downstream business depends.

[0105] The table information includes the table name and table fields. The table name is the name information of a data table, and the table field is the field information contained in the data table.

[0106] In this embodiment, for multiple downstream businesses carried out in the downstream business system, one or more data tables that each downstream business depends on may be obtained.

[0107] S320: Determine a dependency table corresponding to a data source table-level change type according to a business identifier of at least one downstream business and table information corresponding to the business identifier.

[0108] In this embodiment, for each downstream business, a mapping relationship between the business identifier and various types of table information and processing logic can be constructed, that is, each downstream business can obtain a corresponding data strip. Thus, by summarizing the data strips corresponding to all downstream businesses, a dependency table corresponding to the data source table-level change type can be obtained. This dependency table can be stored in a preset storage space for subsequent calls. The advantage of this setting is that by pre-constructing a dependency table at the data source table level, when a data change event at the data source table level occurs, the target downstream business that will be affected by the data change event can be efficiently determined.

[0109] S330: When the data change event is a change in table information of a data table, determine that the target change type is a data source table-level change type.

[0110] In this embodiment, when a trigger event of a change behavior of a data table name or a field in a data table (e.g., adding a field, reducing a field, or modifying a field) occurs in a data table of a data source associated with a data warehouse, the target change type is determined to be a data source table-level change type. In particular, this step can read the data source metadata and compare the metadata with the fields of the database operational data storage layer (ODS) to monitor the specific change content.

[0111] S340: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0112] The change content may include table information change, which may be a change in the data table name or a specific field in the data.

[0113] In this embodiment, after obtaining the target change type of the data source table-level change type, the pre-created dependency table of the data source table-level change type can be retrieved from the preset storage space. Then, based on the target dependency table and the table information changes in the change content, the target downstream business corresponding to the data change event is determined. In this way, when a data change event at the data table level occurs in a data source, the global data impact caused by the data change can be analyzed in a timely manner.

[0114] In another example, if the table name of data table G in data source A is changed (the change content is the table information change), a data change event will also be generated, and the target change type corresponding to the data change event is the data source table-level change type. At this time, the target dependency table of the data source table-level change type can be retrieved from the preset storage space. According to the recorded content in the target dependency table and the relevant information corresponding to the data table G in the change content, for example, the data table identification information corresponding to the data source G, the target downstream business that will be affected by the data change event can be determined.

[0115] The technical solution of the embodiment of the present invention, for at least one downstream business associated with the data warehouse, obtains the table information of at least one data table on which the downstream business depends, and the business identifier of at least one downstream business and the table information corresponding to the business identifier, and determines the dependency table corresponding to the data source table-level change type. Thus, when a data change event occurs in which the table information of a data table of a data source associated with the data warehouse, it is possible to determine, in response to the data change event, that the target change type is a data source table-level change type. Further, the dependency table corresponding to the data source table-level change type is retrieved, and the target downstream business corresponding to the data change event is determined based on the target dependency table and the table information changes in the change content. The technical solution provided by the embodiment of the present invention can timely analyze the global data impact brought about by the data change when a data change event at the data table level occurs in a data source.

[0116] Figure 4 This is a schematic diagram of a data processing method provided by an embodiment of the present invention. Based on the above embodiment, the case of changing the type of data source field level is detailed. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0117] like Figure 4 As shown, the method specifically comprises the following steps:

[0118] S410: For at least one downstream business, obtain the field type of each field in at least one data table on which the downstream business depends.

[0119] The field type is used to represent the data format of the enumeration value under a certain field. For example, the field type can be numeric, character, integer, floating point, long integer, short integer, etc.

[0120] In this embodiment, multiple downstream businesses are carried out in the downstream business system, and the field type of each field in each data table on which each downstream business depends is obtained.

[0121] S420: Determine a dependency table corresponding to a data source field-level change type according to a business identifier of at least one downstream business and a field type of each field in at least one data table corresponding to the business identifier.

[0122] In this embodiment, for each downstream business, a mapping relationship between the business identifier and the field type of each field in each data table and the processing logic can be constructed, that is, each downstream business can obtain a corresponding data strip. Thus, by summarizing the data strips corresponding to all downstream businesses, a dependency table corresponding to the data source field-level change type can be obtained. This dependency table can be stored in a preset storage space for subsequent steps. The advantage of this setting is that by pre-constructing a dependency table of the data source field-level change type, when a data change event at the data source field level occurs, the target downstream business that will be affected by the data change event can be efficiently determined.

[0123] S430: When the data change event is a change in the field type in the data table, determine that the target change type is a data source field-level change type.

[0124] In this embodiment, when the field type of some fields in the data table of a data source associated with the data warehouse changes, the target change type is determined to be a data source field level change type. In particular, this step can read the data source metadata and the fields of the database operation data storage layer to perform metadata comparison and monitor the specific change content.

[0125] S440: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0126] The change content may include a field type change. For example, the field type of a certain field may be changed from a character type to a numeric type, or from an integer type to a floating point type.

[0127] In this embodiment, after obtaining the target change type of the data source field-level change type, the pre-created dependency table of the data source field-level change type can be retrieved from the preset storage space. Then, based on the target dependency table and the field type change in the change content, the target downstream business corresponding to the data change event is determined. In this way, when a field-level fine-grained data change event occurs in a certain data source, the global data impact caused by the data change can be analyzed in a timely manner.

[0128] In another example, if the data type of the first field in the data table G in the data source A changes (the change content is the field type change), a data change event will also be generated, and the target change type corresponding to the data change event is the data source field level change type. At this time, the target dependency table of the data source field level change type can be retrieved from the preset storage space. According to the records in the target dependency table and the relevant information of the specific fields in the change content, the target downstream business that will be affected by the data change event can be determined.

[0129] The technical solution of the embodiment of the present invention, for at least one downstream business associated with the data warehouse, obtains the field type of each field in at least one data table that the downstream business depends on, and determines the dependency table corresponding to the data source field-level change type based on the business identifier of at least one downstream business and the field type of each field in at least one data table corresponding to the business identifier. Thus, when a data change event of a field type change occurs in a data source associated with the data warehouse, the target change type can be determined as the data source field-level change type in response to the data change event. Then, the dependency table corresponding to the data source field-level change type is retrieved, and the target downstream business corresponding to the data change event is determined based on the target dependency table and the field type change in the change content. The technical solution provided by the embodiment of the present invention can timely analyze the global data impact brought about by the data change when a data source-level fine-grained data change event occurs in a data source, providing effective reference information for ensuring the consistency and accuracy of global data.

[0130] Figure 5 This is a schematic diagram of a data processing method provided by an embodiment of the present invention. Based on the above embodiment, the case of changing the type of the enumeration value level of the data source field is described in detail. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here.

[0131] like Figure 5 As shown, the method specifically comprises the following steps:

[0132] S510: For at least one downstream business, obtain an enumeration value of at least one field in at least one data table on which the downstream business depends.

[0133] In this embodiment, the field enumeration value is the content value that can be selected and displayed under the field. In addition, the field enumeration value can also be the edited content value input by the user.

[0134] In this embodiment, multiple downstream businesses are carried out in the downstream business system, and each enumeration value under each field in each data table on which each downstream business depends can be obtained.

[0135] S520: Determine a dependency table corresponding to a field enumeration value change type according to a service identifier of at least one downstream service and an enumeration value of at least one field in at least one data table corresponding to the service identifier.

[0136] In this embodiment, for each downstream business, a mapping relationship between the business identifier and each enumeration value under each field in each data table and the processing logic can be constructed, that is, each downstream business can obtain a corresponding data strip. Thus, by summarizing the data strips corresponding to all downstream businesses, a dependency table corresponding to the field enumeration value change type can be obtained. This dependency table can be stored in a preset storage space for subsequent steps to call. The advantage of this setting is that by pre-constructing a dependency table of the field enumeration value change type, when a data change event occurs at the data source field enumeration value level, the target downstream business that will be affected by the data change event can be quickly and conveniently determined.

[0137] S530: When the data change event is a change in an enumeration value under at least one field in the data table, determine that the target change type is a field enumeration value change type.

[0138] In this embodiment, when the enumeration values ​​under certain fields in the data table of a data source associated with the data warehouse change, the target change type is determined to be the field enumeration value change type.

[0139] Specifically, the specific implementation steps for determining that a data change event is a change in the enumeration value under at least one field in a data table may include: obtaining at least one data table, and collecting the enumeration values ​​under at least one field in at least one data table to obtain actual global enumeration information for each field; when there is a difference between the theoretical global enumeration information and the actual global enumeration information of each field created in advance, determining that the enumeration value under the field has changed.

[0140] In this embodiment, for a certain data table, a full data reference table can be created in advance. The full data reference table can be understood as a standard table. When the enumeration value in the collected data table is inconsistent with the data content in the standard table, it is considered that an enumeration value change event has occurred. The data content under each field recorded in the full data reference table is the theoretical global enumeration information. All enumeration values ​​collected in real time are the actual global enumeration information.

[0141] Specifically, for a data table in a data source, the enumeration values ​​under each field in the data table can be periodically collected based on a pre-set timing task, so as to obtain the actual global enumeration information of each field. Furthermore, for the actual global enumeration information under any field, the consistency result between the actual global enumeration information and the corresponding theoretical global enumeration information is determined; if the consistency result is a difference, it is determined that the enumeration value under the field has changed; otherwise, it is determined that the enumeration value under the field has not changed. The goal of this setting is to effectively identify the extremely fine-grained data changes of the enumeration values ​​in the data source based on the pre-built reference table of the full amount of enumeration values, thereby improving the monitoring accuracy of data changes.

[0142] For example, it is possible to collect enumeration values ​​under each field of each data table of the data source on a daily basis, and record the actual global enumeration information of each field. In particular, an enumeration value definition maintenance function can be provided, for example, business personnel maintain the meaning of the enumeration value and complete the confirmation of the enumeration value information to ensure that the enumeration value is detected as a comparison value. See Table 3 for an example table of some collected information of the enumeration value.

[0143] Table 3 Example of some collected information of enumeration values

[0144]

[0145] Based on the actual global enumeration information, you can obtain a full data reference table, and compare and analyze the actual global enumeration information under each field with the corresponding historical theoretical global enumeration information to view changes in the value content.

[0146] S540: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0147] The changes may include changes in enumeration values, which may include adding new enumeration values, changing enumeration value codes and enumeration value meanings, etc.

[0148] In this embodiment, after obtaining the target change type of the field enumeration value change type, the pre-created dependency table of the field enumeration value change type can be retrieved from the preset storage space. Then, based on the target dependency table and the enumeration value change in the change content, the target downstream business corresponding to the data change event is determined. In this way, when a very fine-grained data change event of a field enumeration value occurs in a certain data source, the global data impact brought about by the data change can be analyzed in a timely manner.

[0149] For example, if the enumeration value under the first field in the data table G in the data source A is increased, decreased, changed, the enumeration value encoding is changed, and the enumeration value meaning is changed (the change content is the enumeration value change), a data change event will also be generated at this time, and the target change type corresponding to the data change event is the field enumeration value change type. At this time, the target dependency table of the field enumeration value change type can be retrieved from the preset storage space. According to the recorded content in the target dependency table and the relevant information of the enumeration value in the change content, the target downstream business that will be affected by the data change event can be determined.

[0150] In another example, when the enumeration value of the data table in the data source changes, the target dependency table can be used to determine the scope of impact of these changes on downstream businesses. For example, whether the downstream business uses the field during processing, including select or where conditions. If logical processing is performed on the field in the select condition, including but not limited to case when, IF, cast, etc., the original value is processed for change conditions; in the where statement, conditional filtering is performed on the field (including but not limited to =, in, <> and other conditions), these are all objects affected by this data content change.

[0151] The technical solution of the embodiment of the present invention, for at least one downstream business associated with the data warehouse, obtains the enumeration value of at least one field in at least one data table on which the downstream business depends, and determines the dependency table corresponding to the field enumeration value change type according to the business identifier of at least one downstream business and the enumeration value of at least one field in the at least one data table corresponding to the business identifier. Thus, when a data change event occurs in a data source associated with the data warehouse in which the enumeration value changes, the target change type can be determined as the field enumeration value change type in response to the data change event. Then, the dependency table corresponding to the field enumeration value change type is retrieved, and the target downstream business corresponding to the data change event is determined according to the target dependency table and the enumeration value change in the change content. The technical solution provided by the embodiment of the present invention can timely analyze the global data impact brought about by the data change when a data change event of extremely fine granularity at the enumeration value level occurs in a data source, providing effective reference information for ensuring the consistency and accuracy of global data.

[0152] Figure 6 A schematic diagram of a data processing method provided by an embodiment of the present invention, based on the above-mentioned embodiment, after obtaining the target downstream business corresponding to the data change event, a change tracking event can be generated based on the target downstream business and the change content in the data change event, and the change tracking event can be displayed or sent to the terminal device corresponding to the target downstream business, and then, in response to the triggering operation of the change tracking event, feedback operation data is received, and when the feedback operation data is the preset operation data, the event state of the change tracking event is adjusted to determine the execution result of the change tracking event based on the event state. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0153] like Figure 6 As shown, the method specifically comprises the following steps:

[0154] S610: In response to a data change event, determine a target change type corresponding to the data change event.

[0155] S620: Retrieve a target dependency table corresponding to the target change type, and determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event.

[0156] S630: Generate a change tracking event based on the target downstream business and the change content in the data change event, and display the change tracking event or send the change tracking event to a terminal device corresponding to the target downstream business.

[0157] The changes may include changed table names, newly added fields, deleted fields, field type changes of target fields, newly added enumeration values, changed enumeration values, changed enumeration value meanings, etc. Change tracking events are computer behaviors that record and track data changes in data sources.

[0158] In this embodiment, for a certain data change event, the relevant information of the corresponding target downstream business (for example, business name, business identification information, etc.) and the descriptive text representing the change content can be encapsulated into a change tracking event, and the change tracking event can be displayed on the display screen of the current server. Alternatively, the change tracking event can be sent to the terminal devices deployed by each target downstream business, so that the management personnel corresponding to the terminal device can be aware of the data change event. In this way, the user corresponding to the target downstream business can be prompted that a data source change event has occurred, attracting the attention of the downstream business users, so as to prompt the downstream business users to synchronize data in a timely manner to ensure the adaptability of the data source change under the global data.

[0159] On the basis of the above embodiment, the change tracking event also includes an adaptation solution corresponding to the change content, and the adaptation solution is used to provide downstream business users with ideas for modifying the adaptability of data changes.

[0160] Specifically, different change contents can correspond to different adaptation schemes. Exemplarily, the correspondence table between the change content and the adaptation scheme can be found in Table 4. As shown in Table 4, the change content includes the name of the changed table, and the adaptation scheme is the data source dependent table name change; the change content is the addition of new fields, and the adaptation scheme is to add new fields to the data table or not to add new fields; the change content is the deletion of fields, and the adaptation scheme includes abnormal deletion and notifying the upstream or invalid field deletion to set the content of the deleted field to empty; the change content is the field type change of the target field, and the adaptation scheme includes copying the original data table to which the target field belongs, updating the field type of the target field to the target type, and deleting the original data table. In this way, after receiving the change tracking event, the downstream business can refer to the adaptation scheme in the change tracking event to assist users in completing data adaptation work quickly and efficiently.

[0161] Table 4 Correspondence between changes and adaptation solutions

[0162]

[0163]

[0164] S640: In response to a triggering operation on a change tracking event, receiving feedback operation data.

[0165] The feedback operation data is the operation data for the change tracking event fed back by the terminal device corresponding to the target downstream business.

[0166] In this embodiment, after the change tracking event is sent to the terminal device corresponding to the target downstream business, the terminal device can display the change tracking event. After consulting the change tracking event, the user corresponding to the target downstream business can complete the data adaptation operation according to the adaptation scheme in the change tracking event, and feedback operation data can be generated at this time. When the user corresponding to the data processing device clicks the control corresponding to the change tracking event, the feedback operation data returned by the target downstream business system can be received.

[0167] S650: When the feedback operation data is the preset operation data, adjust the event state of the change tracking event to determine the execution result of the change tracking event based on the event state.

[0168] The preset operation data is the data content that can be generated after the data adaptation operation is completed according to the corresponding adaptation scheme. The event status can include an unexecuted state and an executed state.

[0169] In this embodiment, when it is detected that the feedback operation data is consistent with the preset operation data, the event state of the change tracking event can be adjusted from an unexecuted state to an executed state. Thus, according to the adjusted event state, it can be determined that the execution result of the change tracking event is an executed result, which indicates that the target downstream business system has completed the data adaptation work corresponding to the data source change event. In this way, by using the change tracking event to follow up on whether the data affected by the data source change has been repaired, by ensuring that the affected downstream business system has repaired the data, the correctness of the data warehouse data is guaranteed, and the correctness of the dependent data analysis results is guaranteed.

[0170] Exemplarily, for multiple different data change events, multiple corresponding change tracking events can be generated, and according to the change impact scope (i.e., target downstream business) and change adaptation rules, a change tracking event list including multiple change tracking events can be automatically generated. See Table 5 for some example tables of the change tracking event list. As shown in Table 5, the list lists multiple data change events, and the data change events include information such as change content, change impact scope, adaptation plan, and event status. For example, the event status in the first row of Table 5 is "executed", indicating that the target downstream business "Tb1" has completed the corresponding data adaptation operation; the event status in the other rows of Table 5 is "not executed", indicating that these target downstream businesses have not completed the corresponding data adaptation operation.

[0171] Table 5 Partial example of change tracking event list

[0172]

[0173] Next, a specific example is used to illustrate the technical solution of the embodiment of the present invention. For a schematic diagram of the specific implementation process of the data processing method provided by the embodiment of the present invention, see Figure 7 .like Figure 7As shown, the data processing method can be integrated into a data processing device, and a data change event can be generated when a data source associated with a data warehouse changes. The data processing device can monitor the data change event, and analyze the impact range of the data change event according to the pre-created dependency table, the data full reference table, and the data change adaptation solution management library, determine the data impact range of the change event (i.e., the target downstream business), and determine the adaptation solution corresponding to the data change time. Further, on the one hand, a change impact notification can be performed, for example, a change impact notification is sent to the person in charge of the data affected by the change event, including the change data source information, the change content, and the change impact; on the other hand, a change tracking event can be generated and sent to the downstream business system, so that the corresponding target downstream business users can take corresponding data adaptation operations for the data change event. Subsequently, the data processing device can also accept the feedback operation data returned by the target downstream business system, and adjust the event status of the change tracking event according to the feedback operation data, so that the execution result of the change tracking event can be determined according to the event status of the data change event.

[0174] The technical solution of the embodiment of the present invention, after obtaining the target downstream business corresponding to the data change event, can also generate a change tracking event based on the target downstream business and the change content in the data change event, and display the change tracking event or send the change tracking event to the terminal device corresponding to the target downstream business, which can prompt the user corresponding to the target downstream business that a data source change event has occurred, attract the attention of the downstream business users, so as to prompt the downstream business users to synchronize data in time to ensure the adaptability of the data source change under the global data. Furthermore, in response to the triggering operation of the change tracking event, feedback operation data is received, and when the feedback operation data is the preset operation data, the event status of the change tracking event is adjusted to determine the execution result of the change tracking event based on the event status, and the change tracking event is used to follow up whether the data affected by the data source change has been repaired. By ensuring that the affected downstream business system has repaired the data, the correctness of the data warehouse data is guaranteed, and the correctness of the dependent data analysis results is guaranteed.

[0175] Figure 8 This is a structural diagram of a data processing device provided by an embodiment of the present invention. The device includes: a change type determination module 710 and a downstream business determination module 720.

[0176] The change type determination module 710 is used to determine, in response to a data change event, a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type;

[0177] The downstream business determination module 720 is used to call the target dependency table corresponding to the target change type, so as to determine the target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event; wherein the dependency is used to characterize at least one data table on which at least one downstream business depends, the source link information of at least one data table, the fields in the data table, and at least one item of the enumeration values ​​under the field.

[0178] On the basis of the above-mentioned device, optionally, the change type determination module 710 is specifically used to determine that the target change type is a data source level change type when the data change event is a change in data source information; when the data change event is a change in table information of a data table, determine the target change type is a data source table level change type; when the data change event is a change in a field type in a data table, determine the target change type is a data source field level change type; when the data change event is a change in an enumeration value under at least one field in a data table, determine the target change type is a field enumeration value change type.

[0179] Based on the above device, optionally, the data source information includes at least one of a data source name, a data source user name, and a data source password, the table information includes a table name and a table field, and the field enumeration value is a content value that can be selected and displayed under the field.

[0180] Based on the above-mentioned device, optionally, the change type determination module 710 is further specifically used to obtain at least one data table, and collect the enumeration values ​​under at least one field in the at least one data table to obtain the actual global enumeration information of each field; when there is a difference between the theoretical global enumeration information and the actual global enumeration information of each field created in advance, it is determined that the enumeration value under the field has changed.

[0181] Based on the above device, optionally, the data processing device further includes: a dependency table determination module;

[0182] A dependency table determination module is used to obtain, for at least one downstream business, at least one data table on which the downstream business depends, source link information of at least one data table, table information of at least one data table, field type of each field in at least one data table, and enumeration value of at least one field in at least one data table; wherein the source link information is used to characterize the information on which the data table used by the downstream business depends; determine a dependency table corresponding to a data source level change type based on a business identifier of at least one downstream business, at least one data table corresponding to the business identifier, and source link information of at least one data table; determine a dependency table corresponding to a data source table level change type based on a business identifier of at least one downstream business and table information corresponding to the business identifier; determine a dependency table corresponding to a data source field level change type based on a business identifier of at least one downstream business and field type of each field in at least one data table corresponding to the business identifier; determine a dependency table corresponding to a field enumeration value change type based on a business identifier of at least one downstream business and enumeration value of at least one field in at least one data table corresponding to the business identifier.

[0183] Based on the above-mentioned device, optionally, the change content includes at least one of a data source change, a table information change, a field type change, and an enumeration value change. The downstream business determination module 720 is specifically used to determine the target downstream business corresponding to the data change event based on the target dependency table and at least one of the data source change, table information change, field type change, and enumeration value change in the change content.

[0184] Based on the above device, optionally, the data processing device further includes: a change event generating module;

[0185] The change event generation module is used to generate a change tracking event based on the target downstream business and the change content in the data change event, and display the change tracking event or send the change tracking event to the terminal device corresponding to the target downstream business.

[0186] Based on the above device, optionally, the change tracking event also includes an adaptation plan corresponding to the change content; wherein, the change content includes the changed table name, and the adaptation plan is the data source dependent table name change; the change content is the addition of new fields, and the adaptation plan is to add new fields to the data table or not to add new fields; the change content is the deletion of fields, and the adaptation plan includes abnormal deletion and notifying the upstream or invalid field deletion to set the content of the deleted field to empty; the change content is the change of the field type of the target field, and the adaptation plan includes copying the original data table to which the target field belongs, updating the field type of the target field to the target type, and deleting the original data table.

[0187] Based on the above device, optionally, the data processing device further includes: an event result determination module;

[0188] The event result determination module is used to receive feedback operation data in response to a trigger operation on a change tracking event; when the feedback operation data is preset operation data, adjust the event state of the change tracking event to determine the execution result of the change tracking event based on the event state.

[0189] According to the technical solution of the embodiment of the present invention, when a data change event occurs in a data source, a target change type corresponding to the data change event can be determined in response to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type. By responding to the data change event, data changes at different granularities of the source level, table level, field level, and enumeration value level of the data source can be captured in a timely manner, thereby improving the capture scope and real-time performance of the data source change content; further, a target dependency table corresponding to the target change type can be retrieved to determine the target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event, wherein the dependency relationship is used to characterize at least one data table, source link information of at least one data table, a field in a data table, and at least one of the enumeration values ​​under the field on which at least one downstream business depends. According to the technical solution of the present embodiment, the downstream business affected by the data change event can be determined through the dependency table, thereby prompting the downstream business to perform data adaptation operations, thereby improving the adaptability and accuracy of the data source change under the global data.

[0190] The data processing device provided in the embodiment of the present invention can execute the data processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0191] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0192] Fig. 9 The present invention is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Fig. 9 A block diagram of an exemplary electronic device 80 suitable for implementing exemplary embodiments of the present invention is shown. Fig. 9 The electronic device 80 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0193] like Fig. 9As shown, the electronic device 80 is in the form of a general computing device. The components of the electronic device 80 may include but are not limited to: one or more processors or processing units 801, a system memory 802, and a bus 803 connecting different system components (including the system memory 802 and the processing unit 801).

[0194] Bus 803 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0195] The electronic device 80 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 80, including volatile and non-volatile media, removable and non-removable media.

[0196] The system memory 802 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 804 and / or cache memory 805. The electronic device 80 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 806 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 9 not shown, usually called a "hard drive"). Although Fig. 9 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 803 via one or more data medium interfaces. The memory 802 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0197] A program / utility 808 having a set (at least one) of program modules 807 may be stored, for example, in the memory 802, such program modules 807 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 807 generally perform the functions and / or methods of the embodiments described herein.

[0198] The electronic device 80 may also communicate with one or more external devices 809 (e.g., keyboards, pointing devices, displays 810, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 80, and / or may communicate with any device that enables the electronic device 80 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 811. Furthermore, the electronic device 80 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 812. As shown, the network adapter 812 communicates with other modules of the electronic device 80 via the bus 803. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 80, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0199] The processing unit 801 executes various functional applications and page processing by running the programs stored in the system memory 802, for example, implementing the data processing method provided by the embodiment of the present invention.

[0200] An embodiment of the present invention further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to perform a data processing method, the method comprising:

[0201] In response to a data change event, determining a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type;

[0202] Retrieving a target dependency table corresponding to the target change type, so as to determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event;

[0203] The dependency relationship is used to characterize at least one data table on which at least one downstream service depends, source link information of the at least one data table, a field in the data table, and at least one item of an enumeration value under the field.

[0204] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0205] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0206] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0207] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0208] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that: include: In response to a data change event, determining a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type; Retrieving a target dependency table corresponding to the target change type, so as to determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event; The dependency relationship is used to characterize at least one data table on which at least one downstream service depends, source link information of the at least one data table, a field in the data table, and at least one item of an enumeration value under the field.

2. The method according to claim 1, characterized in that: The determining the target change type corresponding to the data change event includes: When the data change event is a change in data source information, determining that the target change type is a data source level change type; When the data change event is a change in table information of a data table, determining that the target change type is a data source table-level change type; When the data change event is a change in the field type in the data table, determining that the target change type is a data source field-level change type; When the data change event is a change in an enumeration value under at least one field in the data table, the target change type is determined to be a field enumeration value change type.

3. The method according to claim 2, characterized in that The data source information includes at least one of a data source name, a data source user name, and a data source password; the table information includes a table name and a table field; and the field enumeration value is a content value that can be selected and displayed under the field.

4. The method according to claim 2, characterized in that: Determining that the data change event is a change in the enumeration value of at least one field in the data table includes: Acquire at least one data table, and collect enumeration values ​​under at least one field in the at least one data table to obtain actual global enumeration information of each field; When there is a difference between the theoretical global enumeration information and the actual global enumeration information of each field created in advance, it is determined that the enumeration value under the field is changed.

5. The method according to claim 1, characterized in that The method further comprises: For the at least one downstream business, obtaining at least one data table on which the downstream business depends, source link information of the at least one data table, table information of the at least one data table, field type of each field in the at least one data table, and enumeration value of at least one field in the at least one data table; wherein the source link information is used to characterize information on which the data table used by the downstream business depends; Determine a dependency table corresponding to the data source level change type according to the service identifier of the at least one downstream service, the at least one data table corresponding to the service identifier, and source link information of the at least one data table; Determine, according to the service identifier of the at least one downstream service and the table information corresponding to the service identifier, a dependency table corresponding to the data source table-level change type; Determine a dependency table corresponding to the field-level change type of the data source according to the business identifier of the at least one downstream business and the field type of each field in the at least one data table corresponding to the business identifier; According to the service identifier of the at least one downstream service and the enumeration value of at least one field in the at least one data table corresponding to the service identifier, a dependency table corresponding to the change type of the field enumeration value is determined.

6. The method according to claim 1, characterized in that The change content includes at least one of a data source change, a table information change, a field type change, and an enumeration value change, and determining a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event includes: A target downstream business corresponding to the data change event is determined based on the target dependency table and at least one of a data source change, a table information change, a field type change, and an enumeration value change in the change content.

7. The method according to claim 1, characterized in that The method further comprises: Based on the target downstream business and the change content in the data change event, a change tracking event is generated, and the change tracking event is displayed or sent to a terminal device corresponding to the target downstream business.

8. The method according to claim 7, characterized in that The change tracking event also includes an adaptation solution corresponding to the change content; The change content includes the changed table name, and the adaptation solution is the data source dependent table name change; The changed content is a newly added field, and the adaptation solution is to add the newly added field to the data table or not to add the newly added field; The changed content is a deleted field, and the adaptation scheme includes abnormal deletion and notifying the upstream or invalid field deletion to set the content of the deleted field to empty; The change content is a change in the field type of the target field, and the adaptation solution includes copying the original data table to which the target field belongs, updating the field type of the target field to the target type, and deleting the original data table.

9. The method according to claim 7, characterized in that: The method further comprises: In response to a triggering operation on the change tracking event, receiving feedback operation data; When the feedback operation data is preset operation data, an event state of the change tracking event is adjusted to determine an execution result of the change tracking event based on the event state.

10. A data processing device, characterized in that: include: a change type determination module, configured to determine, in response to a data change event, a target change type corresponding to the data change event, wherein the target change type includes at least one of a data source level change type, a data source table level change type, a data source field level change type, and a field enumeration value change type; A downstream business determination module, used to retrieve a target dependency table corresponding to the target change type, so as to determine a target downstream business corresponding to the data change event based on the target dependency table and the change content in the data change event; The dependency relationship is used to characterize at least one data table on which at least one downstream service depends, source link information of the at least one data table, a field in the data table, and at least one item of an enumeration value under the field.

11. A server, characterized in that: The server includes: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the data processing method as described in any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the data processing method according to any one of claims 1 to 9.

Images