A code value unification method, device, computer equipment and medium

Abstract

The present application provides a code value unification method, device, computer equipment and medium. According to the method, the code value unification device can obtain one or more source tables, and check whether the code value and field corresponding to the data in the source table exist in the standard code value library. If the code value and field exist in the standard code value library, the first classification method and the second classification method are determined, and the code value in the source table is adjusted. Through the above method, data from multiple data sources can be integrated through this method, avoiding problems such as data processing errors and data comprehension errors caused by the inability to integrate data. Furthermore, the method can realize the unified code value standardization process without using complex code logic, thereby improving data processing efficiency. In addition, through this method, the impact of different data code values ​​caused by different classification methods can also be reduced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a code value unification method, device, computer equipment and medium. Background Art

[0002] Faced with complex market competition, many enterprises have proposed plans and implementation plans for establishing enterprise-level data warehouses based on their accumulated data and core businesses over the years, laying the foundation for further development of the enterprise. However, in the process of building a basic data platform, that is, in the process of building an enterprise-level data warehouse, enterprises often encounter inconsistent data code values, which makes data integration difficult, data application communication difficult, and may also lead to data processing errors, ambiguity in data understanding, and other problems.

[0003] In order to solve the above problems, complex code logic is usually used to deal with the problems caused by inconsistent data code values ​​during data integration. Understandably, this processing method is difficult and labor-intensive, and as the code values ​​change, it often leads to unstable codes, frequent modifications, and a series of chain problems.

[0004] Therefore, how to unify the different code values ​​of different data sources in enterprises and data warehouses is a problem that needs to be solved urgently. Summary of the invention

[0005] The present application provides a code value unification method, device, computer equipment and medium. According to the method, the code value unification device can obtain one or more source tables, and check whether the code value and field corresponding to the data in the source table exist in the standard code value library. If the code value and field exist in the standard code value library, the first classification method and the second classification method are determined, and the code value in the source table is adjusted. Through the above method, data from multiple data sources can be integrated, and the use of complex code logic for unified code value processing can be avoided, thereby improving data processing efficiency. In addition, through this method, the impact of different data code values ​​caused by different classification methods can also be reduced.

[0006] In the first aspect, the present application provides a method for unifying code values. The method may include: obtaining a source table; the same source table contains data from the same data source; the data contained in the source table is represented by a first code value; there is a one-to-one correspondence between the first code value and the first-level field; the first-level field is used to represent the specific meaning of the first code value; detecting whether there is a standard field identical to the first-level field in the standard code value library; the standard code value library includes a correspondence between standard fields and standard code values; if there is a standard field identical to the first-level field in the standard code value library, determining a first classification method and a second classification method; the first classification method is a classification method corresponding to the first-level field in the source table; the second classification method is a classification method corresponding to the standard field identical to the first-level field in the standard code value library; comparing the first classification method and the second classification method, and adjusting the first code value.

[0007] In the solution provided by the present application, the code value unification device can call the standard code value library. The standard code value library contains the correspondence between standard fields and standard code values. The code value unification device can obtain one or more source tables, and check whether the code value and field corresponding to the data in the source table exist in the standard code value library. If the code value and field exist in the standard code value library, the first classification method and the second classification method are determined, and the code value in the source table is adjusted. Through this method, data from multiple data sources can be integrated, avoiding problems such as data processing errors and data comprehension errors caused by the inability to integrate data. Furthermore, this method can realize the unified code value standardization processing process without using complex code logic, thereby improving data processing efficiency. In addition, through this method, the impact of different data code values ​​caused by different classification methods can also be reduced.

[0008] In combination with the first aspect, in a possible implementation of the first aspect, the comparing the first classification method and the second classification method and adjusting the first code value specifically include: determining a first classification field and a second classification field; the first classification field is used to represent the classification category corresponding to the first classification method; the second classification field is used to represent the classification category corresponding to the second classification method; comparing the first classification field and the second classification field; in the case that the first classification field and the second classification field have the same field, determining the same field in the first classification field and the second classification field, and the first classification code value and the second classification code value corresponding to the same field; there is a one-to-one correspondence between the first classification code value and the first classification field; there is a one-to-one correspondence between the second classification code value and the second classification field; replacing the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field.

[0009] In combination with the first aspect, in a possible implementation of the first aspect, the method also includes: in a case where the first classification field and the second classification field have some identical fields, recording the second classification field and the second classification code value, and sending prompt information to the data source; the prompt information is used to prompt that the first classification method corresponding to the second classification field and the second classification code value is a standard classification method.

[0010] In combination with the first aspect, in a possible implementation of the first aspect, after replacing the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field, the method also includes: if a new source table input is detected, determining whether there is a field in the new source table that is identical to the second classification field; if there is a field in the new source table that is identical to the second classification field, replacing the code value corresponding to the identical field according to the second classification code value corresponding to the second classification field.

[0011] It is understandable that the data from different data sources may have different corresponding code values ​​when representing the same thing, which may affect subsequent processing. In the solution provided in the present application, if for the existing source table, there is a situation where the first classification field and the second classification field have the same field, this indicates that there may be differences in the classification methods. The code value unification device can record the standard fields and standard code values ​​(i.e., the second classification field and the second classification code value) corresponding to the corresponding standard classification method (i.e., the second classification method). When there is a new source table input, the code value unification device can check the fields corresponding to the data in the new source table. If there is a field that is the same as the second classification field, the code value unification device replaces the code value corresponding to the field that is the same as the second classification field in the new source table with the second classification code value. In this way, possible code value inconsistencies can be screened in advance.

[0012] In combination with the first aspect, in a possible implementation of the first aspect, after replacing the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field, the method also includes: obtaining an updated source table, and storing the updated source table in the data warehouse detail layer.

[0013] In combination with the first aspect, in a possible implementation of the first aspect, after detecting whether there is a standard field identical to the first-level field in the standard code value library, the method also includes: if there is no standard field identical to the first-level field in the standard code value library, determining the first classification method, the first classification field and the first classification code value; the first classification field is used to represent the classification category corresponding to the first classification method; there is a one-to-one correspondence between the first classification code value and the first classification field; and storing the correspondence between the first classification field and the first classification code value in the standard code value library.

[0014] In the solution provided by the present application, if there is no standard field in the standard code value library that is the same as the primary field in the source table, the code value unification device can temporarily use the first classification field in the source table as the standard field, and temporarily use the first classification code value in the source table as the standard code value. The code value unification device can also store the corresponding relationship between the standard field and the standard code value in the standard code value library. In this way, the source table obtained later has a temporary standard code value for reference, which temporarily solves the problem of inconsistent code values.

[0015] In combination with the first aspect, in a possible implementation of the first aspect, after storing the correspondence between the first classification field and the first classification code value in the standard code value library, the method also includes: adding a temporary standard label to the first classification field and the first classification code value; the temporary standard label is used to indicate that the first classification field is a temporary standard field, and the first classification code value is a temporary standard code value.

[0016] In the solution provided in the present application, the code value unification device can also add a temporary standard label to the newly added standard field and standard code value so that the standard field and standard code value can be adjusted later.

[0017] In the second aspect, the present application provides a code value unification device. The device may include: a source table acquisition unit, used to acquire a source table; the same source table contains data from the same data source; the data contained in the source table is represented by a first code value; there is a one-to-one correspondence between the first code value and the first-level field; the first-level field is used to represent the specific meaning of the first code value; a detection unit, used to detect whether there is a standard field identical to the first-level field in the standard code value library; the standard code value library includes a correspondence between standard fields and standard code values; a determination unit, used to determine a first classification method and a second classification method when there is a standard field identical to the first-level field in the standard code value library; the first classification method is a classification method corresponding to the first-level field in the source table; the second classification method is a classification method corresponding to the standard field identical to the first-level field in the standard code value library; a comparison and code value adjustment unit, used to compare the first classification method and the second classification method, and adjust the first code value.

[0018] In combination with the second aspect, in a possible implementation of the second aspect, the comparison and code value adjustment unit, when used to compare the first classification method and the second classification method and adjust the first code value, is specifically used to: determine a first classification field and a second classification field; the first classification field is used to represent the classification category corresponding to the first classification method; the second classification field is used to represent the classification category corresponding to the second classification method; compare the first classification field and the second classification field; when the first classification field and the second classification field have the same field, determine the same field in the first classification field and the second classification field, and the first classification code value and the second classification code value corresponding to the same field; there is a one-to-one correspondence between the first classification code value and the first classification field; there is a one-to-one correspondence between the second classification code value and the second classification field; replace the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field.

[0019] In combination with the second aspect, in a possible implementation of the second aspect, the determination unit is also used to: determine the first classification method, the first classification field and the first classification code value when there is no standard field identical to the first-level field in the standard code value library; the first classification field is used to indicate the classification category corresponding to the first classification method; there is a one-to-one correspondence between the first classification code value and the first classification field; the device also includes: a storage unit, used to store the correspondence between the first classification field and the first classification code value in the standard code value library.

[0020] In combination with the second aspect, in a possible implementation of the second aspect, the device also includes: a label adding unit, used to add a temporary standard label to the first classification field and the first classification code value; the temporary standard label is used to indicate that the first classification field is a temporary standard field, and the first classification code value is a temporary standard code value.

[0021] In a third aspect, the present application provides a computer device, characterized in that it includes: a processor and a memory, the memory storing a computer program, and the processor executes the computer program in the memory to implement the method described in the first aspect and any one of the possible implementation methods of the first aspect.

[0022] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, characterized in that when the above-mentioned computer program runs on a computer, the above-mentioned computer executes the method described in the first aspect and any one of the possible implementation methods of the first aspect.

[0023] It can be understood that the code value unification device provided in the second aspect, the computer device provided in the third aspect, and the computer-readable storage medium provided in the fourth aspect are all used to execute the code value unification method provided in the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects provided in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0025] Figure 1 A flowchart of a code value unification method provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of a code value unification device provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0029] In order to solve the problem that the code values ​​representing the same thing from different data sources are different due to inconsistent encoding methods, the present application provides a code value unification method, device, computer equipment and medium. According to the method, the code value unification device can obtain one or more source tables, and check whether the code value and field corresponding to the data in the source table exist in the standard code value library. If the code value and field exist in the standard code value library, the first classification method and the second classification method are determined, and the code value in the source table is adjusted. Through this method, data from multiple data sources can be integrated, avoiding problems such as data processing errors and data comprehension errors caused by the inability to integrate data. Furthermore, the method can realize the unified code value standardization process without using complex code logic, thereby improving data processing efficiency. In addition, through this method, the impact of different data code values ​​caused by different classification methods can also be reduced.

[0030] The following is a detailed description of a code value unification method provided by the present application. Figure 1 As shown, the code value unification method may include the following steps:

[0031] S101: The code value unification device obtains a source table.

[0032] Specifically, the code value unification device can obtain one or more source tables from multiple data sources. The data contained in the source table can be represented by a first code value. There is a one-to-one correspondence between the first code value and the first-level field. The first-level field is used to represent the specific meaning of the first code value.

[0033] Exemplarily, the source table may include a gender field. That is, the first-level field may be a gender field. Specifically, the first-level field may include male and female. The first code value may include 0 and 1. When the first code value is 0, it may represent male. When the first code value is 1, it may represent female.

[0034] In some embodiments of the present application, the same source table may contain data from the same data source. That is, the source table may include raw data from the same channel. It should be noted that different source tables may include raw data from different channels. It is understandable that the channel mentioned here may refer to a data source. For example, a channel may include a department, a company, etc.

[0035] In some embodiments of the present application, the data contained in the source table may be the original data stored in the ODS layer of the data warehouse. It is understandable that the full name of ODS is Operation Data Store, which means data preparation area in Chinese. The data table of the source system of the data warehouse is usually stored intact, which is called the ODS layer, also known as the preparation area. They are the source of data processed by the subsequent data warehouse layer.

[0036] In some embodiments of the present application, in addition to the first code value, the source table may also include code values ​​of other levels. According to the above, the first code value in the source table corresponds to the first-level field one by one. That is, the first code value in the source table is the code value corresponding to the first-level field.

[0037] It is understandable that the source table may also include secondary fields, and code values ​​corresponding to the secondary fields. In the present application, the code values ​​corresponding to the primary fields may be recorded as primary code values, and the code values ​​corresponding to the secondary fields may be recorded as secondary code values. It should be noted that the primary fields may be understood as primary classifications, and the secondary fields may be understood as secondary classifications. It is understandable that the secondary fields represent the refined categories of the primary fields. The primary fields and the secondary fields have a corresponding relationship. For example, there may be a one-to-one corresponding relationship between the primary fields and the secondary fields. For another example, there may be a one-to-many corresponding relationship between the primary fields and the secondary fields.

[0038] Exemplarily, the first-level field may be a cuisine field. The first-level field may include Sichuan cuisine, Cantonese cuisine, Shandong cuisine, etc. The second-level field may be a detailed category of cuisine. The second-level field corresponding to Sichuan cuisine may include boiled pork slices, mao xu wang, etc. The first-level code value corresponding to Sichuan cuisine is 1, the code value corresponding to boiled pork slices is 11, and the code value corresponding to mao xu wang is 12. It can be understood that the first-level code value 1 corresponds to the second-level code values ​​11 and 12. In other words, the correspondence between the first-level code value 1 and its corresponding second-level code value is one-to-many.

[0039] In some embodiments of the present application, the code value unification device can monitor the acquired source table in real time. That is, the code value unification device can not only actively acquire the source table, but also receive the source table sent in other forms. It can be understood that the code value unification device can monitor the update of the source table in real time, including monitoring the input of new source tables and the deletion of old source tables.

[0040] In some embodiments of the present application, the code value unification device can also monitor the fields and code values ​​in the source table. For example, if there are newly input fields and code values, the code value unification device can obtain and process them in time. For another example, if there is a request to delete a field and code value, the code value unification device can identify and delete it in time.

[0041] In some embodiments of the present application, a technician may also manually input a source table, field, and code value. The code value unification device may manage the source table, field, and code value obtained in this manner.

[0042] S102: The code value unification device detects whether there is a standard field identical to the primary field in the standard code value library.

[0043] Specifically, the code value unification device can detect whether there is a standard field in the standard code value library that is the same as the primary field in the source table. It can be understood that the standard code value library includes the corresponding relationship between the standard field and the standard code value.

[0044] In some embodiments of the present application, the standard code value library can be a UDF package. UDF stands for User Defined Function, i.e., a user-defined function. Users can also create custom functions to meet different computing needs. The UDF package can be understood as a package that includes user-defined functions for implementing code value standardization.

[0045] Exemplarily, the primary fields in the source table include a gender field, and the code value unification device can detect whether the gender field exists in the standard code value library.

[0046] S103: If there is a standard field identical to the primary field in the standard code value library, the code value unification device determines a first classification method and a second classification method.

[0047] Specifically, if the code value unification device detects that there is a standard field in the standard code value library that is the same as the first-level field, the first classification method can be determined based on the first-level field, and the second classification method can be determined based on the standard field in the standard code value library that is the same as the first-level field.

[0048] In some embodiments of the present application, the primary field and / or the first code value in the source table may carry relevant information of the first classification method. The first classification method is the classification method corresponding to the primary field. It is understandable that the primary field may represent a classification category of the first classification method. The relevant information of the first classification method may include the primary fields corresponding to other classification categories of the first classification method, and the first code value corresponding to the primary field.

[0049] In some embodiments of the present application, the standard field and / or the standard code value in the standard code value library may carry relevant information of the second classification method. The second classification method is the classification method corresponding to the standard field. It is understandable that the standard field may represent a classification category of the second classification method. The relevant information of the second classification method may include the standard field corresponding to other classification categories of the second classification method, and the standard code value corresponding to the standard field.

[0050] Exemplarily, there is a primary field X in the source table, and the code value unification device detects that the standard code value library includes the standard field X. The code value unification device can obtain relevant information of the first classification method according to the primary field X, and can also obtain relevant information of the second classification method according to the standard field X. Specifically, the code value unification device can obtain the fields X and Y corresponding to the first classification method, and can obtain the fields X, Y and Z corresponding to the second classification method. It can be understood that X and Y are the first classification fields. X, Y and Z are the second classification fields. The code value unification device can determine the first classification code value. The first classification code value includes 0 and 1. Specifically, the first classification code value corresponding to the first classification field X is 0, and the first classification code value corresponding to the first classification field Y is 1. The code value unification device can determine the second classification code value. The second classification code value includes x, y and z. Specifically, the second classification code value corresponding to the second classification field X is x, the second classification code value corresponding to the second classification field Y is y, and the second classification code value corresponding to the second classification field Z is z. The code value unification device can determine that the same parts of the first classification field and the second classification field are X and Y. The first classification code value corresponding to X is 0, and the second classification code value corresponding to X is x. The first classification code value corresponding to Y is 1, and the second classification code value corresponding to Y is y. It can be understood that the code value unification device can replace the code value 0 corresponding to the primary field X in the source table with x, and replace the code value 1 corresponding to the primary field Y in the source table with y.

[0051] S104: The code value unification device compares the first classification method and the second classification method, and adjusts the first code value.

[0052] It is understandable that after the code value unification device executes step S103, it can continue to execute step S104. The code value unification device can compare the first classification method and the second classification method. Specifically, the code value unification device can determine the first classification field and the second classification field, and compare the first classification field and the second classification field. Among them, the first classification field is used to represent the classification category corresponding to the first classification method; the second classification field is used to represent the classification category corresponding to the second classification method. It is understandable that in the case where the standard code value library includes the same standard field as the first-level field in the source table, the first classification field may include the first-level field mentioned in the above steps S102 and S103, and the second classification field may include the same standard field as the first-level field mentioned in step S103. Of course, the first classification field may also include fields for representing other classification categories. Similarly, the second classification field may also include fields for representing other classification categories.

[0053] It is understandable that there is a one-to-one correspondence between the first classification field and the first classification code value. There is also a one-to-one correspondence between the second classification field and the second classification code value. In some embodiments of the present application, the first code value may include the first classification code value. And the first classification code value may include the first code value corresponding to the primary field mentioned in the above steps S102 and S103.

[0054] In some embodiments of the present application, the code value unification device can determine that there are identical fields in the first classification field and the second classification field. In this case, the code value unification device can determine the identical fields in the first classification field and the second classification field, and the first classification code value and the second classification code value corresponding to the identical fields.

[0055] It is understandable that the code value unification device can adjust the first code value in the source table. Specifically, the code value unification device can replace the first classification code value in the source table with the second classification code value.

[0056] In some embodiments of the present application, when there are some identical fields in the first classification field and the second classification field, the code value unification device can record the second classification field and the second classification code value, and send a prompt message to the data source. The prompt message is used to prompt that the second classification method corresponding to the second classification field and the second classification code value is a standard classification method. It can be understood that the data sources mentioned in the present application can be different companies, different departments, different staff members, etc. The prompt information mentioned here can be used to prompt that when outputting data from the data source, the code value corresponding to the data is determined according to the standard classification method.

[0057] In some embodiments of the present application, if the code value unification device detects the presence of an input of a new source table, it can determine whether there is a field identical to the second classification field in the new source table. If there is a field identical to the second classification field in the new source table, the code value unification device can replace the code value corresponding to the identical field according to the second classification code value corresponding to the second classification field. In some embodiments of the present application, after the first classification code value in the source table is replaced with the second classification code value, a DWD table can be generated, and the DWD table can be stored in the DWD layer of the data warehouse. It can be understood that the full name of DWD is DataWarehouse Detail, which means data warehouse detail layer in Chinese. The DWD layer is the isolation layer between the business layer and the data warehouse, and it can also be understood as the core layer of the data warehouse.

[0058] In some embodiments of the present application, there are fields of other levels (for example, second-level fields) in addition to the first-level fields in the source table. The code value unification device can adjust the code values ​​corresponding to the fields of other levels in the manner of steps S101 to S104. The specific process can be referred to the above embodiments and will not be repeated here.

[0059] In some embodiments of the present application, there is no standard field in the standard code value library that is the same as the primary field in the source table. In this case, the code value unification device can determine the first classification method, the first classification field, and the first classification code value, and store the correspondence between the first classification field and the first classification code value in the standard code value library. In some embodiments of the present application, the code value unification device can add a temporary standard label to the first classification field and the first classification code value stored in the standard code value library. The temporary standard label is used to indicate that the first classification field is a temporary standard field, and the first classification code value is a temporary standard code value.

[0060] In some embodiments of the present application, the code value unification device may include a code value management page so that technicians can uniformly manage the classification of business concepts by different business source ends and facilitate the maintenance of code values ​​by technicians. In this case, the code value unification device can also provide users with code value query services to avoid ambiguity in data usage communication.

[0061] In summary, the code value unification method provided by the present application can integrate data from multiple data sources, avoiding problems such as data processing errors and data misunderstanding errors caused by the inability to integrate data. Furthermore, the method can realize the unified code value standardization process without using complex code logic, thereby improving data processing efficiency. In addition, through this method, the impact of different data code values ​​caused by different classification methods can also be reduced.

[0062] The above embodiment has described in detail the code value unification method provided by the present application. In order to facilitate better implementation of the above solution, correspondingly, the following also provides related devices for cooperating in the implementation.

[0063] like Figure 2 As shown, Figure 2 A schematic diagram of a code value unification device provided in an embodiment of the present application, wherein the code value unification device is used to perform the above Figure 1 The present application does not limit the division of the functional units of the code value unification device, and the various units in the code value unification device can be increased, reduced or merged as needed. In addition, the operations and / or functions of the various units in the code value unification device are respectively to achieve the above Figure 1 For the sake of brevity, the corresponding process of the described method will not be repeated here.

[0064] Figure 2 An exemplary division of functional units is provided: the code value unification device 200 may include a source table acquisition unit 210 , a detection unit 220 , a determination unit 230 and a comparison and code value adjustment unit 240 .

[0065] The source table acquisition unit 210 is used to acquire a source table. It is understandable that the same source table contains data from the same data source. The data contained in the source table is represented by a first code value. There is a one-to-one correspondence between the first code value and the first-level field. The first-level field is used to represent the specific meaning of the first code value.

[0066] The detection unit 220 is used to detect whether there is a standard field identical to the primary field in the standard code value library. The standard code value library includes the correspondence between the standard field and the standard code value.

[0067] The determination unit 230 is used to determine the first classification method and the second classification method when there is a standard field identical to the primary field in the standard code value library. The first classification method is the classification method corresponding to the primary field in the source table. The second classification method is the classification method corresponding to the standard field identical to the primary field in the standard code value library.

[0068] The comparison and code value adjustment unit 240 is used to compare the first classification method and the second classification method, and adjust the first code value.

[0069] In some embodiments of the present application, the comparison and code value adjustment unit 240, when used to compare the first classification method and the second classification method and adjust the first code value, is specifically used to: determine the first classification field and the second classification field, and compare the first classification field and the second classification field. In the case where the first classification field and the second classification field have the same field, the comparison and code value adjustment unit can be used to determine the same field in the first classification field and the second classification field, as well as the first classification code value and the second classification code value corresponding to the same field, and replace the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field. Among them, the first classification field is used to represent the classification category corresponding to the first classification method, and the second classification field is used to represent the classification category corresponding to the second classification method. There is a one-to-one correspondence between the first classification code value and the first classification field. There is a one-to-one correspondence between the second classification code value and the second classification field.

[0070] In some embodiments of the present application, the determination unit 230 is further used to: determine the first classification method, the first classification field and the first classification code value when there is no standard field identical to the primary field in the standard code value library.

[0071] In some embodiments of the present application, the code value unification device 200 may further include: a storage unit 250, configured to store the correspondence between the first classification field and the first classification code value in a standard code value library.

[0072] In some embodiments of the present application, the code value unification device 200 may further include: a label adding unit 260, configured to add a temporary standard label to the first classification field and the first classification code value. It is understood that the temporary standard label is used to indicate that the first classification field is a temporary standard field, and the first classification code value is a temporary standard code value.

[0073] See also Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 300 may be the device 200 for unifying the code value of the dependency relationship in the above-mentioned embodiment. Figure 3 As shown, the electronic device 300 includes: a processor 310 , a communication interface 320 and a memory 330 , and the processor 310 , the communication interface 320 and the memory 330 are interconnected via an internal bus 340 .

[0074] The processor 310, the communication interface 320 and the memory 330 can be connected via a bus, or can communicate via other means such as wireless transmission. The embodiment of the present application takes the connection via bus 340 as an example, wherein the bus 340 can be a peripheral component interconnect standard (Peripheral Component Interconnect, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The bus 340 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0075] The processor 310 may be composed of one or more general-purpose processors, such as a central processing unit (CPU), or a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. The processor 310 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 330, which enables the electronic device 300 to provide a wide variety of services.

[0076] Specifically, the processor 310 can be composed of at least one general-purpose processor, such as a central processing unit (CPU), or a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. The processor 310 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 330, which enables the electronic device 300 to provide a wide variety of services. In some embodiments of the present application, the processor 310 can be used to execute Figure 1 The code value unification method shown.

[0077] The memory 330 may include a volatile memory (Volatile Memory), such as a random access memory (Random Access Memory, RAM); the memory 330 may also include a non-volatile memory (Non-Volatile Memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a hard disk drive (Hard Disk Drive, HDD) or a solid-state drive (SSD); the memory 330 may also include a combination of the above types. Among them, the memory 330 may store application code and program data. The code of the memory 330 may include code to implement the function of the source table acquisition unit, code to implement the function of the detection unit, code to implement the function of the determination unit, and code to implement the function of the comparison and code value adjustment unit. The functions of the source table acquisition unit include Figure 2 The functions of the source table acquisition unit 210 shown in FIG. 2 include, for example, acquiring a source table. The functions of the detection unit include Figure 2 The functions of the detection unit 220 shown in the figure are as follows. For example, it is to detect whether there is a standard field in the standard code value library that is the same as the primary field. The functions of the determination unit include Figure 2 The functions of the determination unit 230 shown in the figure are as follows. For example, when there is a standard field in the standard code value library that is the same as the primary field, the first classification method and the second classification method are determined. The functions of the comparison and code value adjustment unit include Figure 2 The functions of the comparison and code value adjustment unit 240 are shown as follows: For example, the first classification method and the second classification method are compared, and the first code value is adjusted.

[0078] In some embodiments of the present application, the memory 330 may further include the storage unit 250. The code of the memory 330 may further include a code for implementing the functions of the label adding unit. The functions of the label adding unit include the functions of the label adding unit 260. For example, a temporary standard label is added to the first classification field and the first classification code value.

[0079] It is understandable that the code of the memory 330 can be specifically used to execute step S101 and step S102 in the aforementioned embodiment, which will not be described in detail here.

[0080] It is understandable that the memory 330 may store a standard code value library.

[0081] The communication interface 320 can be a wired interface (such as an Ethernet interface), an internal interface (such as a high-speed serial computer expansion bus (Peripheral Component Interconnect express, PCIe) bus interface), a wired interface (such as an Ethernet interface) or a wireless interface (such as a cellular network interface or a wireless local area network interface) for communicating with other devices or modules.

[0082] Need to explain, Figure 3 This is only a possible implementation of the embodiment of the present application. In practical applications, the electronic device may also include more or fewer components, which is not limited here. Figure 1 The relevant explanations in the embodiments are not repeated here. Figure 3 The electronic device shown may also be a computer cluster composed of multiple computing nodes, which is not specifically limited in this application.

[0083] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a processor, Figure 1 The method flow shown is realized.

[0084] The present application also provides a computer program product. When the computer program product is run on a processor, Figure 1 The method flow shown is realized.

[0085] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD), or a semiconductor medium. The semiconductor medium may be an SSD.

[0086] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A code value unification method, characterized in that: The method comprises: Acquire multiple source tables; the same source table contains data from the same data source, and different source tables contain data from different data sources; the data contained in the source table is original data stored in a data preparation area of ​​a data warehouse, and the source table includes a primary field and a first code value; the primary field is used to represent a specific meaning of the first code value; Detecting whether there is a standard field identical to the primary field in a standard code value library; the standard code value library includes a correspondence between standard fields and standard code values, the standard code value library is a user-defined function UDF program package, and the UDF program package is a program package including a user-defined function for implementing code value standardization; If there is a standard field identical to the primary field in the standard code value library, a first classification method is determined according to the primary field, and a second classification method is determined according to the standard field; the first classification method is the classification method corresponding to the primary field in the source table; the second classification method is the classification method corresponding to the standard field identical to the primary field in the standard code value library; Determine a first classification field corresponding to the first classification method, and determine a second classification field corresponding to the second classification method; the first classification field is used to represent one or more classification categories corresponding to the first classification method; the second classification field is used to represent one or more classification categories corresponding to the second classification method; the first classification field includes one or more fields, and the second classification field includes one or more fields; one field in the first classification field corresponds to one first classification code value, and one field in the second classification field corresponds to one second classification code value; comparing the first classification field and the second classification field; In the case that the first classification field and the second classification field have the same fields, determining the same fields in the first classification field and the second classification field, and the first classification code value and the second classification code value corresponding to the same fields; The first classification code value corresponding to the same field in the source table is replaced with the second classification code value corresponding to the same field. After the first classification code value in the source table is replaced with the second classification code value, a data warehouse detail layer DWD table can be generated. The DWD table is stored in the DWD layer of the data warehouse. The DWD layer is an isolation layer between the business layer and the data warehouse. In the case where the first classification field and the second classification field have some identical fields, the second classification field and the second classification code value are recorded, and prompt information is sent to the data source; the prompt information is used to prompt that the second classification method corresponding to the second classification field and the second classification code value is a standard classification method; If there is no standard field identical to the primary field in the standard code value library, determining the first classification method, the first classification field and the first classification code value; the first classification field is used to indicate the classification category corresponding to the first classification method; Storing the correspondence between the first classification field and the first classification code value in the standard code value library; A temporary standard tag is added to the first classification field and the first classification code value; the temporary standard tag is used to indicate that the first classification field is a temporary standard field, and the first classification code value is a temporary standard code value.

2. The method according to claim 1, characterized in that After replacing the first classification code value corresponding to the same field in the source table with the second classification code value corresponding to the same field, the method further includes: If a new source table input is detected, determining whether the new source table has a field identical to the second classification field; If there is a field identical to the second classification field in the new source table, the code value corresponding to the identical field is replaced according to the second classification code value corresponding to the second classification field.

3. A code value unification device, characterized in that: The device comprises a unit for executing the method according to claim 1 or 2, the device comprising: A source table acquisition unit, used for acquiring a source table; the same source table contains data from the same data source; the data contained in the source table is represented by a first code value; there is a one-to-one correspondence between the first code value and the first-level field; the first-level field is used to represent the specific meaning of the first code value; A detection unit, used to detect whether there is a standard field identical to the primary field in a standard code value library; the standard code value library includes a correspondence between standard fields and standard code values; a determining unit, configured to determine a first classification method and a second classification method when there is a standard field identical to the primary field in the standard code value library; the first classification method is a classification method corresponding to the primary field in the source table; the second classification method is a classification method corresponding to the standard field identical to the primary field in the standard code value library; The comparison and code value adjustment unit is used to compare the first classification method and the second classification method, and adjust the first code value.

4. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program in the memory to implement the method according to claim 1 or 2.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is run on a computer, the computer is caused to execute the method as claimed in claim 1 or 2.

Images