Method and system for extracting features of personnel master data

Abstract

The present application relates to big data processing technical field, especially personnel main data feature extraction method and system for according to preset data feature rule carries out management to data warehouse personnel information, through reading the metadata of all data tables in the data warehouse, the key information in the personnel main data is used as the basic feature, according to the basic feature and specified associated feature to set corresponding feature check function;The basic feature position is located by using the basic feature check function, and the basic feature set stored in the position is extracted;The specified associated feature position is located by using the specified associated feature check function, and the mapping combination relationship of the basic feature and the specified associated feature is extracted;The specified associated feature relationship weight is obtained based on the mapping combination relationship grouping according to the basic feature, and the relationship weight is used to determine the associated feature default value and the extension value.The present application can automatically extract the data warehouse personnel main data feature information, and the personnel main data resource directory classification hierarchical management is facilitated.

Description

Technical Field

[0001] This invention relates to the field of big data processing technology, and in particular to a method and system for extracting features from personnel master data. Background Technology

[0002] As data elements, as a new driving force, enter various fields of the digital economy, while driving industrial innovation and development, the personal information of citizens in data elements needs more compliance management to protect personal information in data circulation, especially the master data of personnel, which plays a basic standardization role in personnel data sharing. The master data of personnel generally has the following characteristics: (1) Uniqueness of identification: It has a unique identification mark, such as citizen ID number, mobile phone number and E-mail address. (2) Consistency of features: It is highly consistent in different departments and different business systems. (3) Long-term validity: It runs through the entire data life cycle. (4) Value range stability: It is not modified with the process of environmental and business changes. (5) Non-derivative: It does not need to be obtained through complex calculation or analysis. It can be seen that protecting the feature information of personnel master data is an important part of personal information protection. At present, the extraction of feature information of personnel master data mainly relies on data dictionary description information or exploration of data content to sort out and clarify the distribution of feature information in the data warehouse, and further support the classification and hierarchical management of personnel information, such as: hierarchical management of personal feature information according to sensitivity. The discrepancies between data dictionary descriptions and actual content, the dynamic updates of data warehouses, and the inefficiency of manually exploring data content to revise data catalogs all significantly impact the compliant and standardized application of personal information. In particular, sensitive features such as ID numbers and mobile phone numbers in personnel master data play a fundamental role in personnel data sharing. Manually exploring and extracting these features is difficult to do comprehensively, promptly, accurately, and securely, and may result in situations where mobile phone numbers cannot be linked, leading to prominent issues in personal information sharing. Summary of the Invention

[0003] To address this, the present invention provides a method and system for extracting personnel master data features. This method automatically extracts personnel master data feature information from a data warehouse through data content verification, facilitating the classification and hierarchical management of personnel master data resource directories in various data tables of the data warehouse.

[0004] According to the design scheme provided by this invention, a method for extracting personnel master data features is provided, used to manage personnel information in a data warehouse based on preset data feature rules, comprising:

[0005] Read the metadata of all data tables in the data warehouse, take the key information in the personnel master data that expands the range of personnel master data feature information through mapping relationship as the basic feature, and set the corresponding feature verification function based on the basic feature and the specified associated feature;

[0006] The basic feature verification function is used to locate the basic feature position in the data warehouse data table and extract the basic feature set stored at that position; the specified associated feature verification function is used to locate the specified associated feature position in the data warehouse data table where the basic feature position is located and extract the mapping combination relationship between the basic feature and the specified associated feature.

[0007] Based on the mapping and combination relationship, a set of records of basic personnel characteristics and specified related characteristics are extracted from the data warehouse. The records are then grouped according to the basic characteristics. The weight of the specified related characteristics is obtained based on the grouping results. The related characteristic value with the highest weight is used as the default value of the related characteristic, and the related characteristic value with a lower weight is used as the extended value of the related characteristic.

[0008] As a master data feature extraction method of the inventors, further, the metadata of all data tables in the data warehouse is read, including:

[0009] First, set a time period for regularly probing the data warehouse, and within each time period, probe the metadata of all data tables in the data warehouse;

[0010] Then, based on the exploration results, the information of each data table is read and the latest record count of the data table is counted. The data table information read includes at least: table name, table comments and partition type information.

[0011] As a method for extracting master data features, the latest record count of the data table is further analyzed according to different partition types. When the data table is a full partition table, the latest full partition of the data table is used to count the number of records. When the data table is an incremental partition table, all partitions of the data table are used to count the number of records. When the data table is a non-partition table, the entire content of the data table is used to count the number of records. The field names, field comments, and field data types of the data table are obtained from the metadata. The obtained field names, field comments, and field data types are merged with the table name, table comments, partition type, and the latest record count of the table and saved as a single record. This process is repeated until all data tables in the data warehouse are traversed. The obtained records are written to the full partition table of the exploration results, and the partition values ​​are set according to the date of the traversal.

[0012] As a master data feature extraction method of the inventors, further, in setting corresponding feature verification functions based on basic features and specified associated features, the corresponding basic feature verification functions and specified associated feature verification functions are set according to the basic feature information rules and specified associated feature information rules, respectively.

[0013] As a master data feature extraction method of the inventors, further, the method utilizes a basic feature verification function to locate the position of basic features in the data warehouse data table and extracts the set of basic features stored at that position, including:

[0014] First, the basic feature verification function is used to traverse the data content of each field in the data table of the data warehouse to initially locate the basic feature position;

[0015] Then, by comparing the basic feature information content in the initially located basic feature location with the first preset threshold, the effective location of the basic feature of the data warehouse personnel is further determined, and at the same time, the set of basic features stored in the effective location of the basic feature of the data warehouse personnel is extracted.

[0016] As a master data feature extraction method of the inventors, further, the method utilizes a specified associated feature verification function to locate the specified associated feature position in the data warehouse data table where the basic feature position is located and extracts the mapping combination relationship between the basic feature and the specified associated feature, including:

[0017] First, the specified association feature verification function is used to traverse the data content of each field in the data table where the basic feature is located, and the location of the specified association feature is initially located.

[0018] Then, the content of associated feature information in the initially determined specified associated feature position is compared with the second preset threshold. Based on the comparison result, the effective position of the specified associated feature of the data warehouse personnel is extracted, and the information mapping combination relationship between the basic features of the personnel and the specified associated features is obtained based on the field relationship in the same data table.

[0019] As a master data feature extraction method of the inventors, further, based on the grouping results, the weights of specified association feature relationships are obtained, the association feature value with the highest weight is used as the default value of the association feature, and the association feature values ​​with non-highest weights are used as extended values ​​of the association feature, including:

[0020] First, based on the grouping results, the number of repetitions of different feature values ​​of the specified associated feature is obtained, and the number of repetitions is used as the weight of the relationship of the specified associated feature.

[0021] Then, the associated feature value with the highest weight is used as the default value of the associated feature, and the associated feature value with a lower weight is used as the extended value of the associated feature.

[0022] Furthermore, the present invention also provides a personnel master data feature extraction system for managing personnel information in a data warehouse according to preset data feature rules, comprising: a data exploration module, a mapping and combination module, and a feature acquisition module, wherein,

[0023] The data exploration module is used to read the metadata of all data tables in the data warehouse, take the key information in the personnel master data that expands the range of personnel master data feature information through mapping relationship as the basic feature, and set the corresponding feature verification function based on the basic feature and the specified associated feature.

[0024] The mapping and combination module is used to locate the position of basic features in the data warehouse data table using the basic feature verification function and extract the set of basic features stored at that position; and to locate the position of specified related features in the data warehouse data table where the basic feature is located using the specified related feature verification function and extract the mapping and combination relationship between the basic feature and the specified related feature.

[0025] The feature acquisition module is used to extract a set of records of basic features and specified related features of personnel in the data warehouse based on the mapping and combination relationship, and group them according to the basic features. Based on the grouping results, the weight of the specified related feature relationship is obtained, and the related feature value with the highest weight is used as the default value of the related feature, and the related feature value with a lower weight is used as the extended value of the related feature.

[0026] The beneficial effects of this invention are:

[0027] This invention automatically obtains the storage location of personnel characteristic information through data content verification. It can automatically identify the location of personnel master data characteristics in massive amounts of data, confirm personnel master data characteristics with verified data content characteristics, and support the improvement of personnel master data resource catalog classification and hierarchical management by automatically extracting personnel master data characteristic information. It can timely and accurately discover, identify and protect sensitive information such as personal identification numbers and mobile phone numbers. It is applicable to personal characteristic information with clear composition rules, including but not limited to personnel identification numbers, mobile phone numbers, email addresses and other information, which facilitates the classification and hierarchical management of personnel master data resource catalogs in various data tables of the data warehouse. Attached image description:

[0028] Figure 1 This is a schematic diagram of the personnel master data feature extraction process in the embodiment;

[0029] Figure 2 This is a schematic diagram of the algorithm for automatically extracting personnel master data feature information in the embodiment;

[0030] Figure 3 This is a schematic diagram of the process for exploring the metadata of a data warehouse table in an embodiment;

[0031] Figure 4 This is a schematic diagram of the process for extracting basic feature information in the embodiment;

[0032] Figure 5 This is a schematic diagram of the process for extracting associated feature information in the embodiment. Detailed implementation method:

[0033] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0034] To address the problems of low efficiency, incomplete or distorted data table descriptions leading to misjudgments, and the difficulty in timely, accurate, and secure manual updating and feature extraction when personnel master data is updated, this invention provides an embodiment, see [link to embodiment]. Figure 1 As shown, a method for extracting personnel master data features is provided, which is used to manage personnel information in a data warehouse according to preset data feature rules, including:

[0035] S101. Read the metadata of all data tables in the data warehouse, take the key information in the personnel master data that expands the range of personnel master data feature information through mapping relationship as the basic feature, and set the corresponding feature verification function based on the basic feature and the specified associated feature.

[0036] S102. Use the basic feature verification function to locate the basic feature position in the data warehouse data table and extract the basic feature set stored at that position; use the specified associated feature verification function to locate the specified associated feature position in the data warehouse data table where the basic feature position is located and extract the mapping combination relationship between the basic feature and the specified associated feature.

[0037] S103. Extract the set of records of basic personnel features and specified related features in the data warehouse based on the mapping and combination relationship, and group them according to the basic features. Obtain the weight of the specified related feature relationship based on the grouping results. Use the related feature value with the highest weight as the default value of the related feature, and use the related feature value with a non-highest weight as the extended value of the related feature.

[0038] It can automatically identify the location of personnel master data features in massive amounts of data, confirm personnel master data features using verified data content features and automatically associate and extract them, support the classification and hierarchical management of personnel master data resource catalogs, and timely and accurately discover, identify and protect sensitive information such as personal identification numbers and mobile phone numbers, thus promoting the optimization of personnel information database management.

[0039] As a preferred embodiment, the metadata of all data tables in the data warehouse is further read, including:

[0040] First, set a time period for regularly probing the data warehouse, and within each time period, probe the metadata of all data tables in the data warehouse;

[0041] Then, based on the exploration results, the information of each data table is read and the latest record count of the data table is counted. The data table information read includes at least: table name, table comments and partition type information.

[0042] It can be set to read the metadata of all data tables in the data warehouse at a set time every day to explore and obtain information such as the table name, field names, and number of records for each data table. Based on the personnel master data, representative personnel identification numbers are selected as the basic feature information of the personnel master data. On the basis of the basic feature, more feature information of the personnel master data is associated, such as mobile phone numbers, gradually expanding the feature information range of the personnel master data. Each feature information can be mapped to the basic feature, reaching the individual personnel.

[0043] See Figure 3 As shown, when probing the metadata of data warehouse tables daily, the table name, table comments, and partition type information of each table are obtained by traversing and reading the metadata information of all data tables. The latest record count of the data table is calculated based on different partition types: when the data table is a full partition type, the latest full partition statistics record count is retrieved; when the data table is an incremental partition type, the total partition statistics record count is retrieved; and when the data table is a non-partitioned table type, the total content statistics record count is retrieved. The field names, field comments, and field data types of the data table are obtained from the metadata and merged with the table name, table comments, partition type, and the latest record count of the table, saving them as a single record. After traversing all data tables, the obtained record is written to the full partition table of the probing results, with the partition value being the date at the time of traversal. The records generated by probing the metadata are shown in the table below.

[0044]

[0045] In a preferred embodiment, further, in setting the corresponding feature verification function based on the basic features and the specified associated features, the corresponding basic feature verification function and the specified associated feature verification function are set according to the basic feature information rules and the specified associated feature information rules, respectively.

[0046] See Figure 2 As shown, corresponding feature verification functions can be pre-defined based on the rules of feature information. The big data processing engine then traverses and verifies the values ​​of each field in the data table to obtain the location of the master data feature data of the data warehouse personnel, including the data table and fields where the feature data appears. According to feature priority, they are divided into: basic features and related features.

[0047] Taking the most representative personnel identification number as an example, a feature data verification function for the identification number is constructed. According to the relevant regulations on identification numbers, the identification number can be a feature combination code, consisting of a seventeen-digit or more body code and a one-digit check code. The last digit of the identification number can be the check code, calculated using the formula: ∑(Ai×Wi)(mod 11), where i represents the position number of the number characters from right to left, including the check code; Ai represents the value of the number character at position i; and Wi represents the weighting factor at position i (its value is already determined), calculated using the formula Wi=2^(i-1)(mod 11). The corresponding values ​​for each digit are as follows: 7 9 10 5 8 4 2 1 6 3 7 9 10 5 8 4 2. Since the data itself may contain spaces, line breaks, or noisy characters, in order to avoid the data from being damaged by interference, interference filtering can be added to the verification function before the data is verified according to the information rules of the ID number. This will replace non-capsular 'x' or non-numeric characters with empty strings, thereby improving the error tolerance of the ID number verification.

[0048] Furthermore, in this embodiment, the basic feature verification function is used to locate the basic feature position in the data warehouse data table and extract the basic feature set stored at that position, including:

[0049] First, the basic feature verification function is used to traverse the data content of each field in the data table of the data warehouse to initially locate the basic feature position;

[0050] Then, by comparing the basic feature information content in the initially located basic feature location with the first preset threshold, the effective location of the basic feature of the data warehouse personnel is further determined, and at the same time, the set of basic features stored in the effective location of the basic feature of the data warehouse personnel is extracted.

[0051] Using a data verification function based on personnel identification number features, the data content of each field in the data warehouse table is traversed to initially locate the position of personnel identification numbers. The information content of identification number features at each position is calculated, and by setting an automatic extraction threshold, the positions of personnel identification numbers in the data warehouse are automatically extracted, along with the set of identification numbers stored at the feature positions.

[0052] See Figure 4As shown, the process iterates through the data content of each field in the data warehouse tables. When a field passes validation, the table name and field name of the record are recorded, with a possible annotation "true_idcard". The validation then moves to the next field, continuing until all fields in all tables have been validated. For each field marked "true_idcard", the ID number validation function is used to iterate through all records in that field. The percentage of valid records is calculated, and a threshold is set and compared to this percentage. For example, fields with a percentage exceeding 50% are considered to have information content matching the ID number characteristics and are designated as valid ID number fields. After traversing all locations, the valid locations of personnel ID numbers in the data warehouse are extracted, along with the complete set of ID numbers stored at the characteristic locations.

[0053] Furthermore, by utilizing a specified correlation feature validation function, the location of the specified correlation feature is determined in the data warehouse table where the basic feature is located, and the mapping relationship between the basic feature and the specified correlation feature is extracted, including:

[0054] First, the specified association feature verification function is used to traverse the data content of each field in the data table where the basic feature is located, and the location of the specified association feature is initially located.

[0055] Then, the content of associated feature information in the initially determined specified associated feature position is compared with the second preset threshold. Based on the comparison result, the effective position of the specified associated feature of the data warehouse personnel is extracted, and the information mapping combination relationship between the basic features of the personnel and the specified associated features is obtained based on the field relationship in the same data table.

[0056] Based on the data table containing the basic feature locations, the system uses a correlation feature data validation function to traverse the data content of each field in the table to initially locate the positions of specified correlation features. It then calculates the information associated with the correlation features at each location and, by setting an automatic extraction threshold, automatically extracts the positions of correlation features in the data warehouse, as well as the combination relationships between basic and correlation features.

[0057] Using a specified associated feature data verification function, in this embodiment, taking the mobile phone number of an associateable person as an example, a mobile phone number feature data verification function is constructed. See [link to relevant documentation]. Figure 5As shown, based on the data table containing the basic feature locations, the mobile phone number verification function is used to traverse the data content of each field in the data table. When a field being verified passes the verification, the field name containing that record is recorded and marked "true_phone," and the verification of the next field begins, until the verification of each field in the data table containing the basic feature locations is completed. For each field marked "true_phone," the mobile phone number verification function is used to traverse all records in the field, calculate the percentage of records that pass the verification out of the total number of records in the data table, set a threshold, and compare it with the percentage. For example, for a field with a percentage exceeding 50%, the information content of that field is considered to meet the mobile phone number feature and is recorded as a valid mobile phone number field. After completing the traversal of all locations, the valid locations of mobile phone numbers of personnel in the data warehouse are extracted. Based on the relationship between fields in the same data table, the information mapping relationship between personnel mobile phone numbers and ID numbers is extracted, that is, the combination relationship of basic features and related features is obtained.

[0058] In a preferred embodiment, the weights of specified association features are further obtained based on the grouping results. The association feature value with the highest weight is used as the default value of the association feature, and the association feature values ​​with lower weights are used as extended values ​​of the association feature, including:

[0059] First, based on the grouping results, the number of repetitions of different feature values ​​of the specified associated feature is obtained, and the number of repetitions is used as the weight of the relationship of the specified associated feature.

[0060] Then, the associated feature value with the highest weight is used as the default value of the associated feature, and the associated feature value with a lower weight is used as the extended value of the associated feature.

[0061] Based on the combination relationship, the sets of extracted basic feature information and associated feature information are grouped according to the basic features. The relationship weight is defined by calculating the repetition frequency of different information in the associated features, with the highest weight associated feature value used as the default value, and the other weighted associated feature values ​​used as extended values. If the personnel master data needs to be associated with new specified features, the new feature information can be obtained by setting a data validation function for the new feature and executing the above steps.

[0062] Taking ID card numbers and mobile phone numbers as examples, the data warehouse extracts a set of records for ID card numbers and mobile phone numbers of personnel. The set is then grouped by ID card number, and the frequency of repetition of mobile phone number information is calculated. Weights are defined based on the frequency of each mobile phone number and the associated personnel. The mobile phone number with the highest weight is used as the default value, while those with lower weights are merged into extended values, thus minimizing the extraction of basic and associated features of the personnel master data. When new features need to be associated with the personnel master data, a data validation function for the new feature is set, such as associating with personnel's email address. The associated feature extraction steps are then repeated to obtain the feature information of the personnel master data.

[0063] In this embodiment, by automatically acquiring the storage location of personnel characteristic information, the problem of low exploration efficiency, missing or distorted data table descriptions leading to misjudgments caused by previous manual methods of identifying personnel ID numbers, mobile phone numbers, and other characteristic information is solved. Furthermore, when personnel master data is updated, manual updates and extraction of characteristic information are difficult to achieve in a timely, accurate, and secure manner. This solution can automatically identify the location of personnel master data characteristics in massive amounts of data, confirm personnel master data characteristics using verified data content features, and automatically extract personnel master data characteristic information to support the improvement of personnel master data resource catalog classification and grading, enabling timely and accurate discovery and identification of sensitive information such as identification numbers (passport, work ID, etc.), mobile phone numbers, email addresses, and social status.

[0064] Furthermore, the present invention also provides a personnel master data feature extraction system for managing personnel information in a data warehouse according to preset data feature rules, comprising: a data exploration module, a mapping and combination module, and a feature acquisition module, wherein,

[0065] The data exploration module is used to read the metadata of all data tables in the data warehouse, take the key information in the personnel master data that expands the range of personnel master data feature information through mapping relationship as the basic feature, and set the corresponding feature verification function based on the basic feature and the specified associated feature.

[0066] The mapping and combination module is used to locate the position of basic features in the data warehouse data table using the basic feature verification function and extract the set of basic features stored at that position; and to locate the position of specified related features in the data warehouse data table where the basic feature is located using the specified related feature verification function and extract the mapping and combination relationship between the basic feature and the specified related feature.

[0067] The feature acquisition module is used to extract a set of records of basic features and specified related features of personnel in the data warehouse based on the mapping and combination relationship, and group them according to the basic features. Based on the grouping results, the weight of the specified related feature relationship is obtained, and the related feature value with the highest weight is used as the default value of the related feature, and the related feature value with a lower weight is used as the extended value of the related feature.

[0068] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0069] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0070] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0071] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0072] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for extracting personnel master data features, used to manage personnel information in a data warehouse according to preset data feature rules, characterized in that, Include: Read the metadata of all data tables in the data warehouse, take the key information in the personnel master data as the basic feature, the key information is the information that expands the range of personnel master data feature information through mapping relationship, and set the corresponding feature verification function based on the basic feature and the specified associated feature; The basic feature verification function is used to locate the basic feature position in the data warehouse data table and extract the basic feature set stored at that position. The specified association feature verification function is used to traverse the data content of each field in the data table where the basic feature position is located, and initially locate the specified association feature position. The association feature information content in the initially determined specified association feature position is compared with a second preset threshold. If the association feature information content is greater than the second preset threshold, the valid position of the specified association feature of the data warehouse personnel is extracted, and the information mapping combination relationship between the personnel basic feature and the specified association feature is obtained based on the field relationship in the same data table. Based on the mapping and combination relationship, the data warehouse extracts the basic characteristics of personnel and the set of records with specified related characteristics, and groups them according to the basic characteristics. Based on the grouping results, the number of repetitions of different characteristic values ​​of the specified related characteristics is obtained. The number of repetitions is used as the weight of the relationship of the specified related characteristics. The related characteristic value with the highest weight is used as the default value of the related characteristic, and the related characteristic value with a lower weight is used as the extended value of the related characteristic.

2. The method for extracting personnel master data features according to claim 1, characterized in that, Read the metadata of all tables in the data warehouse, including: First, set a time period for regularly probing the data warehouse, and within each time period, probe the metadata of all data tables in the data warehouse; Then, based on the exploration results, the information of each data table is read and the latest record count of the data table is counted. The data table information read includes at least: table name, table comments and partition type information.

3. The method for extracting personnel master data features according to claim 2, characterized in that, The latest record count in the statistical data table is calculated based on different partition types. When the data table is a full partitioned table, the latest full partition is used to count the record count. When the data table is an incremental partitioned table, all partitions are used to count the record count. When the data table is a non-partitioned table, the entire table is used to count the record count. The table's field names, field comments, and field data types are obtained from the metadata. These are then merged with the table name, table comments, partition type, and the latest record count into a single record. This process is repeated until all tables in the data warehouse are traversed. The resulting records are written to the full partitioned table of the exploration results, and partition values ​​are set based on the date of the traversal.

4. The method for extracting personnel master data features according to claim 1, characterized in that, In setting corresponding feature verification functions based on basic features and specified associated features, the corresponding basic feature verification functions and specified associated feature verification functions are set according to the basic feature information rules and specified associated feature information rules, respectively.

5. The method for extracting personnel master data features according to claim 1, characterized in that, The basic feature verification function is used to locate the basic feature position in the data warehouse data table and extract the basic feature set stored at that position, including: First, the basic feature verification function is used to traverse the data content of each field in the data table of the data warehouse to initially locate the basic feature position; Then, by comparing the basic feature information content in the initially located basic feature location with a first preset threshold, if the basic feature information content is greater than the first preset threshold, the effective location of the basic feature of the data warehouse personnel is determined, and the set of basic features stored in the effective location of the basic feature of the data warehouse personnel is extracted at the same time.

6. A personnel master data feature extraction system, used to manage personnel information in a data warehouse according to preset data feature rules, characterized in that, The method described in claim 1 comprises: a data exploration module, a mapping and combination module, and a feature acquisition module, wherein... The data exploration module is used to read the metadata of all data tables in the data warehouse, take the key information in the personnel master data that expands the range of personnel master data feature information through mapping relationship as the basic feature, and set the corresponding feature verification function based on the basic feature and the specified associated feature. The mapping and combination module is used to locate the position of basic features in the data warehouse data table using the basic feature verification function and extract the set of basic features stored at that position; and to locate the position of specified related features in the data warehouse data table where the basic feature is located using the specified related feature verification function and extract the mapping and combination relationship between the basic feature and the specified related feature. The feature acquisition module is used to extract a set of records of basic features and specified related features of personnel in the data warehouse based on the mapping and combination relationship, and group them according to the basic features. Based on the grouping results, the weight of the specified related feature relationship is obtained, and the related feature value with the highest weight is used as the default value of the related feature, and the related feature value with a lower weight is used as the extended value of the related feature.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is configured to execute a program stored in memory and, when the program is executed, implement the method described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 5.

Images