Method, system, and program product for identifying entities in a database system
By introducing multi-value keys and naming rule priority sorting into the database system, combined with hash value determination, the problem of automatic identification and merging of duplicate data items in the database system is solved, realizing efficient data management and automated processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INTERNATIONAL BUSINESS MACHINE CORPORATION
- Filing Date
- 2021-03-05
- Publication Date
- 2026-05-05
AI Technical Summary
Existing database systems struggle to seamlessly and effortlessly identify and manage duplicate data objects in enterprise and technology applications, especially when dealing with multiple entries, where duplicate data entries are difficult to identify and merge.
By using multi-value keys and naming rules for priority sorting in the database system, combined with the determination of hash values, duplicate data items are automatically identified and merged. Primary key uniqueness is maintained by using multi-value primary keys and row-IDs, and index trees are used to accelerate searches.
It enables automatic identification and merging of duplicate data items, simplifies data management in database systems, and improves the automation and efficiency of data management.
Smart Images

Figure CN115516441B_ABST
Abstract
Description
Background Technology
[0001] This invention generally relates to database systems, and more specifically to a computer-implemented method for explicitly identifying entities in a database system. The invention further relates to a related database system for explicitly identifying entities in a database system, and a computer program product suitable for performing the method.
[0002] Enterprise information management remains a key theme for enterprise IT (Information Technology) organizations. This applies not only to large global 2000 companies but also to small and medium-sized enterprises. The reason is simple: in the information age, the number of sources for data management and the absolute amount of data to be managed successfully are constantly growing. One approach to addressing this challenge is to create an enterprise data catalog within the context of a data warehouse project and to use data management concepts. However, reality has shown that this approach is quite difficult because new data sources and new types of data bombard IT organizations even in shorter timeframes. Therefore, a more dynamic approach is needed to address this common problem of recurring data objects across many enterprise applications.
[0003] For example, the same customer may be entered into an ERP system multiple times, with slightly different names or employer identification numbers (e.g., tax IDs) placed in different formats. Creating some unit constraints is insufficient because the same customer name may be stored in uppercase or mixed cases, or with the full company name or a short name. Over time, those subjects may be discovered, but merging other related data is often difficult, time-consuming, and sometimes impossible. Over time, invoices, orders, and many other related data objects may be created in the ERP system, and also in related systems such as Customer Relationship Management (CRM), Supply Chain Management (SCM), and logistics systems. Moreover, all these systems will use duplicate customer records. For example, if you want to offer customer discounts based on accumulated order values, it needs to aggregate stored orders with different customer data, a feature that may not be available in today's ERP systems.
[0004] However, the problems described exist not only in databases that store customer data, but also in databases used for highly technical applications, such as configuration management databases (CMDBs) used to control, prioritize, and allow or deny access to computing resources.
[0005] Several disclosures exist regarding computer-implemented methods for deduplicating entries in an entry storage system. Document US2017 / 0308557 A1 discloses a method and system for sanitizing and deduplicating data in a database. The method includes filtering junk records from multiple records based on data fields, and applying sanitization rules to create a sanitized database. Similarity vectors are generated, where each vector corresponds to a pairwise comparison of distant data entries in the sanitized database. Matching rules are applied to label each vector as one of three: matched, unmatched, or unclassified.
[0006] Furthermore, document US 2017 / 0011088 A1 discloses a method for finding pairs in a database, which includes calculating hash values for at least two field groups of records in the database, wherein the field groups include at least two fields of the records, and the hash values of a minority group of records are based on values of the corresponding field groups stored in the corresponding records for at least two years.
[0007] However, as mentioned above, these known methods for solving the problem cannot address the difficulties faced by enterprise information organizations in seamlessly and effortlessly handling multiple entries in databases that support enterprise and / or technology applications.
[0008] Therefore, it may be necessary to overcome the aforementioned technical problems and provide an advanced solution to properly store and clearly identify entities involving the same and / or different data objects. Summary of the Invention
[0009] According to one aspect of the invention, a computer-implemented method for explicitly identifying entities in a database system can be provided. The method may include: storing data items in tables of the database, the data items being stored as records including multiple attributes; storing naming rules for selected combinations of attributes of the data items; and prioritizing the naming rules. The method may further include determining a hash value for each of the selected combinations of attributes of the data items, and using the determined hash values and the prioritized naming rules to identify duplicate data items.
[0010] According to another aspect of the invention, a database system for explicitly identifying entities in a database system can be provided. The database system may include: a first storage device adapted to store data items in tables of the database system, the data items being stored as records including multiple attributes; a second storage device adapted to store naming rules for selected combinations of attributes of the data items; and a prioritization sorting device adapted to prioritize the naming rules. The database system may further include a determination device adapted to determine a hash value for each of the selected combinations of attributes of the data items, and an identification device adapted to identify duplicate data items using the determined hash values and the prioritized naming rules.
[0011] The proposed method for a computer implementation of stateless services on nodes in a workload environment offers several advantages, technical benefits, contributions, and / or improvements:
[0012] The problem of duplicate data entries where two records identify the same physical entity is successfully solved. The proposed solution is based on using a hash value that determines a selected combination of attributes of a data item using multi-value keys and naming rules, as well as the priority order of the naming rules, in order to explicitly identify entities in the database. This enables the resolution of problems in classic enterprise data management systems (e.g., business databases) and technical databases (e.g., CMDBs) that already store multiple records intended to identify the same physical entity.
[0013] Describing the same unique physical entity makes it easier for enterprise information management to eliminate duplicate but slightly different entries in database systems. The proposed method and system rely on this concept and work automatically and autonomously once established. For example, duplicate entries can be automatically identified and merged to the necessary and predefined level without any additional manual intervention.
[0014] Unlike deduplication efforts in backup or archiving systems, the proposed concept can be particularly useful for online transaction database systems. However, the proposed method can be advantageously applied to other environments. Given the ever-growing number of sources of additional data in enterprise computing environments, the proposed concept can help better manage enterprise intellectual property related to customer, prospective, and product identities, as well as technical data used to manage large and complex enterprise IT equipment or networks (including personal devices and on-premises computing systems) and remotely accessible computing resources (e.g., in the sense of cloud computing).
[0015] The following will describe other embodiments of the inventive concept applicable to the present invention and related database systems:
[0016] According to a useful embodiment of this method, the database system can be a relational database system. This represents the most common application. However, other types of database or storage concepts can be used for the proposed technical ideas, such as object-oriented databases, graph databases, hierarchical databases, flat file systems for storing data records, etc.
[0017] According to another useful embodiment of the method, the database system can be a configuration management database (CMDB). Such a database can be the foundation of a specific internal organization and is particularly useful for managing technical equipment and applications within a data center. Therefore, the proposed concept can be readily applied to various technical fields.
[0018] According to an advantageous embodiment, the method may further include merging identified duplicate data items by maintaining the determined hash value as a multi-value key for the merged (i.e., combined) data items. Therefore, the concept of multi-value keys in a database system can provide a solid technical foundation for implementing the concepts presented herein.
[0019] According to a higher-level embodiment, the method may further include merging other data items that are compositely related to the identified data item. Therefore, not only can the applied data fields themselves be directly related to each other, but related data—that is, those data that are compositely related to the identified duplicate data item—can be merged, so that all data describing the characteristics of the entity can be accessed jointly and explicitly.
[0020] According to another advantageous embodiment, the method may further include maintaining a pointer to the same row identifier, i.e., row-ID, for the merged (i.e., combined) data item pointing to the determined hash value. This maintains primary key uniqueness, and the row-ID can become the true primary key in the database table. However, they are completely invisible to the database user. Alternatively, they can be visible to the database user but cannot be changed by the database user.
[0021] According to a further advanced embodiment, the method may also include maintaining the table's index and maintaining pointers in the search tree associated with the index, such that for combined data items, the pointer values can point to the same record identifier. Therefore, the concept presented herein can also be elaborated for fast searching in a database. The idea of a centralized application does not slow down the functionality of the search tree. This is achieved by giving pointers in different branches of the index tree the same value.
[0022] According to a more advanced embodiment, the method may also include using a "create SQL statement," which is hereby redefined and adapted to create naming rules and their associated priorities. Thus, a user or application (e.g., via an API) can create naming rules and associated priorities as required by a specific application or naming convention typically used to identify entities, as described by the data item.
[0023] According to one licensed embodiment, the method may also include classifying records in a database table using a multi-valued primary key. Therefore, the proposed concept is not typically used in the manner of normal database table operations and is generally not applicable to traditional database system tables.
[0024] According to another licensed embodiment of the method, multi-valued primary keys can be used to cluster clustered data on a multi-node database engine. Therefore, the proposed concept can also be extended to more complex and distributed types of databases.
[0025] According to another useful embodiment of the method, a multi-valued primary key can be compared with a single-valued column data item. Therefore, known operations such as comparisons of data fields between tables—as known from conventional databases—can also be applied under the technical concept presented herein.
[0026] According to an additional improved embodiment, the method may further include collecting statistical database data of data blocks containing single-valued primary keys and multi-valued primary keys. Therefore, the technical concept of this method is not only to manage database systems according to best practices, but also to allow the use of statistical methods to better manage underlying database management systems that include multi-valued keys.
[0027] Furthermore, embodiments may take the form of an associated computer program product accessible from a computer-usable or computer-readable medium, which provides program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable medium may be any device that can contain means for storing, communicating, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. Attached Figure Description
[0028] It should be noted that embodiments of the invention are described with reference to different subject matter. Specifically, some embodiments are described with reference to method type claims, while others are described with reference to apparatus type claims. However, those skilled in the art will conclude from the above and below description that, unless otherwise indicated, any combination of features relating to different subject matter (specifically, features of method type claims and features of apparatus type claims) is also considered to be disclosed in this document, except for any combination of features belonging to one type of subject matter.
[0029] The aspects defined above and other aspects of the invention will be apparent from examples of the embodiments described below, and will be explained with reference to examples of the embodiments, but the invention is not limited thereto.
[0030] The following detailed description, given by way of example and not intended to limit the exemplary embodiments thereto, will be best understood in conjunction with the accompanying drawings, in which:
[0031] Figure 1 A block diagram illustrating an embodiment of a creative computer-implemented method for explicitly identifying entities in a database system is shown.
[0032] Figure 2a A table showing different records for the same physical entity is displayed.
[0033] Figure 2b A table illustrating the discount issue for total order volume is shown.
[0034] Figure 3a , Figure 3b and Figure 3c A database table with a hierarchical organization of exemplary fields, name, address, and order data is shown.
[0035] Figure 4a and Figure 4b The entries for merging database table entries based on unique attributes are shown.
[0036] Figure 4c and Figure 4d The entries for merging database table entries based on naming rules are shown.
[0037] Figure 5a and Figure 5b This shows the entries for the database table that will be merged along with the relevant index tree.
[0038] Figure 6a and Figure 6b This shows the database table entries that are to be merged with the relevant index tree under the naming rules.
[0039] Figure 7 A block diagram of an embodiment of a database system for explicitly identifying entities in a database system is shown.
[0040] Figure 8 The illustration depicts a scenario including, according to, an exemplary embodiment. Figure 7 An exemplary block diagram of the hardware components of the computing system for a database system.
[0041] Figure 9 A cloud computing environment according to an exemplary embodiment is shown.
[0042] Figure 10 An abstract model layer according to an exemplary embodiment is shown.
[0043] The accompanying drawings are not necessarily drawn to scale. They are schematic representations only and are not intended to depict specific parameters of exemplary embodiments. The drawings are intended only to describe typical exemplary embodiments. In the drawings, the same reference numerals denote the same elements. Detailed Implementation
[0044] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it should be understood that the disclosed embodiments are merely illustrative of the claimed structures and methods, and they may be implemented in various forms. Exemplary embodiments are merely illustrative; however, they may be embodied in many different forms and should not be construed as limiting oneself to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided to make this disclosure thorough and complete, and to fully convey the scope covered by the exemplary embodiments to those skilled in the art. Details of well-known features and techniques may be omitted in the description to avoid unnecessarily obscuring the presented embodiments.
[0045] References to "an embodiment," "embodiment," "exemplary embodiment," etc., in this specification indicate that the described embodiment may include specific features, structures, or characteristics; however, each embodiment may not necessarily include specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is believed that implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) is within the knowledge of those skilled in the art.
[0046] To avoid obscuring the presentation of exemplary embodiments, some processing steps or operations known in the art may have been combined for presentation and illustrative purposes in the following detailed description, and in some cases may not have been described in detail. In other cases, some processing steps or operations known in the art may not be described at all. It should be understood that the following description focuses on distinguishing features or elements according to various exemplary embodiments.
[0047] In the context of this specification, the following conventions, terms and / or expressions may be used:
[0048] The term 'explicitly identified' refers to the ability to uniquely identify a physical item through related data items in a non-interchangeable manner. Simply put, no two data items should point to the same physical entity.
[0049] The term 'entity' can refer to a physical object that will be identified and / or addressed by data items (e.g., records in a database).
[0050] The term 'database system' can refer to a storage device—physical or logical—suitable for storing data items. Examples of systems can be relational database systems, hierarchical database systems, flat file systems, graph databases, and / or object databases, or combinations thereof. Typically, the proposed concept can work in a wide variety of different databases.
[0051] The term 'relational database system' can refer to a database system organized according to the rules of CF Codd. A relational database system can also refer to a storage device that enables data to be preferably stored in records, each with a fixed or variable length. Typically, and especially in relational database systems, data is organized in tables with rows. Each row can represent a record with attributes. Different records in different tables can be configured relatedly by applying selective routing to attributes.
[0052] The term 'attribute' can refer to a field in a record of a table in a database system.
[0053] The term 'naming rules' can refer to a constraint or interpretation framework that will be applied to the attributes of a record. Naming rules can be formulated in a way that relates similar but different data items to each other.
[0054] The term 'prioritizing naming rules' can be used to define a sequence of available naming rules based on their importance to a particular attribute or combination thereof.
[0055] The term 'hash value' can represent a mapping from a variable-length data field (e.g., a text field) to a data field with a fixed length (e.g., an integer value). This mapping is typically performed using a hash function or a hash table.
[0056] The term 'duplicate data item' refers to the process of eliminating data items that involve the same physical entity but may have similar values in a particular attribute rather than identical values. However, it should be noted that similar values of attributes will not be eliminated; only redundant information and other attributes will be removed.
[0057] The term 'multi-valued key' refers to the concept of having more than one value in a key attribute field, rather than just one value according to traditional rules. Uniqueness may no longer be defined by that specific attribute, but rather by the row-ID, which the user may not have the right to manipulate.
[0058] The term 'composition relationship' can be interpreted in the context of UML (Generalized Markup Language). A composition relationship can represent a specific situation of association. In aggregation and composition, an object of a class "owns" another object of a higher class. As an example, a room might not exist without a house. Therefore, the two objects, room / house, have a composition relationship.
[0059] The term 'row identifier' can refer to a unique identifier in a specific table of a database that uniquely identifies a particular record. A row identifier (i.e., row-ID) can be used as a replacement for the original primary key that has been replaced by a multi-valued primary key.
[0060] The term 'index' can refer to a mechanism that increases the speed of access to individual records in a database table, especially in search operations. Index leaves can include pointers to specific records (e.g., pointers to specific row IDs).
[0061] The term 'SQL statement' refers to the commands used in a programming language called Structured Query Language (SQL) for databases.
[0062] A detailed description of the accompanying drawings is given below. All instructions in the drawings are schematic. First, a block diagram of an embodiment of a method for explicitly identifying entities in a database system is given. Subsequently, other embodiments and implementations of the database system for explicitly identifying entities in a database system will be described.
[0063] Figure 1 A block diagram of a preferred embodiment of a computer-implemented method 100 for explicitly identifying entities in a database system is shown. Method 100 includes storing data items 102 in tables of the database, specifically as records in a relational database, but also in any other storage format. Data items are stored as records, each comprising a plurality of fixed- or variable-length attributes.
[0064] Method 100 includes naming rules for selected combinations of attributes that store 104 data items (specifically, at least two rules for two combinations of attributes). It can be noted that a combination may also be associated with only one attribute.
[0065] Furthermore, method 100 includes prioritizing the naming rules 106, that is, defining the sequence in which they are applied or the order in which the rules are defined. The priority order should depend on their importance to entity identification.
[0066] Then, method 100 includes determining 108 hash values for each of the selected combinations of attributes of the data items using a hash function, and identifying 110 duplicate data items, in particular at least one record, using the determined hash values and a priority-sorted naming rule.
[0067] In other words, a naming rule is a unique way to identify a given entity. Naming rules are based on entity attributes. For example, to identify a customer, their name can be used. In most cases, the customer's name alone is sufficient to distinguish them. Thus, this can be the first naming rule. The next naming rule can use the customer's tax identifier, sometimes called the Employer Identification Number (EIN).
[0068] Naming rules can also be based on a set of attributes. For example, a good naming rule would be based on the customer name and its associated EIN.
[0069] Naming rules for a given entity should be categorized based on their respective importance. For example, naming rules based on EIN are better because it's highly unlikely that two customers have the same EIN. This is also more important because if someone who creates records for a customer in the system has access to the EIN database (or database table), they are also likely to have access to other related customer data. Therefore, someone who only knows a customer's name but not their associated EIN is not a very good source of customer data. Consequently, naming rules based solely on customer names should have relatively low priority.
[0070] Naming rules are created for a given table and are automatically considered and applied by the database engine. When records are entered into a table, the database system lists all applicable naming rules (i.e., names based on attributes with non-nullable attribute values). The database engine then determines that each naming rule value is a hash value based on the combined attribute values.
[0071] For example, for a customer, the first naming rule described above can be used to determine its value as follows:
[0072] NR1 = hash(customer name) = hash(ABC) = UUID1, and
[0073] NR2 = hash(customer EIN) = hash(EIN1234) = UUID2.
[0074] Therefore, two values can be used as the primary key for a given record. A record can contain many names, and as a result, many primary keys can exist (contrary to the original historical definition of primary keys in the database). The related record can now be found and identified by each of them, so each of them is a primary key of equal importance. A solution to this potential front-end conflict is to use a unique row ID as the true unique primary key; however, these primary keys are typically not accessible to database users.
[0075] Because it has multiple values as a primary key, it can also be used as a foreign key. Therefore, to create a foreign key constraint, you can use the row-ID (unique row identifier), which is a given internal identifier for the row. In some cases, this row-ID may be visible to the database user, but it should not be used in the database user's queries. The database engine or database system can disable this activity.
[0076] Figure 2aThe diagram shows database table entries displaying different records for the same physical entity. Further explanatory text related to this diagram can be found in the background section of this document. Column PK refers to the shown primary key customer identifiers CID1, CID2, and CID3 associated with the customer names ABC, ABC, and ABC Ltd. In this example, these customer names refer to the same physical entity, i.e., the same customer. In this example, the customer is shown as having two similar but not identical EIN numbers, EIN1234 and EIN-1234.
[0077] Figure 2b This shows database table entries illustrating the discount issue related to order quantity. The text describing this issue can be found in the background section of this document. Therefore, Figure 2b This shows a table of order entries with a primary key order entry (IODi), order number, customer identifier, and order value for each column. The link between the two tables is based on the customer identifier, as indicated by the arrow. All other examples of table and database entry records should be interpreted in the sense described.
[0078] Figure 3a , Figure 3b and Figure 3c A database table with a hierarchical organization of exemplary fields—name, address, and order data—is shown. Figure 3a This shows two customers (possibly the same) registered in the database. Figure 3b It shows Figure 3a The customer's relevant address. Each of them has, for example: Figure 3c The order shown.
[0079] Let's assume someone wants to create a new customer named "ABC" with "EIN1234". This is illustrated in the following diagram.
[0080] Figure 4a and Figure 4b The table entries to be merged based on unique attributes are shown. As a result of this record merging, two existing customer records need to be merged, specifically... Figure 4a One of the first rows of the table, and Figure 4a A table that has a third record, such as Figure 4a As shown in the right-hand ellipse. The "Name" attribute comes from the third row, created using a naming rule that defines higher priority (note that the EIN attribute with higher priority is set in the third row), ensuring that "ABC" remains the name. The EIN field is empty in the first row, ensuring that the value from the second row is retained in the merged record. Therefore, an entry has dual primary keys CNR1#A and CNR2#B but a truly unique identifier shown as the row ID (row identifier).
[0081] Figure 4c and Figure 4d This shows the database table entries to be merged based on naming rules in the parent context. Typically, addresses are "named" in the context of the client. This means they cannot exist without a parent object (i.e., the object "client"). This is represented in the address naming rules, where "client" is defined as the first attribute. Since the parent object (client) should be merged, it needs to be determined whether dependent objects (objects named in the client's context, i.e., addresses in this case) also need to be merged.
[0082] Figure 4c and Figure 4d This involves merging the attributes "street" and "city" to create multi-valued primary keys ANR1#A and ANR1#B under the unique row ID ARID1.
[0083] Therefore, it can be determined that the merged customer has two addresses with the same attributes "street" and "city" that allow for merging. The old primary key value ANR1#B is also retained because it can be used as a reference in other objects or even some external systems. However, since the previous row's -ID is used for determination, there is no risk of conflicts with other primary key values that could lead to over-merging. Over-merging occurs when two different objects (which will represent different entities) are merged, even though they should not be merged.
[0084] Figure 5a and Figure 5b The diagram shows the database table entries that will be merged with the relevant index tree. In addition to the merging techniques discussed above, primary key uniqueness, despite its multi-valued nature, also needs to be supported. In principle, it is necessary to store multiple primary key values pointing to the same row-ID.
[0085] In a given table, aliases can be stored as a collection of aliases in a single column. When an index is created on that column to maintain the primary key constraint, the list of aliases is separated into individual aliases, and each of them is used as a single value in the index. All of these indexes point to the same record in the table. This is in Figure 5a and Figure 5b As shown in [the image]. Figure 5a In the index tree, the two leaves involve different rows from a non-managed table, where... Figure 5b In the middle, the two identical index leaves now point to the same record with a multi-value key and a unique row-ID.
[0086] Figure 6a and Figure 6bThe diagram illustrates the database table entries to be merged with the relevant index tree under the naming rules. These diagrams address the case of child tables. In this case, an additional index is created based on the parent (or superparent) alias. The list of parent names is separated into individual aliases, and each of them is used as a single value in the index. Those values are stored in leaf nodes and point to a single record in the child table.
[0087] Figure 6a and Figure 6b This shows the database table entries to be merged with the relevant index tree under the naming rules. For addresses, there is an additional index based on the parent attribute "customer," such as... Figure 6a As shown.
[0088] Of course, if there are 3 child records and 2 parent primary keys, then it ends with 6 leaf nodes, because it needs to be able to find all 3 rows of parent primary keys CNR1#A and CNR2#B, such as... Figure 6b As shown in the image.
[0089] Figure 7 A block diagram of an embodiment of a database system 700 for explicitly identifying entities in a database system is shown. Database system 700 includes a first storage device (specifically, a first storage unit 702) adapted to store data items in tables of the database system, wherein these data items are stored as records including multiple attributes.
[0090] The database system 700 also includes a second storage device (specifically, a second storage unit 704) adapted to store naming rules for selected combinations of attributes of data items, a priority sorting device (specifically, a priority sorting module 706) adapted to prioritize the naming rules, and a determining device (specifically, a determining unit 708) adapted to determine the hash value of each of the selected combinations of attributes of data items.
[0091] Last but not least, the database system 700 includes an identification device (specifically, identification module 710) adapted to identify duplicate data items using a determined hash value and a priority-ordered naming rule.
[0092] It can be noted that modules, units, and / or devices can be interconnected for signal and / or message exchange between them. This connection can be implemented in a one-to-one manner or using the database system internal bus system 712. The first storage unit 702, the second storage unit 704, the priority sorting module 706, the determination unit 708, and the identification module 710 can be directly or indirectly connected to this database system internal bus system 712.
[0093] Figure 8 A block diagram of a device within the present invention according to exemplary embodiments is depicted. It should be understood that... Figure 8 This illustration provides only one possible implementation and does not imply any limitation regarding the environment in which different embodiments may be implemented. Many modifications can be made to the depicted environment.
[0094] The devices used herein may include one or more processors 02, one or more computer-readable RAMs 04, one or more computer-readable ROMs 06, one or more computer-readable storage media 08, device drivers 12, read / write drivers or interfaces 14, network adapters or interfaces 16, all interconnected via communication structure 18. Communication structure 18 may be implemented using any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system.
[0095] One or more operating systems 10 and one or more application programs 11 are stored on one or more computer-readable storage media 08 for execution by one or more processors 02 via one or more corresponding RAMs 04 (which typically include cache memory). In the illustrated embodiment, each of the computer-readable storage media 08 may be a disk storage device such as an internal hard disk, CD-ROM, DVD, Memory Stick, magnetic tape, magnetic disk, optical disk, semiconductor storage device such as RAM, ROM, EPROM, flash memory, or any other computer-readable tangible storage device capable of storing computer programs and digital information.
[0096] The device used herein may also include an R / W drive or interface 14 for reading from and writing to one or more portable computer-readable storage media 26. An application program 11 on the device may be stored on one or more portable computer-readable storage media 26, and read from and loaded onto the corresponding computer-readable storage media 08 via the appropriate R / W drive or interface 14.
[0097] The device used herein may also include a network adapter or interface 16, such as a TCP / IP adapter card or a wireless communication adapter (such as a 4G wireless communication adapter using OFDMA technology). The application 11 on the computing device can be downloaded to the computing device from an external computer or external storage device via a network (e.g., the Internet, a local area network, or other wide area networks or wireless networks) and the network adapter or interface 16. The program can be loaded from the network adapter or interface 16 onto a computer-readable storage medium 08. The network may include copper wire, fiber optic, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers.
[0098] The devices used herein may also include a display screen 20, a keyboard or keypad 22, and a computer mouse or touchpad 24. The device driver 12 interfaces with the display screen 20 for imaging, the keyboard or keypad 22, the computer mouse or touchpad 24, and / or the display screen 20 for pressure sensing for alphanumeric character input and user selection. The device driver 12, R / W driver or interface 14, and network adapter or interface 16 may include hardware and software (stored on computer-readable storage media 08 and / or ROM 06).
[0099] The programs described herein are identified based on applications that implement them in specific exemplary embodiments within the exemplary embodiments. However, it should be understood that any particular program terminology used herein is merely for convenience, and therefore the exemplary embodiments should not be limited to use only in any particular application identified and / or implied by such terminology.
[0100] Based on the foregoing, a computer system, method, and computer program product have been disclosed. However, various modifications and substitutions can be made without departing from the scope of the exemplary embodiments. Therefore, exemplary embodiments have been disclosed by way of example rather than limitation.
[0101] It should be understood that while this disclosure includes a detailed description of cloud computing, the implementation of the teachings cited herein is not limited to cloud computing environments. Rather, exemplary embodiments can be implemented in conjunction with any other type of computing environment now known or developed hereafter.
[0102] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing power, storage, applications, virtual machines, and services), which can be rapidly provisioned and released with minimal management effort or interaction with the service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.
[0103] The features are as follows:
[0104] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring human interaction with the service provider.
[0105] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client or thick client platforms (e.g., mobile phones, laptops, and PDAs).
[0106] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify a location at a higher level of abstraction (e.g., country, state, or data center).
[0107] Rapid flexibility: The ability to provide capacity quickly and flexibly, automatically scaling down and up rapidly in some situations to scale up rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.
[0108] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.
[0109] The service model is as follows:
[0110] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from different client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.
[0111] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created by the consumer or acquired using programming languages and tools supported by the provider onto cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environment.
[0112] Infrastructure as a Service (IaaS): The capabilities offered to consumers are processing, storage, networking, and other basic computing resources that enable consumers to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).
[0113] The deployment model is as follows:
[0114] Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.
[0115] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.
[0116] Public cloud: Makes cloud infrastructure available to the public or large industry groups and is owned by an organization that sells cloud services.
[0117] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported (e.g., cloud bursting for load balancing between clouds).
[0118] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.
[0119] See now Figure 9 This describes an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 40 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. Nodes 40 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as services that cloud consumers do not need to maintain on their local computing devices. It should be understood that... Figure 9 The types of computing devices 54A-N shown are intended to be illustrative only, and computing node 40 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connectivity (e.g., using a web browser).
[0120] See now Figure 10 This demonstrates the 50 (cloud computing environment) Figure 9 This provides a set of functional abstractions. It should be understood beforehand. Figure 10 The components, layers, and functions shown are intended to be illustrative only, and the exemplary embodiments are not limited thereto. As described, the following layers and corresponding functions are provided:
[0121] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.
[0122] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.
[0123] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources based on anticipated future needs according to the SLA.
[0124] Workload tier 90 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that can be provided from this tier include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics and processing 94; transaction processing 95; and multimedia modification 96.
[0125] This invention can be a system, method, and / or computer program product with any possible level of technical detail integration. The computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.
[0126] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.
[0127] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.
[0128] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages (such as Smalltalk, C++, etc.) and procedural programming languages (such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.
[0129] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0130] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.
[0131] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two blocks shown consecutively may actually be completed as a single step, executed simultaneously, substantially simultaneously, or with partial or complete temporal overlap, or the blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.
[0133] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements over those found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
[0134] This invention can be embodied as a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.
[0135] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system used for propagation. Examples of computer-readable media can include semiconductor or solid-state memory, magnetic tape, removable computer disks, random access memory (RAM), read-only memory (ROM), rigid disks, and optical discs. Current examples of optical discs include compact disc-read-only memory (CD-ROM), compact disc-read / write (CD-R / W), DVDs, and Blu-ray discs.
[0136] Computer-readable storage media can be tangible means for retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.
[0137] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network), or to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.
[0138] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages (such as Smalltalk, C++, etc.) and conventional procedural programming languages (such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may be personalized to execute computer-readable program instructions by utilizing state information from the computer-readable program instructions in order to perform aspects of this invention.
[0139] The present invention will now be described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0140] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.
[0141] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce computer-implemented processing, such that the instructions executed on the computer, other programmable apparatus, or other device perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0142] The flowcharts and / or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.
[0143] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well. It should also be understood that when the terms “comprises” and / or “comprising” are used in this specification, they specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof.
[0144] All means or steps in the following claims, plus corresponding structures, materials, actions, and equivalents of the functional elements, are intended to include any structure, material, or action for performing the said function in conjunction with other claimed elements as specifically claimed. The description of the invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. These embodiments were chosen and described in order to best explain the principles and practical application of the invention, and to enable others skilled in the art to understand different embodiments of the invention with different modifications, as suited to the particular intended use.
Claims
1. A computer-implemented method for explicitly identifying entities in a database system, the method comprising: Data items are stored in tables in a database, whereby the data items are stored as records that include multiple attributes; A naming rule for storing selected combinations of attributes for the data item, wherein the naming rule is based on the plurality of attributes, and wherein the combination includes at least the customer name and the customer-related employer identification number (EIN). Based on the importance of the naming rules to entity identification, the naming rules are prioritized by defining a sequence of applying the naming rules or the order in which the naming rules are defined, wherein the combination of the customer name and the customer-related EIN has a relatively higher priority than the customer name alone; Determine the hash value of each of the selected combinations of the attributes of the data item; Duplicate data items are identified using the determined hash value and a priority-sorted naming rule; and The identified duplicate data items are merged into a single merged data item, and the attribute source of the merged data item is defined using a higher-priority naming rule.
2. The method according to claim 1, wherein, The database system is a relational database system, and an entry in the relational database system automatically triggers the database engine to create a naming rule associated with that entry.
3. The method according to claim 1, wherein, The database system is a configuration management database, which serves as the foundation for a specific internal organization and is used to manage technical equipment and applications across multiple data centers.
4. The method of claim 1, further comprising merging the identified duplicate data items by maintaining the determined hash value as a multi-value key for the merged data items.
5. The method of claim 4, further comprising merging other data items that are in a compositional relationship with the identified data item.
6. The method of claim 4, further comprising maintaining a pointer to the same row identifier for one of the merged data items for the determined hash value.
7. The method of claim 1, further comprising: Maintain the indexes of the table; as well as A pointer is maintained in the search tree associated with the index such that the pointer points to the same record identifier of the combined data item, wherein the combined data item is determined based on splitting a parent name into two or more separate aliases, and each of the two or more separate aliases is used as a single value in the index.
8. The method of claim 1, further comprising: Use creation SQL statements that are appropriate for creating naming rules and their associated priorities.
9. The method of claim 1, further comprising: Use multi-valued primary keys to sort records in a database table.
10. The method according to claim 1, wherein, Multi-valued primary keys are used to cluster clustered data on multi-node database engines.
11. The method of claim 9, wherein multi-valued primary keys and single-valued column data items are comparable.
12. The method of claim 1, further comprising: Collect statistical database data for data blocks with single-value primary keys and multi-value primary keys.
13. A computer system for explicitly identifying entities in a database system, the computer system comprising: One or more computer processors, one or more computer-readable storage media, and program instructions stored on the one or more computer-readable storage media for execution by at least one of the one or more processors capable of performing a method comprising: Data items are stored in tables in a database, whereby the data items are stored as records that include multiple attributes; A naming rule for storing selected combinations of attributes for the data item, wherein the naming rule is based on the plurality of attributes, and wherein the combination includes at least the customer name and the customer-related employer identification number (EIN). Based on the importance of the naming rules to entity identification, the naming rules are prioritized by defining a sequence of applying the naming rules or the order in which the naming rules are defined, wherein the combination of the customer name and the customer-related EIN has a relatively higher priority than the customer name alone; Determine the hash value of each of the selected combinations of the attributes of the data item; Duplicate data items are identified using the determined hash value and a priority-sorted naming rule; and The identified duplicate data items are merged into a single merged data item, and the attribute source of the merged data item is defined using a higher-priority naming rule.
14. The computer system of claim 13, wherein the database system is a relational database system, and wherein, An entry in the relational database system will automatically trigger the database engine to create a naming rule associated with that entry.
15. The computer system of claim 13, wherein the database system is a configuration management database, the configuration management database being the foundation of a particular internal organization and used to manage technical equipment and applications in multiple data centers.
16. The computer system of claim 13, further comprising merging identified duplicate data items by maintaining the determined hash value as a multi-value key for merging data items.
17. The computer system of claim 16, further comprising merging other data items that are in a compositional relationship with the identified data item.
18. The computer system of claim 16, further comprising maintaining a pointer to the same row identifier for one of the merged data items for the determined hash value.
19. The computer system of claim 13, further comprising: Maintain the indexes of the table; as well as A pointer is maintained in the search tree associated with the index such that the pointer points to the same record identifier of the combined data item, wherein the combined data item is determined based on splitting a parent name into two or more separate aliases, and each of the two or more separate aliases is used as a single value in the index.
20. A computer program product for explicitly identifying entities in a database system, the computer program product comprising: One or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable storage media capable of executing a method, the method comprising: Data items are stored in tables in a database, whereby the data items are stored as records that include multiple attributes; A naming rule for storing selected combinations of attributes for the data item, wherein the naming rule is based on the plurality of attributes, and wherein the combination includes at least the customer name and the customer-related employer identification number (EIN). Based on the importance of the naming rules to entity identification, the naming rules are prioritized by defining a sequence of applying the naming rules or the order in which the naming rules are defined, wherein the combination of the customer name and the customer-related EIN has a relatively higher priority than the customer name alone; Determine the hash value of each of the selected combinations of the attributes of the data item; Duplicate data items are identified using the determined hash value and a priority-sorted naming rule; and The identified duplicate data items are merged into a single merged data item, and the attribute source of the merged data item is defined using a higher-priority naming rule.
Citation Information
Patent Citations
Method and system for cleansing and de-duplicating data
US20170308557A1
Entity processing method and device and storage medium
CN109684625A
Method and System for Managing and Reconciling Information Technology Assets in a Configuration Database
US20120096163A1
Finding doublets in a database
US20170011088A1