Efficient re-ranking and classification methods for fuzzy inputs in deep hierarchical structures

Abstract

The present disclosure relates to a classification method for classifying records by relevance to nodes of a hierarchical structure representing classification schemes for different classification options. The classification method includes receiving an input record having a plurality of record content features, and a context attribute indicating the received context of the input record. The method includes retrieving relationship data indicating one or more nodes of the hierarchical structure associated with the context attribute of the received input record. The method also includes determining a relevance score of one or more nodes in the nodes of the hierarchical structure to classify the input record. The relevance score of each of the one or more nodes is determined based on a comparison of the plurality of record content features of the input record relative to the classification option represented by the node, and further, wherein the relevance score of each of the one or more nodes is determined based on the retrieved relationship data.

Description

Technical Field

[0001] The present disclosure generally relates to a method for classifying input records according to a hierarchical structure. In particular, the present disclosure relates to efficient (re)ordering and classification of fuzzy inputs in a deep hierarchical structure. Aspects of the present disclosure relate to a classification method, a controller, and a non-transitory computer-readable storage medium. Background Art

[0002] It is generally desirable to classify the recorded data according to a sparse and deep hierarchical structure (such as a tree or directed acyclic graph). In this way, the data can be classified at a desired level of detail, and the classified data can be efficiently searched and classified for efficient processing and storage. Such a hierarchical structure includes a series of nodes or classification options that are arranged into levels or branches connected together by corresponding edges or connections. In this way, each edge in the hierarchical structure connects a node at a higher level to a set of more granular nodes on a lower level. This forms a so-called "parent node" at a higher level and forms a set of "child nodes" or more detailed classification options at a lower level.

[0003] Methods for classifying data records into such hierarchical structures are well known in the art of computer science. However, challenges arise in classifying data records into such hierarchical structures when the nodes at the desired classification level are very similar to each other and / or the data of the record is otherwise difficult to classify into one option or another.

[0004] This difficulty in classifying records can mean that an increasing number of records need to be classified to construct an accurate ordering within the structure. This suffers from computational inefficiencies, as a larger number of inputs and outputs, with a greater number of iterations, must be processed to achieve an ordering. Furthermore, when records cannot be accurately classified using known computational methods, time-consuming manual intervention may be required to perform the classification, which is undesirable. Therefore, a more accurate and efficient method for classifying records within a hierarchical structure is needed.

[0005] It is in this context that the present disclosure has been designed. Summary of the Invention

[0006] According to one aspect of the present disclosure, a classification method is provided for classifying records by relevance to nodes of a hierarchical structure representing classification schemes for different classification options. The classification method includes receiving an input record having a plurality of record content features, and a context attribute indicating a received context of the input record. The method includes retrieving relationship data indicating one or more nodes of the hierarchical structure associated with the context attribute of the received input record. The method also includes determining a relevance score of one or more nodes in the nodes of the hierarchical structure to classify the input record. The relevance score of each of the one or more nodes is determined based on a comparison of the plurality of record content features of the input record relative to the classification options represented by the nodes. The relevance score of each of the one or more nodes is also determined based on the retrieved relationship data.

[0007] The classification method may include determining relationship data based on a database of historical records.

[0008] The relational data may be determined by applying an ontology engineering algorithm to a database of historical records to determine a knowledge graph that models the relevance of contextual attributes of the historical records to nodes of a hierarchical structure.

[0009] The relationship data may be determined by applying a semantic reasoning algorithm to the knowledge graph to determine a set of contextual attributes that are logically related to corresponding nodes of the hierarchy.

[0010] Relationship data may be retrieved based on matching context attributes of the input record with corresponding context attributes selected from the set of context attributes determined by the semantic reasoning algorithm.

[0011] A knowledge graph may be determined by dividing a database of historical records into one or more subject groups, object groups, and / or predicate groups.

[0012] In some examples, determining the relevance score for each of the one or more nodes includes: determining an initial relevance score based on a comparison of the plurality of record content features of the input record relative to the classification options represented by the node; and updating the initial relevance score based on the retrieved relationship data.

[0013] In some examples, increasing or decreasing the initial relevance score based on the retrieved relational data includes: increasing the initial relevance score of a first node at a desired classification level of the hierarchical structure based on the relational data indicating that the first node is associated with a contextual attribute of the received input record; and decreasing the initial relevance score of a second node different from the first node at the desired classification level.

[0014] In some examples, determining the relevance score for each of the one or more nodes includes: determining a confidence score for the input record indicating uncertainty in an initial relevance score; and updating the initial relevance score according to the determined confidence score.

[0015] The relevance score of one or more nodes in the hierarchical structure can be determined based at least in part on a combination of a local similarity score of the node and local similarity scores of one or more child nodes subordinate to the node, each local similarity score indicating a relevance of the input record to the corresponding node.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium having instructions stored thereon is provided, which, when executed by a computer, causes the computer to perform the above classification method.

[0017] According to another aspect of the present disclosure, a classification system is provided for classifying records by relevance to nodes of a hierarchical structure representing a classification scheme for multiple classification options. The classification system includes an input module configured to receive an input record having multiple record content features and a context attribute indicating a received context of the input record. The classification system includes a relevance evaluation module configured to determine a relevance score for one or more nodes in the nodes of the hierarchical structure to classify the input record. The relevance score of each of the one or more nodes is determined based on a comparison of the multiple record content features of the input record relative to the classification options represented by the nodes. The relevance score of each of the one or more nodes is also determined based on retrieved relationship data indicating one or more nodes of the hierarchical structure associated with the context attribute of the received input record. The classification system includes an output module configured to output a classification of the input record based on the determined relevance score of one or more nodes in the nodes of the hierarchical structure.

[0018] The classification system may further include a contextualizing module configured to determine relationship data based on the database of historical records.

[0019] The contextualization module may be configured to determine the relational data by applying an ontology engineering algorithm to the database of historical records to determine a knowledge graph that models the relevance of contextual attributes of the historical records to nodes of the hierarchical structure.

[0020] The contextualization module may be configured to determine the relationship data by applying a semantic reasoning algorithm to the knowledge graph to determine a set of contextual attributes that are logically related to corresponding nodes of the hierarchical structure.

[0021] The contextualization module may be configured to retrieve the relationship data based on matching context attributes of the input record with corresponding context attributes selected from the set of context attributes determined by the semantic reasoning algorithm.

[0022] It will be appreciated that preferred and / or optional features of each aspect of the present disclosure may also be incorporated into other aspects of the present disclosure, either alone or in appropriate combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Examples of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0024] Figure 1 A schematic diagram illustrating an exemplary classification system according to an embodiment of the present disclosure is shown;

[0025] Figure 2 Schematically illustrates the operation of an embodiment according to the present disclosure Figure 1 An exemplary method of the classification system shown;

[0026] Figure 3 Schematically shows Figure 2 Exemplary sub-steps of the steps in the illustrated method;

[0027] Figure 4 Schematically shows Figure 2 an exemplary sub-step of another step in the method shown; and

[0028] Figure 5 Schematically shows Figure 2 Exemplary sub-steps of further steps in the method shown. DETAILED DESCRIPTION

[0029] Embodiments of the present disclosure relate to a classification system and method for classifying input records, such as a taxonomy of objects, based on their relevance to nodes of a hierarchical structure.

[0030] Each node may represent a corresponding classification option within the classification scheme, and the classification system may be configured to determine a set of scores, referred to as “relevance scores,” that indicate the relevance of the input record to the corresponding node. For example, each relevance score may represent a probability that the input record is relevant to the corresponding node.

[0031] Advantageously, to reduce uncertainty in the determined relevance scores, the classification system is configured to utilize a history of relationship data that can be processed (semantically and contextually) to determine relationship data that indicates how contextual attributes of received records relate to one or more nodes of the hierarchical structure. For example, the relationship data may indicate that input records received at a particular time, generated from a particular source, or received as part of a particular set of input records are related to one or more corresponding nodes of the hierarchical structure. The relationship data may be semantically and / or contextually related data that may have hidden relevance connections.

[0032] Thus, the classification system may receive an input record and a context attribute indicating a context of the input record, and retrieve corresponding relationship data that matches the context attribute to determine the relevance of the input record to a node of the hierarchical structure.

[0033] In this way, uncertainty in the determined relevance score may be reduced and a more accurate relevance score may be determined, as will become clear below.

[0034] Thus, the contribution of the present disclosure can be considered a classification system and method that uses external contextual information about the record in addition to internal information describing the subject matter of the record to classify the record. This arrangement provides enhanced classification capabilities and enables a more accurate relevance score to be determined for each input record.

[0035] Figure 1 An exemplary classification system 1 for determining the relevance of an input record to a hierarchical structure is schematically illustrated.

[0036] The classification system 1 includes an input module 2, a contextualization module 4, a relevance evaluation module 6, and an output module 8. That is, in the example described, four main functional elements, units, or modules are shown. Each of these units or modules can be provided by suitable software running on any suitable computing substrate using conventional or custom processors and memory. Some or all of the units or modules can use a common computing substrate (for example, they can run on the same server) or separate substrates, or different combinations of modules can be distributed among multiple computing devices.

[0037] The input module 2 is configured to receive and / or store input records and contextual attributes of the input records. The input records may relate to, for example, objects, events, or transactions, and may include one or more record content features describing the subject matter of the input records. For example, the one or more record content features may include attributes or values ​​of multiple variables describing the subject matter of the input records.

[0038] The context attributes indicate the context of the input record, or in other words, the circumstances under which the input record was generated or received at the classification system. Thus, the context attributes go beyond record content features that describe the subject matter of the input record and may indicate a range of relevant contextual information that may define one or more context parameters, such as: the source of the input record; the time and / or date of generation or receipt of the input record; one or more accompanying input records received with the input record, for example, as part of a group of input records; and / or the relevance of the one or more accompanying input records to a node of the hierarchical structure.

[0039] Thus, it will be appreciated that contextual attributes may be provided in the form of contextual information from which the classification system 1 can derive the context or situation of the input record.

[0040] The input module 2 is further configured to receive and / or store the hierarchical structure. The hierarchical structure in the example of the classification system 1 can take different forms, including a directed acyclic graph, a tree or a forest of trees and / or a plurality of directed acyclic graphs.

[0041] In each case, the hierarchical structure includes multiple nodes that represent corresponding classification options at successive layers of classification (called classification levels). Multiple nodes are then connected together across the levels by effectively mapping connections between related nodes and forming corresponding edges that branch between classification options at different classification levels.

[0042] In this way, each edge in the hierarchical structure connects a higher-level node to a set of more granular nodes on a lower classification level, forming a "parent node" on the higher classification level and a set of "child nodes" or more detailed classification options on the lower classification level. Due to this structure, the relevance of input records can be evaluated at a desired level of detail.

[0043] For this purpose, the input module 2 may include a memory storage module, such as a cloud storage system or a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium). A computer-readable storage medium may include any mechanism for storing information in a form readable by a machine or electronic processor / computing device, including but not limited to: magnetic storage media (e.g., floppy disks); optical storage media (e.g., CD-ROMs); magneto-optical storage media; read-only memory (ROM); random access memory (RAM); erasable programmable memory (e.g., EPROM and EEPROM); flash memory; or any other type of electronic or other medium for storing such information / instructions.

[0044] The input module 2 may receive the hierarchical structure from any suitable source, including a memory storage device and / or a computing device. Similarly, the input module 2 may receive input records from any suitable source, including a memory storage device, a computing device, and / or one or more data capture systems configured to generate the set of input records. For example, the data capture system may be configured to generate a set of input records including the input records based on corresponding events or transactions, and output the set of input records to the input module 2.

[0045] The contextualisation module 4 is configured to process a database of historical records (ie records that have been previously received and / or processed) to determine relationship data indicating that one or more contextual attributes are associated with respective nodes of the hierarchical structure.

[0046] In particular, the contextualization module 4 may be configured to process the database of historical records using one or more data processing algorithms configured to establish a relationship between the context of the historical records and the relevance of those historical records to the nodes of the hierarchy.

[0047] For this purpose, the database of historical records may include relevance data indicating the historical relevance of each historical record to one or more nodes of the hierarchical structure and context data indicating the context of each historical record, such as the time, date or a set of historical records in which the historical record was received. The contextualization module 4 may integrate such data into an ontology such as a knowledge graph to which data processing algorithms such as semantic reasoning algorithms may be applied to derive one or more contextual attributes associated with the corresponding nodes of the hierarchical structure. For example, the determined relationship data may include a list of contextual attributes associated with the corresponding nodes of the hierarchical structure. For example, the list of contextual attributes may be determined based on trends, frequency counts and / or other suitable metrics.

[0048] In this way, the contextualization module 4 can determine, based on the accumulation of historical records received at a specific reception time that have historical relevance to a corresponding node of the hierarchical structure, that a contextual attribute describing that time is associated with the same node.

[0049] The relevance of each contextual attribute to the corresponding node of the hierarchy may be in the form of an n-ary classification for each of the nodes, such as a relevance score, or a binary classification for each of the nodes.

[0050] It should be understood that the contextualization module 4 may include a memory storage device configured to store a database of historical records including context data and relevance data associated with the historical records.

[0051] The relevance evaluation module 6 is configured to evaluate the relevance of the input record to the nodes of the hierarchical structure by determining relevance scores of multiple nodes of the hierarchical structure. The relevance score of each node may indicate the relevance or relative relevance of the node to the record content feature of the input record.

[0052] For example, each relevance score can represent a probability that an input record is relevant to a corresponding node of the hierarchical structure. In this way, differences between relevance scores associated with corresponding input records can indicate that certain nodes are more or less relevant to one input record than to another input record.

[0053] In one example, the relevance evaluation module 6 is configured to determine relevance scores for some or all of the nodes of the hierarchical structure relative to the input record. For example, the relevance evaluation module 6 can be configured to determine relevance scores for the input record for a plurality of nodes arranged at one or more desired classification levels of the hierarchical structure.

[0054] In another example, the classification system 1 may advantageously be configured to determine a relevance score for the input record for each node in the hierarchy, thereby providing a complete assessment of the relevance of the input record to the hierarchy.

[0055] As will become clear, the relevance evaluation module 6 can be configured to determine the relevance score for each node using one or more methods, and advantageously, the relevance evaluation module 6 can be configured to determine the relevance score for one or more nodes of the hierarchical structure based at least in part on relationship data determined from the database of historical records, and in particular, based on relationship data relating contextual attributes of the input record to one or more nodes of the hierarchical structure.

[0056] To this end, relevance evaluation module 6 may match contextual attributes of the input record with corresponding contextual attributes of the relationship data and determine a relevance score for one or more nodes of the hierarchy based at least in part on the indicated relevance of the nodes to the matched contextual attributes.

[0057] It will be appreciated that the relevance score may be determined based at least in part on the relevance of the node to the indication of the matching contextual attribute using one or more suitable methods or algorithms.

[0058] For example, the relevance evaluation module 6 may include the relevance of the input record to the corresponding nodes of the hierarchical structure (indicated by the relationship data) as an input or variable in the algorithm used to determine the relevance score. In another example, the relevance evaluation module 6 may determine the relevance score in stages and increase or decrease the initial relevance score determined based on the record content characteristics according to the relevance of the input record to those nodes (as indicated by the relationship data).

[0059] It should be understood that conventional classification methods do not use contextual attributes of input records to classify them. This is because such contextual information is outside of the record content features that describe the subject matter of the input record. However, in this classification system, contextual attributes of input records are used to determine a relevance score. Because these contextual attributes describe the context of the input record, the relevance score can be used to infer, rather than explicitly determine, the relevance of the input record to a node in the hierarchical structure.

[0060] In this manner, the classification system is able to provide more accurate relevance scores and enhanced classification capabilities. Furthermore, because the context of the record is used in determining the relevance score, irregular or otherwise unusual input records can be advantageously classified into a hierarchical structure based on the context in which they occurred. Thus, such classification methods can provide enhanced classification capabilities.

[0061] The relevance evaluation module 6 can also be configured to generate a relevance map for the input record based on the determined relevance scores. For example, the relevance map can include a copy of the hierarchical structure and corresponding relevance scores determined for nodes of the hierarchical structure. For example, the relevance map can include a relevance score for each node of the hierarchical structure associated with the input record. In this manner, the relevance map can provide a probabilistic map indicating nodes of the hierarchical structure that are relevant to the input record.

[0062] It should be understood that in the following description, reference to a relevance map for an input record is in fact interchangeable with reference to the set of relevance scores associated with the input record.

[0063] The output module 8 is configured to receive an input record and a determined relevance map or set of relevance scores, and output a classification of the input record based on the determined relevance scores. For example, the output module 8 may output a classification of the input record at one or more classification levels. In this manner, the output module 8 may output a series of increasingly granular classifications of the input record (e.g., corresponding to respective classification options in successive levels of the hierarchical structure).

[0064] Technical benefits of the classification system 1 include efficiency gains from reducing the amount of input and output required to construct an accurate ranking / classification, and computational improvements from reducing the iterations required to classify input records.

[0065] Now we will refer to Figures 2 to 5 The operation of the classification system 1 is described.

[0066] Figure 2 An exemplary method 20 of operating the classification system 1 to classify input records according to their relevance to nodes of a hierarchical structure is shown.

[0067] In step 22 , the classification system 1 receives: an input record for comparison with the hierarchical structure; and a context attribute indicating a received context of the input record.

[0068] To provide context, the hierarchical structure can take the form of a tree representing a taxonomy of objects. In one non-limiting example, the hierarchical structure includes nodes representing fasteners such as screws and washers, and nodes representing mechanical devices, including nodes for washing machines, tumble dryers, and dishwashers, among other machines.

[0069] In this manner, the hierarchical structure may contain subtrees (or branches) corresponding to various types of screws, subtrees (or branches) corresponding to various types of washers, and subtrees (or branches) corresponding to various types of washing machines.

[0070] The input record may relate to a transaction, and the input record may include one or more record content features describing properties of washers involved in the transaction. For example, such record content features may include cost, inventory, and a classification symbol for washers labeled "wash."

[0071] The context attribute may indicate contextual information describing the circumstances under which the input record was generated or received at the classification system 1. For example, the context attribute may indicate that the input record was received along with a companion input record in a set of input records related to a transaction. Thus, the input record may form a first input record of the set of input records, and the companion input record may form a second input record of the set of input records.

[0072] The second input record may include one or more record content features that describe the characteristics of the screws involved in the transaction (such as sheet metal screws). For example, such record content features may include the size, material, strength, and identification of the sheet metal screws.

[0073] It should be understood that the classification system 1 can classify each of the first input record and the second input record according to a classification method substantially as described in the following steps. Therefore, the context attribute of the second input record can indicate context information related to the corresponding circumstances in which the second input record is generated or received at the classification system 1, including the fact that the second input record is received together with the first input record.

[0074] It should also be understood that this example is not intended to limit the scope of classification system 1.

[0075] The first input record and the second input record may be received at and / or stored in the input module, e.g., having been determined by one or more computing devices or data capture systems. For example, the first input record and the second input record may be generated in response to respective transactions or events involving the described object.

[0076] In step 22 , the hierarchical structure may have been previously determined, for example based on one or more user inputs, and the hierarchical structure may also be stored in the input module 2 or otherwise received at the input module.

[0077] In step 22, the classification system 1 may also receive a desired classification level of the hierarchical structure for classifying the first input record and the second input record. For example, the classification system 1 may receive one or more user inputs defining the desired classification level, or the desired classification level may be stored in the input module 2, for example.

[0078] In this simplified example, a desired classification level of the hierarchy may include a first node representing a washer, a second node representing a washing machine, and a third node representing a screw.

[0079] In step 24 , the classification system 1 is configured to determine, based on the database of historical records, relationship data relating one or more context attributes to respective nodes of the hierarchical structure.

[0080] The classification system 1 may use one or more methods to determine the relationship data based on the database of historical records. Therefore, it should be understood that the following examples are not intended to limit the scope of the classification system 1.

[0081] In one example, the contextualization module 4 may integrate the database of historical records into an ontology such as a knowledge graph, to which data processing algorithms such as semantic reasoning algorithms may be applied to derive relational data.

[0082] Thus, method 20 may include Figure 3 Sub-steps 26 and 28 for determining relationship data are shown and will now be described in more detail.

[0083] In sub-step 26, the contextualization module 4 may determine a knowledge graph based on the database of historical records using one or more graph mapping algorithms configured to map record content features, context data, and relevance data in the database of historical records into the knowledge graph.

[0084] Knowledge graphs organize information in a way that preserves semantic knowledge, such as including similarity distance scores that indicate the similarity of contextual data, relevance data, and record content features.

[0085] Knowledge graphs are well known in the field of graph theory and are not discussed in further detail here to avoid obscuring the contributions of the present disclosure.

[0086] However, to give an example, the contextualization module 4 may divide a database including a history of recorded content features, context data, and relevance data into triples including a subject group, an object group, and a predicate group. For example, the contextualization module 4 may then populate a knowledge graph with the triples to create an ontological knowledge base with semantic relationships linking the groups together.

[0087] In such forms, the knowledge graph may be adapted to logically derive relationship data indicating that certain contextual attributes are associated with corresponding nodes of the hierarchical structure.

[0088] In sub-step 28 , the contextualization module 4 may apply one or more semantic reasoning algorithms to the knowledge graph to determine relational data.

[0089] Such semantic reasoning algorithms are not described in detail herein, but it should be understood that semantic reasoning algorithms can use the subject, object, and predicate groups of a knowledge graph to determine logic-based relationships between context attributes and corresponding nodes of a hierarchical structure, where the context attributes are derived from context data in a database of historical records.

[0090] The relationships between different nodes in the hierarchy may be hidden or otherwise unknown and need to be inferred from other shared features. That is, historical data that has been linked through the determined knowledge graph is used to infer commonalities between nodes in the hierarchy. The determined relationship data may, for example, include a list of contextual attributes associated with the corresponding nodes of the hierarchy. For example, the list of contextual attributes may be sorted based on normalized frequency and / or standardized frequency.

[0091] To give an example, the determined relationship data may associate a first contextual attribute of the received record with a node of the hierarchical structure representing a washer. For the node contextually associated with the received record, the first contextual attribute may indicate that the received record should have been received in a set of records along with a companion record associated with a node of the hierarchical structure representing a screw.

[0092] Back to Figure 2 , in step 30 , the classification system 1 determines a relevance score for each input record.

[0093] The following description focuses on the steps involved in determining the relevance score of a first input record. However, it should be understood that the relevance score of a second input record can be determined in substantially the same manner.

[0094] As previously mentioned, the relevance evaluation module 6 may be configured to determine the relevance score of the first input record using one or more methods, and thus the following examples are not intended to limit the scope of the present disclosure.

[0095] As will become clear, in one example, relevance evaluation module 6 can determine relevance scores in stages. For example, relevance evaluation module 6 can determine a set of initial relevance scores based on record content features of a first input record. Relevance evaluation module 6 can then increase or decrease the initial relevance scores based on the contextual relevance of the first input record to those nodes, as provided by the relationship data. Determining relevance scores in this manner can advantageously minimize the processing requirements associated with determining relevance scores for nodes of a hierarchical structure.

[0096] For this purpose, the method 20 may also include: Figure 4 The sub-steps 32 to 36 for determining the initial relevance score shown, and Figure 5 The sub-steps 38 to 50 for modifying the initial relevance score are shown as will now be described in more detail below.

[0097] In this example, the relevance assessment module 6 is configured to determine a relevance score for each node of the hierarchical structure.

[0098] Therefore, in sub-step 32 ( Figure 4 As shown in FIG, the relevance evaluation module 6 evaluates the relevance of the first input record to each node of the hierarchical structure according to the corresponding evaluation dimension.

[0099] For example, the relevance evaluation module 6 may determine a dimensional similarity score DS for each node N of the hierarchical structure with respect to each record content feature (or dimension of evaluation D) of the first input record.

[0100] It should be understood that the dimension similarity score for each node can be determined by one or more methods. For example, the calculation of the dimension similarity score for each node can consider any matches between keywords, values, or measurements extracted from the record content features and any keywords, values, or measurements associated with the corresponding node.

[0101] Thus, in the non-limiting example described, the relevance evaluation module 6 may determine relatively high dimensional similarity scores for the first and second nodes of the hierarchical structure, which are respectively related to washers and washing machines, based on the record content feature including the classification symbol "wash." This is because the classification symbol "wash" can reasonably refer to either a washer or a washing machine.

[0102] However, the relevance assessment module 6 may determine a much lower dimensional similarity score for the same record content feature relative to the third node of the hierarchy relating to screws, since the classification symbol "wash" would not apply.

[0103] To give another example, relevance evaluation module 6 may determine a high-dimensional similarity score for a first node related to a gasket based on record content features describing dimensions of objects that fall within a desired dimensional range for the gasket.

[0104] In sub-step 34, the relevance evaluation module 6 may determine a local relevance or similarity score for the first input record of each node in the hierarchical structure based on the sum or weighted sum of the dimensional similarity scores for the node. For example, the weights used to determine the weighted sum of the dimensional similarity scores may be stored in the relevance evaluation module 6, and the weights may vary for different levels of the hierarchical structure.

[0105] In sub-step 36, the relevance evaluation module 6 may determine an initial relevance score for each node in the hierarchy by propagating the local similarity scores upward in the hierarchy from child nodes to parent nodes. Thus, the initial relevance score of a parent node may be a combination of the local similarity score of the parent node and the aggregated local similarity scores of some or all of its child nodes.

[0106] For example, the relevance evaluation module 6 may recursively determine the initial relevance score of the node based on the aggregation of the local similarity scores of each node and the selected child nodes belonging to the node. For example, the selected child nodes may be the top K child nodes ranked based on their local similarity scores, where K is a predetermined integer.

[0107] To illustrate this, the following example describes determining relevance scores for a first and second lowest level of a hierarchy.

[0108] First, the relevance evaluation module 6 may determine an initial relevance score for each node at the lowest level of the hierarchical structure. The initial relevance score for each node at the lowest level of the hierarchical structure may be equal to the local similarity score of the node, since the node is at the end of a branch and has no child nodes subordinate to it.

[0109] In order to propagate the local similarity scores upward in the hierarchical structure, the relevance evaluation module 6 can then determine the initial relevance score for each node in the second level of the hierarchical structure by: i) sorting the child nodes based on the relevance scores of the child nodes subordinate to the node (i.e., sorting those nodes subordinate to the node on the lowest level of the hierarchical structure); and adding together the local similarity score of the node and the determined initial relevance scores of the top K ranked child nodes subordinate to the node.

[0110] The relevance evaluation module 6 may then continue in this manner to recursively determine an initial relevance score for each node in the hierarchy, progressing through the levels in the hierarchy in substantially the same manner.

[0111] In one example, once an initial relevance score has been determined for each node in the hierarchy, the relevance evaluation module 6 may generate a relevance map for the first input record based on the hierarchy and the set of relevance scores determined for the input record.

[0112] It will be appreciated that the relevance map provides a convenient combination of a hierarchy and the set of relevance scores for efficient data processing.

[0113] Referring to the desired classification hierarchy, the relevance evaluation system 6 may determine a high initial relevance score for a first node related to washers and a relatively high initial relevance score for a second node related to washing machines. However, the relevance evaluation system 6 may determine a relatively low initial relevance score for a third node related to screws.

[0114] In addition, although not described in detail, it should be understood that the record content features of the second input record are clearly related to screws, and therefore the relevance evaluation system 6 can determine a high relevance score for the second input record for the third node and a low relevance score for each of the first node and the second node.

[0115] In this manner, the relevance score of each input record represents a scale (from most relevant to least relevant) of the relevance of the corresponding input record to a node of the hierarchy.

[0116] However, while it may be clear that the first input record is not specifically related to the screw node, it is less clear whether the first input record is related to a washer or a washing machine. Thus, due to the ambiguous nature of record content features (such as the text "wash" in this example, which may refer to a washer or a washing machine), there may be considerable uncertainty in one or more of the relevance scores.

[0117] To reduce this uncertainty, the classification system 1 may be configured to refine the initial relevance scores using the relationship data, and the classification system 1 may focus on improving those initial relevance scores with the greatest uncertainty or those for nodes at a desired classification level.

[0118] For example, in sub-steps 38 to 50 (e.g. Figure 5 ), the classification system 1 may evaluate the confidence of the relevance score determined for each input record at the desired classification level of the hierarchy.

[0119] The confidence of the determined relevance score at the desired classification level can be evaluated by determining a confidence score for each input record.

[0120] The confidence score indicates a relative assurance that the determined relevance score for the input record is accurate, where a high confidence score indicates that there is low uncertainty in the determined relevance score and a low confidence score indicates that there is high uncertainty in the determined relevance score.

[0121] It will be appreciated that one or more methods may be used to determine such confidence scores.

[0122] To give an example, in sub-step 38 , the relevance evaluation module 6 may receive a desired classification level corresponding to a level of a node in the hierarchical structure.

[0123] In sub-step 40, the relevance evaluation module 6 may determine a normalized relevance score for each input record for each node in the desired classification hierarchy. The normalized relevance score for each input record may be determined for each node by dividing the initial relevance score for that node by the total relevance score for the desired classification hierarchy. The total relevance score for the desired classification hierarchy may be calculated by adding together the initial relevance scores for the nodes at the desired classification hierarchy.

[0124] In sub-step 42 , the relevance assessment module 6 may determine a confidence score for each input record using one or more confidence assessment algorithms based on the normalized relevance scores of the nodes at the desired classification level.

[0125] For example, the relevance evaluation module 6 may use the Shannon entropy formula to determine a confidence score for each input record based on the positive (non-zero) normalized relevance scores of nodes at the desired classification level.

[0126] It will be appreciated that the Shannon entropy formula is well known in the field of information theory and is not described in detail herein to avoid obscuring the present method.

[0127] Nevertheless, for clarity, the Shannon entropy formula can be applied as:

[0128]

[0129] Among them, P i is the normalized relevance score of each node i at the desired classification level with a positive relevance score. It should be understood that the confidence score takes a positive value not greater than 1.

[0130] It will be appreciated that in other examples, this method of evaluating the confidence of the determined relevance scores may be repeated for each classification level of the hierarchy.

[0131] In addition, in the case where the hierarchical structure includes multiple connected components (i.e., a forest), according to the above-mentioned sub-steps 38 to 42, a confidence score can be determined for each connected component (i.e., for each tree or directed acyclic graph), and the total confidence score of the input record can be determined as the minimum confidence score determined for each connected component.

[0132] To return to the earlier example, the relevance evaluation module 6 may determine a relatively low confidence score for the first input record at the desired classification level because the relevance scores of the first node and the second node are relatively high due to ambiguous record content features. This means that the classification system cannot classify the first input record as being related to washers or washing machines with high confidence.

[0133] Meanwhile, the relevance evaluation module 6 may determine a high confidence score for the relevance score of the second input record at the desired classification level because a high relevance score was determined for the third node and low relevance scores were determined for the first and second nodes.

[0134] As will become clear, the classification system 1 may then use such confidence scores to optimize the determined relevance scores.

[0135] In sub-step 44 , the classification system 1 may retrieve relationship data associated with the contextual attributes of the first input record in order to optimize the relevance score.

[0136] As previously described, relationship data indicates that one or more contextual attributes are associated with corresponding nodes in the hierarchical structure. In this example, the relationship data may indicate that records associated with "screws" are often accompanied by other records associated with "washers." For example, such relationship data has been determined from the database of historical records if the following conditions are met: i) the database of historical records includes multiple pairs of records received in corresponding groups; and ii) one record in each group is highly associated with "screws," and another record in each group is highly associated with "washers."

[0137] Thus, if a context attribute indicates that an input record was received with a companion input record related to a third node representing a screw, the relationship data may indicate that the input record is related, through its context, to a first node representing a washer, and vice versa.

[0138] Therefore, in sub-step 44 , the relevance evaluation module 6 may match the context attributes of the first input record with the relationship data and identify a corresponding relationship between the first input record and the first node related to the gasket.

[0139] Such a relationship can be identified based on the following conditions: i) the first input record is received together with the second input record; ii) the second input record and the third node (related to the screw) to which the relationship data relates are most correlated; iii) the confidence score of the first input record is low, while the confidence score of the second input record is high; and iv) the first node related to the washer is the one among the nodes for which the first input record has the highest correlation score.

[0140] The identified relationship between the first input record and the first node can then be used to adjust the initial relevance score of the first input record. In this way, the classification system 1 can use the determined contextual relationship between the "screw" and "washer" nodes to adjust the initial relevance score of the first input record.

[0141] In another example, a database of historical records may include multiple records received from the same source at the same time on consecutive days (e.g., from a particular store at 7 a.m.). Each of these historical records may be highly correlated with the first node (related to washers), and thus the relationship data may indicate that the records received from the source at that time are contextually related to "washers."

[0142] Thus, if the context attribute indicates that the first input record was received from the store at 7 a.m., the relationship data will indicate that the first input record is related to the first node through its context. Therefore, the context attribute will relate the first input record to the first node, and this relationship will be used to adjust the initial relevance score.

[0143] The classification system 1 may then use one or more methods to update the initial relevance scores based on the nodes indicated by the relationship data.

[0144] For example, method 20 may include sub-steps 46 to 50 for optimizing an initial relevance score of an input record.

[0145] The following description relates to the steps taken to optimize the relevance score at the desired classification level for a first input record, but it will be appreciated that the same steps may be repeated mutatis mutandis as required for a second input record.

[0146] In sub-step 46, the relevance evaluation module 6 may increase the relevance score of the node indicated by the relationship data based on the contextual attributes of the input record. For example, the relevance evaluation module 6 may increase the relevance score of the most relevant node indicated by the relationship data by a predetermined value α. In this example, the relevance evaluation module 6 may therefore increase the relevance score of the first node related to the gasket by α.

[0147] In sub-step 48, relevance evaluation module 6 may reduce the relevance scores of the other nodes at the desired classification level of the hierarchical structure. For example, the update module may reduce the relevance score of each of the other nodes by a predetermined value β. Thus, in this example, relevance evaluation module 6 may reduce the relevance scores of the second node associated with a washing machine and the third node associated with a screw by β. It should be understood that β may be equal to α.

[0148] In sub-step 50 , the correlation evaluation module 6 may then repeat sub-steps 38 to 48 as many times as necessary, thereby decreasing the α and / or β values ​​with each iteration until the α and / or β values ​​are less than the corresponding threshold values.

[0149] In this manner, the relevance assessment module 6 effectively reduces uncertainty in the relevance score of the first input record based on previously determined relevance of the database of historical records. Thus, relevance knowledge is effectively transferred from the historical records (which may include less ambiguous record content features) to the input record based on the shared context of occurrences that logically link the records together.

[0150] Back to Figure 2 In step 52 , the classification system 1 may classify the input records according to the classification options of the hierarchy based on the determined relevance scores.

[0151] For example, the output module 8 may classify the first input record as being related to washers because the maximum relevance score at the desired classification level is determined for the first node. Similarly, the output module 8 may classify the second input record as being related to screws.

[0152] In this way, the classification system 1 is able to classify an ambiguous set of input records according to a hierarchy or classification scheme, and the output module 8 can output such classification, eg, for efficient processing and / or storage of the input records.

[0153] Many modifications may be made to the examples described above without departing from the scope of the appended claims.

[0154] In another example, the input record may be in the form of an image and the classification system 1 may be used as part of an image classification system to classify the content of the image according to a hierarchical structure. In this case, the record content features may be in the form of pixels and the classification system 1 may include one or more image processing algorithms for determining an initial relevance score based on the pixels in sub-steps 32 to 36. The classification system may then use contextual attributes of the input record (such as the image set in which the image was received) in sub-steps 38 to 50 to optimize the initial relevance score and reduce uncertainty in the image classification. Such an image classification system may be used as a vision system with, for example, a robot or robotic arm so that context that can be derived from the surrounding environment can be taken into account. In image or visual classification, enhancing the image caption text extracted and inferred from the image is also an option.

[0155] In another example, in sub-steps 44 to 50, the relevance evaluation module 6 may optimize the initial relevance scores only for those input records whose confidence scores are less than a confidence threshold. There is significant uncertainty in the initial relevance scores determined for those input records, and it is necessary to use contextual attributes and relationship data to further optimize those initial relevance scores. Therefore, before using the relationship data to optimize the initial relevance scores in sub-steps 44 to 50, the relevance evaluation module 6 may compare the confidence score determined for each input record with the confidence threshold in sub-step 43.

[0156] Once the initial relevance score has been optimized, the process may be iteratively repeated in step 50 until the confidence score for the input record is greater than or equal to a threshold confidence level.

[0157] Those input records with confidence scores greater than or equal to the confidence threshold may be considered sufficiently accurate, and the relevance scores of the input records may be output to the output module 8 .

[0158] In another example, the relevance evaluation module 6 may use the relationship data to optimize the initial relevance scores, substantially as described in steps 44 to 50, but may increase / decrease the initial relevance scores of nodes at the desired classification level based on the relative relevance of the contextual attributes to those nodes. For example, the relevance evaluation module 6 may increase / decrease the initial relevance scores in proportion to the relative relevance of the contextual attributes to the nodes, as indicated by the relationship data, e.g., in the form of historical relevance scores for the nodes.

[0159] In this regard, it should be understood that the relational data described in the above examples is binary, ie (screw, washer), but in other examples, higher dimensional relationships are possible.

[0160] In another example, the classification system 1 may be configured to optimize the initial relevance score for each classification level of the hierarchy based on the relationship data, substantially as described in sub-steps 38 to 50 .

[0161] Furthermore, in step 52 , the output module 8 may output a series of increasingly granular classifications corresponding to the respective nodes in successive classification levels of the hierarchy.

Claims

1. A computer-implemented classification method for classifying records by relevance to nodes of a hierarchical structure representing a classification scheme for a plurality of classification options, the classification method comprising: receiving an input record having a plurality of record content features, and a context attribute indicating a context of said receipt of said input record; A knowledge graph is generated by dividing a database including a history record of content features, context data, and relevance data into triples, the knowledge graph being populated with the triples, thereby generating an ontology knowledge base having semantic relationships linking the triples together; Determining relational data based on a database of historical records by applying an ontology engineering algorithm to the knowledge graph, the knowledge graph modeling the relevance of contextual attributes of the historical records to nodes of the hierarchical structure; retrieving the relationship data, the relationship data indicating one or more nodes of the hierarchical structure associated with the contextual attribute of the received input record; determining a relevance score for one or more of the nodes of the hierarchical structure to classify the input record, wherein the relevance score for each of the one or more nodes is determined based on a comparison of the plurality of record content features of the input record relative to the classification options represented by the nodes, and further wherein the relevance score for each of the one or more nodes is determined based on the retrieved relationship data; as well as A classification of the input record is output based on the determined relevance scores of one or more of the nodes of the hierarchy.

2. A classification method according to claim 1, wherein the relationship data is determined by applying a semantic reasoning algorithm to the knowledge graph to determine a set of contextual attributes that are logically related to the corresponding nodes of the hierarchical structure. 3 . The classification method according to claim 2 , wherein the relationship data is retrieved based on matching the context attributes of the input record with corresponding context attributes selected from the set of context attributes determined by the semantic reasoning algorithm.

4. A classification method according to any one of claims 1 to 3, wherein the knowledge graph is determined by dividing the database of historical records into one or more subject groups, object groups and / or predicate groups.

5. The classification method according to any one of claims 1 to 3, wherein determining the relevance score of each of the one or more nodes comprises: determining an initial relevance score based on the comparison of the plurality of record content features of the input record relative to the classification options represented by the nodes; and updating the initial relevance score based on the retrieved relationship data.

6. The classification method according to claim 5, wherein increasing or decreasing the initial relevance score based on the retrieved relationship data comprises: increasing the initial relevance score of a first node at a desired classification level of the hierarchy based on the relationship data indicating that the first node is associated with the context attribute of the received input record; as well as The initial relevance score of a second node different from the first node at the desired classification level is reduced.

7. The classification method of claim 5, wherein determining the relevance score of each of the one or more nodes comprises: determining a confidence score for the input record indicating uncertainty in the initial relevance score; and updating the initial relevance score according to the determined confidence score.

8. A classification method according to any one of claims 1-3, wherein the relevance score of one or more nodes in the nodes of the hierarchical structure is determined at least in part based on a combination of a local similarity score of the node and a local similarity score of one or more child nodes subordinate to the node, each local similarity score indicating the relevance of the input record to the corresponding node.

9. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by a computer, cause the computer to perform the classification method according to any preceding claim.

10. A classification system for classifying records by relevance to nodes of a hierarchical structure representing a classification scheme for a plurality of classification options, the classification system comprising: an input module configured to receive an input record having a plurality of record content characteristics and a context attribute indicating a context of said receipt of said input record; a contextualization module configured to generate a knowledge graph by dividing a database including historical records recording content features, context data, and relevance data into triples, the knowledge graph being populated with triples, thereby generating an ontology knowledge base having semantic relationships linking the triples together; and determining relational data based on a database of historical records by applying an ontology engineering algorithm to the knowledge graph, the knowledge graph modeling the relevance of contextual attributes of the historical records to nodes of the hierarchical structure; a relevance evaluation module configured to determine a relevance score for one or more of the nodes of the hierarchical structure to classify the input record; wherein the relevance score for each of the one or more nodes is determined based on a comparison of the plurality of record content features of the input record relative to the classification options represented by the nodes, and further wherein the relevance score for each of the one or more nodes is determined based on retrieved relationship data indicating one or more nodes of the hierarchical structure associated with the contextual attribute of the received input record; and An output module is configured to output a classification of the input record based on the determined relevance scores of one or more of the nodes of the hierarchical structure.

11. A classification system according to claim 10, wherein the contextualization module is configured to determine the relationship data by applying a semantic reasoning algorithm to the knowledge graph to determine a set of contextual attributes that are logically related to the corresponding nodes of the hierarchical structure.

12. The classification system of claim 11, wherein the contextualization module is configured to retrieve the relationship data based on matching the context attributes of the input record with corresponding context attributes selected from the set of context attributes determined by the semantic reasoning algorithm.

Images