Semantic matching system and method

Abstract

A semantic matching system and method is disclosed. A computer-based system and method for determining similarity between at least two heterogeneous unstructured data records and for optimizing processing performance. A plurality of professional data records are generated, and for each of the professional data records, a corresponding vector is created to represent the professional data record. Each of the vectors is sharded into a plurality of chunks. Thereafter, semantic matching of the chunks occurs in parallel to compare at least one professional data record and at least one other professional data record simultaneously and in substantially real-time. Thereafter, a value representing similarity between at least two of the professional data records is output.

Description

[0001] Cross-references to related applications

[0002] This patent application is based on and claims priority to U.S. Patent Application Serial No. 16 / 045,902, filed July 26, 2018, entitled Classifier System and Method, the entire contents of which are incorporated herein by reference as if their entire contents were expressly set forth herein. Technical Field

[0003] This patent application generally relates to the field of electronic information matching, and more specifically, to computer-implemented systems, methods, and computer program products for comparing at least two heterogeneous data records contained in different datasets to determine similarity. Background Technology

[0004] Semantic matching of data is known. For example, in professional fields, semantic matching is primarily based on keyword-based methods, in which data is searched for words that literally correspond to a given set of keywords. The more keywords found, the better the match is considered. In some cases, specific algorithms and / or advanced word vectors are used, which are considered here as one of the Natural Language Processing (“NLP”) similarity techniques. For example, one or more algorithms use word similarity purely at the string and sentence level, thereby largely ignoring, if not completely, the underlying semantic context, specific cultural characteristics, and multilingual differences.

[0005] Furthermore, the importance of certain occupational criteria that match a specific job (e.g., one or more specific skills, expertise, and / or experience) cannot be considered by simply comparing text strings and sentences. Therefore, differences related to specific occupations, regions, cultures, and / or languages ​​are also not taken into account, although these criteria may influence the relevance of a particular criterion. Consider two open job vacancies whose profiles differ only in two criteria: occupational titles (which may be very similar) and skills. A purely keyword-based approach is ineffective in determining which of the two candidates is a good fit for the position because specific information about the relevance of their respective skills to the target vacancy is required. Furthermore, manual classification or individual self-prioritization is tedious and impractical, especially for finding matches on large datasets that may include the collection of data records, such as occupational data records.

[0006] Furthermore, considering various technical aspects, semantic matching on very large datasets presents significant challenges. For example, vector operations are performed in high-dimensional vector spaces, requiring data preparation and efficient execution of numerous computations, issues that existing systems may not address optimally, particularly in terms of performance. Consequently, keyword-based methods and NLP similarity techniques exhibit significant weaknesses when comparing heterogeneous data records across culturally diverse and multilingual domains (such as occupational domains). This is due to information gaps reducing the accuracy of these matching results, and additionally, these methods may introduce probabilistic errors regarding contextual correctness.

[0007] Based on these and other considerations, this article presents publicly available content. Summary of the Invention

[0008] In one or more implementations, this application includes systems and methods for determining the similarity between at least two heterogeneous unstructured data records and for optimizing processing performance. Furthermore, at least one processor configured to generate a plurality of occupational data records is configured to execute code stored on a non-transitory processor-readable medium. The at least one processor can create a respective vector to represent each of the occupational data records. The at least one processor can fragment each of the vectors into multiple chunks and perform semantic matching on each chunk in parallel to compare at least one occupational data record with at least one other occupational data record simultaneously and substantially in real time. Furthermore, the at least one processor can output values ​​representing the similarity between at least two of the occupational data records.

[0009] In one or more implementations, each of the vectors has a magnitude and a direction.

[0010] In one or more embodiments, the at least one processor can create an n-dimensional non-orthogonal unit vector space.

[0011] In one or more implementations, an n-dimensional nonorthogonal unit vector space is created by computing the dot product between unit vectors corresponding to concepts from the ontology.

[0012] In one or more implementations, each vector resides in a high-dimensional nonorthogonal unit vector space.

[0013] In one or more embodiments, the at least one processor may apply correlation coefficients derived from information provided by the ontology.

[0014] In one or more implementations, the at least one processor may be a concept represented by a weighted vector.

[0015] In one or more embodiments, the at least one processor may store information associated with dot products that are greater than zero or at least equal to a predetermined threshold.

[0016] In one or more embodiments, the matching step includes performing an asymmetric comparison.

[0017] In one or more implementations, asymmetric comparisons are based on cosine similarity.

[0018] In one or more implementations, the outputs are sorted based on the degree of similarity.

[0019] These and other aspects, features, and advantages can be understood from the accompanying description and drawings of certain embodiments of the invention. Attached Figure Description

[0020] Figure 1 This is a simplified block diagram illustrating the matching process associated with an example implementation of this application;

[0021] Figure 2 This is a high-level diagram illustrating an example topology and configuration of a system according to an example implementation disclosed herein;

[0022] Figure 3A This is a block diagram illustrating multiple data storage structures, including three schemes, and available in one or more processes shown and described herein;

[0023] Figure 3B The illustration depicts a series of steps associated with the process of creating occupational data records according to an example implementation of this application;

[0024] Figure 3C It is a high-level flowchart illustrating routines for vector matching according to one or more implementations, including routines for parallel execution;

[0025] Figure 4 It is a block diagram illustrating an example of a unit vector representing an ontology, from which the dot product is determined; and

[0026] Figure 5 This is an illustration of an exemplary concept graph according to an example implementation, the concept graph comprising a directed graph of concept nodes, each concept node corresponding to a respective concept.

[0027] Figure 6 Describe an orthogonal basis in a 2D vector space, where nonorthogonal basis vectors are expressed. Detailed Implementation

[0028] As an overview and introduction, this disclosure details a system and method for comparing at least two heterogeneous occupational data records contained in different datasets up to a very large number of occupational data records, and generating numerical scores representing the degree of similarity between the information contained therein as a function of a predefined set of criteria. As used herein, the term "heterogeneity," particularly when used in conjunction with occupational data records, refers to data records whose criteria (e.g., concepts) may differ in number and type. Such differences may stem from, for example, different languages, different occupational descriptions, or different skills. Comparisons between occupational data records are performed by representing each occupational data record as a vector in a high-dimensional, non-orthogonal unit vector space and applying correlation coefficients derived from empirical human expertise provided by an ontology.

[0029] As used in this paper, the term ontology typically refers to a complex data structure containing a large number of occupational concepts and the logical relationships between them. Relationships can be hierarchical (e.g., parent-child, grandparent-grandchild), or they can express different types and degrees of similarity.

[0030] Subsequently, a ranked list of standardized scores, typically in the range of 0 to 1, is calculated for the degree of cosine similarity between the vectors of occupational data records, and is output. In one or more embodiments, a value of 1 indicates a perfect match. According to this application, examples of such occupational data records include open or vacant job postings, job searches for candidates, profiles of workers in a company, or other units of occupational information. As mentioned above, the data records compared during the matching process may include a set of criteria or data points, which are generally referred to herein as "concepts".

[0031] In operation, prior to the matching step, the concepts vector-represented in each occupational data record are weighted according to a customizable weighting system that depends on the specificity (OC) level of the occupational description. By sharding the indexed occupational data record vectors into data chunks, parallel processing enables the comparison of large numbers of occupational data records in virtually real-time. In one or more embodiments of this application, the module acquisition, which operates as a virtual engine, is generally referred to herein as a semantic matching engine.

[0032] As used herein, "matching" can refer to an operator that employs two graph-like structures and generates semantically corresponding mappings between the nodes of these graphs. Semantic relations can be computed to determine equivalence and to analyze the meaning (concepts, not labels) elements and structure of patterns. The ontology of this application derives correlation coefficients from which semantic matching is performed and includes large-granularity and diversity of semantic relations. For example, a class of relations commonly referred to herein as "same but different" can represent several different degrees of similarity. This allows for coverage of a broader spectrum than simply equivalence, overlap, or discontinuity. Furthermore, the direction of comparison and the perspective of comparison are also considered in the ontology of this application.

[0033] The semantic matching engine of this application is particularly effective in the field of occupational data analysis, encompassing a diverse range of areas including occupation, industry, language, country, and culture. This semantic matching engine represents a significant improvement over known similarity techniques used in natural language processing, which fail to bridge the information gap between knowledge of a specific occupational background and the information obtainable from the interrelationships between occupational data points. This application, including the semantic matching engine, operates to take these corresponding data into account and perform improved data analysis.

[0034] In one or more implementations, semantic context is determined as a function of relevance coefficients derived from the ontology. Based on the ontology, a set of concepts and categories in a subject domain or domain, and their respective relationships, can be represented, and access to information contained in the terminological relationships between terms can be determined. In addition to professional terms and sentences, this allows various terms to be represented in the ontology as concepts describing specific professions, skills, specializations, education, experience, etc.

[0035] Furthermore, in a semantic context, information related to professions, such as architects, is associated with other skills, such as process management, project time and cost planning, quality assurance, and general management. This paper recognizes that information associated with this semantic context, as well as the nuanced information related to various cultural backgrounds, cannot be retrieved simply through keyword-based matching or NLP similarity.

[0036] Ontology-derived relevance coefficients can be further grounded in the experience of subject matter experts, which can improve the accuracy of similarity comparisons. Unlike keyword-based matching or methods that only use NLP similarity techniques, ontology-based relevance coefficients generated according to the teachings of this paper do not suffer from the drawbacks associated with semantic misunderstandings (such as bias errors), thus constituting an added value to semantic matching.

[0037] Furthermore, and based on the relevant occupational and priority occupational criteria, the semantic matching engine of this application may include a complex weighting system, including weighting concepts belonging to occupational data records. Weights can be applied differently to different types of concepts depending on the occupational category assigned to the occupational data record. Accordingly, different occupations should be considered when only weights are given to specific skills, expertise, experience, education, and other criteria that are truly important to the specific occupation. As an example, the ability to stand upright in the same position for extended periods may be crucial for a dentist, but completely irrelevant for a cashier. Such regional variations in weighting distribution can be addressed by customizing a weighting table.

[0038] Occupational datasets, such as those related to job seekers and job providers, are typically complex, unstructured or semi-structured datasets and not specific standardized classification systems associated with corresponding categories. Given a standardized classification system, each set in the occupational dataset undergoes semantic interpretation and analysis to identify one or more defined standardized "concepts" that best match the given occupational dataset from the classification system. Furthermore, exemplary systems and methods are further configured to transform unclassified datasets into structured records of standardized occupational data, where occupational descriptions are expressed according to one or more prescribed classification systems. Additionally, occupational datasets can be augmented or enhanced by intelligently annotating the data with additional standardized occupational data inferred from the analysis.

[0039] Accordingly, it is understood that, through the specific technical solutions described herein for classification and standardization, embodiments of this disclosure translate unstructured and non-standardized sets of information that are unsuitable for analysis using existing data processing and analysis systems due to their inconsistent and unclassified nature, and that are structured and defined according to any of many defined classification systems. Therefore, embodiments of this disclosure are specifically configured to generate new and enhanced sets of occupational data that are more suitable for further data analysis processing, such as benchmarking, matching, or statistical analysis; the generated data from which deeper and more meaningful insights can be derived. Embodiments of this disclosure can similarly be used to analyze and (e.g., “classify”) occupational data that has been standardized according to a particular classification system and translate it into different classification systems to facilitate further processing.

[0040] This application further considers language differences, including generating different, language-specific labels and distinct concepts in the ontology as needed. Additionally, individual proficiency levels in skills, language, education, and experience can be defined based on each occupational data record. Furthermore, this application handles vector operations on high-dimensional vector spaces (e.g., approximately 10k dimensions). In one or more embodiments, known vectors are pre-computed to avoid unnecessary computation, and only relevant values ​​are stored in memory, with parallel processing applied during actual semantic matching.

[0041] More specifically, the matching process can be divided into two main stages: the data pre-computation stage (preparation) and the matching stage. During the preparation stage, occupational data record vectors can be computed in a high-dimensional vector space. By normalizing the occupational data record vectors as early as possible, unnecessary dot product calculations can be avoided. Furthermore, by not storing zero values ​​(which may be due to the concept of disjointness), unnecessary comparisons are not performed during matching. Additionally, inappropriate occupational data records are pre-filtered by using predefined filtering criteria. Pre-computation saves time during the actual matching process, but to ensure optimal performance during the actual comparison of occupational data records, the pre-computed data is chunked, and these chunks are then processed in parallel to compute the cosine similarity metric in essentially real-time.

[0042] As used herein, an Occupational Data Record (“ODR”) describes a closely related unit of occupational data and may include a list of occupational concepts and additional attributes such as the Occupational Data Record type. Occupational Data Records can be created manually or automatically from information describing new job vacancies, job application information from candidates, information related to resumes, worker profiles, or any other type of occupational information. Information can be mapped to concepts, which are then used to form Occupational Data Records.

[0043] Referring to the accompanying drawings, similar reference numerals denote similar elements. Figure 1 This is a simplified block diagram illustrating the matching process related to an example implementation of this application. (e.g.) Figure 1 As shown, two main phases are illustrated (preparation phase 102 and matching phase 104). These two phases can be executed asynchronously, so that the initial preparation phase occurs before the matching phase.

[0044] During preparation, an n-dimensional non-orthogonal unit vector space can be created, for example, by computing the dot product between all unit vectors corresponding to concepts from the ontology. All dot products that result in zero values ​​and / or values ​​below a certain custom threshold are preferably not stored. Individual factors in computing the dot products may be unknown; however, the dot products themselves are known, at least because they correspond to a given correlation coefficient from the ontology. Subsequently, irrelevant ODRs can be filtered out from the dataset, for example, based on a predefined list of filtering criteria. Examples of such filtering criteria could include location, industry, and contract type.

[0045] An ODR vector, which may include a linear combination of unit vectors in a nonorthogonal vector space, can be generated by assigning weights to each component (e.g., concept) in the ODR vector using a custom weight table. The weights on the table may be based on a classification of the ODR to an occupational category (OC). As used herein, an occupational category typically refers to a level of specificity used to describe an occupation. For example, whether or not a concept belongs to an occupational title ranges from 1 – very specific (e.g., a Python programmer) to 5 – very vague / broad (e.g., consultant). An OC categorizes occupations into levels of specificity, such as from 1 – very specific (e.g., "embedded C / C++ developer") to 5 – very vague (e.g., "consultant" or "project manager"). In one or more embodiments of this application, for each concept in the ODR, the vector component is multiplied by one or more assigned weights. Additionally, when the ODR is represented as a vector, it may include information about an individual's proficiency level, previously set by the user, such as skills, languages, education, and experience. The process of representing an ODR vector in a nonorthogonal vector space includes filtering out previous vectors of irrelevant ODRs and assigning weights to the concepts of the ODR vector, which are generally referred to herein as indexes.

[0046] Continue to refer to Figure 1 After preparation phase 102 is completed, matching 104 can be performed. The ODR vector, pre-computed using empirical correlation coefficients and implicitly derived from the ontology (e.g., the dot product of unit vectors), is an adapted form of cosine-like computation, often referred to in this paper as the soft cosine metric, performed while comparing two datasets relative to each other. The two datasets consist of a first dataset containing a single ODR and a second dataset containing n ODRs (where n = 1… many). This results in a list of cosine similarity metrics, typically ranging between 0 and 1. The resulting list is sorted, with the best metric (e.g., best match) shown at the top.

[0047] Now for reference Figure 2 A block diagram illustrating the topology and high-level architecture (system 200) of an exemplary embodiment according to this application is shown. The exemplary computer system in... Figure 2The diagram is shown as a block diagram, presenting a high-level view illustrating the configuration of a system for classifying occupational data according to an embodiment of the present invention. In this arrangement, system 200 includes an application / system server 205. Remote computing devices communicating with system server 205 are also shown, including a third-party computing system 204 and a user's personal computing device 206. The system server and one or more other remote computing devices may also communicate with one or more data storage devices, such as a database server 280 and a remote data source 208.

[0048] System server 205 is intended to represent various forms of digital computing devices and / or data processing apparatuses, such as servers, blade servers, mainframes and other suitable computers and / or networked or cloud-based computing systems capable of communicating with remote computing devices, data storage devices and computing networks, including receiving, sending and storing electronic information, and processing information, as further described herein. Database server 280 and third-party system 204 are also intended to represent similar computing devices to implement their respective functions.

[0049] As shown in the figure, user equipment 206 enables a user to interact with remote computing devices such as system server 205 and database server 280 via a network. User equipment 206 can be any device capable of communicating with the server and receiving input directly from the user, such as a personal computer, tablet computing device, personal digital assistant (PDA), mobile phone, or other type of computing device, as those skilled in the art will understand.

[0050] Database server 280 may contain and / or maintain various data items and elements utilized in various operations of system 200. The information stored by database server 280 may include, but is not limited to, information related to one or more ontology 281 (including multiple concept graphs), ODR repository 282, ODR index 283, filtering capabilities 284, and matching results 285. Database server 280 may also store or additionally maintain one or more sets of rules, including semantic interpretation rules and classification rules, which processor 210 at server 205 may apply to evaluate data input into the system and classify the data according to one or more given classification systems, as further described herein. It should also be noted that although database server 280 is described as being configured external to system server 205, in some embodiments, database server 280 and / or any data elements stored therein may be locally located on system server 205 or other remote computing devices in a manner known to those skilled in the art.

[0051] Server 205 may be equipped with various hardware and software components capable of operating system 200, including hardware processor 210, memory 220, and communication interface 250. Processor 210 is used to execute software instructions that can be loaded into and retrieved from memory 220. Depending on the specific deployment of the system, processor 210 may contain one or more processors, multiprocessor cores, or some other type of hardware processor.

[0052] Preferably, memory 220 and / or memory is accessible to processor 210, enabling processor 210 to receive and execute instructions stored on memory 220 and / or memory. Memory 220 may be, for example, random access memory (RAM) or any other suitable volatile or non-volatile computer-readable storage medium. Furthermore, memory 220 may be fixed or removable. Depending on the specific implementation, memory can take various forms. For example, memory may contain one or more components or devices such as hard disk drives, flash memory, rewritable optical discs, rewritable magnetic tapes, or combinations thereof. Memory can also be fixed, removable, or remote, such as a cloud-based data storage system.

[0053] One or more software modules are encoded in memory and / or memory 220. A software module may contain one or more software programs or application programs having a set of computer program code or instructions that are executed by processor 210. The software module may be tightly integrated with the operation and configuration of the physical hardware aspects of one or more embodiments herein.

[0054] Such computer program code or instructions for implementing the operational aspects of the systems and methods disclosed herein can be written in any combination of one or more programming languages. The program code may execute entirely on server 205, partially on server 205, as a standalone software package, partially on system server 205, partially on a remote computer / device (e.g., database server 280), or entirely on a remote computing device. In the latter case, the remote device may be connected to system server 205 via any type of network, including a local area network (LAN) or wide area network (WAN), or may be connected to an external computing system (e.g., via the Internet through an Internet service provider).

[0055] It can also be stated that the program code of the software modules and one or more non-transitory computer-readable storage devices (such as memory 220 and / or storage) form a computer program product, which, as known to those skilled in the art, can be made and / or distributed in accordance with this disclosure. It should be understood that, in some illustrative embodiments, one or more software modules may be downloaded to the storage via a network from another device or system (e.g., a remote data storage device) via communication interface 250 for use within system 200. Furthermore, it should be noted that other information and / or data related to the operation of this system and method may also be stored in the storage.

[0056] The communication interface 250 is also operatively connected to the processor 210 and can be any interface capable of communicating between the server 205 and external devices, machines, and / or components. Preferably, the communication interface 250 includes, but is not limited to, a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter / receiver (e.g., Bluetooth, cellular, NFC), a satellite communication transmitter / receiver, an infrared port, a USB connection, and / or any other such interface for connecting the system server 105 to other computing devices and / or communication networks (such as private networks and the Internet). Such connection can include wired or wireless connections (e.g., using the IEEE 802.11 standard), although it should be understood that the communication interface 250 can in practice be any interface capable of communicating with the server 205.

[0057] Although system 200 is described with reference to a single device (such as server 205), it should be understood that the system is configured to interact with any number of local and remote computing devices, providing data to and receiving information from such devices. It should be understood that... Figure 2 Any remote computing devices depicted herein may communicate directly with each other or with server 205, communicate indirectly with each other or with server 205, and / or communicate and coordinate with each other or system server 205 through a computer network such as the Internet, LAN, or WAN.

[0058] Now go to Figure 3AThe diagram illustrates data table 302, which includes three schemes 302A, 302B, and 30C, and can be configured as a data structure (e.g., a table in a database) and used in one or more processes shown and described herein. As described herein, two main processes are used: one for creating the ODR and one for matching. Scheme 302A may include records associated with a non-orthogonal unit vector space, scheme 302B may include records associated with an ODR index, and scheme 302C may include records associated with an ODR filter. Two additional data structures are further supported: an ODR repository and a data structure containing matching results, which will be described in more detail below.

[0059] Data structure 302A (Non-Orthogonal Unit Vector Basis) can include the dot product between any two concepts in a graph (asymmetric). As used herein, the term asymmetric (or asymmetric) refers to matching scores that differ depending on a particular comparison direction, and concepts with different correlation coefficients between them, such as depending on the direction of the relation. Two variations can be supported, including when searching for vacancies and when searching for people. The vector space can be defined by stating all the correlations between every possible combination of concepts found in the ontology. In turn, correlations can be defined by the dot product between any two concepts in the knowledge graph, as shown and described in more detail below. Once the vector space is established, the ODR can be defined based on the unit vector space as a linear combination of non-orthogonal unit vectors. Figure 4 An example is shown in the figure, representing a unit vector of the ontology, from which the dot product is determined.

[0060] It should be understood that the dot product is not necessarily commutative or interchangeable, at least because there are at least two different viewpoints. For example, consider two professionals: a programmer and a Python programmer. The correlation (e.g., similarity) between them can be calculated, for example, depending on the direction of the comparison:

[0061] PYTHON programmer -> programmer = 0.8

[0062] Programmer -> Python Programmer = 0.6

[0063] Continuing this example, while a Python programmer may be very similar to an average programmer, an average programmer may not be as proficient in Python and therefore less similar to a Python programmer. Depending on the direction of the comparison, this example of identifying industry characteristics demonstrates why comparing two specific ODRs can lead to different match scores. For example, match scores may differ for companies offering vacancies to candidates, as addressed by variations of Scenario 302A for candidates with company vacancies, even if the variations are not limited to those listed under Scenario 302A.

[0064] Although variations exist as described above, it is conceivable to perform specific matching in one direction. Typically, the direction is from a single ODR (in one dataset) to a large number of other ODRs (in another dataset). However, it should be understood that this application also supports many-to-many comparisons in addition to typical one-to-many comparisons.

[0065] Regarding the creation of a nonorthogonal unit vector space, the data structure containing unit vector space information (e.g., a table) contains information implicit from the concepts in the ontology, but not necessarily the ODR itself. In this respect, the structure containing unit vector space information is therefore independent of the ODR index and ODR filter. In one or more embodiments, the nonorthogonal unit vector space data structure is recalculated on a periodic and / or rule-based basis, such as when creating new concepts in the ontology. In one or more embodiments, for example, although many dot products between concepts are equal to 0, the dot products that are equal to 0 between concepts are not stored in data structure 302A.

[0066] Now for reference Figure 3B This describes the process of creating an ODR, which can be considered as the process and prerequisites for matching. When creating and indexing the ODR, data structures 302B and 302C are populated. Figure 3A ).exist Figure 3B The simplified block diagram illustrates steps 330 associated with the process of creating an ODR.

[0067] In various embodiments of this application, the ODR can be created by receiving input from a user (e.g., manual input) (step 332), or it can be created substantially automatically (e.g., machine input). The fields to be specified can be associated with occupational topics, such as occupation, skills, contract type, etc. Subsequently, an indexing process for the ODR occurs (step 334), where predefined fields can be matched against the ontology. For any text that does not yet exist, new concepts are automatically created in the latter. Additionally, for filters used in this ODR, filter entries are created accordingly in data scheme 302C. Continue to refer to Figure 3B The example procedure for index ODR 334 shown here involves matching the ODR field with the ontology. Subsequently, a new concept is created in the ontology for the text that does not yet exist. Furthermore, a new ODR filter entry is created in data scheme 302C.

[0068] During indexing, each ODR in the ODR repository can be represented as a vector, whose components are unit vectors in a vector space defined by data structure 302A. Furthermore, each vector component in the ODR can be multiplied by weights based on a weighting system that is based on the occupational category associated with the ODR, such as those described in greater detail below. For example, whether an ODR's occupational description concept belongs to OC 1 for "mathematics teacher of advanced mathematics" or to OC 5 for "project manager" plays a crucial role when weighting the concepts in the ODR. In other words, depending on the occupational category, weights can be distributed differently within the ODR for a given concept, for example, considering the importance of a skill, specialty, user experience, etc. Furthermore, the allocation of weights can consider the importance of multiple skills to a specific occupation. Table 1 below includes exemplary weights that can be used during the indexing process of the ODR. For example, Table 1 provides occupational category values, occupational weights, functional weights, specialization weights, skill weights, and language weights, along with corresponding textual descriptions.

[0069]

[0070] Table 1

[0071] When ODRs are indexed, data structures 302B and 302C are essentially automatically populated. In one or more implementations, the data structures can be populated using batches of SQL database queries operating on 1000 ODRs. Two queries are executed, such as one for the index structure (302B) and one for the filter structure (302C). In one or more implementations, the ODR index table depends on the corresponding data structure (data structure 302A) representing the vector space, and the ODR index is also refreshed on a periodic and / or rule-based basis. This also means that if one or more data changes occur in data structure 302A, such as changes due to new concepts or changes to one or more concepts, re-indexing is performed with the latest ontology-based state. During one or more matching processes, ODRs can be dynamically created to be compared and indexed with one or more (potentially all) other ODRs. In one or more implementations, vectors are not stored in the ODR index table or in the ODR filters. Instead, data is only used temporarily during matching.

[0072] Referring now to ODR filters, data structure 302C serves as a standard store for filters existing in the ODR repository. Example filter types can include location, industry, contract type (e.g., full-time, part-time, contract, etc.), tags, exclusion of specific concepts, keywords or non-keywords, ODR type, and publication date. When an ODR is created, the filters applied to it can be stored largely automatically in the ODR filter table (e.g., data structure 302C).

[0073] Now go to Figure 3C A high-level flowchart is shown, illustrating routines 350 for vector matching according to one or more implementations, including parallel execution with respect to system 100. In addition... Figure 3C Beyond the high-level process illustrated, this paper also describes specific details associated with one or more algorithms related to vector matching, including cosine similarity, matching / search, normalization, and cosine similarity constant shift.

[0074] exist Figure 3CThe process begins by creating a new dataset, or using a pre-existing dataset containing (typically a large number of) all the ODRs used, for which individual ODRs are compared (step 352). Each ODR in the corresponding dataset is represented as a vector in a non-orthogonal unit vector space, where the indexed ODR vector is stored in the ODR index, and the filter is in the ODR filter (step 354). For example, a second dataset can be started, where only one ODR to be compared with all other ODRs is dynamically created from an existing ODR or by creating a new temporary ODR at query time. Subsequently, in step 356, individual ODR vectors are created (indexed) simultaneously with the ODR vectors without storing them in the ODR index. Similarly, custom vector components for filtering individual vectors are created without storing them in the ODR filter. Subsequently, when calculating the cosine similarity between individual ODR vectors and every other ODR vector, the ODR index vector data is partitioned into slices to enable parallel processing (step 358). An example application formula for cosine similarity calculation is defined by Equation 5 below. After calculating the cosine metric, the result is inserted into the matching results table (step 360). A normalization function based on the magnitude of all ODR vectors is then applied (step 362), and the result is output to the sorted score results list (step 364). The values ​​in the score results list ideally fall within the range of 0 and 1. As used herein, the term "magnitude" can generally refer to the weight applied to a specific occupational concept in an occupational data record based on a given occupational category. Additionally, the term "occupational concept" as used herein is generally derived from the ontology and represents a data point representing occupational information. Occupational concepts can include different attributes (e.g., labels for different languages) and, in the case of a concept representing an occupational description, can include associated occupational categories. Thus, an occupational concept can represent specific skills, soft skills, specialization, occupational titles, experience, industry, language, salary range, working conditions, or any other occupational information. Concepts and their relationships are stored in the ontology. In one or more implementations, concepts are formatted to include a unique ID.

[0075] As mentioned above, an ontology represents a conceptual hierarchy within one or more occupational classification systems, and importantly, the relationships between concepts and concept maps are exemplary logical representations of the relationships between concepts in an ontology. Figure 4 An example of a unit vector representing an ontology is shown, from which a dot product is determined. Knowledge graph 402 illustrates the relationships between concepts (e.g., nodes) C0-C5. Relevance factor 404 identifies the corresponding dot product between two non-orthogonal unit vectors of a concept. For example, in... Figure 4 In the concepts C0 and C1, the dot product is 0.8.

[0076] Figure 5 This is a simplified illustration of an exemplary concept graph containing concept nodes, each concept node corresponding to a specific concept. More specifically, Figure 5 This demonstrates the correlation values ​​of common or similar concepts (502A, 502B, 502C, and 502D) between two ODRs just before cosine similarity is calculated. Returning to the earlier example of a Python programmer, 502B represents a Python programmer compared to a general programmer (502A), and 502C represents the skill associated with writing SQL statements compared to designing relational databases (502D). Before computation, these common concepts and their respective correlation coefficients can be identified as functions of a pre-calculated correlation coefficient table. At this point, the relationships between concepts can already be represented, thus eliminating the need to traverse the knowledge graph during the matching process. The correlation coefficients of the identified common concepts derived from 302A can be temporarily stored at execution time (e.g., in RAM) using a hash graph (or other suitable function) for computation.

[0077] The output of the semantic matching process is a list of cosine similarity scores, first sorted by the highest results. These scores include a custom threshold, listing only results on the client system that exceed this threshold. The resulting cosine metric is returned as a percentage value derived from the normalized results, where 100% represents a perfect match. As previously mentioned, various parameters can be adjusted to tailor the matching process to the conditions of the local job market, such as filtering criteria and by defining different weights for various concept types, including jobs.

[0078] This application provides the output of a semantic matching process, which may include a list of cosine similarity scores. The list can be ordered in various suitable ways, such as first by the highest results. These results include a custom threshold, providing only the list results on the client system that are above the custom threshold. The resulting cosine metric is returned as a percentage value obtained from the normalized results, where a value of 100% represents a perfect match. As described herein, multiple parameters can be adjusted to adapt the matching process to the conditions of the local job market, such as filtering criteria and by defining different weights for various concept types, such as occupations.

[0079] This application will now be described in more detail using example implementations including vector matching algorithms, matching and search algorithms, and algorithms related to cosine similarity constant shifting. Reference objects, such as occupational concepts, are used to describe other features of this application. Some objects may be quite complex, such as the comprehensive acquisition of complex vectors and / or related concepts.

[0080] Referring to the example supplier matching algorithm, in one or more implementations, matching is performed by representing objects as vectors and measuring the proximity between vectors using a cosine similarity metric. This document typically refers to it as "cosine similarity" or "soft cosine metric." For illustrative purposes, the supplier matching algorithm is described using common linear algebra, and accordingly, one skilled in the art will recognize that the supplier matching algorithm can be general. When objects are represented as vectors, the matching vector is equivalent to the matching object. When such objects can be represented as vectors, virtually any type of entity can be matched using algorithms such as those shown and described herein.

[0081] Now, referring to an example supplier matching algorithm, given two vectors... and Represent an entity that has size and orientation and has a dot product:

[0082] (Equation 1)

[0083] in It is a vector Size or length, and yes and The angle between directions. Examples of dot product properties, such as:

[0084] and Vectors are said to be orthogonal, independent, or perpendicular. The magnitude of a vector can be calculated using its dot product:

[0085] (Equation 2)

[0086] Vector magnitude, direction and θ ab It can be visualized in three dimensions, but the same properties can be extended to higher dimensions.

[0087] From the two equations above, and The cosine similarity between them is:

[0088] (Equation 3)

[0089] In the above equations, express A unit (length) vector in the direction. It is obvious to the average person that the dot product is the fundamental operation used to calculate cosine similarity. In the case of non-unit vectors, a triple dot product can be used, while in the case of unit vectors, the cosine similarity can be the dot product itself. Considering that the dot product can be a rather expensive operation, significant practical savings can be achieved by normalizing all vectors as quickly as possible; this also simplifies the equations.

[0090] Given a complete vector basis covering the entire vector space of interest, consisting of K unit vectors that are not necessarily orthogonal, , , ..., Composition, then vector and It can be expressed (not necessarily uniquely) as:

[0091]

[0092]

[0093] Accordingly, the practical formula for calculating the dot product is:

[0094] (Equation 4)

[0095] in , It is a real number. , It might not be directly known, but rather their dot product. It is known that they are real numbers in the range [-1.0, +1.0].

[0096] For a unit vector, combining equations 3 and 4:

[0097] (Equation 5)

[0098] According to the example implementation of this application, a reference vector matching algorithm is used, given two sets of unit vectors:

[0099] and Use (Equation 5) to calculate all triples. , m =1... M , n =1... N A list sorted in descending order Preferably, the best result will be found at the beginning of the list, where the best possible result is... If it exists, it will indicate a perfect match. In many practical cases, set A consists of a single vector, while set B is very large.

[0100] For the average technician, it is obvious that the more familiar and intuitive vector basis is that which chooses the minimum number of orthogonal unit vectors to cover a vector space of a given dimension: that is, two-dimensional vectors corresponding to the x and y axes and three-dimensional vectors corresponding to the x, y, and z axes. This is commonly referred to as an orthogonal basis. When two vectors are expressed in an orthogonal basis, given (1) and (2) above, equation (4) simplifies to the more familiar dot product formula in an orthogonal basis:

[0101] (Equation 4B)

[0102] Even in such a linear vector space that is very high-dimensional, it still possesses an orthogonal basis, in which non-orthogonal basis vectors... It will be expressed. Figure 6 This is described in 2D vector space.

[0103] exist Figure 6 The vector shown is:

[0104]

[0105]

[0106]

[0107]

[0108] in and yes Orthogonal basis: unit vectors along the positive x-axis and y-axis.

[0109] Continuing with this example, vectors and This can be achieved by solving linear systems based on and To express:

[0110]

[0111]

[0112] Right now,

[0113]

[0114]

[0115] In the expression above, we only know the two equations plus... The value, Equation (4) can be used to calculate. Equation 4B can be used to calculate. However, real values ​​in the range of [-1.0, +1.0] can actually be assigned, as described below in conjunction with matching / search algorithms.

[0116]

[0117]

[0118] This can be verified using Equation 4B, as shown below:

[0119]

[0120] By representing each individual object as a vector, the vector matching algorithm described above can be applied to matching and searching for objects. In Equation 5 above, and This represents two distinct objects. Two object vectors can be defined as a linear combination of K non-orthogonal unit basis vectors corresponding to K concepts in a concept graph. (This article references...) Figure 5 The examples are shown and described. In Equation 5, and This represents two different concepts in a concept map. express Unit vector of the object in the direction The i Each component, and similarly, express Unit vector of the object in the direction The j Each component. The following describes the separated parts: basis vectors and object vectors.

[0121] First, referring to the basis vectors, the non-orthogonal unit vector basis is a set of K unit vectors, with one corresponding to each concept in the concept map: , , ..., .although It may be unknown, but all All of these are known. These dot products are almost exclusively computed from the relationships defined in the concept graph. For example, direct parent-child relationships have higher values, close to the maximum of 1.0, while more distant grandparent-grandchild relationships have lower values. These empirically determined rules provide good quality results and do not cover the commutative property of the dot product, i.e., .

[0122] In addition to formulas that rely on the relationships illustrated in the concept map, Other values ​​can also be added manually. Although there may be specific calculation methods, The collection of all explicitly computed values, and the implicit value of all missing combinations being zero, provides the basis for computational equation (5) and thus performs the vector matching algorithm. In one or more embodiments, Most values ​​are implicitly zero. Correspondingly, non-zero values ​​are stored as triples. .

[0123] The following example (Example 2) connects two concepts considering the application of this application: programmer With Python programmers In which PYTHON programmer => programmer = 0.8 Programmer => Python programmer = 0.6 Preferably, the triples (358, 121, 0.8) and (121, 358, 0.6) are stored.

[0124] In one or more implementations, the object vector set comprises a set of N vectors, one for each object in the system: The information about the corresponding object matches the corresponding concept in the concept map: , where the subscripts n and k refer to a specific object and concept, respectively. In one or more implementations, You can multiply by the weight. The weights are then added to the object vector. Weights can be a combination of rules, including considerations of the relative importance of given information, and weights specified by the user. Accordingly, the object vector can be defined as follows:

[0125]

[0126] Moreover, as described in this article, object vectors can be normalized, stored, and further processed early as object unit vectors:

[0127]

[0128] In the embodiments of this application, the vast majority w nk It equals zero; therefore, in one or more implementations, after normalization, only the non-zero components are considered as triples. Components are stored in data structures (e.g., tables in a database). Components that are not stored have an implicit value of zero.

[0129] The following example (Example 3) relates to the application of this application, which is associated with the work of a PYTHON programmer in Paris, France. The weights can include a weight with an application value of 3.0 and a weight of 2.0 for French-speaking countries. However, programmers could also be considered. He might not know that the PYTHON programmer has a weight of 0.5.

[0130] The resulting equation could be:

[0131]

[0132] Can be saved Components, such as rows in a data table, , and As can be understood by ordinary technicians, based on equations 2 and 4, Depending on the relevance between Python programmers and programmers, as described in the previous example, that is, and As the dot product changes, the rows (or other data storage) are recalculated. In one or more embodiments, a corresponding database table can be configured to store directly input information about the object, rather than necessarily information implied through a concept graph, which may be stored in a separate data source (data table). Moreover, in one or more embodiments, the vector matching algorithm according to this application can be applied to combine both information sources when calculating the dot product, such as using equation (5).

[0133] Now we turn to the discussion of cosine similarity constant offset, where low-cost filtering is added to the vector matching algorithm shown and described in this paper. More specifically, given orthogonal to... and Unit vectors of both For the real number D, the following equation (6) is applicable:

[0134] (Equation 6)

[0135] Referring to the vector matching algorithm described earlier, small sets , m = 1… M and very large sets , n = 1… N Accordingly, two new sets can be created. and Where D is a real number > 1, and It is a filtering function, if If a certain condition is met, the filtering function evaluates to 1; otherwise, it evaluates to -1. Equation 6 can then be applied, and the vector matching algorithm for A' and B' can generate triples. , m = 1… M , n = 1… N Sort the list. Discard all negative results and subtract D from the remaining results to produce a smaller list with the same sorting. This does not alter the cosine similarity. In fact, obtaining set A' from A does not cause performance issues because the set is small, consisting of a single vector. When set B can be very large, this technique results in providing efficient filtering and matching operations. For example, by performing a union query with multiple tables during the matching / search process to dynamically add a single row for each distinct object unit vector (denoted by n). This is used to obtain set B' from B. Furthermore, F is a completely independent basis vector corresponding to all i=1...K concepts in the concept graph. ,Right now The constant.

[0136] While cosine similarity provides good and fair matching results, there are situations where the real-world use of the features described in this application is not well represented. This can happen to users providing important information such as skills, language, and education. For example, during a search for a specific job position, the returned search results increase, although the calculated score for each job will be lower than the score for a user targeting only one job. However, this can be disadvantageous for job providers looking for potential candidates to fill vacant positions. Candidates with richer datasets (who are arguably better suited for a particular position) may score lower than candidates with only a few concepts but more closely matching their respective job descriptions. In turn, this can have unintended effects, penalizing those who are overly descriptive in their profiles, even though they might be instructed to be as descriptive as possible.

[0137] Accordingly, alongside the cosine similarity matching process, one or more embodiments of this application may apply the following normalization factors to correct for shortcomings, such as those described above:

[0138]

[0139] in It is the search vector, and This represents the vector provided for each pair of search results before normalization. The size of the unnormalized vector can be used, for example, these vectors can be cached during use.

[0140] Continuing the normalization process, the cosine similarity equation (e.g., equation 3) is used... and Replace each and Fractional adjustment factors can be applied, as in the following example:

[0141]

[0142] If we use search vectors Perform a search and display the matching vectors. The score can be expressed by the following equation:

[0143]

[0144]

[0145] in, They can be stored, for example, in a data table, from which they are indexed when provided. Furthermore, when providing vectors... The search vector being searched and used for matching , S The fraction can be changed as follows:

[0146]

[0147]

[0148] in, The calculation can be performed before the matching query is executed and the result is used in the query.

[0149] The exemplary systems and methods for matching described herein can be applied to any number of real-world scenarios. Use cases can be extended beyond comparisons between open vacancies and job searches, and can include comparisons between open vacancies, comparisons between candidates (e.g., finding successors within a company, or evaluating continuing education programs), comparisons between resumes, comparisons between worker profiles, benchmarking, gap analysis, and more.

[0150] In one example use case, a government labor official needed a matching system to mine data and match the profiles of unemployed workers in her country with each other. The labor official further needed to match them with open vacancies to identify gaps between the skills they possessed and the requirements of company job openings. The system and methodology described in this paper enabled her to match people to jobs and identify potential education and training needs.

[0151] In another illustrative use case, the matching systems and methods shown and described herein can be used by companies that match job requirements in their own advertisements with job requirements in the advertisements of their competitors, as they want to compare their skill sets with benchmarks for the occupations in their respective industries. For example, the company might attempt to identify the five most important skills for each occupation and compare them with the five most desired skills from other companies in the corresponding sector. The teachings of this document provide the tools to support such comparative analysis.

[0152] In another illustrative use case, job seekers use the teachings in this article to match their skills with those of other job seekers in the same occupation. This supports conducting a comparative study to identify where job seekers should enhance their skill sets and experience, and then improve their profiles and employment opportunities.

[0153] In another illustrative use case, a university applied the teachings of this paper to align the skills and subjects offered in its curriculum with the job requirements of current labor-intensive industries. This enabled comparative analysis, including identifying future curriculum changes to better prepare students for industry needs.

[0154] In yet another illustrative use case, a recruiter working for a company whose corporate strategy is moving towards greater diversity and inclusion uses the teachings in this article to compare the specific requirements of her company's job openings with many potential candidates globally. This expands upon previously available reach and functionality.

[0155] On a larger scale, exemplary systems and methods for matching can be performed on different types of occupational data, say, based on two sets of occupational data defined by two different systems, in order to transform the different system data sets into a common paradigm (e.g., a common set of concepts), thereby enabling the performance of additional benchmarking, comparative analysis, and other such operations across the matching system.

[0156] In this regard, it should be noted that although the foregoing description is largely directed to exemplary systems and methods for classifying occupational data and exemplary real-world use cases, the systems and methods disclosed herein can be similarly deployed and / or implemented in scenarios, situations, and settings that extend far beyond the reference scenario.

[0157] It should be understood that similar figures in the accompanying drawings represent similar elements in several drawings, and not all embodiments or arrangements require reference to all components and / or steps described and shown in the drawings. It should also be understood that embodiments and / or arrangements of the systems and methods disclosed herein can be incorporated as software algorithms, applications, programs, modules, or code residing on hardware, firmware, and / or computer-usable media (including software modules and browser plugins), which can be executed in a processor of a computer system or computing device to configure the processor and / or other elements to perform the functions and / or operations described below. It should be understood that, according to at least one embodiment, when performing the methods of the invention, one or more computer programs or applications do not need to reside on a single computer or processor, but can be distributed in a modular manner across a large number of different computers or processors to implement various aspects of the systems and methods disclosed herein.

[0158] Therefore, illustrative embodiments and arrangements of this system and method provide computer-implemented methods, computer systems, and computer program products for assessing the level of risk in prescription behavior records. The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the system, method, and computer program product according to various embodiments and arrangements. In this regard, each block in a flowchart or block diagram may represent a module, fragment, or code section containing one or more executable instructions for implementing multiple specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the blocks may appear in a different order than those marked in the figures. For example, depending on the functions involved, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It will also be noted that each block illustrated in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware or a combination of dedicated hardware and computer instructions that performs the specified functions or actions.

[0159] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “described” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should further be understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0160] It should be noted that ordinal terms, such as "first," "second," and "third," are used. In claims, modifying a claim element itself does not imply any priority, precedence, or order of one claim element relative to another, or the chronological order of the method actions, but is merely used as a label to distinguish one claim element with a specific name from another element with the same name (but for the purpose of using ordinal terms) to differentiate claim elements.

[0161] Furthermore, the wording and terminology used herein are for descriptive purposes and should not be construed as limiting the use of "including," "contains," or "has," "includes," "involves," and variations thereof to include the items listed thereafter and their equivalents as well as additional items.

[0162] The subject matter described above is provided for illustrative purposes only and should not be construed as limiting. Various modifications and changes may be made to the subject matter described herein without following the exemplary embodiments and applications illustrated and described, and without departing from the true spirit and scope of the invention as set forth in the following claims.

Claims

1. A computer-based method for determining the similarity between at least two heterogeneous unstructured data records and for optimizing processing performance, the method comprising: A plurality of occupational data records are generated by at least one processor, the at least one processor being configured to execute code stored on a non-transitory processor-readable medium, wherein the plurality of occupational data records are heterogeneous unstructured data records; The at least one processor generates an n-dimensional non-orthogonal unit vector space, wherein the ontology includes multiple unit vectors corresponding to multiple occupational concepts from the ontology, the ontology is generated by measuring the associations between combinations of occupational concepts, and the n-dimensional non-orthogonal unit vector space is generated by calculating the dot product between unit vectors corresponding to concepts from the ontology. The at least one processor creates a corresponding vector representing each of the occupational data records in the n-dimensional non-orthogonal unit vector space by calculating the dot product between unit vectors corresponding to concepts from the ontology; Each of the vectors is divided into multiple chunks by the at least one processor; The at least one processor performs semantic matching on each of the blocks in parallel to compare at least one occupational data record with at least one other occupational data record; as well as The at least one processor outputs a value representing the similarity between at least two of the occupational data records.

2. The method according to claim 1, wherein, Each of the corresponding vectors has a magnitude and a direction.

3. The method of claim 1, further comprising applying correlation coefficients derived from information provided by the ontology.

4. The method of claim 1, further comprising the concept of a weighted vector representation.

5. The method of claim 1, further comprising storing information associated with dot products greater than zero or at least equal to a predetermined threshold.

6. The method of claim 1, wherein, The matching step includes performing asymmetric comparisons.

7. The method of claim 6, wherein, The asymmetric comparison is based on cosine similarity.

8. The method of claim 1, wherein, The outputs are sorted based on the degree of similarity.

9. A computer-based system for determining the similarity between at least two heterogeneous unstructured data records and for optimizing processing performance, the system comprising: At least one processor, configured to access a non-transitory processor-readable medium, and when executing instructions stored on the non-transitory processor-readable medium, the at least one processor is further configured to: Generate multiple occupational data records, wherein the multiple occupational data records are heterogeneous unstructured data records; Generate an n-dimensional non-orthogonal unit vector space, wherein the ontology includes multiple unit vectors corresponding to multiple occupational concepts from the ontology, the ontology is generated by measuring the associations between combinations of occupational concepts, and the n-dimensional non-orthogonal unit vector space is generated by calculating the dot product between unit vectors corresponding to concepts from the ontology; By calculating the dot product between unit vectors corresponding to concepts from the ontology, a corresponding vector representing the occupational data record is created in the n-dimensional non-orthogonal unit vector space for each of the occupational data records; Each of the vectors is divided into multiple blocks; Semantic matching is performed in parallel on each of the blocks to compare at least one occupational data record with at least one other occupational data record; and The output represents a value indicating the similarity between at least two of the occupational data records.

10. The system of claim 9, wherein, Each of the corresponding vectors has a magnitude and a direction.

11. The system of claim 9, wherein, The at least one processor is further configured to: The application derives the correlation coefficients from the information provided by the ontology.

12. The system of claim 9, wherein, The at least one processor is further configured to: The concept of weighted vector representation.

13. The system of claim 9, wherein, The at least one processor is further configured to: Store information associated with dot products that are greater than zero or at least equal to a predefined threshold.

14. The system of claim 9, wherein, The matching step includes performing asymmetric comparisons.

Images