Method and system for generating reference data structure for anonymization of textual data

Abstract

A method and system for automatically generating a reference data structure for a K-anonymity model using machine learning. A vector space is generated from reference text data, wherein the vector space is defined by numerical vectors representing semantic meanings of the reference text words. Input text words are converted to numerical vectors using the vector space. Word clusters are formed according to semantic similarities between the input text words, wherein the semantic similarity between pairs of input text words is represented by a metric value determined from pairs of numerical vectors. The word clusters define nodes of the reference data structure. A text label is applied to each node of the reference data structure, wherein the text label represents a semantic meaning shared by elements of the word cluster.

Description

[0001] Cross-citation of related applications

[0002] This application claims priority to U.S. Patent Application Serial No. 16 / 913,711, filed June 26, 2020, entitled “Method and System for Generating Reference Data Structures for Anonymizing Text Data,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to methods and systems for generating reference data structures, and more particularly to methods and systems for generating reference data structures for text data anonymization. Background Technology

[0004] With the advent of technology, data generation across all disciplines is growing at an unprecedented rate. One of the major challenges arising from this rapid increase in data volume is providing data privacy. Data anonymization is one such technology that can be used to protect privacy and ensure the prevention of the leakage of personal identity or sensitive information.

[0005] One approach to achieving data anonymization is the K-anonymity model, which, broadly speaking, is a generalization technique used to describe anonymized datasets at a certain level. The K-anonymity model assumes that the records in a given dataset can be in a structured format, arranged in a table, where rows represent individual records and columns contain the attributes of each record. The anonymization process involves replacing attribute values ​​that serve as specific identifiers with less specific, generalized terms. Therefore, the primary goal of the K-anonymity model is to transform the dataset such that the association between a record and its corresponding entity cannot be determined with a probability greater than 1 / K (where K is the number of records). To achieve this, any K-anonymity model requires dividing the records in the dataset into clusters, each cluster containing at least K records, so that each record is indistinguishable from at least (K–1) other records. To improve data quality, it is also desirable that the records within a given cluster are as similar as possible to each other. This helps reduce data distortion when records within a cluster are modified to have the same generalized terms.

[0006] Some attributes, such as those containing numerical values, can easily generalize to numerical ranges. Other attributes, such as those with text-based values, are more difficult to generalize. Typically, when k-anonymization is applied to a dataset containing text-based attributes, the text values ​​of some or all attributes will be generalized and replaced with similar values—e.g., semantically consistent but less specific.

[0007] However, overgeneralization can exacerbate data distortion. One technique that can be used to enhance K-anonymity implementation is to generate a reference data structure from the data, such as a hierarchical classification tree. This reference data structure can cluster similar entities together in such a way that entities in one cluster are more similar to each other than entities in other clusters. More importantly, each cluster in the reference data structure can be identified by a generalized term, which can also serve as a meaningful name for the entire cluster and can be used to replace any member of the same cluster. The generalized terms of clusters at the same reference data structure level can then be further generalized to higher levels, thereby helping to reduce data distortion.

[0008] Currently, the reference data structure for K-anonymity models is typically generated manually, a time-consuming task. Each new domain requires a different reference data structure, necessitating extensive domain expertise to generate the required generalized terms. Pre-fabricated reference data structures for different domains may need to be provided along with the dataset, increasing the data size.

[0009] Furthermore, a thorough understanding of word classification and the semantic meaning of words and phrases is required. A major challenge in generating pre-existing reference data structures for text data is grouping words with multiple context-dependent meanings.

[0010] Therefore, an improved method and system for generating K-anonymous reference data structures is needed. Summary of the Invention

[0011] In various examples, and in at least one aspect, the present invention describes a method for automatically generating a K-anonymous reference data structure for text-based data. The disclosed method can utilize machine learning techniques to generate a vector space that can be used to convert input text data into numerical values, automatically cluster similar data records, and generate a reference data structure having a meaningful identifier for each cluster, wherein the identifier can semantically represent the cluster.

[0012] On the other hand, the method disclosed in this paper can automatically generate K-anonymous reference data structures, not limited to English, and can support text-based data in multiple languages.

[0013] In another aspect, the method for automatically generating reference data structures can save processing costs and / or eliminate the need for intensive manual labor. Specifically, the method disclosed herein eliminates the time-intensive manual labor required to manually create reference data structures. Furthermore, compared to prior art, the automatically generated reference data structures according to the present invention can also be updated in a shorter time (e.g., to reflect updated text corpora).

[0014] In another aspect, by retraining the vector space (representing the potential code space of the text sequence) on any given text context, new or additional information (e.g., new semantic meaning of the text sequence) can be represented relatively easily in the updated reference data structure.

[0015] In another respect, the reference data structure generated according to the present invention can be generated as needed without providing additional files to the client anonymized using K-anonymity.

[0016] In another aspect, the method according to the invention can generate a reference data structure based on input text data without needing to create large reference data structures for different semantic contexts. This allows for a reduction in the size of the reference data structure (compared to existing technologies), which can be used to enhance its search capabilities (e.g., requiring fewer processing resources to perform searches of the reference data).

[0017] In another aspect, the present invention relates to a method for generating a reference data structure for input text data comprising a plurality of input text words. The method includes: generating a vector space from the reference text data comprising a plurality of reference text words, wherein the vector space is defined by numerical vectors representing the semantic meaning of the reference text words; converting the plurality of input text words into corresponding numerical vectors using the vector space; forming word clusters from the plurality of input text words according to semantic similarity between the input text words, the word clusters defining corresponding nodes of the reference data structure, the semantic similarity between pairs of input text words being represented by corresponding metric values ​​determined from corresponding numerical vector pairs, the metric values ​​being used to determine whether a clustering criterion is satisfied; applying text labels to each node of the reference data structure, the text labels representing semantic meanings shared by elements of the word clusters; and storing the reference data structure.

[0018] In any of the foregoing, the method may include replacing each given word in the input text data with a text label of the word cluster to which the given word belongs.

[0019] In any of the above, the metric can be one of Euclidean distance, squared Euclidean distance, Manhattan distance, maximum distance, and Mahalanobis distance.

[0020] In any of the preceding steps, forming word clusters from the plurality of words in the input text data may include: initializing the word clusters such that each root word among the plurality of root words derived from the plurality of words in the input text data forms an element of each word cluster in the word clusters; iteratively forming the word clusters, each iteration including: determining a metric between each word cluster pair based on the numerical vector of each word cluster in the word cluster pair; and merging the two word clusters into a single word cluster when the metric determined from two word clusters in the word clusters satisfies a clustering criterion.

[0021] In any of the preceding steps, forming a word cluster from the plurality of words in the input text data may include: initializing a single word cluster comprising all root words generated from the plurality of words in the input text data, the single word cluster being defined as a parent cluster; iteratively forming the word cluster, each iteration comprising: identifying potential word clusters from the parent cluster by applying a flat clustering algorithm; and separating one of the potential word clusters from the parent cluster to form a new word cluster.

[0022] In any of the above, the clustering criterion may be one of the following: maximum link clustering, minimum clustering, unweighted average link clustering, weighted average link clustering, minimum energy clustering, the sum of variances within all clusters, Ward's criterion, V-linking, graph degree linking, and the increment of a cluster descriptor.

[0023] In any of the above, the text tag may be a common hyponym of the elements of the word cluster.

[0024] In any of the foregoing, the input text data may be in a first language, and the method may include: translating words in the input text data from the first language to a second language; using the translated words in the second language to determine common hyponyms in the second language; translating the common hyponyms in the second language into the first language, wherein the translated words in the first language are used as text tags for the word cluster.

[0025] In any of the foregoing, generating the vector space may include: collecting a document corpus to form the reference text data; converting the plurality of words in the reference text data into root words; and mapping the root words to the numerical vector.

[0026] In any of the preceding steps, converting the plurality of words in the reference text data into root words may include: formatting the plurality of words in the reference text data to remove data that is not useful to the semantic meaning of the reference text words; separating the formatted plurality of words into tokens; and normalizing the tokens into root words.

[0027] In any of the foregoing, normalizing the symbol to a root word may include at least one of the following: truncating affixes from the symbol; or converting the symbol to its basic form.

[0028] In some aspects, the present invention describes a system comprising: a processing unit for executing instructions to cause the system to perform any of the methods described herein.

[0029] In some aspects, the present invention describes a computer-readable medium that tangibly stores instructions. When executed by a processing unit of a system, the instructions cause the system to perform any of the methods described herein. Attached Figure Description

[0030] The accompanying drawings, which illustrate exemplary embodiments of this application, will now be shown by way of example, in which:

[0031] Figure 1 A block diagram of an exemplary processing system that can be used to implement the methods and systems disclosed herein is shown;

[0032] Figure 2 A flowchart is shown, illustrating an exemplary method for generating a reference data structure for data anonymization, as provided by one aspect of the present invention.

[0033] Figure 3 It shows Figure 2 A flowchart of an example of step 110;

[0034] Figure 4 It shows Figure 3 A flowchart of an example of step 114;

[0035] Figure 5 It shows that it can be used Figure 2 The flowchart of the coagulation method implemented in step 130;

[0036] Figure 6 It shows that it can be used Figure 2 The flowchart of the division method implemented in step 130;

[0037] Figure 7 A table containing exemplary methods for determining vector distances is shown;

[0038] Figure 8 A table containing exemplary clustering criteria is shown;

[0039] Figure 9 A tree diagram of the unlabeled reference data structure provided by the present invention is shown in the form of a binary tree.

[0040] Similar reference numerals can be used to denote similar components in different accompanying drawings. Detailed Implementation

[0041] To aid in understanding the examples discussed in this article, an exemplary processing system is first described. The processing system may be implemented as a workstation, server, or other suitable computing system.

[0042] Figure 1 This is a simplified block diagram of an exemplary processing system 10 that can be used to implement the embodiments disclosed herein. Other processing units may be adapted to implement the embodiments described herein and may include components different from those discussed below. Although Figure 1 A single instance of each component is shown, but multiple instances of each component may exist in the processing system 10.

[0043] The processing system 10 may include one or more processing devices 12 (also referred to as processing units), such as processors, microprocessors, graphics processing units (GPUs), tensor processing units (TPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), special-purpose logic circuits, or combinations thereof. The processing system 10 may optionally include one or more input / output (I / O) interfaces 14 to support connectivity with one or more optional input devices 16 and / or output devices 18. The processing system 10 may include one or more network interfaces 20 for wired or wireless communication with a communication network (not shown) or for peer-to-peer communication with other processing systems. The one or more network interfaces 20 may include wired links (e.g., Ethernet cables) and / or wireless links (e.g., one or more antennas) for intranet and / or extranet communication.

[0044] The processing system 10 may also include one or more storage units 22, which may include mass storage units such as solid-state drives, hard disk drives, disk drives, and / or optical disk drives. In some exemplary embodiments, the one or more storage units 22 may include a reference text data repository 24 for storing reference input data, as described in further detail below. Although Figure 1 One or more storage units 22 including a reference text data repository 24 are shown; however, in alternative embodiments, the reference text data repository 24 may be included in one or more remote storage units that can be remotely accessed via a wireless or wired communication network. Details regarding the reference text data will be discussed further below.

[0045] Processing system 10 may include one or more non-transitory memories 26, which may include volatile or non-volatile memories (e.g., flash memory, random access memory (RAM), and / or read-only memory (ROM)). The one or more non-transitory memories 26 may store instructions executed by one or more processing devices 12, for example, to perform the exemplary methods described in this invention. The one or more memories 26 may store other software (e.g., instructions for execution by one or more processing devices 12), such as operating systems and other applications / functions. In some embodiments, one or more datasets and / or modules may be provided by external memory (e.g., an external driver communicating with processing system 10 via wired or wireless communication) or by transient or non-transitory computer-readable media. Examples of non-transitory computer-readable media include RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, CD-ROM, or other portable storage. In one embodiment, one or more memories 26 store a reference data structure generation system 28, which is a software system including machine-readable instructions stored in one or more memories 26 and executable by the processing device 12. The reference data structure generation system 28 may include a root format converter 30, a vector converter 32, a cluster builder 34, and a tag generator 36, which are software units (or software subsystems) of the system 28. The vector converter 32 may be a machine learning-based software unit that implements a learning model to convert text data into numerical vectors, as described in further detail below. In other embodiments, the vector converter 32 may implement an algorithm to convert text data into numerical vectors. In some embodiments, the system 28 may be implemented as a single hardware device, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a system-on-a-chip (SoC). The hardware device includes electronic circuitry that performs the functions of units 30, 32, 34, and 36. In other exemplary embodiments, the system 28 may be implemented as multiple hardware devices (e.g., multiple ASICs, FPGAs, and / or SoCs). Each hardware device includes electronic circuitry that performs the function of one of the execution units 30, 32, 34, and 36.Details of the root format converter 30, vector converter 32, cluster constructor 34, and tag generator 36 will be discussed further below. It should be understood that units 30, 32, 34, and 36 are not necessarily individual units of system 28, and the illustration of 30, 32, 34, and 36 as individual blocks within system 28 may only be a conceptual representation of the overall operation of system 28.

[0046] A bus 38 may exist to provide communication between components of the processing system 10, including one or more processing devices 12, one or more optional I / O interfaces 14, one or more network interfaces 20, one or more storage units 22, and / or one or more memories 26. The bus 38 may be any suitable bus architecture, including, for example, a memory bus, a peripheral bus, or a video bus.

[0047] exist Figure 1 In this embodiment, one or more input devices 16 (e.g., keyboard, mouse, microphone, touchscreen integrated into a display device, which may include a UI and / or keypad) and one or more optional output devices 18 (e.g., display devices that may include a UI, speakers, and / or printers) are shown as external to the processing system 10. In other examples, one or more of the input devices 16 and / or one or more output devices 18 may be internal components of the processing system 10. The one or more input devices 16 may include a display device with a display screen and a user interface (UI) navigation device (e.g., touchscreen input, mouse, or handheld controller) for enabling a user to interact with items displayed on the display device. The one or more output devices 18 may also include a display device with a display screen and a user interface (UI) navigation device for displaying the generated results of the methods disclosed herein.

[0048] Figure 2 This is a flowchart of an exemplary method 100, which can be executed by the reference data structure generation system 28 of the present invention to generate a reference data structure for data anonymization. Specifically, method 100 can be used to automatically generate a reference data structure for a K-anonymity model of text input data.

[0049] In step 110, a vector space is generated from reference text data that includes multiple text words. Figure 3This is a flowchart illustrating an exemplary step that can be performed as part of step 110. Specifically, in step 112, a document corpus may be collected to form reference text data. Typically, the reference text data may include unstructured text that may contain punctuation, symbols, and uppercase / lowercase letters. In some embodiments, the reference text data may include words collected from different domains, which can more comprehensively cover the lexical semantic meaning of words in a given language. The reference text data contains contextual information for each of the multiple words. As a non-limiting example, for a given word in the reference text data, its surrounding words or contextual words can provide contextual information, such as the semantic meaning of the given word. Factors such as semantic consistency, lexical coverage, and the universality of the reference text data may also affect the accuracy of the vector transformation step, as discussed in more detail below.

[0050] In some embodiments, reference text data may be collected from a third-party source via network interface 20 or input by input device 16 via I / O interface 14. The collected reference text data may be stored in a reference text data repository 24 of one or more storage units 22 for easy access by various system components via bus 38. It should be understood that the reference text data may be obtained through one or more of the methods described above or any other suitable method.

[0051] In step 114, the root format converter 30 converts multiple words from the reference text data into root words.

[0052] Figure 4 A flowchart is shown of exemplary steps that can be performed as part of exemplary step 114. In the illustrated embodiment, conversion step 114 may further include the following steps: formatting data 114a using a data formatter 30a; separating the formatted data into symbols 114b using a symbol separator 30b; and converting the symbols into basic forms using a basic form converter 30c.

[0053] In the illustrated embodiment, in the data formatting step 114a, irrelevant elements contained in the reference text data that may be useless to the semantic meaning of words or phrases are removed, and the formatting of the unstructured text is normalized. For example, step 114a may include removing HTML tags and URLs, extra whitespace characters, special characters (e.g., "$" and "%") and punctuation marks, converting accented characters to unaccented characters (e.g., "café" to "cafe"), expanding and contracting characters (e.g., "don't" to "do not"), making all text lowercase, and / or converting numeric words to numeric forms or removing numbers. The data formatting step 122 may also include removing stop words, which are generic or common words that do not contribute much to the semantic meaning of words or phrases. For example, words including "the" and "a" generally do not contribute much to the semantic meaning of the text content. It should be understood that other appropriate steps may be implemented in the data formatting step 114a.

[0054] In symbol separation step 114b, the formatted reference text data from step 114a is separated or segmented into smaller fragments or "symbols" according to a segmentation strategy. Various segmentation strategies can be employed to define the boundaries between symbols. By way of non-limiting examples, segmentation boundaries can be defined as non-alphanumeric characters, whitespace characters, and / or punctuation marks. In other embodiments, segmentation boundaries can be defined by changes in character entropy, which determine the conditional probability of all transitions occurring between characters to further determine the number of possible next neighbors, and the reduction in the probability of exceeding a threshold given each consecutive character that can serve as a boundary. In some other embodiments, a manually created list of symbol boundary definitions can be used. It should be understood that other segmentation strategies for defining symbol boundaries are possible. Segmentation boundaries can depend on the language of the reference text data. For example, whitespace characters may serve as suitable segmentation boundaries for languages ​​such as English, but it may not be suitable for East Asian languages ​​because there are no spaces between characters in East Asian languages.

[0055] After the symbol separation step 114b, the symbols can be normalized in step 114c to convert the separated symbols into root format, such that two symbols with surface differences in the character sequence, or families of derived related words with similar meanings (e.g., “studying”, “studies”, “studied”), can be matched with the same root word (e.g., “study”).

[0056] The normalization step 114c may include a word truncation step 115a, wherein each symbol may be further truncated to remove affixes (e.g., suffixes, prefixes, infixes, exfixes) to obtain a truncated version of the symbol, or a “stem” as known in the art. In some embodiments, the word truncation step 115a (also referred to in the art as “stem extraction”) is implemented by a basic form converter 30c via a heuristic process that simply removes word endings and typically includes the removal of derived affixes. For example, the words “depression” and “depressed” can be converted to the word “depress” by truncating the suffixes “ion” and “ed”, respectively. Other possible algorithms that can be used to implement the word truncation step 115a include the Porter stemming algorithm, the Lancaster stemming algorithm, the single-pass Lovins stemming algorithm, and the Paice / Husk stemming algorithm, etc.

[0057] The symbol normalization step 114c may also include a basic form transformation step 115b, wherein each symbol, as an inflectional or conjugate form of a word, can be transformed into a basic or canonical form, which is defined as the simplest form used for dictionary input. This basic form is also known as a "lemma". For example, the word "better" can return its basic form, the word "good". The algorithm for step 115b typically relies on a rule-based approach, where basic form transformation rules can be manually defined according to a specific language, or the rules can be learned using machine learning methods. Existing basic form transformation algorithms or lemmas, including NLTK lemmas, Wordnet lemmas, Spacy lemmas, TextBlob, CLiPS patterns, Stanford CoreNLP, Gensim lemmas, and TreeTagger, can be used to implement step 115b.

[0058] In some embodiments, word truncation step 115a may require fewer computational resources and less execution time because its method of trunculating the affixes of symbolic words is relatively simple. However, in some cases, the results may be less accurate. For example, after word truncation step 115a, the word "better" may become "bett," which does not accurately reflect the lexical meaning of the original word. Instead, the basic form transformation step 115b can produce a more accurate normalized form of the word "better," "good." However, improving the accuracy of step 115b may come at the cost of computational resources and execution time. Therefore, it should be understood that steps 115a and 115b can be used independently or in combination in the normalization step 115, depending on, for example, computational resource constraints or execution time constraints.

[0059] Following symbol normalization step 115, symbols from the reference text data are converted into root words, which are then mapped to numerical vectors by vector converter 32 in step 116. The vector values ​​of a word can indicate its semantic meaning relative to other words. A metric is a measure of vector proximity between two vectors, where vector proximity represents the semantic relationship between the words that generated the numerical vectors. A semantic relationship can be defined as an association between the meanings of words, phrases, or sentences. For example, synonyms with similar meanings can be called semantically similar, and vice versa; antonyms with different meanings can be called semantically dissimilar. As discussed in more detail, the numerical values ​​of each vector are trained to reflect the semantic meaning of a given word based on the contextual information presented in the reference text data. A class of techniques that maps words to real-valued numerical vectors is also known as word embedding. Preferably, words with similar semantic meanings are represented by similar numerical vectors.

[0060] The numerical vectors have a dimension of size n, and all numerical vectors collectively define the vector space. The dimension of each vector can affect its ability to capture the semantic meaning of the root word. If every word and every part of a word is represented by a numerical vector, then a high-dimensional vector will be required for a vocabulary with a large number of words. The resulting vector that a given word can be mapped to will consist primarily of zeros, with only one non-zero entry (value 1) at the index corresponding to the given word. This is an example of sparse representation, called one-hot encoding. In some embodiments of the invention, each numerical vector is an n-dimensional dense vector, where n is smaller than the dimension of its corresponding one-hot encoding. The dense vector may include floating-point values ​​that can indicate the semantic meaning of a particular word. A higher dimension size n may be able to capture the semantic meaning of a word better at the expense of computational efficiency. Conversely, a lower dimension size n may provide computational efficiency but may be less efficient in capturing semantic meaning. The dimension size n can be set as a parameter during training.

[0061] In step 116, the vector converter 32 can implement various methods for mapping root words to numerical vectors. In some embodiments, step 116 can be performed using machine learning methods.

[0062] In some embodiments, the execution of step 116 of vector converter 32 can be implemented using the word2vec algorithm, which uses a shallow two-layer neural network to map discrete objects (words) to n-dimensional real-valued feature vectors in such a way that feature vectors with similar semantic meanings have similar representations. It should be understood that other algorithms for mapping words to numerical vectors, such as Global Vector (GloVe) and doc2vec, are also possible.

[0063] In the case of word2vec, instead of training on the input word through reconstruction, it trains on words adjacent to the target input word in the reference text data that provides contextual information about the target input word. The training model can be implemented in one of two ways: a continuous bag-of-words (CBOW) model or a skip-gram model. The CBOW model may be suitable for faster model training and provides better representations for more frequently occurring words. The skip-gram model handles small amounts of training data well and improves the representation of rare words or phrases.

[0064] The CBOW model architecture attempts to predict the current target word (center word) from source context words or surrounding words that provide contextual information about the target word. It should be understood that the size of the context to be considered can vary. Specifically, the goal of the CBOW model is to maximize the following equation (1), where wt is the word to be predicted from the surrounding context words wt–1, wt–2, ..., wt–c and wt+1, wt+2, ..., wt+c that provide contextual information about the target word wt. The variable |V| represents the total number of root words in the reference text data, and c is the context size. For example, for a c value of 4, the four words before and four words after the target word wt are considered.

[0065]

[0066] Equation (1)

[0067] The CBOW model uses context words as input, which are passed to the embedding layer as a weight matrix initialized with randomized weight values. The embedding layer generates feature vectors for the context words passed to the λ-layer, where these feature vectors are summed and averaged to form a single average context vector. This average context vector is then passed to dense softmax layers to predict the target word. The softmax layer essentially implements the softmax function, which takes K real-valued vectors as input and normalizes them to a probability distribution consisting of K probabilities proportional to the exponent of the input number. The prediction is then compared to the actual target word t to calculate a loss or error, which is backpropagated to update the weights of the embedding layer.

[0068] Alternatively, the skip-gram model attempts to achieve the opposite effect of the CBOW model by trying to predict the context words (surrounding words) of a given target word. The goal of the skip-gram model is to maximize the following equation (2), where |V| is the total number of root words in the reference text data and c is the context size.

[0069]

[0070] Equation (2)

[0071] In some embodiments, the task of predicting context words using a skip-gram model can be simplified to word pairs consisting of a target word and context words. The word pairs and relevance indicators are the inputs to the skip-gram model. The relevance indicator represents the relevance between the target word and the context words. For example, a context indicator value of "1" indicates a semantically similar pair, and a value of "0" indicates that the two words are semantically different. Each word pair, along with the relevance indicator, is passed to the corresponding embedding layer as a weight matrix initialized with random weight values. The embedding layer then generates numerical feature vectors for the target word and the context words, and these vectors are passed to a merging layer where the dot product of the two vectors is determined. This dot product value is then passed to a dense sigmoid layer, which predicts either 1 or 0 depending on whether the two words are context-related or simply random words. The relevance prediction is compared to the actual relevance indicator value to calculate a loss or error, which is backpropagated to update the weight values ​​of the embedding layer.

[0072] Upon completion, the embedding layer with updated weight values ​​in the form of an embedded weight matrix can be used for numerical vector transformation. In some embodiments, the root word is stored along with its corresponding row identifier, wherein the row identifier identifies the row of the embedded weight matrix, and the column value or updated weight value is the numerical value of its corresponding numerical vector.

[0073] In some embodiments of the invention, an extension of the continuous skip-gram model can be employed, subsampling frequently occurring words during training. In some other embodiments, step 116 can also employ noise contrast estimation to train the skip-gram model, which can result in faster training and more accurate vector representations.

[0074] In some embodiments, a term frequency (TF) and inverse document frequency (IDF) weighted algorithm can be used, wherein the function of the IDF factor is to eliminate or minimize the influence of commonly used words such as propositions, so that the TF factor can more accurately represent the topic of the document being processed.

[0075] It should be understood that the training process can be applied multiple times by returning to step 112 to obtain additional reference text data.

[0076] In some embodiments, step 116 may also include sub-steps such as dimensionality reduction to further enhance the performance of method 100 to obtain more accurate root word representations. Specifically, as mentioned above, the analysis of high-dimensional vectors can be time-consuming and computationally resource-intensive. Therefore, it may be desirable to reduce the dimensionality of the vectors while retaining as much meaningful information as possible. Other advantages of dimensionality reduction may include reducing the space required for data storage and shortening computation / training time. Furthermore, dimensionality reduction makes it easier to visualize or display data graphically. One possible dimensionality reduction algorithm is the t-distributed stochastic neighbor embedding (t-SNE) algorithm. The t-SNE algorithm first constructs a probability distribution on high-dimensional object pairs such as numerical vectors in a way that similar objects have a high probability of being selected, while different points have a very low probability of being selected. Then, the t-SNE algorithm defines a similarity probability distribution on points in a low-dimensional map and minimizes the Kullback-Leibler divergence (KL divergence) between the two distributions with respect to point locations. Other suitable dimensionality reduction algorithms may include latent semantic analysis (LSA) and uniform manifold estimation and projection (UMAP).

[0077] Other sub-steps of step 116 may include analogical reasoning (or evaluating embeddings).

[0078] Back Figure 2 In step 120, the input text data, which includes multiple words, is converted into a numerical vector using a vector space generated from the reference text data. Specifically, the reference data structure generation system 28 receives the input text data, which includes multiple words. The input text data can be in a structured data format, such as a table or dataset with multiple columns, where one or more columns contain data values ​​to be anonymized. Each column that needs to be anonymized can consist of multiple rows.

[0079] Based on the vector space derived from step 110, one or more of the words to be anonymized from the input text data are converted into numerical vectors. Before performing the numerical vector conversion, the words to be anonymized from the input text data are first converted into root words. In some embodiments, the same step 114 can be used (i.e., using... Figure 4The same steps 114a to 114c) shown convert words from the input text data into root words, such that the same words from the input text data can be converted into the same root words as the words in the reference text data used to generate the vector space. Therefore, a numerical vector for a given root word in the input text data can be obtained by searching for the root word in the embedding weight matrix. As a non-limiting example, the word "better" from the reference text data may have been converted into the root word "good," which can then be mapped to a corresponding numerical vector, such as [0.1, 0.23, 0.45, 0.78, 0.05], which could be row 50 in the embedding weight matrix. The root word "good" can be stored using row identifier 50. Then, by applying the same step 120 to the input text data, the word "better" appearing in the input text data can also be converted into the root word "good." The vector converter 32 can then simply look up the entry for the root word "good" and retrieve the row identifier stored with the root word. Using row identifier 50, the numerical vector for the root word "good" can be retrieved from row 50 of the embedding weight matrix and used to convert the word "better" from the input text data into its corresponding numerical vector [0.1, 0.23, 0.45, 0.78, 0.05]. In some embodiments, the input text data may contain words or phrases not encountered in the reference text data, or words outside the vocabulary. In this case, searching for the root word of a word outside the vocabulary may not return a corresponding numerical vector from the embedding weight matrix. There are many suitable methods for handling words outside the vocabulary, including returning a zero vector or generating a corresponding vector using subword information (e.g., character n-grams).

[0080] After the root words of the input text data have been converted into numerical vectors, in step 130, the root words from the input text data are iteratively and hierarchically grouped into word clusters by the cluster builder 34. Specifically, word clusters are formed based on the metric values ​​of representative vectors that satisfy the similarity conditions set by the clustering criteria, where each word cluster defines a node of the reference data structure. The representative vectors may depend on the clustering criteria. The formation of word clusters can be achieved using a bottom-up approach or agglomerative clustering method, where each word of the input text data starts from its own word cluster and is iteratively merged together in a hierarchical manner until a single cluster appears. Alternatively, grouping can be achieved using a top-down approach or a splitting clustering method, where all root words of the input text data start from a single word cluster and are recursively split into smaller word clusters until, for example, each word cluster includes at least K root words that the K-anonymity model might require.

[0081] Figure 5 A flowchart of a grouping cohesion method 250 that can be implemented in step 130 is shown.

[0082] During step 252, N word clusters are initialized such that each of the N root words of the input text data forms an element of a unique word cluster.

[0083] In step 254, a metric is determined between the numerical vectors of two word clusters out of the N word clusters, as described in more detail below. The metric can represent the semantic similarity between the two word clusters.

[0084] In step 256, two word clusters out of N are considered to be closest to each other when the metric values ​​of their representative vectors satisfy the clustering criteria. The representative vectors can be determined by the clustering criteria. Therefore, the two closest word clusters are merged to form a new word cluster.

[0085] Steps 254 and 256 are repeated iteratively until a single word cluster appears at the top of the reference data structure in step 258.

[0086] Figure 6 A flowchart of a split clustering method 350 for forming word clusters, which can be implemented in step 130, is shown. In step 352, all root words of the input text data are initialized to form a single word cluster. In step 354, a flat clustering algorithm can be applied to identify potential word clusters. By way of a non-limiting example, a K-means flat clustering algorithm can be implemented, where K is the number of desired clusters. The goal of the K-means flat clustering algorithm is to minimize a metric, such as the mean squared difference, between a data point and its assigned centroid. The algorithm first randomly assigns K centroids to parent clusters. Based on the vector distance to each centroid, the data point closest to the centroid is assigned, or in this case, the root word. Then, each centroid is recalculated as the mean of the vector assigned to the centroid during reassignment. This process is repeated iteratively until a stopping condition is met. Upon completion of step 354, the K centroids of the K potential clusters are identified. In step 356, the best potential word cluster is selected from the K potential clusters based on clustering criteria and separated from the parent clusters, for example, by forming links between the newly formed clusters and the parent clusters. Each term cluster defines a node in the reference data structure. Steps 354 and 356 are repeated iteratively until the completion criteria are met in step 358, such as each cluster containing only a single element or multiple desired clusters.

[0087] In some embodiments, split clustering method 350 may be more complex than agglomerative clustering method 250 because split clustering method 350 requires a flattening clustering method. However, split clustering may be more efficient when the incomplete reference data structure does not extend to a single data leaf. Agglomerative clustering has a time complexity of O(n^2). 3 In Big O notation, it can be optimized down to O(n). 2For split clustering, the number of patterns and clusters is linear. Furthermore, split clustering may be more accurate, at least because agglomerative clustering makes decisions by considering local patterns or neighboring points rather than first considering the global distribution of the data. In contrast, split clustering considers the global distribution of the data when making top-level partitioning decisions.

[0088] As described above, the metric used in step 130 indicates vector proximity, which represents the semantic similarity between words. Multiple metrics can be suitable as indicators of numerical vector proximity, thereby indicating the semantic similarity between word clusters.

[0089] Figure 7 Table 700 shows some metrics that can be used in step 130.

[0090] The metric includes Euclidean distance 702a, which is used to calculate the straight-line distance between two points in Euclidean space. As an illustrative example, for vectors a and b of dimension i, the distance between these two points can be determined using formula 702b, as follows:

[0091]

[0092] The squared Euclidean distance 704a (which could be another metric that can be used in step 130) is determined using equation 704b, similar to the Euclidean distance 702b, but without taking the square root. Therefore, clustering using a Euclidean squared distance metric may be faster than clustering using regular Euclidean distance.

[0093] Possible metrics could also include the Manhattan distance 706a, named after the grid-like layout of most streets on Manhattan Island. This might be analogous to calculating the distance from one data point to another if a grid-like path is followed. The Manhattan distance between two items is the sum of the differences between their corresponding components, as shown in Equation 706b.

[0094] Another possible metric that can be used in step 130 is the maximum distance 708a, which is determined by formula 708b as the maximum difference between two vectors along any coordinate dimension.

[0095] Another possible metric that can be used in step 130 includes Mahalanobis distance 710a, which calculates the distance between two points in a multivariate space. In conventional Euclidean space, variables (e.g., x, y, z) are represented by axes drawn perpendicular to each other. However, if two or more variables are correlated, the axes are no longer perpendicular. The Mahalanobis distance, determined by formula 710b, measures the distance between points, even for correlated points with multiple variables.

[0096] It should be understood that other suitable metrics, such as Hamming distance or Levenshtein distance, can also be used in step 130.

[0097] After the metric values ​​are determined, clustering criteria are used to determine the formation of specific word clusters based on the metric values ​​determined in steps 254 and 354. More specifically, the criterion determines the similarity between word clusters as a function of pairwise metrics between them. For example, in each step, two clusters separated by the shortest Euclidean vector distance or any other metric are combined into a single word cluster. The definition of "shortest distance" distinguishes different agglomerative clustering methods.

[0098] Figure 8 Table 800 contains a list of various clustering criteria, any one of which, among others, can be adapted for implementation in step 256. The clustering criteria represent semantic similarity conditions used to merge word clusters based on metrics representing vectors of the word clusters in question, as discussed in more detail below with respect to each criterion.

[0099] Criterion 802a is the maximum or fully linked clustering criterion. In fully linked clustering, each element of one cluster is linked to a corresponding element of another cluster, forming multiple element pairs. The vector distance between two clusters is defined as equal to the distance between their representative vectors, which are the vectors of the element pairs that are furthest apart from each other or have the longest link. During each clustering iteration, the two clusters with the shortest links are merged into one cluster. In Equation 802b, the vector distance between clusters A and B is defined as the maximum value of the vector distance between element pairs a and b, where a and b are elements of clusters A and B, respectively. The vector distance formula d can be... Figure 7 Any of the formulas shown or other suitable methods for determining vector distance.

[0100] Criterion 804a is a minimum or single-link clustering criterion that merges a cluster with its nearest neighbor. Criterion 804a can generate a minimum span hierarchical structure tree. Also known as a single-link clustering criterion, this criterion groups one cluster with another at a given similarity (or distance) level, possibly requiring only that each of the two clusters to be agglomerated is linked to the others at that level. In Equation 804b, the vector distance between clusters A and B is defined as the minimum value of the vector distance between the vectors (representing vectors) of element pairs a and b, where a and b are elements of clusters A and B, respectively. The vector distance formula d can be... Figure 7 Any of the formulas shown or other suitable methods for determining vector distance.

[0101] Another criterion is Unweighted Average Linked Clustering 806a, also known as Group Average Ordination or Unweighted Arithmetic Average Pair-Group Method (UPGMA). This criterion gives equal weight to the original similarities and assumes that the elements in each group constitute a representative sample of the corresponding larger group of objects in the reference population. Therefore, UPGMA clustering criterion 806a may be well-suited for simple random or systematic sampling designs if the results are extrapolated to a larger reference population. UPGMA criterion 806a allows objects to be added to groups or clusters by the average distance between the object and all members of the group. Two clusters are merged by the average distance between all members of one group and all members of the other group. In Equation 806b, the average of all distances determined by formula d, which is the distance between the pair of objects x in cluster A and y in cluster B, is averaged over cluster sizes |A| and |B|.

[0102] The weighted average linked clustering criterion 808a can be used to represent cases where groups of elements (and thus potentially different groups) are represented by unequal numbers of elements. In this case, the unweighted UPGMA criterion 806a described above may be distorted when large and small clusters are merged. In Equation 808b, in each merge iteration, the two nearest clusters (e.g., i and j) are merged into a higher-level cluster i∪j. Its distance to another cluster k is then defined as the arithmetic mean of the average distances between all members of k and i, and between k and j (as determined using distance formula d). Therefore, all vectors representing all members of clusters i and j are representative vectors.

[0103] The centroid-linked clustering criterion 810a, also known as the unweighted centroid clustering criterion (UPGMC), is another criterion that can be adopted. In a cluster of elements, the centroid, as the representative vector of the cluster, is a point with the average coordinates of all objects in the cluster. As shown in Equation 810b, UPGMC 810a determines their centroid C. s With C t The centroid distance between clusters is determined, and if the centroid distance value is the smallest among all cluster pairs, the two clusters are merged. Methods for determining the centroid of an object's clusters are well known in the art.

[0104] It should be recognized that although Table 800 lists several clustering criteria, other appropriate clustering criteria may also be used, such as the sum of variances within all clusters; the variance increment of the merged clusters (Ward's criterion); the probability that candidate clusters arise from the same distribution function (V-link); the product of in-degree and out-degree on the k-nearest neighbor graph (graph degree link); and the increment of some cluster descriptors after merging two clusters (i.e., a quantity defined to measure cluster quality).

[0105] Upon completion of step 130, an unlabeled reference data structure is generated. Figure 9 A tree diagram of an exemplary unlabeled reference data structure 900 in the form of a binary tree, which can be generated after step 250 using Euclidean distance 702a and minimum or single-link clustering criterion 804a, is shown. Multiple word clusters 902 (each including one of the root words of the input text data) form the leaf nodes of the binary tree at the lowest level.

[0106] For each pair of word clusters, a Euclidean vector distance is determined as a metric. Clusters 902a and 902b (each containing a single word representing its respective cluster) are determined to have the minimum Euclidean distance, as demonstrated by the lowest link formation height on the dendrogram. The minimum Euclidean distance indicates that clusters 902a and 902b are semantically most similar among all clusters. Therefore, clusters 902a and 902b are merged into cluster 904a according to the single-link clustering criterion. In the next iteration of steps 254 and 256, clusters 902c and 902d are determined to have the minimum Euclidean distance and are merged into cluster 904b according to the single-link clustering criterion. The iterations are repeated until a single cluster 906 is found.

[0107] As described above, the reference data structure generated from step 130 is unlabeled. In step 140, the label generator 36 generates a label for each word cluster. In some embodiments, the label is a text label that represents similarity in semantic meaning shared by members of each word cluster.

[0108] In some embodiments, an external database (such as WordNet, a large lexical database that can generate hyponyms of words in their English forms) can be used to generate text labels for each word cluster. Several WordNet-based word similarity algorithms are implemented in a Perl package called WordNet::Similarity and a Python package called NLTK. Other more sophisticated WordNet-based similarity techniques include ADW, which is now available in Java. By merging one of the WordNet-based programming packages, the root word of a given word cluster can be used to generate the lowest common hyponym of the root word. Hyponyms can be used as text labels for word clusters. For example, the root words “cat” and “tiger” can generate the hyponym “feline”, which can be used as text labels for word clusters including “cat” and “tiger”. Then, when word clusters are merged into new word clusters, the text labels of the word clusters can be used to represent the word clusters to generate text labels for the new word clusters. Step 160 continues until all word clusters are labeled.

[0109] In some embodiments of the invention, it is possible to support the generation of reference data structures for text input data in a second language other than English. As a non-limiting example, the second-language input text data is used to generate the reference data structure using the same method 100. After completing step 130, the root words of the second language can be converted to English by an appropriate translator, such an automatic translation algorithm, or by manual means. The translated English input text data root words can then be used to generate hyponyms, which can be translated back into the second language for use as text tags for word clusters. It is understood that in some embodiments, the translation can be performed by an external source (e.g., equivalent to WordNet) capable of providing hyponyms of words in the second language.

[0110] In step 150, the marked reference data structure may be stored locally in a computer-readable medium, such as data storage 22, or remotely via network interface 20, such as on a remote server or in a cloud database.

[0111] In step 160, the tagged reference data structure can be used to anonymize the raw input text data. As mentioned above, the input text data is typically in the form of structured data, such as a table, where the data to be anonymized is the row values ​​of certain columns of the table. To achieve K-anonymization, each row value to be anonymized is replaced with a cluster label from the reference data structure, such that at least K row values ​​are indistinguishable from each other. In some embodiments, for a given row value to be anonymized, a reference data structure generated based on row value traversal is used to locate the corresponding cluster containing at least K root words. The row value can then be replaced with the text label of the located cluster. Step 180 continues until all values ​​to be anonymized have been replaced with cluster labels, such that each value is indistinguishable from at least K other values.

[0112] Although the present invention describes methods and processes by steps performed in a certain order, one or more steps in the methods and processes may be omitted or modified as appropriate. Where appropriate, one or more steps may be performed in an order other than that described.

[0113] Although the invention has been described at least partially in terms of method, those skilled in the art will understand that the invention is also directed to various components for performing at least some aspects and features of the method, whether by hardware components, software, or any combination thereof. Accordingly, the technical solutions of the invention can be embodied in the form of a software product. Suitable software products can be stored in pre-recorded storage devices or other similar non-volatile or non-transitory computer-readable media, including DVDs, CD-ROMs, USB flash drives, removable hard drives, or other storage media. The software product includes instructions tangibly stored thereon, which enable a processing device (e.g., a personal computer, server, or network device) to perform examples of the methods disclosed herein.

[0114] The invention may be implemented in other specific forms without departing from the subject matter of the claims. The exemplary embodiments described are merely illustrative in all respects and not restrictive. Features selected from one or more of the above embodiments may be combined to create alternative embodiments not explicitly described, and features suitable for such combinations will be understood within the scope of the invention.

[0115] All values ​​and sub-ranges within the scope of disclosure are also disclosed. Furthermore, although the systems, devices, and processes disclosed and shown herein may include a specific number of elements / components, the systems, devices, and components may be modified to include more or fewer of such elements / components. For example, although any element / component disclosed may be referenced as a single quantity, embodiments disclosed herein may be modified to include multiple such elements / components. The subject matter described herein is intended to cover and encompass all appropriate technical changes.

[0116] All published papers identified in this invention are incorporated herein by reference.

Claims

1. A method for generating a reference data structure for input text data including multiple input text words, said reference data structure being a tree structure including at least one node, wherein, The leaf nodes of the at least one node include at least one root word corresponding to a reference text word, characterized in that the method includes: A vector space is generated from reference text data including multiple reference text words, wherein the vector space is defined by numerical vectors representing the semantic meaning of the reference text words; The vector space is used to convert the multiple input text words into corresponding numerical vectors; The multiple input text words are formed into a word cluster based on the semantic similarity between the input text words. The word cluster defines the corresponding node of the reference data structure. The semantic similarity between input text word pairs is represented by the corresponding metric value determined from the corresponding numerical vector pair. The metric value is used to determine whether the clustering criterion is met. Text labels are applied to each node of the reference data structure, whereby the text labels represent the semantic meaning shared by the elements of the word cluster; Store the reference data structure.

2. The method according to claim 1, characterized in that, It also includes replacing each given word in the input text data with the text label of the word cluster to which the given word belongs.

3. The method according to claim 1 or 2, characterized in that, The metric is one of Euclidean distance, squared Euclidean distance, Manhattan distance, maximum distance, and Mahalanobis distance.

4. The method according to claim 1, characterized in that, The step of forming word clusters from the multiple words in the input text data further includes: Initialize the word clusters such that each root word among the multiple root words transformed from the multiple words in the input text data forms an element of each word cluster in the word cluster; The word clusters are formed iteratively, with each iteration including: Based on the numerical vector of each word cluster in the word cluster pair, determine the metric value between each word cluster pair; When the metric determined from two word clusters in the given word cluster satisfies the clustering criteria, the two word clusters are merged into a single word cluster.

5. The method according to claim 1, characterized in that, The step of forming word clusters from the multiple words in the input text data further includes: Initialization includes generating a single word cluster of all root words from the plurality of words in the input text data, and the single word cluster is defined as a parent cluster; The word clusters are formed iteratively, with each iteration including: Potential word clusters are identified from the parent clusters by applying a flat clustering algorithm; One of the potential word clusters is separated from the parent cluster to form a new word cluster.

6. The method according to claim 1, characterized in that, The clustering criterion is one of the following: maximum link clustering, minimum link clustering, unweighted average link clustering, weighted average link clustering, minimum energy clustering, sum of variances within all clusters, Ward's criterion, V-linking, graph degree linking, and increment of a cluster descriptor.

7. The method according to claim 1, characterized in that, The text tag is a common hyponym of the elements of the term cluster.

8. The method according to claim 7, characterized in that, The input text data is in a first language, and the method further includes: Translate the words in the input text data from the first language to the second language; The translated words in the second language are used to determine the common hyponyms of the second language; The common hyponyms of the second language are translated into the first language, and the translated words in the first language are used as the text tags of the word cluster.

9. The method according to claim 1, characterized in that, The generation of the vector space further includes: Collect a document corpus to form the reference text data; Convert the multiple words in the reference text data into root words; Map the root word to the numerical vector.

10. The method according to claim 9, characterized in that, The step of converting the multiple words in the reference text data into root words also includes: The plurality of words in the reference text data are formatted to remove data that is not useful to the semantic meaning of the reference text words; Separate the formatted words into tokens; The symbols are normalized to root words.

11. The method according to claim 10, characterized in that, The normalization of the symbol to a root word also includes at least one of the following: truncating affixes from the symbol; converting the symbol to a basic form.

12. A system, characterized in that, The system includes a processing unit and a memory for storing instructions, which, when executed by the processing unit, cause the system to: A vector space is generated from reference text data including multiple reference text words, wherein the vector space is defined by numerical vectors representing the semantic meaning of the reference text words, and the reference data structure is a tree structure including at least one node, wherein the leaf nodes of the at least one node include at least one root word corresponding to the reference text word; The vector space is used to convert multiple input text words into corresponding numerical vectors; The multiple input text words are formed into a word cluster based on the semantic similarity between the input text words. The word cluster defines the corresponding node of the reference data structure. The semantic similarity between input text word pairs is represented by the corresponding metric value determined from the corresponding numerical vector pair. The metric value is used to determine whether the clustering criterion is met. Text labels are applied to each node of the reference data structure, whereby the text labels represent the semantic meaning shared by the elements of the word cluster; The reference data structure is stored in the memory.

13. The system according to claim 12, characterized in that, When the instruction is executed by the processing unit, the system also causes the system to: replace each given word in the input text data with the text label of the word cluster to which the given word belongs.

14. The system according to claim 12 or 13, characterized in that, When the instruction is executed by the processing unit, it also causes the system to: Initialize the word clusters such that each root word among the multiple root words transformed from the multiple words in the input text data forms an element of each word cluster in the word cluster; The word clusters are formed iteratively, with each iteration including: Based on the numerical vector of each word cluster in the word cluster pair, determine the metric value between each word cluster pair; When the metric determined from two word clusters in the given word cluster satisfies the clustering criteria, the two word clusters are merged into a single word cluster.

15. The system according to claim 12, characterized in that, When the instruction is executed by the processing unit, it also causes the system to: Initialization includes a single word cluster of all root words generated from the multiple words in the input text data, and the single word cluster is defined as a parent cluster; The word clusters are formed iteratively, with each iteration including: Potential word clusters are identified from the parent clusters by applying a flat clustering algorithm; One of the potential word clusters is separated from the parent cluster to form a new word cluster.

16. The system according to claim 12, characterized in that, The text tag is a common hyponym of the elements of the term cluster.

17. The system according to claim 16, characterized in that, The input text data is in a first language, and the instructions, when executed by the processing unit, also cause the system to: Translate the words in the input text data from the first language to the second language; The translated words in the second language are used to determine the common hyponyms of the second language; The common hyponyms of the second language are translated into the first language, and the translated generalized words of the first language are used as the text tags of the word cluster.

18. The system according to claim 12, characterized in that, When generating the vector space, the instructions, when executed by the processing unit, also cause the system to: Collect a document corpus to form the reference text data; Convert the multiple words in the reference text data into root words; Map the root word to the numerical vector.

19. The system according to claim 12, characterized in that, When converting the plurality of words in the reference text data into root words, the instruction, when executed by the processing unit, also causes the system to: The plurality of words in the reference text data are formatted to remove data that is not useful to the semantic meaning of the reference text words; Separate the formatted words into symbols; The symbols are normalized to root words.

20. A computer-readable medium tangibly storing instructions, characterized in that, When the instruction is executed by the system's processing unit, the system: A vector space is generated from reference text data including multiple reference text words, wherein the vector space is defined by numerical vectors representing the semantic meaning of the reference text words, and the reference data structure is a tree structure including at least one node, wherein the leaf nodes of the at least one node include at least one root word corresponding to the reference text word; The vector space is used to convert multiple input text words into corresponding numerical vectors; The multiple input text words are formed into a word cluster based on the semantic similarity between the input text words. The word cluster defines the corresponding node of the reference data structure. The semantic similarity between input text word pairs is represented by the corresponding metric value determined from the corresponding numerical vector pair. The metric value is used to determine whether the clustering criterion is met. Text labels are applied to each node of the reference data structure, whereby the text labels represent the semantic meaning shared by the elements of the word cluster; The reference data structure is stored in the system's memory.

Images