Method, device, equipment, storage medium and product for standardizing the names of data elements

Abstract

The present application discloses a method, device, equipment, storage medium and product for name standardization of data elements, and relates to the field of artificial intelligence technology. The method includes: obtaining the original corpus of the target data element and the relevant key corpus of the target data element; extracting the synonyms of the original corpus from the relevant key corpus; respectively calculating the similarity between the original corpus and each synonym; for each synonym, determining the weight of the synonym, and multiplying the similarity by the weight to obtain the final result value; taking the target synonym with the highest final result value among the synonyms as the standardized name of the target data element. Compared with the conventional method of name standardization of data elements by similarity, the present application also calculates the weight corresponding to each synonym, and then uses the weight to perform weighted calculation of the similarity, which further improves the accuracy of the degree of correlation between the synonyms and the original corpus, thereby improving the accuracy of data element name standardization.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, equipment, storage medium and product for standardizing the names of data elements. Background Art

[0002] With the advancement of informatization in various industries and fields, massive data are stored in electronic form. In the process of data management and information exchange, the standardization of data elements is crucial to ensure the consistency and interoperability of data.

[0003] Currently, data element name standardization usually relies on established standard lexicons and domain-specific terminologies, and then searches the lexicon for the word with the highest similarity to the original corpus of the data element, and uses the word with the highest similarity as the standardized name of the data element.

[0004] However, when the field described by the data element is unclear or there is no standard word library for the corresponding field, the existing standardization method cannot be directly applied. In addition, in the standardization process, only the similarity of the name itself is often considered, and the research dimension used in standardization is relatively single, resulting in poor accuracy of the standardization result. Therefore, how to improve the accuracy of data element name standardization has become an urgent problem to be solved in this field.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a method, device, equipment, storage medium and product for standardizing the names of data elements, aiming to solve the technical problem of how to improve the accuracy of data element name standardization.

[0007] To achieve the above objectives, the present application proposes a method for standardizing the names of data elements, the method comprising:

[0008] Acquire the original corpus of the target data element and the key corpus related to the target data element;

[0009] Extracting synonyms of the original corpus from the relevant key corpus;

[0010] Calculating the similarity between the original corpus and each of the synonyms respectively;

[0011] For each of the synonyms, determine the weight of the synonym, and multiply the similarity by the weight to obtain a final result value;

[0012] The target synonym with the highest final result value among the synonyms is used as the standardized name of the target data element.

[0013] In one embodiment, the step of extracting synonyms of the original corpus from the relevant key corpus includes:

[0014] Splitting the relevant key corpus into text units and obtaining a context window;

[0015] Counting the frequencies of co-occurrence of each of the text units in the relevant key corpus and the original corpus in the same context window;

[0016] The target text unit in each of the text units whose frequency exceeds a preset frequency threshold is used as a synonym of the original corpus.

[0017] In one embodiment, before the step of splitting the relevant key corpus into text units, the method further includes:

[0018] Extracting pronouns from the relevant key corpus;

[0019] Finding multiple candidate nouns in the text preceding the pronoun;

[0020] Determining a target candidate noun corresponding to the pronoun from among the candidate nouns by using a preset syntactic analysis structure tool;

[0021] The pronoun in the relevant key corpus is replaced by the target candidate noun.

[0022] In one embodiment, the step of respectively calculating the similarity between the original corpus and each of the synonyms comprises:

[0023] Calculating the word form similarity between the original corpus and each of the synonyms based on the word form of the original corpus and the word form of each of the synonyms;

[0024] Mapping the original corpus and each of the synonyms into a vector space to obtain an original corpus vector and multiple synonym vectors;

[0025] Calculating the cosine similarity between the original corpus vector and each of the synonym vectors respectively;

[0026] Counting the number of times each of the synonyms appears in the same sentence as the original corpus;

[0027] For each of the synonyms, the largest one among the word form similarity, cosine similarity and the number of times corresponding to the synonym is taken as the similarity between the synonym and the original corpus.

[0028] In one embodiment, the step of determining the weight of the synonyms includes:

[0029] For each of the synonyms, calculating the mutual information between the synonym and the position of the synonym in the relevant key corpus;

[0030] Calculate the penalty factor by using the preset penalty factor coefficient;

[0031] The mutual information minus the penalty factor is used to obtain the weight of the synonym.

[0032] In one embodiment, after the step of using the target synonym with the highest final result value among the synonyms as the standardized name of the target data element, the method further includes:

[0033] When there are at least two target synonyms, a standardized name of the target data element is selected from each of the target synonyms by using a preset expert rule model.

[0034] In addition, to achieve the above purpose, the present application also proposes a data element name standardization device, the data element name standardization device comprising:

[0035] A corpus acquisition module, used to acquire the original corpus of the target data element and the key corpus related to the target data element;

[0036] A synonym extraction module, used for extracting synonyms of the original corpus from the relevant key corpus;

[0037] A similarity calculation module, used to calculate the similarity between the original corpus and each of the synonyms;

[0038] A weighting module, for determining the weight of each synonym, and multiplying the similarity by the weight to obtain a final result value;

[0039] The standardization module is used to use the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a data element name standardization device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the data element name standardization method as described above.

[0041] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the data element name standardization method as described above are implemented.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the data element name standardization method as described above.

[0043] The present application provides a method for standardizing the names of data elements. The present application first obtains the original corpus of the target data element and the relevant key corpus related to the target data element, extracts synonyms with similar meanings to the original corpus of the data element from the relevant key corpus, and then calculates the similarity between each synonym and the original corpus respectively, and then calculates the weight of each synonym, and multiplies the weight by the similarity to obtain the final result value of the degree of correlation between the original corpus and the synonym. Among all the synonyms, the synonym with the largest final result value is used to standardize the data element.

[0044] In summary, the present application calculates the correlation between the synonyms in the relevant key corpus and the target data element by the weighted value of similarity and weight in the relevant key corpus of the target data element, and uses the synonyms with the highest correlation to perform name standardization on the data element. Compared with the conventional method of performing name standardization on the data element by similarity, the present application also calculates the weight corresponding to each synonym, and then uses the weight to perform weighted calculation on the similarity, thereby further improving the accuracy of the correlation between the synonyms and the original corpus, thereby improving the accuracy of the name standardization of the data element. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of the first embodiment of the method for standardizing the names of data elements of the present application;

[0048] Figure 2 A schematic diagram of the implementation structure provided for the third embodiment of the method for standardizing the names of data elements of the present application;

[0049] Figure 3 A schematic diagram of a weight structure provided in Example 4 of the method for standardizing the names of data elements of the present application;

[0050] Figure 4A schematic diagram of a sorting structure provided in Example 4 of the method for standardizing the names of data elements of the present application;

[0051] Figure 5 A schematic diagram of the module structure of a device for standardizing the names of data elements in an embodiment of the present application;

[0052] Figure 6 A schematic diagram of the device structure of the hardware operating environment involved in the method for standardizing the names of data elements in the embodiments of the present application.

[0053] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: obtaining the original corpus of the target data element and the relevant key corpus of the target data element; extracting the synonyms of the original corpus from the relevant key corpus; respectively calculating the similarity between the original corpus and each of the synonyms; for each of the synonyms, determining the weight of the synonym, and multiplying the similarity by the weight to obtain a final result value; and taking the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

[0057] In this embodiment, for the convenience of description, the following description is made by taking the name standardization device of the data element as the execution subject.

[0058] With the advancement of informatization in various industries and fields, massive data are stored in electronic form. In the process of data management and information exchange, the standardization of data elements is crucial to ensure the consistency and interoperability of data.

[0059] Currently, data element name standardization usually relies on established standard lexicons and domain-specific terminologies, and then searches the lexicon for the word with the highest similarity to the original corpus of the data element, and uses the word with the highest similarity as the standardized name of the data element.

[0060] However, when the field described by the data element is unclear or there is no standard word library for the corresponding field, the existing standardization method cannot be directly applied. In addition, in the standardization process, only the similarity of the name itself is often considered, and the research dimension used in standardization is relatively single, resulting in poor accuracy of the standardization result. Therefore, how to improve the accuracy of data element name standardization has become an urgent problem to be solved in this field.

[0061] In response to the above problems, the present application provides a method for standardizing the names of data elements. The present application first obtains the original corpus of the target data element and the relevant key corpus related to the target data element, extracts synonyms with similar meanings to the original corpus of the data element from the relevant key corpus, and then calculates the similarity between each synonym and the original corpus respectively, and then calculates the weight of each synonym. The weight is multiplied by the similarity to obtain the final result value of the degree of correlation between the original corpus and the synonym. Among all the synonyms, the synonym with the largest final result value is used to standardize the data element.

[0062] In summary, the present application calculates the correlation between the synonyms in the relevant key corpus and the target data element by the weighted value of similarity and weight in the relevant key corpus of the target data element, and uses the synonyms with the highest correlation to perform name standardization on the data element. Compared with the conventional method of performing name standardization on the data element by similarity, the present application also calculates the weight corresponding to each synonym, and then uses the weight to perform weighted calculation on the similarity, thereby further improving the accuracy of the correlation between the synonyms and the original corpus, thereby improving the accuracy of the name standardization of the data element.

[0063] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a data element name standardization device, etc. The following takes the data element name standardization device as an example to illustrate this embodiment and the following embodiments.

[0064] Based on this, the present application embodiment provides a method for standardizing the name of a data element, referring to Figure 1 , Figure 1 Schematic diagram of the flow chart of the first embodiment of the method for standardizing the names of data elements of the present application.

[0065] In this embodiment, the method for standardizing the name of the data element includes steps S10 to S50:

[0066] Step S10, obtaining the original corpus of the target data element and the key corpus related to the target data element;

[0067] It should be noted that, in this embodiment, the relevant key corpus of the target data element may be policies, standards, laws and regulations, dictionaries, important documents, etc. related to the target data element. The above materials may be materials manually sorted by technical personnel, or they may be files related to the target data element found through web crawler technology.

[0068] In this embodiment, when standardizing the names of data elements, the target data elements that need to be standardized are first identified, and then the original text data containing the target data elements are collected from relevant documents, databases or other data sources, and further text data closely related to the target data elements are collected. These data can be texts containing similar data elements, or contextual information closely related to the target data elements.

[0069] Specifically, first prepare the original corpus of the data element. You need to prepare in advance the original corpus to which the data element belongs, that is, the original corpus of the data element that needs to be standardized. For example, a certain policy document, the extracted data element, such as "filing date". Then prepare the relevant key corpus, collect the corpus related to the data element itself, including policies, standards, laws and regulations, dictionaries, important documents, etc., to build a knowledge base for the data element.

[0070] Step S20, extracting synonyms of the original corpus from the relevant key corpus;

[0071] In this embodiment, the synonyms of the original corpus are extracted from the relevant key corpus by using the natural language processing (NLP) technology.

[0072] Step S30, respectively calculating the similarity between the original corpus and each of the synonyms;

[0073] In this embodiment, in order to improve the latitude considered when calculating similarity, one or more similarity measurement methods are selected to calculate the similarity between the original corpus and the synonym candidates, and then the similarity scores between the original corpus and each synonym candidate are calculated separately according to the selected similarity measurement methods.

[0074] Step S40, for each of the synonyms, determining the weight of the synonym, and multiplying the similarity by the weight to obtain a final result value;

[0075] In this embodiment, a weight is assigned to each synonym, and the weight can be determined based on the frequency of word co-occurrence, domain knowledge, and other relevant information. The final result value of each synonym is obtained by multiplying the similarity by the weight.

[0076] Step S50: taking the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

[0077] In this embodiment, the final result values ​​of all synonyms are compared, and the synonym with the highest score is selected as the standardized name of the target data element.

[0078] Furthermore, in a feasible implementation manner, after the above step S50, the method may further include step S60:

[0079] Step S60: When there are at least two target synonyms, a standardized name of the target data element is selected from each of the target synonyms by using a preset expert rule model.

[0080] It should be noted that, in this embodiment, the expert rule module is a set of rules formulated based on the knowledge and experience in the field. For example, it can be stipulated that terms with high frequency of occurrence and commonly used in the industry have a higher priority.

[0081] In this embodiment, when there are multiple target synonyms with the highest similarity scores, an expert rule model is needed to assist in decision-making. Target synonyms with similar scores are evaluated according to the preset expert rule model, and the most appropriate synonym is selected as the standardized name of the target data element based on the evaluation results of the expert rule model.

[0082] In this embodiment, the present application calculates the degree of correlation between synonyms in the relevant key corpus and the target data element by using the weighted value of similarity and weight in the relevant key corpus of the target data element, and uses the synonyms with the highest degree of correlation to perform name standardization on the data element. Compared with the conventional method of performing name standardization on the data element by similarity, the present application also calculates the weight corresponding to each synonym, and then uses the weight to perform weighted calculation on the similarity, thereby further improving the accuracy of the correlation between the synonyms and the original corpus, thereby improving the accuracy of the data element name standardization.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. On this basis, step S20 can include steps S21~S23:

[0084] Step S21, splitting the relevant key corpus into text units and obtaining a context window;

[0085] In this embodiment, when extracting synonyms, the relevant key corpus is first split into smaller text units, which can be words or phrases. For example, if the relevant key corpus contains the sentence "Today's temperature is very high, very hot", it can be split into text units such as "today", "temperature", "very high", and "very hot". Then, a context window size is set, such as 5 words before and after the target word. The size of the context window can be adjusted according to the actual situation. A larger window can capture a wider range of context information, but may introduce more noise.

[0086] Step S22, counting the frequencies of co-occurrence of each of the text units in the relevant key corpus and the original corpus in the same context window;

[0087] In this embodiment, the frequency of co-occurrence of each text unit and the original corpus in the same context window is counted, and then the specific situation of co-occurrence of each text unit and the original corpus is recorded, including the number of co-occurrences and the context.

[0088] Step S23: taking the target text unit whose frequency in each of the text units exceeds a preset frequency threshold as a synonym of the original corpus.

[0089] In this embodiment, a frequency threshold is set according to the specific situation, for example, 3 times, and then the text units whose co-occurrence frequency exceeds the threshold are used as synonym candidates of the original corpus, so that multiple synonyms of the original corpus can be obtained.

[0090] Specifically, when extracting synonyms, we can use the method of vocabulary co-occurrence, that is, by analyzing the frequency of occurrence and co-occurrence relationship of words in the corpus, we can extract synonyms and similar words. For example, we can use part-of-speech taggers and N-gram models to find frequently co-occurring words; or we can use a dictionary-based method: use dictionary tools such as WordNet or Roget's Thesaurus to find synonyms and similar words. In addition, we can also use the synonym or antonym list in the dictionary for extraction, and form the synonym vocabulary N (n1, n2, n3, ...) of the data element in this link.

[0091] Furthermore, in a feasible implementation manner, before the above step S21, the method may further include steps A10 to A40:

[0092] Step A10, extracting pronouns from the relevant key corpus;

[0093] In this embodiment, the pronouns in the relevant key corpus are identified, such as demonstrative pronouns: this, that, etc., and the position of each pronoun in the text is recorded for subsequent processing.

[0094] Step A20, searching for multiple candidate nouns in the text preceding the pronoun;

[0095] In this embodiment, the text preceding the pronoun is searched for nouns that may be referred to by the pronoun, and the found candidate nouns are recorded. These nouns may be the objects referred to by the pronoun.

[0096] Step A30, determining the target candidate noun corresponding to the pronoun from among the candidate nouns by using a preset syntactic analysis structure tool;

[0097] In this embodiment, a preset syntactic analysis structure tool (such as a syntactic analyzer in a natural language processing library) is used to analyze the text structure, and based on the result of the syntactic analysis, the noun that is most likely to be referred to by the pronoun is determined.

[0098] Step A40: Replace the pronoun in the relevant key corpus with the target candidate noun.

[0099] In this embodiment, the pronoun is replaced with the target candidate noun to eliminate the ambiguity caused by the pronoun, update the relevant key corpus, and use the replaced text for subsequent processing.

[0100] Specifically, before extracting synonyms, it is necessary to identify the pronouns in the relevant key corpus. In addition to "this", "that", "it", there are also abbreviations of the word or approximate pronouns. The method of extracting pronouns can be to match common pronouns, such as "this", "that", "it", etc. by writing rules. It can be based on information such as vocabulary, grammar, and context. For example, when "this" appears in a sentence, you can check whether a noun appears in front of it. If so, "this" may be a pronoun; or based on syntactic analysis: through syntactic analysis, you can find the pronouns in the sentence. You can use a syntactic analyzer or dependency syntactic analysis to analyze the structure of the sentence and find the relationship between pronouns and other words. In this link, a synonym vocabulary D (d1, d2, d3, ...) of the data element is formed.

[0101] In this embodiment, the present application uses NLP technology to extract synonyms or pronouns of the data element from policies, standards, laws and regulations, dictionaries, important documents and other corpora related to the target data element, and forms a synonym library through semantic similarity analysis. There is no need for an existing domain standard vocabulary as a prerequisite, thereby improving the versatility of the data element name standardization method.

[0102] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those of the first embodiment and / or the second embodiment can be referred to the above description, and will not be described in detail later. On this basis, the above step S30 can include steps S31 to S35:

[0103] Step S31, calculating the word form similarity between the original corpus and each of the synonyms based on the word form of the original corpus and the word form of each of the synonyms;

[0104] In this embodiment, a word form similarity calculation method (such as Levenshtein distance, Jaro-Winkler distance, etc.) is used to calculate the word form similarity between the original corpus and each synonym, and the word form similarity score between each synonym and the original corpus is recorded.

[0105] Step S32, mapping the original corpus and each of the synonyms into a vector space to obtain an original corpus vector and multiple synonym vectors;

[0106] In this embodiment, word vectorization technology (such as Word2Vec, GloVe, etc.) is used to map the original corpus and each synonym into a vector space to obtain the original corpus vector and each synonym vector.

[0107] Step S33, respectively calculating the cosine similarity between the original corpus vector and each of the synonym vectors;

[0108] In this embodiment, the cosine similarity is used to calculate the similarity between the original corpus vector and each synonym vector, and the cosine similarity score between each synonym and the original corpus is recorded.

[0109] Step S34, counting the number of times each of the synonyms appears in the same sentence with the original corpus;

[0110] In this embodiment, the number of times each synonym and the original corpus appear in the same sentence is counted, and the number of co-occurrences between each synonym and the original corpus is recorded.

[0111] Step S35: for each of the synonyms, the largest one among the word form similarity, cosine similarity and the number of times corresponding to the synonym is taken as the similarity between the synonym and the original corpus.

[0112] In this embodiment, for each synonym, the largest value among word form similarity, cosine similarity and co-occurrence times is selected as the similarity between the synonym and the original corpus, and the final similarity score between each synonym and the original corpus is recorded.

[0113] As an example, when calculating similarity, please refer to Figure 2, the extracted synonyms or pronouns, such as N (n1, n2, n3, ...), D (d1, d2, d3, ...) are unified to form an approximate word library, and the selected data element original corpus O and the approximate word library are calculated based on word form, word meaning and pragmatic statistics to obtain three values, and the maximum value is selected as the similarity value and then sorted. Specifically, the similarity is first calculated based on word form. This method uses the writing form of Chinese words to calculate the similarity. For example, the similarity of two Chinese characters can be evaluated by calculating the number of strokes, stroke order, components, semantic information, etc. For example, an On1-1 can be obtained through the "stroke sorting" function of Excel. Then the similarity is calculated based on word meaning, and the semantic information of Chinese words is used to calculate the similarity. For example, word embeddings technology, such as Word2Vec, GloVe, etc., can be used to map Chinese words into vector space, and their similarity can be evaluated by calculating the cosine similarity between vectors.

[0114] The specific calculation method is as follows:

[0115] Set the original data element as vector x, the synonyms to be compared as vector y, and use the cosine metric as the similarity function:

[0116] ;

[0117] Where ||x|| is a vector The Euclidean norm of is defined as , which is the length of vector x, and the same goes for ||y||.

[0118] Therefore, the similarity On1-2 between the two can be calculated through cosine similarity.

[0119] Then, similarity is calculated based on pragmatic statistics, using the frequency of use of Chinese words and contextual information to calculate similarity. For example, the similarity of two Chinese words can be evaluated by calculating the number of times they appear in the corpus, the number of times they appear in the same sentence, etc., to obtain an On1-3. Then, On1-1, On1-2, and On1-3 are sorted, and the maximum value is taken as On1. Finally, a set of similarity values ​​is obtained after calculation.

[0120] Based on the first to third embodiments of the present application, in the fourth embodiment of the present application, the same or similar contents as those of the first to third embodiments can be referred to the above description, and will not be repeated in the following. On this basis, the step of determining the weight of the synonym in the above step S40 includes steps S41 to S43:

[0121] Step S41, for each of the synonyms, calculating the mutual information between the synonym and the position of the synonym in the relevant key corpus;

[0122] In this embodiment, mutual information (MI) is a measure of the correlation between two random variables. In this embodiment, a synonym and its position in the relevant key corpus are regarded as two random variables, and the mutual information between each synonym and its position in the relevant key corpus is calculated. The higher the mutual information, the stronger the correlation between the synonym and the position.

[0123] Step S42, calculating the penalty factor using a preset penalty factor coefficient;

[0124] In this embodiment, the penalty factor is a coefficient used to reduce the redundancy between synonyms to prevent the selection of too many similar synonyms. The penalty factor is calculated using a preset penalty factor coefficient. The penalty factor is usually calculated based on the similarity or mutual information between synonyms.

[0125] Step S43: subtract the penalty factor from the mutual information to obtain the weight of the synonym.

[0126] In this embodiment, the weight of each synonym can be obtained by subtracting the penalty factor from the mutual information calculated in step 1.

[0127] For details, please refer to Figure 3 The position weight of synonyms or pronouns in the corpus needs to be analyzed by technical means. The position weight of each word in the approximate word library in the corpus, that is, the position paragraph is a key position, and the weight is given. The weight is multiplied by the similarity to get the final score. The weight calculation is mainly based on the MIFS algorithm: feature selection based on mutual information. This algorithm takes into account the correlation and redundancy between features. It can not only select features that are closely related to the target variable, but also remove those features that are useless for predicting the target variable.

[0128] The calculation formula of MIFS algorithm is as follows:

[0129] ;

[0130] Where β is the penalty factor. When β takes different values, the performance of the MIFS algorithm fluctuates greatly. When the value of β is in the range of [0.5,1], the algorithm performance is better. The calculated J(f) is the weight V of the synonym. Finally, please refer to Figure 4 , the similarity value and the approximate vocabulary weight value are multiplied accordingly to obtain the final result, which is sorted and the highest one is used as the standardized name of the data element.

[0131] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the name standardization method of the data elements of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0132] This application also provides a data element name standardization device, please refer to Figure 5 , the data element name standardization device comprises:

[0133] A corpus acquisition module 10 is used to acquire the original corpus of the target data element and the key corpus related to the target data element;

[0134] A synonym extraction module 20, used for extracting synonyms of the original corpus from the relevant key corpus;

[0135] A similarity calculation module 30, used to calculate the similarity between the original corpus and each of the synonyms;

[0136] A weighting module 40 is used to determine the weight of each synonym, and multiply the similarity by the weight to obtain a final result value;

[0137] The standardization module 50 is used to use the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

[0138] Optionally, the synonym extraction module is also used to:

[0139] Splitting the relevant key corpus into text units and obtaining a context window;

[0140] Counting the frequencies of co-occurrence of each of the text units in the relevant key corpus and the original corpus in the same context window;

[0141] The target text unit in each of the text units whose frequency exceeds a preset frequency threshold is used as a synonym of the original corpus.

[0142] Optionally, the synonym extraction module is also used to:

[0143] Extracting pronouns from the relevant key corpus;

[0144] Finding multiple candidate nouns in the text preceding the pronoun;

[0145] Determining a target candidate noun corresponding to the pronoun from among the candidate nouns by using a preset syntactic analysis structure tool;

[0146] The pronoun in the relevant key corpus is replaced by the target candidate noun.

[0147] Optionally, the similarity calculation module is further used for:

[0148] Calculating the word form similarity between the original corpus and each of the synonyms based on the word form of the original corpus and the word form of each of the synonyms;

[0149] Mapping the original corpus and each of the synonyms into a vector space to obtain an original corpus vector and multiple synonym vectors;

[0150] Calculating the cosine similarity between the original corpus vector and each of the synonym vectors respectively;

[0151] Counting the number of times each of the synonyms appears in the same sentence as the original corpus;

[0152] For each of the synonyms, the largest one among the word form similarity, cosine similarity and the number of times corresponding to the synonym is taken as the similarity between the synonym and the original corpus.

[0153] Optionally, the weighting module is also used to:

[0154] For each of the synonyms, calculating the mutual information between the synonym and the position of the synonym in the relevant key corpus;

[0155] Calculate the penalty factor by using the preset penalty factor coefficient;

[0156] The mutual information minus the penalty factor is used to obtain the weight of the synonym.

[0157] Optionally, the data element name standardization device is further used to:

[0158] When there are at least two target synonyms, a standardized name of the target data element is selected from each of the target synonyms by using a preset expert rule model.

[0159] The data element name standardization device provided by the present application adopts the data element name standardization method in the above-mentioned embodiment, which can solve the technical problem of how to improve the accuracy of data element name standardization. Compared with the prior art, the beneficial effects of the data element name standardization device provided by the present application are the same as the beneficial effects of the data element name standardization method provided by the above-mentioned embodiment, and the other technical features in the data element name standardization device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.

[0160] The present application provides a data element name standardization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data element name standardization method in the above-mentioned embodiment one.

[0161] Reference below Figure 6 , which shows a schematic diagram of a structure of a name standardization device for data elements suitable for implementing the embodiment of the present application. The name standardization device for data elements in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The data element name standardization device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0162] like Figure 6As shown, the name standardization device for data elements may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the name standardization device for data elements are also stored in RAM1004. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the name standardization device of the data element to communicate wirelessly or wired with other devices to exchange data. Although the figure shows the name standardization device of the data element with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0163] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0164] The data element name standardization device provided by the present application adopts the data element name standardization method in the above embodiment, which can solve the technical problem of how to improve the accuracy of data element name standardization. Compared with the prior art, the beneficial effects of the data element name standardization device provided by the present application are the same as the beneficial effects of the data element name standardization method provided by the above embodiment, and the other technical features in the data element name standardization device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0165] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0166] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0167] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the method for standardizing the names of data elements in the above-mentioned embodiments.

[0168] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0169] The computer-readable storage medium may be included in the name standardization device for data elements; or may exist independently without being assembled into the name standardization device for data elements.

[0170] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the name standardization device of the data element, the name standardization device of the data element: obtains the original corpus of the target data element and the relevant key corpus of the target data element; extracts the synonyms of the original corpus from the relevant key corpus; respectively calculates the similarity between the original corpus and each of the synonyms; for each of the synonyms, determines the weight of the synonym, and multiplies the similarity by the weight to obtain a final result value; and uses the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

[0171] Computer program code for performing the operation of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0172] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0173] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0174] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned data element name standardization method, and can solve the technical problem of how to improve the accuracy of data element name standardization. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the data element name standardization method provided by the above-mentioned embodiment, and will not be repeated here.

[0175] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned data element name standardization method when executed by a processor.

[0176] The computer program product provided by the present application can solve the technical problem of how to improve the accuracy of data element name standardization. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the data element name standardization method provided by the above embodiment, which will not be repeated here.

[0177] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for standardizing the names of data elements, characterized in that: The method comprises: Acquire the original corpus of the target data element and the key corpus related to the target data element; Extracting pronouns from the relevant key corpus; Finding multiple candidate nouns in the text preceding the pronoun; Determining a target candidate noun corresponding to the pronoun from among the candidate nouns by using a preset syntactic analysis structure tool; Replacing the pronoun in the relevant key corpus with the target candidate noun; Splitting the relevant key corpus into text units and obtaining a context window; Counting the frequencies of co-occurrence of each of the text units in the relevant key corpus and the original corpus in the same context window; Taking the target text unit whose frequency in each of the text units exceeds a preset frequency threshold as a synonym of the original corpus; Calculating the similarity between the original corpus and each of the synonyms respectively; For each of the synonyms, determine the weight of the synonym, and multiply the similarity by the weight to obtain a final result value; The target synonym with the highest final result value among the synonyms is used as the standardized name of the target data element.

2. The method for standardizing the names of data elements according to claim 1, characterized in that: The step of respectively calculating the similarity between the original corpus and each of the synonyms comprises: Calculating the word form similarity between the original corpus and each of the synonyms based on the word form of the original corpus and the word form of each of the synonyms; Mapping the original corpus and each of the synonyms into a vector space to obtain an original corpus vector and multiple synonym vectors; Calculating the cosine similarity between the original corpus vector and each of the synonym vectors respectively; Counting the number of times each of the synonyms appears in the same sentence as the original corpus; For each of the synonyms, the largest one among the word form similarity, cosine similarity and the number of times corresponding to the synonym is taken as the similarity between the synonym and the original corpus.

3. The method for standardizing the names of data elements according to claim 1, characterized in that: The step of determining the weight of the synonyms comprises: For each of the synonyms, calculating the mutual information between the synonym and the position of the synonym in the relevant key corpus; Calculate the penalty factor by using the preset penalty factor coefficient; The mutual information minus the penalty factor is used to obtain the weight of the synonym.

4. The method for standardizing the names of data elements according to claim 1, characterized in that: After the step of using the target synonym with the highest final result value among the synonyms as the standardized name of the target data element, the method further includes: When there are at least two target synonyms, a standardized name of the target data element is selected from each of the target synonyms by using a preset expert rule model.

5. A device for standardizing the name of a data element, characterized in that: The device comprises: A corpus acquisition module, used to acquire the original corpus of the target data element and the key corpus related to the target data element; The synonym extraction module is used to extract the pronouns in the relevant key corpus; search for multiple candidate nouns in the text preceding the pronoun; determine the target candidate noun corresponding to the pronoun in each of the candidate nouns by using a preset syntactic analysis structure tool; replace the pronouns in the relevant key corpus with the target candidate noun; split the relevant key corpus into text units and obtain a context window; count the frequencies of each of the text units in the relevant key corpus and the original corpus in the same context window; and use the target text unit in each of the text units whose frequency exceeds a preset frequency threshold as a synonym of the original corpus; A similarity calculation module, used to calculate the similarity between the original corpus and each of the synonyms; A weighting module, for determining the weight of each synonym, and multiplying the similarity by the weight to obtain a final result value; The standardization module is used to use the target synonym with the highest final result value among the synonyms as the standardized name of the target data element.

6. A device for standardizing the names of data elements, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for standardizing the names of data elements according to any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the data element name standardization method according to any one of claims 1 to 4 are implemented.

8. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for standardizing the names of data elements according to any one of claims 1 to 4 are implemented.

Images