A method, device, and storage medium for generating field annotations and understanding strings
By automatically restoring and translating missing field annotations in the database, the problem of inefficient manual processing is solved, and efficient and accurate field annotation generation is achieved.
Patent Information
- Application Number
- CN202010425699.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-19
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2040-05-19
AI Technical Summary
In the prior art, field names with missing field annotations in the database need to be processed manually, resulting in low efficiency and accuracy.
By determining the English abbreviation string in the field name and automatically restoring and translating the English abbreviation to generate field comments based on the mapping relationship between the English abbreviation and the full spelling of the English, the English abbreviation is automatically restored and translated to generate field comments.
Improves the efficiency of field annotation generation, ensures the accuracy of annotations without relying on manual methods.
Smart Images

Figure CN113688614B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of data processing, and in particular, to a method, device, and storage medium for generating field annotations and understanding strings. Background Art
[0002] With the continuous increase in the amount of user data, more and more users have put forward the need for data standardization, hoping to obtain high-quality data assets. An important processing content in the data standardization process is to complete the annotation of field names in the database.
[0003] Currently, it is necessary to manually complete the annotation of field names in the database, and the efficiency and accuracy of this method are relatively low. Summary of the Invention
[0004] Multiple aspects of this application provide a method, device, and storage medium for generating field annotations and understanding strings, which are used to improve the generation efficiency and accuracy of field annotations.
[0005] An embodiment of this application provides a method for generating field annotations, including:
[0006] Obtain the field name to be processed;
[0007] Determine the English abbreviation string included in the field name;
[0008] Based on the mapping relationship between the English abbreviation and the full English spelling, determine the full English spelling corresponding to the English abbreviation string;
[0009] Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the field name to generate a field annotation for the field name.
[0010] An embodiment of this application also provides a method for understanding strings, including:
[0011] Obtain the string to be understood;
[0012] Determine the English abbreviation string included in the string to be understood;
[0013] Based on the mapping relationship between the English abbreviation and the full English spelling, determine the full English spelling corresponding to the English abbreviation string;
[0014] Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result for the string to be understood.
[0015] An embodiment of this application also provides a computing device, including a memory and a processor;
[0016] The memory is used to store one or more computer instructions;
[0017] The processor is coupled to the memory and is configured to execute the one or more computer instructions for:
[0018] Obtain a field name to be processed;
[0019] Determine an English abbreviation string included in the field name;
[0020] Based on the mapping relationship between the English abbreviation and the full English spelling, determine the full English spelling corresponding to the English abbreviation string;
[0021] Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the field name to generate a field annotation of the field name.
[0022] An embodiment of the present application further provides a computing device, including a memory and a processor;
[0023] The memory is used to store one or more computer instructions;
[0024] The processor is coupled to the memory and is configured to execute the one or more computer instructions for:
[0025] Obtain a string to be understood;
[0026] Determine an English abbreviation string included in the string to be understood;
[0027] Based on the mapping relationship between the English abbreviation and the full English spelling, determine the full English spelling corresponding to the English abbreviation string;
[0028] Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result of the string to be understood.
[0029] An embodiment of the present application further provides a computer-readable storage medium storing computer instructions, which, when executed by one or more processors, cause the one or more processors to execute the foregoing field annotation generation method or the foregoing string understanding method.
[0030] In an embodiment of the present application, for a field name lacking a field annotation, an English abbreviation string included therein can be determined, and the English abbreviation string can be restored to restore the English abbreviation in the field name to the full English spelling. On this basis, an English translation can be performed on the field name to generate a field annotation of the field name. Accordingly, in an embodiment of the present application, there is no need to rely on manual methods to generate field annotations, which can effectively improve the generation efficiency of field annotations, and moreover, by accurately restoring the English abbreviations in the field name, the accuracy of the generated field annotations can be ensured. Description of the Drawings
[0031] The accompanying drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:
[0032] Figure 1 is a flowchart of a method for generating field annotations provided for an exemplary embodiment of the present application;
[0033] Figure 2 is a logic block diagram of a method for generating field annotations provided for an exemplary embodiment of the present application;
[0034] Figure 3 is a flowchart of a method for string understanding provided for another exemplary embodiment of the present application;
[0035] Figure 4 is a schematic structural diagram of a computing device provided for yet another exemplary embodiment of the present application;
[0036] Figure 5 is a schematic structural diagram of another computing device provided for yet another exemplary embodiment of the present application. Detailed Description of the Embodiments
[0037] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.
[0038] Currently, it is necessary to manually supplement the annotations of field names in the database. This method has relatively low efficiency and accuracy. To address these technical problems, an embodiment of the present application provides a solution. One of the basic ideas is: for a field name lacking a field annotation, the English abbreviation strings contained therein can be determined and the English abbreviation strings can be restored to restore the English abbreviations in the field name to English full spellings. On this basis, the field name can be translated into English to generate the field annotation of the field name. Accordingly, in the embodiments of the present application, there is no need to rely on manual methods to generate field annotations, which can effectively improve the generation efficiency of field annotations. Moreover, by accurately restoring the English abbreviations in the field name, the accuracy of the generated field annotations can be ensured.
[0039] The following will describe in detail the technical solutions provided by each embodiment of the present application in conjunction with the drawings.
[0040] Figure 1A flowchart of a method for generating field annotations provided by an exemplary embodiment of the present application. Figure 2 A logic block diagram of a method for generating field annotations provided by an exemplary embodiment of the present application. The method for generating field annotations provided in this embodiment can be executed by a field annotation generation device, which can be implemented as software or a combination of software and hardware, and the field annotation generation device can be integrally provided in a computing device. As Figure 1 and 2 shown, the method includes:
[0041] Step 100: Obtain the field name to be processed;
[0042] Step 101: Determine the English abbreviation string included in the field name;
[0043] Step 102: Based on the mapping relationship between the English abbreviation and the full English spelling, determine the full English spelling corresponding to the English abbreviation string;
[0044] Step 103: Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the field name to generate a field annotation for the field name.
[0045] The method for generating field annotations provided in this embodiment can be applied to scenarios such as databases or spreadsheets where there are fields, and this embodiment does not limit the application scenarios. Taking a database as an example, in most cases, the columns in a data table can be called fields, and each field contains information on a certain topic. Taking a spreadsheet as an example, a column in a spreadsheet can also be used as a field.
[0046] A field corresponds to a field name, and the field name serves as the name of the information contained in the field. For example, in a database scenario, the field name can be the name of attribute type information, such as ID card, gender, etc.
[0047] In practical applications, in order to adapt to requirements such as software code technology, field names are usually composed of characters. Moreover, according to the habits of different technical personnel, the writing methods of field names may not be exactly the same, which results in relatively low readability of field names. Therefore, field names usually come with field annotations to explain the meaning of the field names. For example, if the field name is yhsj, a technical person can add a field annotation "user data" to this field name.
[0048] However, at present, there are still a large number of field names lacking field annotations. For field names lacking field annotations, only technical personnel can understand them manually. Especially for technical personnel who did not participate in the original development process, the processing efficiency and accuracy are relatively low.
[0049] In this embodiment, the field name with missing field annotations can be used as the field name to be processed. As mentioned above, in this embodiment, the source of the field name to be processed is not limited.
[0050] In this embodiment, the English abbreviation string included in the field name can be determined.
[0051] Among them, the English abbreviation string can be a string that cannot be translated into English words.
[0052] In practical applications, there are usually separator characters between English abbreviation strings belonging to different English words. For example, in the field name CUST_NO, the two English abbreviation strings are separated by the separator character "_". In this case, it is considered that the field name contains two English abbreviation strings [CUST] and [NO].
[0053] In this embodiment, for each English abbreviation string in the field name, based on the mapping relationship between the English abbreviation and the full English spelling, the full English spelling corresponding to the English abbreviation string can be determined.
[0054] Among them, the mapping relationship between English abbreviations and full English spellings in different industry fields may not be exactly the same. In this embodiment, according to the target industry field to which the field name belongs, based on the mapping relationship between English abbreviations and full English spellings in the target industry field, the full English spelling corresponding to the English abbreviation string can be determined.
[0055] Accordingly, based on the full English spelling corresponding to the English abbreviation string, the field name can be translated into English to generate the field annotation corresponding to the field name.
[0056] As mentioned above, the field name may contain English abbreviation strings and may also contain other characters, such as full English spelling strings. In this embodiment, the English abbreviation strings in the field name can be replaced with full English spellings, and after combining the full English spellings corresponding to the English abbreviation strings with other characters in the field name, English translation can be performed to generate the field annotation of the field name. Of course, the full English spellings corresponding to the English abbreviation strings can also be directly translated into Chinese phrases, and the translation results can be spliced with the understanding results of other characters in the field name to generate the field annotation of the field name.
[0057] For the English words included in the field name, direct English translation can be performed without performing the aforementioned operation of restoring English abbreviations.
[0058] Accordingly, the field annotation of the field name can be generated.
[0059] In this embodiment, for a field name lacking a field annotation, the English abbreviation string contained therein can be determined, and the English abbreviation string can be restored to restore the English abbreviation in the field name to its full English spelling. On this basis, the field name can be translated into English to generate a field annotation for the field name. Accordingly, in the embodiments of the present application, there is no need to rely on manual methods to generate field annotations, which can effectively improve the generation efficiency of field annotations. Moreover, by accurately restoring the English abbreviations in the field name, the accuracy of the generated field annotations can be ensured.
[0060] In the above or following embodiments, the English abbreviation string can be used as the greatest common divisor sequence, and at least one candidate word matching the greatest common divisor sequence can be determined from the English word library.
[0061] In practical applications, the full English spellings, that is, English words, in different industry fields can be collected to form an English word library for different industry fields.
[0062] Based on the English word library for different industry fields, in this embodiment, when the industry field to which the field name belongs is known, at least one candidate word matching the greatest common divisor sequence can be determined from the English word library corresponding to the target industry field to which the field name belongs, using the English abbreviation string as the greatest common divisor sequence.
[0063] Among them, matching the greatest common divisor sequence means that all the English letters contained in the English abbreviation exist in the English word. Although these English letters may not be consecutive in the English word, the order of these English letters in the English word is the same as the order in the English abbreviation string.
[0064] For example, if the English abbreviation string is cd, for the English word code in the English word library, there are English letters c and d, and the order of c and d in code is the same as the order in the English abbreviation string cd. Therefore, the English word code can be determined as a candidate word for the English abbreviation string cd.
[0065] Accordingly, a candidate word set corresponding to the English abbreviation string can be obtained, and the candidate word set contains at least one candidate word.
[0066] For at least one candidate word in the candidate word set, the probability that the English abbreviation string is the English abbreviation of at least one candidate word can be calculated based on the mapping relationship between the English abbreviation and the full English spelling.
[0067] For this purpose, in this embodiment, the English abbreviation string can be input into an English abbreviation prediction model, and in the English abbreviation prediction model, based on the mapping relationship between the English abbreviation and the full English spelling, the probability that at least one candidate word is abbreviated to the English abbreviation string can be calculated.
[0068] The English abbreviation prediction model can traverse each candidate word in the candidate word set and calculate the conditional probability of the candidate word under the English abbreviation string, that is, the probability that the candidate word is abbreviated to the English abbreviation string.
[0069] Among them, in the English abbreviation prediction model, for different industry fields, the mapping relationship between different English abbreviations and their full English spellings can be learned. Here, the target industry field can be input into the English abbreviation prediction model. In the English abbreviation prediction model, based on the mapping relationship between English abbreviations and their full English spellings in the target industry field to which the field name belongs, the probability that at least one candidate word is abbreviated to the English abbreviation string can be determined.
[0070] In order to enable the English abbreviation prediction model to learn the mapping relationship between English abbreviations and their full English spellings in different industry fields, the English abbreviation prediction model can be trained. The training process can be as follows:
[0071] Obtain a sample data set containing sample English words and sample English abbreviations;
[0072] Mark the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set;
[0073] Input the marked sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between English abbreviations and their full English spellings.
[0074] Among them, web crawler technology etc. can be used to obtain training texts from the network. Also, the English abbreviation dictionary and the English word dictionary can be used as training texts. Moreover, the training texts can be classified according to industry fields, and for different industry fields, the English abbreviation prediction model can be trained using not completely the same training texts.
[0075] Among them, in the process of marking the sample data set, the sample English abbreviations can be encoded to obtain the encoded sequence of the sample English abbreviations. The encoded sequence is used to represent the common factor between the sample English abbreviations and their corresponding sample English words; establish the corresponding relationship between the encoded sequence and the sample English words for the English abbreviation prediction model to learn the mapping relationship between the encoded sequence and the full English spellings.
[0076] In practical applications, the sample English abbreviations can be encoded with 0 and 1, and the obtained encoded sequence will be a 01 sequence. Among them, 1 in the encoded sequence can represent that there is a common factor between the sample English abbreviation at this sequence position and its corresponding sample English word. For example, for the sample English abbreviation cd and its corresponding sample English word code, the sample English abbreviation can be encoded as
[1010] .
[0077] In this way, the English abbreviation prediction model can learn the mapping relationship between the coding sequence and the full English spelling from the labeled sample dataset.
[0078] Accordingly, the English abbreviation prediction model can calculate the probability that at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string based on the mapping relationship between the coding sequence and the full English spelling, as the probability that at least one candidate word is abbreviated to the English abbreviation string.
[0079] In practical applications, it is impossible to predict which coding sequence a candidate word may be abbreviated to. For this reason, multiple coding sequences that a candidate word may be abbreviated to can be determined by exhaustive enumeration or according to preset rules, and in the English abbreviation prediction model, based on the learned mapping relationship between the coding sequence and the full English spelling, the probabilities that the candidate word is abbreviated to the foregoing multiple coding sequences are respectively predicted. The multiple coding sequences include the coding sequence corresponding to the English abbreviation string.
[0080] Of course, this is only exemplary. In this embodiment, it is also possible to specify that the English abbreviation prediction model only calculates the probability that a candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string, and no longer calculates the probability that the candidate word is abbreviated to other coding sequences. This embodiment does not make a limitation in this regard.
[0081] In this embodiment, by using the coding sequence to represent the English abbreviation string, the prediction problem of English abbreviations can be converted into a classification problem, making the prediction efficiency higher, and the prediction results more reasonable and accurate.
[0082] In practical applications, the English abbreviation prediction model can adopt a seq2seq model. Of course, the English abbreviation prediction model can also adopt other types of algorithm models, and this embodiment is not limited thereto.
[0083] So far, based on the English abbreviation prediction model, the probability that each of at least one candidate word in the candidate word set is abbreviated to the English abbreviation string can be determined.
[0084] On this basis, based on the Bayesian hypothesis, according to the probability that at least one candidate word output by the English abbreviation prediction model is abbreviated to the English abbreviation string, the probability that each of at least one candidate word is the full English spelling of the English abbreviation string can be calculated.
[0085] Among them, the calculation process based on the Bayesian hypothesis can be represented as a calculation formula:
[0086]
[0087] Among them, P(word|abbr) represents the probability that the candidate word word is the full English spelling of the English abbreviation string abbr, P(abbr|word) represents the probability that the candidate word word is abbreviated to the English abbreviation string abbr (from the aforementioned English abbreviation prediction model), P(word) represents the frequency of occurrence of the candidate word word, and P(abbr) represents the frequency of occurrence of the English abbreviation string.
[0088] Accordingly, the probability that each of at least one candidate word in the candidate word set is the full English spelling of the English abbreviation string can be calculated.
[0089] On this basis, the candidate words that meet the preset requirements can be used as the full English spellings corresponding to the English abbreviation strings. For example, the candidate word with the highest probability can be used as the full English spelling corresponding to the English abbreviation string.
[0090] In this embodiment, an adapted English word library can be constructed for different industry fields, which makes the process of restoring English abbreviations have a more accurate and reasonable restoration range, and can effectively improve the accuracy of restoring English abbreviations. Moreover, by representing the English abbreviation string in the form of a coding sequence, the prediction problem of English abbreviations can be converted into a classification problem, making the prediction efficiency higher, the prediction results more reasonable and accurate, and thus can also effectively improve the accuracy of restoring English abbreviations.
[0091] In the above or following embodiments, the field name may contain delimiter characters.
[0092] In this embodiment, if the field name contains delimiter characters, the field name can be split into multiple character segments according to the delimiter characters; the character segments that do not belong to English words in the multiple character segments can be determined as English abbreviation strings.
[0093] The delimiter characters in the field name mostly play a semantic segmentation role. For example, the delimiter character "_" in the field name CUST_NO plays a semantic segmentation role, splitting the semantics of the field name into customer and number.
[0094] In this embodiment, the field name can be split according to the delimiter characters, and for each of the multiple character segments obtained after splitting, the understanding results can be generated respectively. On this basis, the understanding results of the multiple character segments can be concatenated to generate the field annotation corresponding to the field name.
[0095] In addition, in this embodiment, for the delimiter characters in the field name, they can be retained in the field annotation of the field name. Of course, they can also be directly deleted and no longer appear in the field annotation. This can be flexibly set according to actual needs or user instructions, etc., and this embodiment does not make any limitations in this regard.
[0096] In this embodiment, segmental understanding of the field name can be achieved, which makes the understanding of the field name more accurate. Especially for field names containing multiple semantics, the mutual influence between different semantics can be avoided, so as to effectively ensure that the multiple semantics it contains obtain the most accurate understanding results and effectively improve the accuracy of the finally generated field annotations.
[0097] In the above or following embodiments, based on the field annotations generated for the field name, the field annotations corresponding to the field name can be supplemented to the database where the field name is located.
[0098] Accordingly, the generated field annotations can be applied to the database to add field annotations to the field names in the database.
[0099] In this embodiment, the association relationship between the field name and the field annotation under the database can also be constructed based on the field annotation corresponding to the field name and the field annotations corresponding to other field names in the database where the field name is located.
[0100] Based on this, the association relationship between the field name and the field annotation can be used as an intermediate medium in the process of accessing the database to ensure that the visitors correctly understand the meanings of the field names in the database.
[0101] In practical applications, the association relationship between the field name and the field annotation can be configured in the relevant data access protocol so that the communication parties can perform data processing on the same understanding basis.
[0102] Of course, the application of the field annotation is not limited to this. In this embodiment, the generated field annotation can also be applied to other processing matters, which will not be enumerated here.
[0103] In the above or following embodiments, in the process of determining the English full spelling corresponding to the English abbreviation string based on the mapping relationship between the English abbreviation and the English full spelling, the English abbreviation dictionary can also be used to identify the English abbreviation string. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, then according to the English abbreviation dictionary, the English full spelling corresponding to the English abbreviation string is determined. There is no need to perform operations such as determining candidate words.
[0104] Among them, the English abbreviation dictionary can be a general dictionary in the industry field. Of course, it can also be a general dictionary certified by other official organizations or groups. This embodiment does not make any limitations on this. The English abbreviation dictionary records the corresponding relationship between the English abbreviation and the English full spelling.
[0105] Accordingly, in this embodiment, it can be pre-determined whether the English abbreviation string appears in the English abbreviation dictionary. If so, the English full spelling can be directly determined according to the English abbreviation dictionary. If not, then based on the English abbreviation string used as the greatest common divisor sequence provided in the foregoing embodiment, at least one candidate word matching the greatest common divisor sequence and other subsequent operations can be determined from the English word library to determine the English full spelling of the English abbreviation string. This can effectively improve the efficiency of restoring English abbreviations.
[0106] Figure 3 FIG. 4 is a schematic flowchart of a string understanding method provided for another exemplary embodiment of the present application. The string understanding method provided in this embodiment can be executed by a string understanding device, which can be implemented as software or a combination of software and hardware, and the string understanding device can be integrally set in a computing device. As Figure 3 shown, the method includes:
[0107] Step 300, obtain the string to be understood;
[0108] Step 301, determine the English abbreviation string included in the string to be understood;
[0109] Step 302, based on the mapping relationship between the English abbreviation and the English full spelling, determine the English full spelling corresponding to the English abbreviation string;
[0110] Step 303, based on the English full spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result of the string to be understood.
[0111] The string understanding method provided in this embodiment can be applied to scenarios for understanding strings with unknown meanings, such as database scenarios, spreadsheet scenarios, chat scenarios, journal translation or reading scenarios, search engine scenarios, e-commerce scenarios, etc. This embodiment does not limit the application scenarios.
[0112] For different application scenarios, the types of the strings to be understood may not be exactly the same. The string to be understood may include one or more of field names in a database, strings in chat content, technical terms, and search keywords. For example, in a database scenario, the string to be understood may be a field name, and in a spreadsheet scenario, the string may be the content in any cell. In other scenarios, the string to be understood may also be a string of characters in a text, or a line of code in a code file, etc.
[0113] Accordingly, in this embodiment, the string to be understood can be any string with an unknown meaning, and this embodiment does not limit the source, specification, type, etc. of the string to be understood.
[0114] The string understanding method provided in this embodiment can restore a string with unknown meaning. For example, in an IM tool, when typing and chatting, it can restore the abbreviated strings in the chat content; in academic journals or professional journals, such as hospital journals, it can restore the abbreviations of professional terms; in search scenarios, it can restore the abbreviations of keywords, such as product keywords in an e-commerce scenario or search keywords in a search engine, etc.
[0115] This embodiment is different from Figure 1 the embodiment shown in that the string to be understood in this embodiment is not limited to the field names in the foregoing embodiments.
[0116] Based on a similar inventive concept, in this embodiment, for technical details, reference can be made to the relevant descriptions in the foregoing embodiments regarding the method for generating the final understanding result. To save space, the technical details will not be elaborated further, but this should not cause a loss to the protection scope of this application.
[0117] Only a few representative embodiments will be described exemplarily below.
[0118] In an optional embodiment, the step of determining the English full spelling corresponding to the English abbreviation string based on the mapping relationship between the English abbreviation and the English full spelling includes:
[0119] Using the English abbreviation string as the greatest common divisor sequence, determine at least one candidate word that matches the greatest common divisor sequence from the English word library;
[0120] Based on the mapping relationship between the English abbreviation and the English full spelling, calculate the probability of each of the at least one candidate word being the English full spelling of the English abbreviation string;
[0121] Take the candidate word whose probability meets the preset requirements as the English full spelling corresponding to the English abbreviation string.
[0122] In an optional embodiment, the step of calculating the probability of each of the at least one candidate word being the English full spelling of the English abbreviation string based on the mapping relationship between the English abbreviation and the English full spelling includes:
[0123] Input the English abbreviation string into the English abbreviation prediction model; and in the English abbreviation prediction model, based on the mapping relationship between the English abbreviation and the English full spelling, calculate the probability that at least one candidate word is abbreviated to the English abbreviation string;
[0124] Based on the Bayesian hypothesis, according to the probability that at least one candidate word output by the English abbreviation prediction model is abbreviated to the English abbreviation string, calculate the probability of each of the at least one candidate word being the English full spelling of the English abbreviation string.
[0125] In an alternative embodiment, the English abbreviation prediction model adopts a seq2seq model.
[0126] In an alternative embodiment, before the step of inputting the English abbreviation string into the English abbreviation prediction model, the following steps are further included:
[0127] Obtain a sample data set including sample English words and sample English abbreviations;
[0128] Mark the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set;
[0129] Input the marked sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between English abbreviations and English full spellings.
[0130] In an alternative embodiment, the step of marking the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set includes:
[0131] Encode the sample English abbreviations to obtain an encoded sequence of the sample English abbreviations, and the encoded sequence is used to represent the common factor between the sample English abbreviations and their corresponding sample English words;
[0132] Establish the corresponding relationship between the encoded sequence and the sample English words for the English abbreviation prediction model to learn the mapping relationship between the encoded sequence and the English full spelling.
[0133] In an alternative embodiment, the step of calculating the probability that at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between the English abbreviation and the English full spelling includes:
[0134] Based on the mapping relationship between the encoded sequence and the English full spelling, calculate the probability that at least one candidate word is abbreviated to the encoded sequence corresponding to the English abbreviation string as the probability that at least one candidate word is abbreviated to the English abbreviation string.
[0135] In an alternative embodiment, the step of determining at least one candidate word matching the maximum common factor sequence from the English word library with the English abbreviation string as the maximum common factor sequence includes:
[0136] Determine the target industry field where the string to be processed is located;
[0137] With the English abbreviation string as the maximum common factor sequence, determine at least one candidate word matching the maximum common factor sequence from the English word library corresponding to the target industry field.
[0138] In an alternative embodiment, the step of inputting the English abbreviation string into the English abbreviation prediction model includes:
[0139] Determine the target industry field where the string to be processed is located;
[0140] The English abbreviation string and the target industry field are input into the English abbreviation prediction model;
[0141] Based on the mapping relationship between the English abbreviation and the full English spelling, calculate the probability that at least one candidate word is abbreviated to the English abbreviation string, including:
[0142] Based on the mapping relationship between the English abbreviation and the full English spelling in the target industry field, calculate the probability that at least one candidate word is abbreviated to the English abbreviation string.
[0143] In an alternative embodiment, the step of determining the English abbreviation string included in the string to be processed includes:
[0144] If the string to be processed contains a delimiter character, split the string to be processed into multiple character segments according to the delimiter character;
[0145] Determine the character segments that do not belong to English words among the multiple character segments as the English abbreviation string.
[0146] In an alternative embodiment, the method further includes:
[0147] Supplement the field annotation corresponding to the string to be processed to the database where the string to be processed is located; or
[0148] Based on the field annotation corresponding to the string to be processed and the field annotations corresponding to other strings to be processed in the database where the string to be processed is located, construct the association relationship between the strings to be processed and the field annotations under the database.
[0149] In an alternative embodiment, before the step of using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from the English word library, it further includes:
[0150] Use the English abbreviation dictionary to identify the English abbreviation string. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, then according to the English abbreviation dictionary, determine the full English spelling corresponding to the English abbreviation string;
[0151] If it is determined that the English abbreviation string does not exist in the English abbreviation dictionary, then perform the operation of using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from the English word library.
[0152] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices as the execution subject. For example, the execution subject of steps 100 to 102 can be device A; for another example, the execution subject of steps 100 and 101 can be device A, and the execution subject of step 102 can be device B; and so on.
[0153] In addition, in some processes described in the above embodiments and the accompanying drawings, a plurality of operations appear in a specific order. However, it should be clearly understood that these operations can be executed not in the order in which they appear in this article or in parallel. The operation numbers such as 100, 102, etc. are only used to distinguish different operations, and the numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in order or in parallel.
[0154] Figure 4 The following is a schematic structural diagram of a computing device provided by another exemplary embodiment of the present application. As Figure 4 shown, the computing device includes: a memory 40 and a processor 41.
[0155] The processor 41 is coupled to the memory 40 and is configured to execute a computer program in the memory 40 for:
[0156] Obtain the field name to be processed;
[0157] Determine the English abbreviation string included in the field name;
[0158] Based on the mapping relationship between the English abbreviation and the English full spelling, determine the English full spelling corresponding to the English abbreviation string;
[0159] Based on the English full spelling corresponding to the English abbreviation string, perform an English translation on the field name to generate a field annotation of the field name.
[0160] In an optional embodiment, when the processor determines the English full spelling corresponding to the English abbreviation string based on the mapping relationship between the English abbreviation and the English full spelling, it is configured to:
[0161] Use the English abbreviation string as the greatest common divisor sequence, and determine at least one candidate word that matches the greatest common divisor sequence from the English word library;
[0162] Based on the mapping relationship between the English abbreviation and the English full spelling, calculate the probability of each of the at least one candidate word as the English full spelling of the English abbreviation string;
[0163] Take the candidate word whose probability meets the preset requirements as the English full spelling corresponding to the English abbreviation string.
[0164] In an alternative embodiment, when calculating the probability of each of at least one candidate word being the full English spelling of an English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling, the processor is configured to:
[0165] Input the English abbreviation string into an English abbreviation prediction model; and in the English abbreviation prediction model, calculate the probability that at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling.
[0166] Based on the Bayesian hypothesis, calculate the probability of each of at least one candidate word being the full English spelling of the English abbreviation string according to the probability that at least one candidate word output by the English abbreviation prediction model is abbreviated to the English abbreviation string.
[0167] In an alternative embodiment, the English abbreviation prediction model employs a seq2seq model.
[0168] In an alternative embodiment, before inputting the English abbreviation string into the English abbreviation prediction model, the processor is further configured to:
[0169] Obtain a sample data set including sample English words and sample English abbreviations;
[0170] Mark the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set;
[0171] Input the marked sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between the English abbreviation and the full English spelling.
[0172] In an alternative embodiment, when marking the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set, the processor is configured to:
[0173] Encode the sample English abbreviation to obtain an encoded sequence of the sample English abbreviation, and the encoded sequence is used to represent the common factor between the sample English abbreviation and its corresponding sample English word;
[0174] Establish the corresponding relationship between the encoded sequence and the sample English word for the English abbreviation prediction model to learn the mapping relationship between the encoded sequence and the full English spelling.
[0175] In an alternative embodiment, when calculating the probability that at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling, the processor is configured to:
[0176] Calculate the probability of the encoded sequence corresponding to the English abbreviation string abbreviated by at least one candidate word based on the mapping relationship between the encoded sequence and the full English spelling as the probability that at least one candidate word is abbreviated to the English abbreviation string.
[0177] In an alternative embodiment, when the processor determines at least one candidate word that matches the greatest common divisor sequence from an English word library using the English abbreviation string as the greatest common divisor sequence, it is used for:
[0178] Determine the target industry field where the field name is located;
[0179] Using the English abbreviation string as the greatest common divisor sequence, determine at least one candidate word that matches the greatest common divisor sequence from the English word library corresponding to the target industry field.
[0180] In an alternative embodiment, when the processor inputs the English abbreviation string into the English abbreviation prediction model, it is used for:
[0181] Determine the target industry field where the field name is located;
[0182] Input the English abbreviation string and the target industry field into the English abbreviation prediction model;
[0183] When calculating the probability that at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between English abbreviations and full English spellings, it is used for:
[0184] Calculate the probability that at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between English abbreviations and full English spellings in the target industry field.
[0185] In an alternative embodiment, when the processor determines the English abbreviation string included in the field name, it is used for:
[0186] If the field name contains a delimiter character, split the field name into multiple character segments according to the delimiter character;
[0187] Determine the character segments that do not belong to English words among the multiple character segments as the English abbreviation string.
[0188] In an alternative embodiment, the processor is further used for:
[0189] Supplement the field annotation corresponding to the field name to the database where the field name is located; or
[0190] Based on the field annotation corresponding to the field name and the field annotations corresponding to other field names in the database where the field name is located, construct the association relationship between the field name and the field annotation under the database.
[0191] In an alternative embodiment, before the processor determines at least one candidate word that matches the greatest common divisor sequence from the English word library using the English abbreviation string as the greatest common divisor sequence, it is further used for:
[0192] Identify the English abbreviation string using an English abbreviation dictionary. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, then determine the corresponding full English spelling of the English abbreviation string according to the English abbreviation dictionary.
[0193] If it is determined that the English abbreviation string does not exist in the English abbreviation dictionary, then perform the operation of using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from the English word library.
[0194] It should be noted that for the technical details in the above embodiments of the computing device, reference can be made to the relevant descriptions in the embodiments of the foregoing field annotation generation method. To save space, they will not be elaborated here, but this should not cause loss of the protection scope of this application.
[0195] Furthermore, as Figure 4 shown, the computing device further includes: other components such as a communication component 42 and a power supply component 43. Figure 4 Only some components are schematically shown in Figure 4 and it does not mean that the computing device only includes
[0196] the components shown.
[0197] Figure 5 This is a schematic structural diagram of another computing device provided in another embodiment of this application. As Figure 5 shown, the computing device includes: a memory 50 and a processor 51.
[0198] The processor 51 is coupled to the memory 50 and is configured to execute the computer program in the memory 50 for:
[0199] Obtain the string to be understood;
[0200] Determine the English abbreviation string included in the string to be understood;
[0201] Based on the mapping relationship between the English abbreviation and the full English spelling, determine the corresponding full English spelling of the English abbreviation string;
[0202] Based on the full English spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result of the string to be understood.
[0203] In an optional embodiment, when the processor 51 determines the corresponding full English spelling of the English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling, it is used for:
[0204] Using an English abbreviation string as the greatest common factor sequence, determine at least one candidate word that matches the greatest common factor sequence from an English word library;
[0205] Based on the mapping relationship between English abbreviations and full English spellings, calculate the probability that each of the at least one candidate word is the full English spelling of the English abbreviation string;
[0206] Use the candidate words whose probabilities meet the preset requirements as the full English spellings corresponding to the English abbreviation string.
[0207] In an alternative embodiment, when the processor 51 calculates the probability that each of the at least one candidate word is the full English spelling of the English abbreviation string based on the mapping relationship between English abbreviations and full English spellings, it is used for:
[0208] Input the English abbreviation string into an English abbreviation prediction model; and in the English abbreviation prediction model, calculate the probability that each of the at least one candidate word is abbreviated to the English abbreviation string based on the mapping relationship between English abbreviations and full English spellings;
[0209] Based on the Bayesian hypothesis, calculate the probability that each of the at least one candidate word is the full English spelling of the English abbreviation string according to the probability that each of the at least one candidate word output by the English abbreviation prediction model is abbreviated to the English abbreviation string.
[0210] In an alternative embodiment, the English abbreviation prediction model uses a seq2seq model.
[0211] In an alternative embodiment, before the processor 51 inputs the English abbreviation string into the English abbreviation prediction model, it is also used for:
[0212] Obtain a sample data set containing sample English words and sample English abbreviations;
[0213] Mark the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set;
[0214] Input the marked sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between English abbreviations and full English spellings.
[0215] In an alternative embodiment, when the processor 51 marks the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set, it is used for:
[0216] Encode the sample English abbreviations to obtain an encoded sequence of the sample English abbreviations, and the encoded sequence is used to represent the common factor between the sample English abbreviations and their corresponding sample English words;
[0217] Establish a correspondence between the coding sequence and the sample English words for the English abbreviation prediction model to learn the mapping relationship between the coding sequence and the full English spelling.
[0218] In an alternative embodiment, when the processor 51 calculates the probability that at least one candidate word is abbreviated to an English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling, it is used for:
[0219] Calculate the probability of the coding sequence corresponding to at least one candidate word abbreviated to an English abbreviation string based on the mapping relationship between the coding sequence and the full English spelling, as the probability that at least one candidate word is abbreviated to an English abbreviation string.
[0220] In an alternative embodiment, when the processor 51 determines at least one candidate word that matches the greatest common divisor sequence from the English word library with the English abbreviation string as the greatest common divisor sequence, it is used for:
[0221] Determine the target industry field where the string to be processed is located;
[0222] With the English abbreviation string as the greatest common divisor sequence, determine at least one candidate word that matches the greatest common divisor sequence from the English word library corresponding to the target industry field.
[0223] In an alternative embodiment, when the processor 51 inputs the English abbreviation string into the English abbreviation prediction model, it is used for:
[0224] Determine the target industry field where the string to be processed is located;
[0225] Input the English abbreviation string and the target industry field into the English abbreviation prediction model;
[0226] When calculating the probability that at least one candidate word is abbreviated to an English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling, it is used for:
[0227] Calculate the probability that at least one candidate word is abbreviated to an English abbreviation string based on the mapping relationship between the English abbreviation and the full English spelling in the target industry field.
[0228] In an alternative embodiment, when the processor 51 determines the English abbreviation string included in the string to be processed, it is used for:
[0229] If the string to be processed contains delimiter characters, split the string to be processed into multiple character segments according to the delimiter characters;
[0230] Determine the character segments that do not belong to English words among the multiple character segments as the English abbreviation string.
[0231] In an alternative embodiment, the method is also used for:
[0232] Supplement the field annotations corresponding to the string to be processed to the database where the string to be processed is located; or
[0233] Based on the field annotations corresponding to the string to be processed and the field annotations corresponding to other strings to be processed in the database where the string to be processed is located, construct the association relationship between the strings to be processed and the field annotations under the database.
[0234] In an optional embodiment, before the processor 51 determines at least one candidate word that matches the greatest common factor sequence from the English word library using the English abbreviation string as the greatest common factor sequence, it is further configured to:[[]]
[0235] Use the English abbreviation dictionary to identify the English abbreviation string. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, then according to the English abbreviation dictionary, determine the full English spelling corresponding to the English abbreviation string;
[0236] If it is determined that the English abbreviation string does not exist in the English abbreviation dictionary, then perform the operation of determining at least one candidate word that matches the greatest common factor sequence from the English word library using the English abbreviation string as the greatest common factor sequence.
[0237] It should be noted that for the technical details in the above embodiments of the computing device, reference can be made to the relevant descriptions in the embodiments of the foregoing string understanding method. To save space, they will not be elaborated here, but this should not cause loss of the protection scope of this application.
[0238] Further, as Figure 5 shown, the computing device further includes: a communication component 52, a power supply component 53, and other components. Figure 5 Only some components are schematically shown in Figure 5 and it does not mean that the computing device only includes
[0239] the components shown in
[0240] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed, it can implement the steps executable by the computing device in the above method embodiments. Figure 4 and Figure 5The memory therein is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, pictures, videos, etc. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disks or optical discs.
[0241] Wherein, Figure 4 and Figure 5 The communication component therein is configured to facilitate communication in a wired or wireless manner between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0242] Wherein, Figure 4 and Figure 5 The power component therein provides power for various components of the device where the power component is located. The power component can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device where the power component is located.
[0243] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.
[0244] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a means for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in one block or multiple blocks.
[0245] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in one block or multiple blocks.
[0246] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in one block or multiple blocks.
[0247] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.
[0248] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.
[0249] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media such as modulated data signals and carrier waves.
[0250] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.
[0251] The above are only examples of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of the claims of the present application.
Claims
1. A method for generating field annotations, characterized in that, Including: Obtain the field name to be processed; Determine the English abbreviation string included in the field name; Using the English abbreviation string as the greatest common divisor sequence, determine at least one candidate word that matches the greatest common divisor sequence from the English word library; Using an English abbreviation prediction model, based on the mapping relationship between the coding sequence and the English full spelling, calculate the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string; Based on the Bayesian hypothesis, according to the probability that the at least one candidate word output by the English abbreviation prediction model is abbreviated to the coding sequence corresponding to the English abbreviation string, calculate the probability that each of the at least one candidate word is the English full spelling of the English abbreviation string, where the coding sequence is used to represent the common factor between the English abbreviation and its corresponding English word; Take the candidate word with the highest probability as the English full spelling corresponding to the English abbreviation string; Based on the English full spelling corresponding to the English abbreviation string, perform an English translation on the field name to generate a field annotation for the field name.
2. The method according to claim 1, wherein The English abbreviation prediction model uses a seq2seq model.
3. The method according to claim 1, wherein Before inputting the English abbreviation string into the English abbreviation prediction model, it further includes: Obtain a sample data set including sample English words and sample English abbreviations; Mark the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set; Input the marked sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between the English abbreviation and the English full spelling.
4. The method according to claim 3, wherein The marking the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set includes: Encode the sample English abbreviation to obtain the coding sequence of the sample English abbreviation, where the coding sequence is used to represent the common factor between the sample English abbreviation and its corresponding sample English word; Establish a corresponding relationship between the coding sequence and the sample English word for the English abbreviation prediction model to learn the mapping relationship between the coding sequence and the English full spelling.
5. The method according to claim 1, wherein The using the English abbreviation string as the greatest common divisor sequence, determining at least one candidate word that matches the greatest common divisor sequence from the English word library includes: Determine the target industry field where the field name is located; Using the English abbreviation string as the greatest common divisor sequence, determine at least one candidate word that matches the greatest common divisor sequence from the English word library corresponding to the target industry field.
6. The method according to claim 1, characterized in that It also includes: Determine the target industry field where the field name is located; Input the English abbreviation string and the target industry field into the English abbreviation prediction model; The calculating the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string based on the mapping relationship between the coding sequence and the English full spelling includes: Based on the mapping relationship between the coding sequence and the English full spelling in the target industry field, calculate the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string.
7. The method according to claim 1, wherein Determining the English abbreviation string included in the field name includes: If the field name includes a delimiter character, splitting the field name into multiple character segments according to the delimiter character; Determining the character segments that do not belong to English words among the multiple character segments as the English abbreviation string.
8. The method according to claim 1, wherein It further includes: Supplementary the field annotation corresponding to the field name to the database where the field name is located; Or Based on the field annotation corresponding to the field name and the field annotations corresponding to other field names in the database where the field name is located, constructing the association relationship between the field name and the field annotation under the database.
9. The method according to claim 1, wherein Before determining at least one candidate word that matches the maximum common factor sequence from the English word library with the English abbreviation string as the maximum common factor sequence, it further includes: Using an English abbreviation dictionary to identify the English abbreviation string. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, then determining the English full spelling corresponding to the English abbreviation string according to the English abbreviation dictionary; If it is determined that the English abbreviation string does not exist in the English abbreviation dictionary, then performing the operation of determining at least one candidate word that matches the maximum common factor sequence from the English word library with the English abbreviation string as the maximum common factor sequence.
10. A computing device, characterized in that, It includes a memory and a processor; The memory is used to store one or more computer instructions; The processor is coupled to the memory and is used to execute the one or more computer instructions for: Obtaining the field name to be processed; Determining the English abbreviation string included in the field name; Using the English abbreviation string as the maximum common factor sequence, determining at least one candidate word that matches the maximum common factor sequence from the English word library; Using an English abbreviation prediction model, calculating the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string based on the mapping relationship between the coding sequence and the English full spelling; Based on the Bayesian hypothesis, calculating the probability that each of the at least one candidate word is the English full spelling of the English abbreviation string according to the probability that the at least one candidate word output by the English abbreviation prediction model is abbreviated to the coding sequence corresponding to the English abbreviation string, where the coding sequence is used to represent the common factor between the English abbreviation and its corresponding English word; Taking the candidate word with the maximum probability as the English full spelling corresponding to the English abbreviation string; Based on the English full spelling corresponding to the English abbreviation string, performing an English translation on the field name to generate the field annotation of the field name.
11. The device according to claim 10, characterized in that, The English abbreviation prediction model adopts a seq2seq model.
12. The device according to claim 10, characterized in that, Before the processor inputs the English abbreviation string into the English abbreviation prediction model, it is further used to: Obtaining a sample data set including sample English words and sample English abbreviations; Annotating the corresponding relationship between the sample English words and the sample English abbreviations in the sample data set; Inputting the annotated sample data set into the English abbreviation prediction model for the English abbreviation prediction model to learn the mapping relationship between the English abbreviation and the English full spelling.
13. The device according to claim 12, characterized in that, When labeling the correspondence between sample English words and sample English abbreviations in the sample dataset, the processor is used to: Encode the sample English abbreviation to obtain an encoded sequence of the sample English abbreviation, where the encoded sequence is used to represent the common factor between the sample English abbreviation and its corresponding sample English word; Establish a correspondence between the encoded sequence and the sample English word for the English abbreviation prediction model to learn the mapping relationship between the encoded sequence and the full English spelling.
14. The device according to claim 10, characterized in that, When using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from an English word library, the processor is used to: Determine the target industry field where the field name is located; Use the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from the English word library corresponding to the target industry field.
15. The device according to claim 10, characterized in that, The processor is further used to: Determine the target industry field where the field name is located; Input the English abbreviation string and the target industry field into the English abbreviation prediction model; When calculating the probability that the at least one candidate word is abbreviated to the encoded sequence corresponding to the English abbreviation string based on the mapping relationship between the encoded sequence and the full English spelling, the processor is used to: Calculate the probability that the at least one candidate word is abbreviated to the encoded sequence corresponding to the English abbreviation string based on the mapping relationship between the encoded sequence and the full English spelling in the target industry field.
16. The device according to claim 10, characterized in that, When determining the English abbreviation string included in the field name, the processor is used to: If the field name contains a delimiter character, split the field name into multiple character segments according to the delimiter character; Determine the character segments that do not belong to English words among the multiple character segments as the English abbreviation string.
17. The device according to claim 10, characterized in that, The processor is further used to: Supplement the field annotation corresponding to the field name to the database where the field name is located; or Based on the field annotation corresponding to the field name and the field annotations corresponding to other field names in the database where the field name is located, construct the association relationship between the field names and the field annotations under the database.
18. The device according to claim 10, characterized in that, Before using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from an English word library, the processor is further used to: Use an English abbreviation dictionary to identify the English abbreviation string. If it is determined that the English abbreviation string exists in the English abbreviation dictionary, determine the full English spelling corresponding to the English abbreviation string according to the English abbreviation dictionary; If it is determined that the English abbreviation string does not exist in the English abbreviation dictionary, perform the operation of using the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from an English word library.
19. A method for string understanding, characterized in that, It includes: Obtain the string to be understood; Determine the English abbreviation string included in the string to be understood; Use the English abbreviation string as the greatest common factor sequence to determine at least one candidate word that matches the greatest common factor sequence from an English word library; Using an English abbreviation prediction model, based on the mapping relationship between the coding sequence and the English full spelling, calculate the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string; Based on the Bayesian hypothesis, according to the probability that the at least one candidate word output by the English abbreviation prediction model is abbreviated to the coding sequence corresponding to the English abbreviation string, calculate the probability that each of the at least one candidate word is the English full spelling of the English abbreviation string, wherein the coding sequence is used to represent the common factor between the English abbreviation and its corresponding English word; Take the candidate word with the highest probability as the English full spelling corresponding to the English abbreviation string; Based on the English full spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result of the string to be understood.
20. The method according to claim 19, wherein The string to be understood includes one or more of: field names in a database, strings in chat content, technical terms, and search keywords.
21. A computing device, characterized in that, Includes a memory and a processor; The memory is used to store one or more computer instructions; The processor is coupled to the memory and is used to execute the one or more computer instructions for: Obtain the string to be understood; Determine the English abbreviation string included in the string to be understood; Using the English abbreviation string as the greatest common factor sequence, determine at least one candidate word that matches the greatest common factor sequence from an English word library; Using an English abbreviation prediction model, based on the mapping relationship between the coding sequence and the English full spelling, calculate the probability that the at least one candidate word is abbreviated to the coding sequence corresponding to the English abbreviation string; Based on the Bayesian hypothesis, according to the probability that the at least one candidate word output by the English abbreviation prediction model is abbreviated to the coding sequence corresponding to the English abbreviation string, calculate the probability that each of the at least one candidate word is the English full spelling of the English abbreviation string, wherein the coding sequence is used to represent the common factor between the English abbreviation and its corresponding English word; Take the candidate word with the highest probability as the English full spelling corresponding to the English abbreviation string; Based on the English full spelling corresponding to the English abbreviation string, perform an English translation on the string to be understood to generate an understanding result of the string to be understood.
22. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed by one or more processors, cause the one or more processors to execute the field annotation generation method according to any one of claims 1-9 or the string understanding method according to any one of claims 19-20.
Citation Information
Patent Citations
Table name and field name intelligent completion method based on NLP technology
CN110413972A
Acronym Extraction
US20120109974A1