Data table management method, device, and equipment and storage medium
By scanning and detecting data table fields and performing semantic similarity matching, the problem of inconsistent table fields caused by different developers' abbreviation habits was solved, thus achieving standardized management and efficient maintenance of data tables.
Patent Information
- Application Number
- CN202010901626.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-31
- Publication Date
- 2025-12-30
- Estimated Expiration
- 2040-08-31
AI Technical Summary
Because different developers have different habits of abbreviating table fields, the abbreviations of the same nouns are inconsistent, which affects the efficiency of the data platform in managing data tables.
By scanning and detecting the fields of the data table, the fields to be mapped are determined, and semantic similarity is matched with a pre-built root word library. If a match is found, the English root word is called; otherwise, the root word is mapped according to the mapping rules of the root word library, and a unified English root word is obtained for replacement.
It has achieved a unified naming standard for data table fields, which facilitates the management and testing of data tables by the data platform and improves the efficiency of data table management.
Smart Images

Figure CN112035480B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of metadata management, and more particularly to a data table management method, apparatus, device, and storage medium. Background Technology
[0002] Currently, with the increasing diversity of data, the number of data tables is also growing. Therefore, data management platforms can be used to query the field names, comment information, and lineage relationships between data tables.
[0003] However, in practical use, some data tables contain long, compound nouns as their fields, which can be quite lengthy. Therefore, abbreviations are often used to map these long nouns to shorter roots, making it easier for developers to understand the meaning of the table fields. However, different developers have different abbreviation habits for table fields, and their translation skills also vary. This can lead to different abbreviations for the same noun, hindering the data platform's management of the data tables.
[0004] Therefore, how to standardize table fields to facilitate data platform management of data tables has become an urgent problem to be solved. Summary of the Invention
[0005] This application provides a data table management method, apparatus, device, and storage medium to unify table fields, thereby facilitating data platform management of data tables.
[0006] Firstly, this application provides a data table management method, the method comprising:
[0007] The data table's fields are scanned and detected to identify fields to be mapped. The semantic similarity between the field to be mapped and multiple fields in a pre-built root word library is determined to identify whether a matching field exists in the pre-built root word library. If a matching field exists in the pre-built root word library, its English root is used as the English root of the field to be mapped. If no matching field exists in the pre-built root word library, the field to be mapped is mapped according to the root word mapping rules of the pre-built root word library to obtain an English root. The field to be mapped corresponding to the English root is replaced with the replaced data table for management.
[0008] Secondly, this application also provides a data table management device, the device comprising:
[0009] The system includes a field scanning module for scanning and detecting the fields of a data table to identify fields to be mapped; a field matching module for determining the semantic similarity between the field to be mapped and multiple fields in a pre-built root word library, to determine whether a matching field exists in the pre-built root word library based on the semantic similarity; a root word calling module for calling the English root of the matching field as the English root of the field to be mapped if a matching field exists in the pre-built root word library; a root word mapping module for mapping the field to be mapped according to the root word mapping rules of the pre-built root word library if no matching field exists in the pre-built root word library, to obtain an English root; and a field replacement module for replacing the field to be mapped corresponding to the English root, to obtain a replaced data table for managing the data table.
[0010] Thirdly, this application also provides a computer device, the computer device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the data table management method as described above when executing the computer program.
[0011] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the data table management method described above.
[0012] This application discloses a data table management method, apparatus, device, and storage medium. It involves scanning and detecting the fields of a data table to determine the fields to be mapped. Then, it determines the semantic similarity between the fields to be mapped and each field in a pre-built root word library. Based on the semantic similarity, it determines whether there is a matching field in the pre-built root word library. If a matching field exists, its English root word is directly used as the English root word for the field to be mapped. If no matching field exists, root word mapping is performed according to the root word mapping rules of the pre-built root word library to obtain the English root word. Finally, the field to be mapped is replaced according to the English root word to obtain a replaced data table, thus managing the data table. Scanning the data table fields and using a unified root word mapping rule to map the fields to be mapped ensures consistent naming conventions for the mapped data table fields. Furthermore, it allows for the detection and correction of data tables that have already undergone root word mapping within the data platform, facilitating data table management by the data platform. Attached Figure Description
[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0014] Figure 1 This is a schematic flowchart of a data table management method provided in an embodiment of this application;
[0015] Figure 2 This is a schematic flowchart illustrating the determination of semantic similarity provided in an embodiment of this application;
[0016] Figure 3 yes Figure 1 A flowchart illustrating the sub-steps of a data table management method provided in the document;
[0017] Figure 4 A schematic block diagram of a data table management device provided in an embodiment of this application;
[0018] Figure 5 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0020] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.
[0021] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0023] Embodiments of this application provide a data table management method, apparatus, computer device, and storage medium. The data table management method can be applied to a data management platform to manage the table fields of data tables within the platform, ensuring uniformity of the root word mapping rules used in the table fields and facilitating management.
[0024] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0025] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a data table management method provided in an embodiment of this application. This data table management method achieves uniformity in the word root mapping rules used by the table fields within the data table by pre-constructing a root word library.
[0026] like Figure 1 As shown, the data table management method specifically includes steps S101 to S105.
[0027] S101. Scan and detect the table fields of the data table to determine the fields to be mapped in the table fields.
[0028] The data management platform stores multiple data tables. It scans each field of each table to determine the fields to be mapped in each table. These fields are those for which word root mapping is required.
[0029] In some embodiments, the field to be mapped includes a first field and / or a second field, wherein the first field refers to a Chinese field that has not yet been mapped by word roots, and the second field refers to a field that has been mapped by word roots, but the word root mapping rule used is not a preset word root mapping rule.
[0030] When designing a new data table, all the table fields are Chinese characters. To facilitate the management and storage of the data table, it is necessary to map the Chinese characters in the data table to their corresponding English characters.
[0031] When determining whether a table field contains a first field (i.e., a Chinese field without root mapping), one can use the byte length as an example. Since English letters, numbers, and punctuation marks each occupy one byte with a value between 0 and 255, while Chinese characters occupy two bytes, the byte length characteristic can be used to determine if the table field contains Chinese characters. Alternatively, the Unicode value range of Chinese characters can also be used for this determination.
[0032] When determining whether a second field exists in a table field, English roots in a pre-built root word library can be retrieved based on the table field to determine whether there is an English root word in the pre-built root word library that matches the field to be mapped. If not, the table field is considered to be the field to be mapped.
[0033] The pre-built root word library can be pre-built according to preset root word mapping rules. The pre-built root word library stores Chinese fields and the complete English words and English roots corresponding to the Chinese fields. If a table field is mapped according to the root word mapping rules of the pre-built root word library, then the table field should be searchable in the pre-built root word library. Therefore, the table field in the data table is matched with the English roots in the pre-built root word library to determine whether the table field exists in the pre-built root word library. If the table field does not exist in the pre-built root word library, it can be considered that the table field has not been mapped according to the preset root word mapping rules. The table field is then designated as a field to be mapped so that the root word mapping of the table field can be re-performed, and the resulting table field conforms to the preset root word mapping rules. This ensures that the table fields of each data table in the data management platform are unified, facilitating the management of the data tables.
[0034] S102. Determine the semantic similarity between the field to be mapped and multiple fields in the pre-built root word library, so as to determine whether there is a matching field for the field to be mapped in the pre-built root word library based on the semantic similarity.
[0035] Since the pre-built root word library includes Chinese fields, their corresponding complete English words, and English root words, the semantic similarity between the Chinese fields in the pre-built root word library and the field to be mapped can be calculated. Because the calculated semantic similarity is a real number between 0 and 1, the semantic similarity is 1 when the two words are exactly the same, and close to 0 when the two words are completely different concepts. Using semantic similarity, fields that are exactly the same as or similar to the field to be mapped can be matched from the pre-built root word library. Therefore, semantic similarity can be used to determine whether there is a matching field in the pre-built root word library that matches the field to be mapped.
[0036] Among them, fields similar to the field to be mapped refer to fields that are semantically similar but have different expressions. For example, they can be synonyms or near-synonyms of the field to be mapped, as well as fields that have the same meaning but different word order from the field to be mapped.
[0037] For example, if the field to be mapped is a first-level organization code and the pre-built root word library contains the first-level organization code, then the first-level organization code can be considered a similar field to the first-level organization code.
[0038] Consider the order and meaning of the Chinese expressions of the fields to be mapped, and avoid multiple expressions caused by combining words with the same meaning in different orders, so as to facilitate the management of data tables and the standardization of the mapping of word roots of table fields.
[0039] When calculating semantic similarity, it is necessary to perform the calculation based on the Chinese meaning of the field to be mapped. Since the field to be mapped includes a first field and / or a second field, in some embodiments, please refer to [the relevant documentation / reference needed]. Figure 2 The flowchart illustrates the process of determining semantic similarity. The determination of the semantic similarity between the field to be mapped and multiple fields in a pre-built root word library includes steps S1021 and S1022:
[0040] S1021. Determine the Chinese meaning of the second field, and convert the second field into the first field according to the Chinese meaning.
[0041] When determining the Chinese meaning of the second field, an SQL statement can be used to query the field comments of the second field in the data table. The Chinese meaning of the second field can then be determined based on the field comments, and the second field can be converted into the first field based on the Chinese meaning.
[0042] After converting the second field into the first field, semantic similarity can be calculated based on the first field in the pre-built root word library, thereby determining whether there is a field in the pre-built root word library that matches the first field.
[0043] S1022. Determine the semantic similarity between the first field and multiple fields in the pre-built root word library.
[0044] When calculating the semantic similarity between Chinese fields in a pre-built root word library and fields to be mapped, various calculation methods can be used. For example, it can be calculated based on world knowledge, or it can be calculated using a large-scale corpus.
[0045] When choosing to calculate based on world knowledge, the semantic similarity between two words can be calculated using a formula, such as: Where α is an adjustable parameter, representing the word distance value when the semantic similarity is 0.5, Sim(W1,W2) represents the semantic similarity between the Chinese field and the field to be mapped, and Dis(W1+W2) represents the word distance between the Chinese field and the field to be mapped.
[0046] By calculating semantic similarity, it is possible to match synonyms of the field to be mapped in a pre-built root word library, so that the field to be mapped can match fields in the pre-built root word library that have the same Chinese semantics but different Chinese expressions, thereby improving the accuracy of field matching.
[0047] In some embodiments, determining whether a matching field for the field to be mapped exists in the pre-built root word library based on the semantic similarity includes: determining whether there is a semantic similarity greater than a preset threshold among a plurality of semantic similarities; if there is a semantic similarity greater than the preset threshold, then determining that a matching field for the field to be mapped exists in the pre-built root word library, and using the field with the semantic similarity greater than the preset threshold as the matching field.
[0048] The preset threshold can be pre-set, for example, a preset threshold of 0.9. The semantic similarity between each Chinese field in the pre-built root word library and the field to be mapped is calculated, and the calculated semantic similarities are compared with the preset threshold. It is determined whether any of the calculated semantic similarities are greater than the preset threshold. If there is a semantic similarity greater than the preset threshold, the Chinese field with the semantic similarity greater than the preset threshold is selected as the matching field of the field to be mapped, and it is determined that there is a matching field of the field to be mapped in the pre-built root word library.
[0049] In some embodiments, when there are multiple semantic similarities greater than a preset threshold, the step of using the field with the semantic similarity greater than the preset threshold as the matching field includes: determining the maximum value among the multiple semantic similarities, and using the field corresponding to the maximum value of the semantic similarity as the matching field of the field to be mapped.
[0050] When there are multiple semantic similarities greater than the preset threshold, there will also be multiple matching fields. Therefore, the maximum value among the multiple semantic similarities is determined, and the field corresponding to the maximum semantic similarity is used as the matching field of the field to be mapped.
[0051] For example, fields with semantic similarity greater than a preset threshold can be sorted in descending order of semantic similarity, and then the field corresponding to the maximum semantic similarity can be determined.
[0052] S103. If there is a matching field in the pre-built root word library that matches the field to be mapped, then the English root word of the matching field is called as the English root word of the field to be mapped.
[0053] Since the pre-built root word library contains Chinese fields and their corresponding complete English words and English root words, if it is determined that there is a matching field in the pre-built root word library that matches the field to be mapped, the English root word corresponding to the matching field can be directly called, and the called English root word can be used as the English root word of the field to be mapped, so that the English root word of the field to be mapped is obtained by root word mapping according to the preset root word mapping rules.
[0054] S104. If there is no matching field in the pre-built root word library that matches the field to be mapped, then the field to be mapped is mapped according to the root word mapping rules of the pre-built root word library to obtain English root words.
[0055] If there is no matching field in the pre-built root word library that matches the field to be mapped, then the field to be mapped can be considered a new field, and root mapping needs to be performed on the field to be mapped according to the root word mapping rules in the pre-built root word library.
[0056] In some embodiments, please refer to Figure 3 Step S104 specifically includes steps S1041 and S1042.
[0057] S1041. The field to be mapped is split to obtain at least one word to be mapped.
[0058] First, the field to be mapped needs to be split. For example, a Chinese word segmenter can be used to split the field. The Chinese word segmenter can automatically match the most frequently used word combinations based on the usage weight of each word in the field to be mapped, thereby completing the splitting of the field to be mapped and obtaining the word to be mapped.
[0059] S1042. Obtain the English word corresponding to each of the words to be mapped, and abbreviate the English words according to the abbreviation rules to obtain the English word root corresponding to the field to be mapped.
[0060] After identifying the term to be mapped, the corresponding English field is queried, and then the English field is abbreviated according to the abbreviation rules to obtain the English root word. For example, after the application system is split into two terms, application and system, the corresponding English field is application_system. Abbreviating it according to the abbreviation rules, the corresponding English root word is app_sys.
[0061] The abbreviation rules can be pre-set. For example, when abbreviating words, the abbreviation should be omitted after the consonant and before the vowel. For similar words, the compressed letter method can be used to avoid conflicts between similar words. Function words should be abbreviated in their entirety. Commonly known abbreviations should be selected for country names, etc.
[0062] In some embodiments, if the newly generated English word root conflicts with existing word roots stored in a pre-built word root library, the developers can decide whether the newly generated English word root needs to be split or renamed.
[0063] In some embodiments, the data table management method includes: jointly saving the field to be mapped and the English word roots obtained by mapping the field to be mapped according to the word root mapping rules to the pre-built word root library.
[0064] The fields to be mapped and their corresponding English word roots are stored in a pre-built word root library to expand the content of the pre-built word root library. When translating the same field to be mapped in the future, the word root library can be directly called without splitting the mapping, which improves the convenience of use and the speed of word root mapping.
[0065] S105. Replace the field to be mapped corresponding to the English word root according to the English word root to obtain the replaced data table for management.
[0066] After performing root word mapping on the field to be mapped and obtaining the corresponding English root word, the field to be mapped in the data table can be automatically replaced according to the obtained English root word, resulting in a replaced data table. The table fields in the replaced data table are all obtained by root word mapping according to the preset root word mapping rules, thus realizing the management of the data table.
[0067] In some embodiments, the data table management method includes: marking the replaced data table; and scanning and detecting the table fields of the data table, which includes: scanning and detecting the table fields of the unmarked data table.
[0068] After performing root word mapping and replacing the table fields in the data table, the replaced data table is saved. When saving, the replaced data table is marked to indicate that the table fields in the data table are obtained by root word mapping according to the preset root word mapping rules. When the data platform manages the data table, it is not necessary to scan and detect the table fields again.
[0069] When the data platform scans and checks the fields of the data tables, it can only scan and check the fields of unlabeled data tables to detect the fields to be mapped in the unlabeled data tables. These fields are then mapped and replaced according to preset root word mapping rules, resulting in a replaced data table. Based on this, the data platform can check and correct the field information of all data tables stored within the platform, ensuring that all table fields conform to the preset root word mapping rules, thus unifying the table fields.
[0070] When marking the replaced data table, for example, you can add a marker to the name of the saved data table to indicate that the data table is the replaced data table.
[0071] The data table management method provided in the above embodiments scans and detects the fields of the data table to determine the fields to be mapped. It then determines the semantic similarity between the fields to be mapped and each field in a pre-built root word library. Based on the semantic similarity, it determines whether there is a matching field in the pre-built root word library. If a matching field exists, the English root of the matching field is directly used as the English root of the field to be mapped. If no matching field exists, root mapping is performed according to the root word mapping rules of the pre-built root word library to obtain the English root. Finally, the field to be mapped is replaced according to the English root to obtain the replaced data table, thus managing the data table. Scanning the fields of the data table and using unified root word mapping rules to map the fields to be mapped ensures consistent naming conventions for the fields of the mapped data table. Furthermore, it allows for the inspection and correction of data tables that have already undergone root word mapping within the data platform, facilitating data table management by the data platform.
[0072] Please see Figure 4 , Figure 4 This application also provides a schematic block diagram of a data table management device for executing the aforementioned data table management method. The data table management device can be configured in a server or a terminal.
[0073] The server can be a standalone server or a server cluster. The terminal can be an electronic device such as a mobile phone, tablet, laptop, desktop computer, personal digital assistant, and wearable device.
[0074] like Figure 4 As shown, the data table management device 200 includes: a field scanning module 201, a field matching module 202, a root word calling module 203, a root word mapping module 204, and a field replacement module 205.
[0075] The field scanning module 201 is used to scan and detect the table fields of the data table to determine the fields to be mapped in the table fields.
[0076] The field matching module 202 is used to determine the semantic similarity between the field to be mapped and multiple fields in a pre-built root word library, so as to determine whether there is a matching field for the field to be mapped in the pre-built root word library based on the semantic similarity.
[0077] The field matching module 202 includes a field conversion submodule 2021 and a similarity calculation submodule 2022.
[0078] Specifically, the field conversion submodule 2021 is used to determine the Chinese meaning of the second field, so as to convert the second field into the first field according to the Chinese meaning; the similarity calculation submodule 2022 is used to determine the semantic similarity between the first field and multiple fields in the pre-built root word library.
[0079] The root word calling module 203 is used to call the English root word of the matching field as the English root word of the field to be mapped if there is a matching field in the pre-built root word library that matches the field to be mapped.
[0080] The root word mapping module 204 is used to perform root word mapping on the field to be mapped according to the root word mapping rules of the pre-built root word library if there is no matching field in the pre-built root word library that matches the field to be mapped, so as to obtain English root words.
[0081] The root word mapping module 204 includes a field splitting submodule 2041 and a word abbreviation submodule 2042.
[0082] Specifically, the field splitting submodule 2041 is used to split the field to be mapped to obtain at least one word to be mapped; the word abbreviation submodule 2042 is used to obtain the English word corresponding to each word to be mapped, and abbreviate the English word according to the abbreviation rules to obtain the English word root corresponding to the field to be mapped.
[0083] The field replacement module 205 is used to replace the field to be mapped corresponding to the English word root according to the English word root, so as to obtain the replaced data table for management.
[0084] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the data table management device and its modules described above can be referred to the corresponding processes in the aforementioned data table management method embodiments, and will not be repeated here.
[0085] The aforementioned data table management device can be implemented as a computer program, which can, for example... Figure 5 It runs on the computer device shown.
[0086] Please see Figure 5 , Figure 5 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a server or a terminal.
[0087] See Figure 5The computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.
[0088] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any data table management method.
[0089] The processor provides computing and control capabilities, supporting the operation of the entire computer device.
[0090] Internal memory provides an environment for the execution of computer programs stored in non-volatile storage media. When these computer programs are executed by the processor, the processor can perform any data table management method.
[0091] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0092] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.
[0093] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:
[0094] The data table's fields are scanned and detected to identify fields to be mapped. The semantic similarity between the field to be mapped and multiple fields in a pre-built root word library is determined to identify whether a matching field exists in the pre-built root word library. If a matching field exists in the pre-built root word library, its English root is used as the English root of the field to be mapped. If no matching field exists in the pre-built root word library, the field to be mapped is mapped according to the root word mapping rules of the pre-built root word library to obtain an English root. The field to be mapped corresponding to the English root is replaced with the replaced data table for management.
[0095] In one embodiment, the field to be mapped includes a first field and / or a second field; the processor, when determining the semantic similarity between the field to be mapped and multiple fields in a pre-built root word library, is configured to:
[0096] Determine the Chinese meaning of the second field, and convert the second field into the first field according to the Chinese meaning; determine the semantic similarity between the first field and multiple fields in a pre-built root word library.
[0097] In one embodiment, when the processor determines whether a matching field for the field to be mapped exists in the pre-built root word library based on the semantic similarity, it is configured to:
[0098] Determine whether there is a semantic similarity greater than a preset threshold among the multiple semantic similarities; if there is a semantic similarity greater than the preset threshold, determine that there is a matching field for the field to be mapped in the pre-constructed root word library, and use the field with the semantic similarity greater than the preset threshold as the matching field.
[0099] In one embodiment, the semantic similarity values greater than the preset threshold are multiple; when the processor implements the step of using fields with semantic similarity values greater than the preset threshold as matching fields, it is configured to:
[0100] The maximum value among the multiple semantic similarities is determined, and the field corresponding to the maximum semantic similarity is used as the matching field of the field to be mapped.
[0101] In one embodiment, when the processor performs root mapping on the field to be mapped according to the root mapping rules of the pre-built root word library, it is configured to:
[0102] The field to be mapped is split to obtain at least one word to be mapped; the English word corresponding to each word to be mapped is obtained, and the English word is abbreviated according to the abbreviation rules to obtain the English root word corresponding to the field to be mapped.
[0103] In one embodiment, the processor is configured to: save the field to be mapped and the English word roots obtained by mapping the field to be mapped according to the word root mapping rules together to the pre-built word root library.
[0104] In one embodiment, the processor is configured to: mark the replaced data table; and when performing the scanning and detection of the table fields of the data table, the processor is configured to: perform scanning and detection of the table fields of the unmarked data table.
[0105] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement any of the data table management methods provided in the embodiments of this application.
[0106] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.
[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A data table management method characterized by comprising: The method comprises the following steps: scanning and detecting table fields of a data table to determine to-be-mapped fields in the table fields, wherein the to-be-mapped fields comprise first fields and second fields, the first fields are Chinese fields that have not been subjected to root mapping, and the second fields are fields that have been subjected to root mapping but use a root mapping rule that is not a preset root mapping rule; determining Chinese meanings of the second fields to convert the second fields into the first fields according to the Chinese meanings; determining semantic similarities between the first fields and a plurality of fields in a pre-constructed root library; determining whether there is a matching field of the to-be-mapped field in the pre-constructed root library according to the semantic similarities; if there is a matching field of the to-be-mapped field in the pre-constructed root library, using an English root of the matching field as an English root of the to-be-mapped field; if there is no matching field of the to-be-mapped field in the pre-constructed root library, splitting the to-be-mapped field to obtain at least one to-be-mapped word, and obtaining an English word corresponding to each to-be-mapped word; omitting letters after a consonant and before a vowel of the English word; if the to-be-mapped word is a virtual word, performing full abbreviation on the to-be-mapped word to obtain an English root corresponding to the to-be-mapped field; if the to-be-mapped word is a similar word, processing the to-be-mapped word by a letter compression method; replacing the to-be-mapped field corresponding to the English root with the English root to obtain a replaced data table, so as to manage the data table.
2. The data table management method according to claim 1, characterized by, The determining whether there is a matching field of the to-be-mapped field in the pre-constructed root library according to the semantic similarities comprises the following steps: determining whether there is a semantic similarity greater than a preset threshold value in the plurality of semantic similarities; if there is a semantic similarity greater than the preset threshold value, determining that there is a matching field of the to-be-mapped field in the pre-constructed root library, and taking the field with the semantic similarity greater than the preset threshold value as the matching field.
3. The data table management method according to claim 2, characterized by, The semantic similarity greater than the preset threshold value is a plurality of; and the taking the field with the semantic similarity greater than the preset threshold value as the matching field comprises the following steps: determining a maximum value of the plurality of semantic similarities, and taking a field corresponding to the maximum value of the semantic similarities as the matching field of the to-be-mapped field.
4. The data table management method according to claim 1, characterized by, The method comprises the following steps: collectively storing the to-be-mapped field and an English root of the to-be-mapped field mapped according to the root mapping rule in the pre-constructed root library.
5. The data table management method of claim 1, wherein, The method comprises the following steps: labeling the replaced data table; The scanning and detecting the table fields of the data table comprises the following steps: scanning and detecting table fields of an unlabeled data table.
6. A data table management apparatus characterized by comprising: The method comprises the following steps: The field scanning module is configured to scan and detect table fields of a data table, and process a field to be mapped using a word segmenter to determine fields to be mapped in the table fields, the fields to be mapped including a first field and a second field, wherein the first field refers to a Chinese field that has not been subjected to root mapping, and the second field refers to a field that has been subjected to root mapping but uses a root mapping rule other than a preset root mapping rule. The field matching module is configured to determine a Chinese meaning of the second field, convert the second field into the first field according to the Chinese meaning, determine semantic similarity of the first field and a plurality of fields in a pre-constructed root library, and determine whether a matching field of the field to be mapped exists in the pre-constructed root library according to the semantic similarity. The root calling module is configured to call an English root of the matching field as an English root of the field to be mapped if the matching field of the field to be mapped exists in the pre-constructed root library. The root mapping module is configured to split the field to be mapped to obtain at least one field to be mapped, obtain an English word corresponding to each field to be mapped, omit letters after a consonant and before a vowel of the English word, perform full abbreviation on the field to be mapped if the field to be mapped is a virtual word, and obtain an English root corresponding to the field to be mapped if the field to be mapped is a similar word, and process the field to be mapped by a compression letter method to obtain an English root corresponding to the field to be mapped. The field replacement module is configured to replace the field to be mapped corresponding to the English root with the English root to obtain a replaced data table, and manage the data table.
7. A computer device, comprising: The computer device includes a memory and a processor. The memory is configured to store a computer program. The processor is configured to execute the computer program and implement the data table management method in any one of claims 1 to 5 when the computer program is executed.
8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to make the processor implement the data table management method in any one of claims 1 to 5.
Citation Information
Patent Citations
Data processing method and device, electronic equipment and computer readable storage medium
CN111061833A
Data processing apparatus and method for converting words to abbreviations, converting abbreviations to words, and selecting abbreviations for insertion into text
US7475343B1