Data standard automatic intelligent pickup method and system based on large model
By automatically extracting data standards through large models, the problem of low efficiency and high cost of manually setting data standards is solved, and efficient and accurate data standard setting and large model capability improvement are achieved.
Patent Information
- Application Number
- CN202510852229.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-24
- Publication Date
- 2025-09-23
AI Technical Summary
In existing technologies, formulating data standards requires manual processing, which is inefficient, costly, and has a high error rate, and cannot effectively utilize large models for automated processing.
A data governance system is built using a large model. Through training data sets and iterative learning, the common content of the business system is automatically extracted and a standard data table is summarized, replacing manual and time-consuming work.
It improves the efficiency and accuracy of data standard formulation, reduces costs, ensures the consistency and quality of processing results, and continuously improves the capabilities of large models through iterative training.
Smart Images
Figure CN120688495A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data governance technology, and in particular to a method and system for automatic intelligent picking of data standards based on a large model. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Data assets are playing an increasingly important role as a new production factor. Data assetization requires high data quality, and data standards are the basis for data governance to improve data quality.
[0004] Data governance and defining data standards are the first tasks to be completed. An organization may have multiple business systems with inconsistent data structures. Establishing a unified standard allows for the collaborative use of data across these systems. While there are very few industry and national standards that define data elements, most data systems require analyzing the common content within the data of different business systems and determining the standard based on actual business requirements. This process relies on manual work by data experts familiar with the business. Due to limitations in the expertise of these experts and their limited resources, this process can be inefficient, costly, and prone to errors.
[0005] The large models that have emerged in recent years have strong summarization and learning capabilities, making them particularly suitable for handling data governance-related tasks. Their application in this area has already been explored. However, these approaches cannot replace manual work with large models. In the absence of standard documentation, a feasible process for directly extracting and summarizing standards based on business data is required. Summary of the Invention
[0006] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method and system for automatic intelligent picking of data standards based on a large model. The present invention adopts a large model to replace manual labor to perform process processing in data standard picking. It can directly extract common content from the data structure of the business system and summarize the data standard table, replacing manual labor to complete the most tedious and time-consuming part, and can improve the processing speed.
[0007] In order to achieve the above object, the present invention adopts the following technical solutions: The first aspect of the present invention provides a method for automatic intelligent picking of data standards based on a large model.
[0008] A method for automatic intelligent picking of data standards based on a large model, comprising: Build business system training data sets and deploy large business system data governance models; Based on the first prompt word input, the business system data management model is trained using the business system training data set to construct a Chinese name and a word dictionary list for translation; if the word dictionary list is inaccurate, a second prompt word is added and the business system data management model is retrained to obtain the first output result; Based on the third prompt word input, the Chinese name of the first output result is segmented, and the field root is found from the word dictionary. If the field root is inaccurate, a fourth prompt word is added, and the business system data governance model is retrained to obtain the second output result; Based on the third prompt word input and the second output result, summarize the data standard table of field roots. If the data in the data standard table does not meet business requirements, add a fifth prompt word and retrain the business system data governance model. Deploy and apply the trained business system data governance model.
[0009] Furthermore, the constructing of the Chinese name and the word dictionary list used for translation includes: constructing the Chinese name and the word dictionary list used for translation with standard words-Chinese, standard words-English, approximate words-Chinese and approximate words-English as columns respectively.
[0010] Furthermore, the inaccurate word dictionary list includes: inaccurate English translation and inaccurate Chinese Pinyin translation; the second prompt word includes: correct English translation, correct Chinese Pinyin translation and merged synonyms or near synonyms.
[0011] Furthermore, the third prompt word includes: outputting Chinese root, English root and data type, and the fourth prompt word includes: correct field root.
[0012] Furthermore, the data standard table summarizing the field word roots includes: constructing the data standard table with English word roots, Chinese word roots and data types as columns respectively.
[0013] Furthermore, the method for deploying and applying the trained business system data governance model includes: continued iterative application within the same project and cross-project replication application; wherein, continued iterative application within the same project includes: Based on other table structure information that requires mapping standards outside the business system training data set, use the trained business system data governance model. If the model output is inaccurate, perform additional iterative training according to the training process of the business system data governance model. If the model output results are accurate, export the data standard table and the field mapping standard table. Based on the mapping relationship of the field roots, check whether the data in the field mapping standard table is consistent with the mapping standard. If not, perform additional iterative training according to the training process of the business system data governance large model.
[0014] The second aspect of the present invention provides a data standard automatic intelligent picking system based on a large model.
[0015] A data standard automatic intelligent picking system based on a large model, comprising: The data and model preparation module is configured to: build business system training data sets and deploy a large business system data governance model; The dictionary training module is configured to: train the business system data management model using the business system training data set based on the first prompt word input, and construct a dictionary list of Chinese names and words used in translation; if the dictionary list of words is inaccurate, add a second prompt word and retrain the business system data management model to obtain a first output result; The root word training module is configured to: segment the Chinese name of the first output result based on the third prompt word input, find the field root word from the word dictionary, add the fourth prompt word if the field root word is inaccurate, and retrain the business system data governance model to obtain the second output result; The data standard training module is configured to: summarize the data standard table of field roots based on the input third prompt word and the second output result; if the data in the data standard table does not meet the business requirements, add the fifth prompt word and retrain the business system data governance model; The application module is configured to deploy and apply the trained business system data governance model.
[0016] A third aspect of the present invention provides a computer device, comprising: a processor adapted to execute a computer program; Computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the method for automatic intelligent picking of data standards based on a large model as described in the first aspect above are implemented.
[0017] The fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which is suitable for being loaded by a processor and executing the steps in the large model-based data standard automatic intelligent picking method as described in the first aspect above.
[0018] A fifth aspect of the present invention provides a computer program product or computer program.
[0019] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the method for automatic intelligent picking of data standards based on a large model as described in the first aspect above.
[0020] Compared with the prior art, the present invention has the following beneficial effects: The present invention is aimed at the situation where there is no standard document and it is necessary to directly pick and summarize standards based on business data. By using a large model instead of manual work, the process processing work in data standard picking can be carried out. Common content can be directly extracted from the data structure of the business system, and a data standard table can be summarized. This replaces manual work to complete the most tedious and time-consuming part, which can improve the processing speed. At the same time, as a type of computer program, even if there is a large amount of repetitive data input that needs to be processed, it can still ensure that the repeated data obtains consistent results, avoid omissions and differences in manual processing, and improve the quality of the processing results. After completing the initial training process, only the review, confirmation, and modification of the processing results need to be completed by experts, which greatly reduces the cost compared to manual processing. The large model continuously improves the processing accuracy through iterative learning. Different experts participate in iterative training of the large model, so that the large model can integrate the knowledge of different experts and continuously improve the ability of the large model. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0022] Figure 1 1 is a flow chart of a method for automatic intelligent picking of data standards based on a large model according to an embodiment of the present invention; Figure 2 is a flow chart of data standard picking and application shown in this embodiment; Figure 3 1 is a structural diagram of a data standard automatic intelligent picking system based on a large model according to an embodiment of the present invention; Figure 4 It is a structural diagram of a computer device shown in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0024] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.
[0025] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0026] As described in the background, summarizing existing data in business systems and establishing data standards requires manual effort. The primary method for summarizing and establishing standards is to search for identical content within the field information of the business system database and categorize and merge it. However, field information in business systems often contains significant gaps. For example, only the database field names are available, while Chinese aliases and business descriptions are often missing. This requires expert expertise to supplement and standardize the data. For example, the financial field "Local Currency Amount" may have various abbreviations in the business system, such as pinyin (BBJE) or English abbreviations (LOCAL_CCY_BAL). Manual verification is required to automatically complete the aliases and automatically calculate consistent Chinese aliases. This process relies heavily on personal experience. Given the varying expertise of different individuals organizing the same data, the time and accuracy of the results can vary significantly. Furthermore, the summarization process is tedious. The amount of field information to be categorized and merged is often substantial, with tens of thousands of fields often having to be sorted. Furthermore, fields originate from different business systems, with information provided by these systems and manually supplemented. This can result in fields with the same business meaning having different Chinese aliases. For example, currency may have multiple names, such as "currency," "currency number," and "currency code." Therefore, manual processing is time-consuming and prone to artifacts and errors.
[0027] Based on this, the present invention provides a method and system for automatic intelligent picking of data standards based on a large model. The following describes the solution of the present invention in detail through several embodiments: Figure 1 This is a flow chart of the method for automatic intelligent picking of data standards based on a large model shown in this embodiment, with reference to Figure 1 , the method comprising: Build business system training data sets and deploy large business system data governance models; Based on the first prompt word input, the business system data management model is trained using the business system training data set to construct a Chinese name and a word dictionary list for translation; if the word dictionary list is inaccurate, a second prompt word is added and the business system data management model is retrained to obtain the first output result; Based on the third prompt word input, the Chinese name of the first output result is segmented, and the field root is found from the word dictionary. If the field root is inaccurate, a fourth prompt word is added, and the business system data governance model is retrained to obtain the second output result; Based on the third prompt word input and the second output result, summarize the data standard table of field roots. If the data in the data standard table does not meet business requirements, add a fifth prompt word and retrain the business system data governance model. Deploy and apply the trained business system data governance model.
[0028] By leveraging the summarizing and learning capabilities of the big model, the present invention designs a data standard intelligent picking method based on the big model. This method can extract common content from the data structure of the business system and summarize a data standard table, replacing the most tedious and time-consuming manual work. Only the review, confirmation, modification and improvement work needs to be completed by experts, which can greatly improve efficiency and reduce costs. At the same time, the standard content that has been manually reviewed and confirmed can be used for iterative training of the big model, thereby continuously improving the capabilities of the big model.
[0029] Figure 2 This is a flowchart of data standard picking and application shown in this embodiment, refer to Figure 2 Taking the example of a data governance professional service agency performing data standard extraction and application on a data table in a financial enterprise's business system, the method includes: Step (1): Preparation for large model training Step (1-1): Select a large, general-purpose model that supports local deployment, such as DeepSeek, and deploy it locally as the training target. DeepSeek has been trained on a vast amount of knowledge and has learned a certain level of specialized terminology in the financial field. Subsequent standard extraction relies primarily on specialized knowledge related to specific terms.
[0030] Steps (1-2): Select representative core business tables from the business system and export the table structure as training data. The structure consists of a table with "table name," "field name," "field type," "length," and "precision." This process requires manual screening. For example, export core business tables such as margin and vouchers from the guarantee system as training data.
[0031] Step (2): Training step (1): Give the Chinese names of tables and fields and the word dictionary used for translation to the training data, and correct the translation errors to obtain an accurate dictionary. The specific steps are: Step (2-1): Explain the processing requirements to the large model, such as the prompt example: "For a table describing the following table structure for a financial business system, add a column after the table name column and fill in the translated Chinese table name. Add a column for field descriptions after the field name column and fill in the translated Chinese field name. The field description should be combined with the table Chinese name to identify the complete business meaning. Prioritize translation based on English words or English abbreviations, followed by full pinyin or abbreviations of proprietary names related to the financial industry. For untranslatable names, fill in the original table and field names. Also, compile and list the words used in the translation, as shown in Table 1. This list includes four columns: Standard Words - Chinese, Standard Words - English, Approximate Words - Chinese, and Approximate Words - English." After the prompt, send the training data to the large model to obtain the results.
[0032] Table 1 Word dictionary
[0033] Step (2-2): Check the accuracy of the English translation in the word list. For example, if the word "voucher" is translated as "voucher," which is the most common meaning in general scenarios, rather than "voucher" in financial services, enter the prompt "In financial services, voucher is a voucher. Correct and re-output all content." into the big model. In actual use, all findings should be sent to the big model.
[0034] Steps (2-3): Check the accuracy of the Chinese pinyin translation in the word list. For example, if the field name is "BBJE," an experienced business expert can directly identify this field as "Local Currency Amount" and enter the prompt "BBJE is the local currency amount. Correction is required and the modified content must be re-output" into the large model. In actual use, all issues of this type found during the inspection should be sent to the large model.
[0035] Steps (2-4): Merge and organize dictionaries. First, merge according to Chinese words. If there are multiple corresponding English words with the same or similar meanings, these English words are synonyms. For example, enter the prompt word "English synonyms of coding include ID / NO / CODE and need to be merged."; then, according to whether there are multiple corresponding Chinese words for an English word, if so, analyze whether the Chinese words are synonyms or near-synonyms, and merge those that meet the requirements, such as entering the prompt word "Chinese synonyms corresponding to CODE include coding, numbering, and internal code and need to be merged."
[0036] Step (3): Following the training step (2), we segment the Chinese names of the fields, find the root words from the dictionary, and then check if they are reasonable and make corrections. The main steps are: Step (3-1): Explain the processing requirements for the large model, such as the prompt "To describe the table structure, add a root column at the end. Identify the root within the dictionary range and fill it in." Recognition results, such as "Local Currency Amount" and "Margin Amount," identify their root as "amount," and all preceding qualifiers. Synonyms can be automatically identified based on the dictionary, such as "Voucher Internal Code," where the root is identified as number.
[0037] Step (3-2): For fields with special business requirements that cannot be directly judged according to grammatical rules, manual corrections are required through prompts. For example, fields such as "Currency," "Currency Number," and "Currency Code" actually store currency codes. Currency codes are based on the ISO 4217 international standard and require a three-letter uppercase format, which differs significantly from general numbering requirements. Therefore, corrections are required through prompts, such as "Currency number and synonym fields require their root word to be set to currency. Please correct and re-enter the modified content." After one or more cycles of this process, a root word that meets business requirements is obtained for each field.
[0038] Step (4): Training step (3), summarize the data standards corresponding to the root word, that is, the field type, length, and precision requirements for the field. After finding the root word, pick up the data standards from the field information according to the same root word. This is the last step of the standard picking that requires the large model to process. It is necessary to use the understanding and induction ability of the large model. The specific steps mainly include finding the root word, standardizing the field name and mapping the root word, and merging and organizing the data standards by root word. The specific steps include: Step (4-1): Merge and pick according to the rules. The prompt word example is "According to the standard data format of picking roots from field information with the same root. If there is only one field corresponding to one root, directly refer to the field setting. If there are multiple fields corresponding to one root, set it according to the field with the most repetitions. If there are still multiple fields with the most repetitions, take the longest one for character fields and the most accurate one for numeric fields. Finally, output the data standard table, which includes three items: standard root - Chinese, standard root - English, and data type. The field type meets the requirements of Oracle database."
[0039] Step (4-2): If the picking criteria do not meet the business requirements, make corrections. For example, if there are five fields with the root word "currency", one of them is NVARCHAR2(3), three are NVARCHAR2(10), and one is NVARCHAR2(20). The automatically picked result is NVARCHAR2(10). However, according to the ISO 4217 standard, it should be set to the NVARCHAR2(3) format for storing three uppercase English letters. Therefore, modify it by entering a prompt, such as "The currency data format should be NVARCHAR2(3). Please correct and re-output the modified content."
[0040] Step (5): Standard application, mainly including continued iterative application of the same project and cross-project copy application.
[0041] Step (5-1): Continue iterating the application of the same project and export the table structure of other tables that need to be mapped to the standard except for the training data.
[0042] Step (5-1-1): Use the trained standard for mapping. For example, the prompt is "Use the existing dictionary to supplement the Chinese name, field description, and root column of the following table structure information according to the processing logic just now. Leave blanks if there is no corresponding column in the dictionary." Then, provide the table structure information and obtain the processing result. Check whether the root information and other information are correct. If there are any blanks or the field descriptions, root information, etc. are incorrect, start the training step (1) again for supplementary iterative training on the problematic content.
[0043] Step (5-1-2): After confirming that all word roots are correctly mapped, export the data standard (Table 3) and the field mapping standard (Table 2). Based on the mapping relationship of the word roots, check whether the data type, length, and precision in Table 2 are consistent with the mapping standard. This process is implemented using a Java program because the processing logic is clear and fixed. It is more efficient than calling a large model through prompt words, and inconsistent results can be directly written to databases such as Oracle for recording.
[0044] Table 2 Original database fields and standardization results
[0045] Table 3 Data standards
[0046] Step (5-2): Cross-project replication and application. Because DeepSeek is deployed locally, other similar projects in the financial industry can directly replicate and deploy the model, starting from the application steps. Initial training is no longer required, reducing costs. Furthermore, when used across multiple projects, different business experts can participate in training, allowing for continuous iteration. This also enables the integration of knowledge from multiple experts to create larger models with higher capabilities.
[0047] The present invention can greatly improve the efficiency of data standard formulation and reduce costs; at the same time, it can iteratively train large models and continuously improve the capabilities of large models.
[0048] Combination of the above Figure 1 The method for automatic intelligent picking of data standards based on a large model provided by an embodiment of the present invention is introduced in detail. Next, the system for automatic intelligent picking of data standards based on a large model provided by an embodiment of the present invention will be introduced in conjunction with the accompanying drawings.
[0049] Figure 3 This is a schematic diagram of the structure of the data standard automatic intelligent picking system based on the large model shown in an embodiment of the present invention, referring to Figure 3 , the system of the present invention comprises: The data and model preparation module is configured to: build business system training data sets and deploy a large business system data governance model; The dictionary training module is configured to: train the business system data management model using the business system training data set based on the first prompt word input, and construct a dictionary list of Chinese names and words used in translation; if the dictionary list of words is inaccurate, add a second prompt word and retrain the business system data management model to obtain a first output result; The root word training module is configured to: segment the Chinese name of the first output result based on the third prompt word input, find the field root word from the word dictionary, add the fourth prompt word if the field root word is inaccurate, and retrain the business system data governance model to obtain the second output result; The data standard training module is configured to: summarize the data standard table of field roots based on the input third prompt word and the second output result; if the data in the data standard table does not meet the business requirements, add the fifth prompt word and retrain the business system data governance model; The application module is configured to deploy and apply the trained business system data governance model.
[0050] In some embodiments, the constructing of the Chinese name and the word dictionary list used for translation includes: constructing the Chinese name and the word dictionary list used for translation with standard words-Chinese, standard words-English, approximate words-Chinese and approximate words-English as columns respectively.
[0051] In some embodiments, the word dictionary list is inaccurate, including: inaccurate English translation and inaccurate Chinese Pinyin translation; the second prompt word includes: correct English translation, correct Chinese Pinyin translation and merged synonyms or near synonyms.
[0052] In some embodiments, the third prompt word includes: outputting Chinese roots, English roots and data types, and the fourth prompt word includes: correct field roots.
[0053] In some embodiments, the data standard table of the summary field word roots includes: constructing the data standard table with English word roots, Chinese word roots and data types as columns.
[0054] In some embodiments, the application module is specifically configured to: continue to iteratively apply the trained business system data governance model in the same project and replicate and apply the trained business system data governance model across projects.
[0055] Among them, the continued iterative application of the trained business system data governance big model in the same project includes: based on other table structure information that requires mapping standards outside the business system training data set, the trained business system data governance big model is used. If the model output result is inaccurate, supplementary iterative training is performed according to the training process of the business system data governance big model; if the model output result is accurate, the data standard table and the standard table for field mapping are exported, and according to the mapping relationship of the field root, the data in the standard table for field mapping is checked to see if it is consistent with the mapping standard. If not, supplementary iterative training is performed according to the training process of the business system data governance big model.
[0056] The present invention provides an automatic intelligent picking system for data standards based on a large model. With the help of the summarizing and learning capabilities of the large model, common content can be extracted from the data structure of the business system, and a data standard table can be summarized, replacing the most tedious and time-consuming manual work. Only experts need to complete the review, confirmation, modification and improvement work, which can greatly improve efficiency and reduce costs. At the same time, the standard content that has been manually reviewed and confirmed can be used for iterative training of the large model, thereby continuously improving the capabilities of the large model.
[0057] According to an embodiment of the present invention, the data standard automatic intelligent picking system based on the large model can correspond to the method described in the embodiment of the present invention, and the above and other operations and / or functions of each module of the data standard automatic intelligent picking system based on the large model are respectively to achieve Figure 1 For the sake of brevity, the corresponding processes of each method in are not repeated here.
[0058] See also Figure 4The structural diagram of the computer device shown in the figure includes a processor, a communication interface and a computer-readable storage medium. The processor, the communication interface and the computer-readable storage medium can be connected via a bus or other means. The communication interface is used to receive and send data. The computer-readable storage medium can be stored in the memory of the computer device, and the computer-readable storage medium is used to store a computer program. The computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer-readable storage medium. The processor (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding steps in the embodiment of the automatic intelligent picking method of data standards based on large models.
[0059] This embodiment provides a computer-readable storage medium (Memory). This computer-readable storage medium is a memory device within a computer device that is used to store programs and data. It should be understood that the computer-readable storage medium herein may include both built-in storage media within the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device. Furthermore, the storage space also stores one or more instructions suitable for being loaded and executed by the processor. These instructions may be one or more computer programs (including program code). It should be noted that the computer-readable storage medium herein may be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage device; alternatively, it may be at least one computer-readable storage medium located remotely from the processor.
[0060] In one embodiment, one or more instructions are stored in the computer-readable storage medium; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the corresponding steps in the above-mentioned embodiment of the large model-based data standard automatic intelligent picking method.
[0061] This embodiment provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding steps of the embodiment of the method for automatic intelligent picking of data standards based on large models.
[0062] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.
[0063] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0064] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0066] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).
[0067] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A method for automatic intelligent picking of data standards based on a large model, characterized in that: include: Build business system training data sets and deploy large business system data governance models; Based on the first prompt word input, the business system data management model is trained using the business system training data set to construct a Chinese name and a word dictionary list for translation; if the word dictionary list is inaccurate, a second prompt word is added and the business system data management model is retrained to obtain the first output result; Based on the third prompt word input, the Chinese name of the first output result is segmented, and the field root is found from the word dictionary. If the field root is inaccurate, a fourth prompt word is added, and the business system data governance model is retrained to obtain the second output result; Based on the third prompt word input and the second output result, summarize the data standard table of field roots. If the data in the data standard table does not meet business requirements, add a fifth prompt word and retrain the business system data governance model. Deploy and apply the trained business system data governance model.
2. The method for automatic intelligent data standard picking based on large models according to claim 1 is characterized in that: The constructing of the Chinese name and the word dictionary list used for translation includes: constructing the Chinese name and the word dictionary list used for translation with standard words-Chinese, standard words-English, approximate words-Chinese and approximate words-English as columns respectively.
3. The method for automatic intelligent data standard picking based on large models according to claim 1 is characterized in that: The inaccurate word dictionary list includes: inaccurate English translation and inaccurate Chinese Pinyin translation; the second prompt word includes: correct English translation, correct Chinese Pinyin translation and merged synonyms or near synonyms.
4. The method for automatic intelligent data standard picking based on large models according to claim 1 is characterized in that: The third prompt word includes: outputting Chinese root, English root and data type, and the fourth prompt word includes: correct field root.
5. The method for automatic intelligent data standard picking based on large models according to claim 1 is characterized in that: The data standard table summarizing field word roots includes: constructing a data standard table with English word roots, Chinese word roots and data types as columns.
6. The method for automatic intelligent data standard picking based on large models according to claim 1 is characterized in that: The method for deploying and applying the trained business system data governance model includes: continued iterative application within the same project and cross-project replication application; wherein, continued iterative application within the same project includes: Based on other table structure information that requires mapping standards outside the business system training data set, use the trained business system data governance model. If the model output is inaccurate, perform additional iterative training according to the training process of the business system data governance model. If the model output results are accurate, export the data standard table and the field mapping standard table. Based on the mapping relationship of the field roots, check whether the data in the field mapping standard table is consistent with the mapping standard. If not, perform additional iterative training according to the training process of the business system data governance large model.
7. A data standard automatic intelligent picking system based on a large model, characterized in that: include: The data and model preparation module is configured to: build business system training data sets and deploy a large business system data governance model; The dictionary training module is configured to: train the business system data management model using the business system training data set based on the first prompt word input, and construct a dictionary list of Chinese names and words used in translation; if the dictionary list of words is inaccurate, add a second prompt word and retrain the business system data management model to obtain a first output result; The root word training module is configured to: segment the Chinese name of the first output result based on the third prompt word input, find the field root word from the word dictionary, add the fourth prompt word if the field root word is inaccurate, and retrain the business system data governance model to obtain the second output result; The data standard training module is configured to: summarize the data standard table of field roots based on the input third prompt word and the second output result; if the data in the data standard table does not meet the business requirements, add the fifth prompt word and retrain the business system data governance model; The application module is configured to deploy and apply the trained business system data governance model.
8. A computer device, characterized in that: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein the computer program, when executed by the processor, implements the steps of the method for automatic intelligent picking of data standards based on a large model as described in any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded by a processor and executing the steps of the large model-based data standard automatic intelligent picking method according to any one of claims 1 to 6.
10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the method for automatic intelligent picking of data standards based on a large model according to any one of claims 1 to 6.