Data standard processing method and device, electronic equipment and storage medium
Through standardized processing, classification and theme framework optimization of data standards, the problem of low data standard management efficiency is solved, efficient retrieval and management of data standards is achieved, and the accuracy and real-timeness of data standards are improved.
Patent Information
- Application Number
- CN202411117111.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, the management efficiency of data standards is low and error-prone, resulting in low data retrieval and management efficiency, and it is impossible to effectively cope with the rapidly growing data volume and diversified data standards.
By obtaining multiple pending data standards, standardizing processing and classification, building a theme framework, and optimizing based on preset information and expert review, the theme framework is updated to improve the retrieval and management efficiency of data standards.
It realizes efficient retrieval and management of data standards during application, improves the accuracy and real-timeness of data standards, reduces manual operations, and improves work efficiency.
Smart Images

Figure CN119004183A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data standard processing method and device, electronic equipment and storage medium. Background Art
[0002] In related technologies, data standards are the main basis for data standardization, and data standardization is an important part of data governance. With the rapid development of information technology, the amount of data generated by various industries has increased dramatically, and the number and types of corresponding data standards have also increased. The existing method is to manually manage various data standards. This process of manually managing data standards is inefficient and prone to errors, resulting in low efficiency in retrieval and management in subsequent applications.
[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention
[0004] The main purpose of the embodiments of the present application is to propose a data standard processing method and device, electronic device and storage medium, which can effectively improve the retrieval and management efficiency of data standards during the application process.
[0005] To achieve the above objective, an embodiment of the present application provides a method for processing data standards, the method comprising the following steps:
[0006] Acquire a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types;
[0007] Performing standardization processing on the first data standard to be processed to obtain a second data standard to be processed, wherein the data structures of a plurality of the second data standards to be processed are the same;
[0008] Classifying all the second data standards to be processed to obtain data standard sets of various types;
[0009] Constructing a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards;
[0010] Optimizing the first subject framework according to preset information to obtain a second subject framework, wherein the preset information includes expert review information and target object behavior information;
[0011] Collecting a third data standard to be processed according to the source of the first data standard to be processed;
[0012] The second subject framework is updated according to the third data standard to be processed.
[0013] In some embodiments, the obtaining of a plurality of first data standards to be processed includes:
[0014] crawling a third data standard to be processed at a preset time interval by a crawler algorithm, wherein the third data standard to be processed includes a data standard chart or a data standard document;
[0015] Evaluate the authority and reliability of the third data standard to be processed according to the source of the third data standard to be processed to obtain an evaluation result;
[0016] The third data standard to be processed is preprocessed according to the evaluation result to obtain the first data standard to be processed.
[0017] In some embodiments, the step of performing standardization processing on the first data standard to be processed to obtain a second data standard to be processed includes:
[0018] Parsing the first data standard to be processed by a preset algorithm to obtain key information;
[0019] The key information is standardized to obtain the second data standard to be processed.
[0020] In some embodiments, the classifying all the second data standards to be processed to obtain data standard sets of various types includes:
[0021] Classifying the second to-be-processed data standard into predefined types to obtain a classification result;
[0022] generating a data standard entry of the second data standard to be processed according to the classification result;
[0023] Generate data labels for the data standard items through text classification and word vector algorithms;
[0024] Associating the data standard entry with the data tag to obtain a first association relationship;
[0025] Generate data standard sets of various types according to the classification result and the first association relationship.
[0026] In some embodiments, constructing a first subject framework according to the data standard set includes:
[0027] Extract the core theme of each data standard in the data standard set through natural language processing and knowledge graph algorithm;
[0028] Associating the core topic with the business information to obtain a second association relationship;
[0029] Constructing a third theme framework according to the second association relationship;
[0030] The third subject framework is reviewed and optimized to obtain the first subject framework, and each data standard on the first subject framework corresponds to a preset code.
[0031] In some embodiments, optimizing the first theme framework according to preset information to obtain the second theme framework includes:
[0032] Obtain expert review opinions on the data standards in the first subject framework;
[0033] The data standards in the first subject framework are optimized according to the expert review opinions to obtain the second subject framework.
[0034] In some embodiments, updating the second subject framework according to the third to-be-processed data standard includes:
[0035] Calculate a weighted evaluation value of the third data standard to be processed according to the standard mapping frequency, timeliness and importance;
[0036] Determining the priority of the third to-be-processed data standard according to the weighted evaluation value;
[0037] The second subject framework is updated according to the priority of the third data standard to be processed.
[0038] To achieve the above object, another aspect of an embodiment of the present application provides a data standard processing device, the device comprising:
[0039] A first module is used to obtain a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types;
[0040] A second module is used to perform standardization processing on the first data standard to be processed to obtain a second data standard to be processed, and the data structures of multiple second data standards to be processed are the same;
[0041] A third module is used to classify all the second data standards to be processed to obtain data standard sets of various types;
[0042] A fourth module is used to construct a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards;
[0043] A fifth module is used to optimize the first subject framework according to preset information to obtain a second subject framework, wherein the preset information includes expert review information and target object behavior information;
[0044] A sixth module is used to collect a third data standard to be processed according to the source of the first data standard to be processed;
[0045] The seventh module is used to update the second subject framework according to the third standard of data to be processed.
[0046] To achieve the above object, another aspect of an embodiment of the present application provides an electronic device, including:
[0047] at least one processor;
[0048] at least one memory for storing at least one program;
[0049] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.
[0050] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.
[0051] The embodiments of the present application include at least the following beneficial effects: The present application provides a data standard processing method and device, an electronic device and a storage medium. The scheme classifies all the second data standards to be processed after standardizing multiple first data standards to be processed to obtain second data standards to be processed, and then constructs a first subject framework including the affiliation and priority of multiple data standards according to the classified data standard set, and then optimizes the first subject framework according to preset information to obtain a second subject framework, and then updates the second subject framework according to the collected third data standard to be processed, so that during the application process, the data standards can be retrieved and managed based on the second subject framework, effectively improving the retrieval and management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Figure 1 is a flow chart of a method for processing data standards provided in an embodiment of the present application;
[0053] Figure 2 yes Figure 1 Flow chart of step S110 in FIG.
[0054] Figure 3 yes Figure 1 Flow chart of step S120 in FIG.
[0055] Figure 4 yes Figure 1 Flow chart of step S130 in FIG.
[0056] Figure 5 yes Figure 1Flow chart of step S140 in FIG.
[0057] Figure 6 yes Figure 1 Flow chart of step S150 in FIG.
[0058] Figure 7 yes Figure 1 Flowchart of step S170 in FIG.
[0059] Figure 8 It is an overall flow chart of the data standard processing method provided in the embodiment of the present application;
[0060] Fig. 9 is a structural diagram of a data standard processing device provided in an embodiment of the present application;
[0061] Fig.10 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application.
[0063] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0064] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.
[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0066] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:
[0067] Data standards: are standards for defining, classifying and unifying data formats, naming and management to ensure data quality, and to ensure data consistency and interoperability between different systems, applications and organizations. Data standards specify requirements for data structure, format, encoding, naming rules, units, scope, etc. to ensure that data can be accurately understood, shared and used.
[0068] Data standardization: It is the process of processing specific data based on data standards.
[0069] In the related art, with the development of information technology, the amount of data in various industries has increased dramatically, and the types and data of corresponding data standards have also increased. Since the formats of various types of data standards are different, the existing method is to process various types of data standards manually. Since the manual processing method is inefficient and prone to errors, the retrieval and management efficiency of the subsequent application process is low.
[0070] In view of this, the embodiments of the present application provide a data standard processing method and device, an electronic device and a storage medium. The present application can effectively improve the retrieval and management efficiency of data standards in the application process by formatting and classifying multiple types of data standards, constructing a subject framework, and updating and optimizing the subject framework.
[0071] The data standard processing method provided in the embodiment of the present application relates to the field of data processing technology. The data standard processing method provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application of a data standard processing method, etc., but is not limited to the above forms.
[0072] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.
[0073] Figure 1 is an optional flowchart of the data standard processing method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S170:
[0074] Step S110, obtaining a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types;
[0075] Step S120: standardize the first data standard to be processed to obtain a second data standard to be processed, wherein the data structures of the plurality of second data standards to be processed are the same;
[0076] Step S130: classify all second data standards to be processed to obtain data standard sets of various types;
[0077] Step S140: construct a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards;
[0078] Step S150: optimizing the first subject framework according to preset information to obtain a second subject framework, wherein the preset information includes expert review information and target object behavior information;
[0079] Step S160, collecting the third data standard to be processed according to the source of the first data standard to be processed;
[0080] Step S170: Update the second subject framework according to the third standard of data to be processed.
[0081] In the embodiments of the present application, Figure 2 As shown, the process of obtaining multiple first data standards to be processed includes but is not limited to steps S210 to S230:
[0082] Step S210: crawling a third data standard to be processed at a preset time interval by a crawler algorithm, wherein the third data standard to be processed includes a data standard chart or a data standard document;
[0083] Step S220, evaluating the authority and reliability of the third data standard to be processed according to the source of the third data standard to be processed, and obtaining an evaluation result;
[0084] Step S230: pre-process the third data standard to be processed according to the evaluation result to obtain the first data standard to be processed.
[0085] It is understandable that the web crawler algorithm can be used to regularly crawl the relevant icons, documents and other data of the newly released or updated structured, semi-structured and unstructured data standards from various public channels as the third data standard to be processed to ensure the real-time nature of the data. Among them, the public channels include but are not limited to industry associations, standardization organizations, preset websites, etc. In this embodiment, when using the crawler algorithm, this embodiment will comply with relevant laws, regulations and ethical standards, and respect the rights and privacy of the target website.
[0086] Exemplarily, the crawler algorithm code of this embodiment is as follows, and needs to be adjusted according to information such as domain name and crawled data type during application:
[0087]
[0088]
[0089] f.write(item['content'])
[0090] return item
[0091] In an embodiment of the present application, for the third data standard to be processed that cannot be automatically crawled, or the third data standard to be processed that requires special supplementation, it can be manually uploaded through the target object.
[0092] In an embodiment of the present application, after obtaining the third data standard to be processed, the third data standard to be processed can be stored, backed up and version managed to provide data support for subsequent updates. At the same time, after evaluating the authority and reliability of the third data standard to be processed according to the source of the third data standard to be processed (such as a public data source, manual upload, etc.), the third data standard to be processed is pre-processed. Specifically, when the evaluation result determines that the authority or reliability of the third data standard to be processed does not pass, the third data standard to be processed is deleted; when the evaluation result determines that the authority or reliability of the third data standard to be processed passes, the third data standard to be processed is retained. Then, the retained third data standard to be processed is subjected to operations such as format conversion, deduplication and indexing to facilitate subsequent analysis and processing.
[0093] In the embodiments of the present application, Figure 3 As shown, the process of standardizing the first data standard to be processed to obtain the second data standard to be processed includes but is not limited to steps S310 to S320:
[0094] Step S310: parsing the first data standard to be processed by a preset algorithm to obtain key information;
[0095] Step S320: Standardize the key information to obtain a second standard of data to be processed.
[0096] It is understandable that the preset algorithm includes but is not limited to natural language processing (NLP), optical character recognition (OCR) or machine learning algorithm. In this embodiment, the first data standard to be processed is intelligently parsed by the preset algorithm to obtain key information, and the key information is formed into structured content. Among them, the key information is used to guide the process of standardizing specific data. It is understandable that after obtaining the key information corresponding to each first data standard to be processed, the present embodiment can standardize all key information through the large model algorithm, and integrate it into a unified data format to obtain the second data standard to be processed, so as to facilitate the subsequent data processing and analysis process.
[0097] In the embodiments of the present application, Figure 4 As shown, the process of classifying all second data standards to be processed to obtain data standard sets of various types includes but is not limited to steps S410 to S450:
[0098] Step S410, classify the second to-be-processed data standard into predefined types to obtain a classification result;
[0099] Step S420, generating a data standard entry of a second data standard to be processed according to the classification result;
[0100] Step S430, generating data labels for data standard items through text classification and word vector algorithm;
[0101] Step S440: Associating the data standard item with the data tag to obtain a first association relationship;
[0102] Step S450: Generate data standard sets of various types according to the classification results and the first association relationship.
[0103] It is understandable that this embodiment can classify the second data standard to be processed into predefined types after classifying the analyzed semantic model through a hierarchical clustering machine learning algorithm. Specifically, the predefined types of this embodiment include basic data standards, indicator data standards, public data standards, and standard terminology. Among them, the basic data standard is a standardized specification formulated for data directly generated in the process of business development; the indicator data standard is a standardized specification formulated for data generated by processing basic data to meet internal analysis management needs and external regulatory needs; the public data standard is a standardized specification formulated for defining the codes used for sharing data between public institutions and organizations; standard terminology is the unification and standardization of professional terms for corporate business and data, so as to avoid ambiguity and misunderstanding.
[0104] In an embodiment of the present application, after completing the classification of the second data standard to be processed, a specific data standard entry is generated based on the classification result. Among them, the data standard entry includes detailed information such as definition and attributes. Specifically, the basic data standard entry includes alias, business definition, business rule, standard source, data type, data length, whether it can be empty, corresponding code, and responsible department; the indicator data standard entry includes alias, business definition, standard source, calculation formula, statistical dimension, statistical frequency, data length, data accuracy, whether it can be empty, and responsible department; the public data standard entry includes data item, data length, value range, and responsible department; the standard terminology entry includes terminology, concept, and responsible department.
[0105] After obtaining the data standard entry of each second data standard to be processed, the text classification and word vector model are used to automatically generate data tags related to each data standard entry, and then the data tags are associated with the data standard entries, so that the data standards can be quickly retrieved and managed during the application process. In this embodiment, the data tags include but are not limited to standard types, applicable fields, importance levels, etc., such as "industry standards", "important", "manufacturing", "business analysis", etc., to facilitate rapid retrieval and application of target objects.
[0106] In the embodiments of the present application, Figure 5 As shown, the process of constructing the first subject framework according to the data standard set includes but is not limited to steps S510 to S540:
[0107] Step S510: extract the core theme of each data standard in the data standard set through natural language processing and knowledge graph algorithm;
[0108] Step S520: Associating the core topic with the business information to obtain a second association relationship;
[0109] Step S530: construct a third theme framework according to the second association relationship;
[0110] Step S540: review and optimize the third theme framework to obtain a first theme framework, wherein each data standard on the first theme framework corresponds to a preset code.
[0111] It is understandable that the content of each data standard in the data standard set is analyzed by natural language processing and knowledge graph algorithm, and then the core theme of the corresponding data standard is extracted, and the input business information is associated with the core theme to ensure that the theme framework is highly matched with the business needs. After completing the association matching, a preliminary theme framework is formed by clustering similar themes through a machine learning algorithm. Then, a hierarchical theme framework is formed as the third theme framework based on the association relationship between the core theme and the business information, and the affiliation and priority between each core theme are clarified in the third theme framework. Specifically, the present embodiment can use a large model algorithm to further subdivide each core theme to form a detailed theme framework structure. Exemplarily, when the first theme framework includes three layers, it can be a primary classification, a secondary classification, and a tertiary classification. Then the formed third theme framework is reviewed, and the final theme framework is formed as the first theme framework after being optimized according to the review feedback structure.
[0112] In the embodiment of the present application, after obtaining the first subject framework, each data standard in the first subject framework is encoded by the encoding rule. It is understandable that the encoding rule can adopt the following rules:
[0113] The overall structure of the first theme framework:
[0114] Code length: 11 bits (including separator);
[0115] Composition: first-level classification (2 letters) + second-level classification (2 letters) third-level classification (2 letters) + separator (1 digit) + serial number (3 digits);
[0116] Detailed description of the coding:
[0117] Separator (1 digit): short dash “-”.
[0118] First-level classification (2 digits): Indicated by a combination of uppercase English letters (AY).
[0119] Secondary classification (2 digits): Indicated by a combination of uppercase English letters (AY).
[0120] Level 3 classification (2 digits): Indicated by a combination of uppercase English letters (AY).
[0121] Serial number (3 digits): Indicated by numbers (001-999) and used to distinguish different data standards or versions under the same three-level classification.
[0122] For example:
[0123] Assume that “customer management” is a first-level category, coded as “KH”;
[0124] "Customer Information" is a secondary category under "Customer Management" and is coded as "XX";
[0125] “Basic information” is the third-level classification under “customer information” and is coded as “JC”;
[0126] If this is the first data standard under the three-level classification, the serial number is "001", then the entire code is "KHXXJC-001";
[0127] If there are other data standards in the future, the serial number will increase, such as "KHXXJC-002", "KHXXJC-003", etc.
[0128] This embodiment encodes each data standard in the first subject framework through the above encoding rules to improve the efficiency of subsequent retrieval and management processes.
[0129] In the embodiments of the present application, Figure 6 As shown, the process of optimizing the first theme framework according to the preset information to obtain the second theme framework includes but is not limited to steps S610 to S620:
[0130] Step S610, obtaining expert review opinions on data standards in the first subject framework;
[0131] Step S620: Optimize the data standards in the first subject framework according to the expert review opinions to obtain the second subject framework.
[0132] It can be understood that the expert review opinions in this embodiment can be the result of experts in the corresponding fields of the data standards reviewing all the data standards in the first subject framework one by one, mainly reviewing the scientificity and reliability of the data standards, and providing modification opinions as expert review opinions, thereby ensuring the accuracy and authority of the data standards.
[0133] In an embodiment of the present application, the natural language, large model algorithm or machine learning algorithm can also be iteratively optimized according to the usage behavior of the target object, such as clicks, searches, feedback, etc., thereby improving the retrieval and management efficiency of data standards.
[0134] In the embodiments of the present application, Figure 7 As shown, the process of updating the second subject framework according to the third to-be-processed data standard includes but is not limited to steps S710 to S730:
[0135] Step S710, calculating a weighted evaluation value of the third to-be-processed data standard according to the standard mapping frequency, timeliness and importance;
[0136] Step S720: Determine the priority of the third to-be-processed data standard according to the weighted evaluation value;
[0137] Step S730: Update the second subject framework according to the priority of the third data standard to be processed.
[0138] It is understandable that an automatic monitoring mechanism can be established through a web crawler algorithm and an API interface, and different monitoring frequencies can be set according to the importance and release rate of different data standard sources, so that the sources of each first data standard to be processed can be checked regularly, and the latest released or updated data standard can be obtained as the third data standard to be processed. Then, based on the three dimensions of standard mapping frequency, timeliness and importance, each data standard is weighted and evaluated, and a comprehensive score is calculated for each data standard based on the weighted evaluation formula as a weighted evaluation value, and then all data standards are divided into different priorities according to different weighted evaluation values and actual conditions.
[0139] In the embodiment of the present application, in order to perform an effective weighted evaluation of the data standard, the present embodiment designs a weighted evaluation formula that comprehensively considers the four dimensions of the data standard source authority, mapping frequency, timeliness and importance, as follows:
[0140] S = w1*A+w2*F+w3*T+w4*I;
[0141] Among them, S represents the weighted evaluation value of the data standard, A represents the authority score of the standard source; F represents the standard mapping frequency score; T represents the standard timeliness score; I represents the standard importance score; w1, w2, w3 and w4 represent the weights of each score, satisfying w1+w2+w3+w4=1.
[0142] Specifically, the score for each dimension is determined in the following way:
[0143] 1. Standard source authority score A:
[0144] The authority of a source is assessed based on the level of the issuing organization, with the higher the level, the higher the authority. It can be scored on the following scale (out of 10):
[0145] National standard: 10 points;
[0146] Industry standard: 8 points;
[0147] Group standard: 6 points;
[0148] Enterprise standard: 4 points;
[0149] Additional points may be awarded if the specific source is particularly authoritative.
[0150] 2. Standard mapping frequency score F:
[0151] The mapping frequency reflects how often the standard is used in practice. It can be scored based on the number of times the standard is cited and applied in the system:
[0152] High frequency use (more than 100 times / month): 10 points;
[0153] Moderate frequency of use (50-100 times / month): 7 points;
[0154] Low frequency use (10-50 times / month): 4 points;
[0155] Very low frequency of use (less than 10 times / month): 2 points.
[0156] 3. Standard timeliness score I3:
[0157] Timeliness reflects the time interval between the publication or update of the standard. Newer standards are scored higher (out of 10):
[0158] Published or updated within the last 6 months: 10 points;
[0159] Published or updated within the last year: 7 points;
[0160] Published or updated within the last 3 years: 4 points;
[0161] Not updated for more than 3 years: 2 points.
[0162] 4. Criteria Importance Rating I:
[0163] The importance of a standard is scored based on its influence and criticality in the relevant field or industry. It can be determined by combining expert evaluation and user feedback (out of 10 points):
[0164] Key criteria, directly affecting business or compliance: 10 points;
[0165] Important criteria, significantly affecting business processes: 7 points;
[0166] Secondary criterion, which has some influence but is not critical: 4 points;
[0167] General standard, minor impact: 2 points.
[0168] Weights w1, w2, w3 and w4 reflect the importance of each scoring item in the overall evaluation. They are set according to actual needs and expert opinions, for example:
[0169] Authority of standard source: w1=0.3;
[0170] Standard mapping frequency: w2 = 0.2;
[0171] Standard timeliness: w3 0.2;
[0172] Standard importance: w4 = 0.3;
[0173] Through the above weight setting and scoring item calculation, a comprehensive score S of each standard can be obtained to determine its update priority.
[0174] Assume that the scoring items of a certain standard are as follows:
[0175] Source authority A=8 (industry standard);
[0176] Mapping frequency F = 7 (medium frequency use);
[0177] Timeliness T = 10 (published within the last 6 months);
[0178] Importance I = 7 (important criterion);
[0179] According to the set weights, the weighted evaluation value S is calculated as follows:
[0180] S=0.3*8+0.2*7+0.2*10+0.3*7=7.9;
[0181] The weighted assessment value of 7.9 indicates that the standard has a higher update priority in the current environment, and the system should give priority to updating.
[0182] Through this weighted evaluation formula and detailed scoring mechanism, the update requirements of each data standard can be effectively evaluated to ensure that the standards in the system are always kept up to date and most relevant.
[0183] The standard update priority is divided into three categories: high, medium, and low according to the weighted evaluation value S:
[0184] High priority: score 7.5 and above;
[0185] Medium priority: score between 4-7.5 points;
[0186] Low priority: score below 4 points;
[0187] Based on the evaluation results, data standards are automatically updated and replaced. Standards with higher update priorities are updated first, and standards with too low scores are recommended to be taken offline to ensure the timeliness and accuracy of data standards in the subject framework within the database.
[0188] In the embodiments of the present application, Figure 8 As shown, this embodiment provides a complete embodiment process of the method, including but not limited to steps S810 to S860:
[0189] Step S810: Acquire multiple first data standards to be processed, and store and back up the first data standards to be processed; wherein the multiple first data standards to be processed include but are not limited to graphs, tables, and documents corresponding to structured, semi-structured, and unstructured data standards;
[0190] Step S820: Intelligently analyze the plurality of first data standards to be processed by natural language and large model algorithm to obtain second data standards to be processed;
[0191] Step S830: classify all second data standards to be processed, generate a data standard entry for each second data standard to be processed, add a data tag to each data standard entry, and associate the data tag with the data standard entry;
[0192] Step S840: Generate a subject framework corresponding to the classified data standard as a first subject framework, and encode the data standard according to the core subject corresponding to the data standard;
[0193] Step S850: Optimize the first subject framework by receiving the review opinions of experts on the data standards within the first subject framework and the behavior information of the target object to obtain a second subject framework;
[0194] Step S860: Update the data standards in the second subject framework based on the data standard update status of each data standard source and the three dimensions of mapping frequency, timeliness and importance corresponding to the data standards after weighted evaluation.
[0195] In summary, the embodiments of the present application combine natural language processing algorithms, large model algorithms, and various classification algorithms to realize the collection, analysis, classification, and management of data standards, thereby significantly reducing manual operations and improving work efficiency. At the same time, the data standards within the subject framework are optimized and updated in combination with expert opinions, thereby improving the accuracy and real-time performance of the data standards, and further effectively improving the retrieval and management efficiency of the data standards during the application process.
[0196] Reference Fig. 9 , an embodiment of the present application provides a data standard processing device, the device comprising:
[0197] The first module 910 is used to obtain a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types;
[0198] The second module 920 is used to perform standardization processing on the first data standard to be processed to obtain a second data standard to be processed, and the data structures of the plurality of second data standards to be processed are the same;
[0199] The third module 930 is used to classify all second data standards to be processed to obtain data standard sets of various types;
[0200] A fourth module 940 is used to construct a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards;
[0201] The fifth module 950 is used to optimize the first theme framework according to preset information to obtain a second theme framework, wherein the preset information includes expert review information and target object behavior information;
[0202] The sixth module 960 is used to collect the third data standard to be processed according to the source of the first data standard to be processed;
[0203] The seventh module 970 is used to update the second subject framework according to the third standard of data to be processed.
[0204] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0205] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above data standard processing method when executing the computer program. The electronic device can be any smart terminal including a tablet computer, a car computer, etc.
[0206] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0207] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0208] The processor 1010 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0209] The memory 1020 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1020 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1020, and the processor 1010 calls and executes the data standard processing method of the embodiment of this application;
[0210] Input / output interface 1030, used to implement information input and output;
[0211] The communication interface 1040 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);
[0212] Bus 1050 , which transmits information between various components of the device (e.g., processor 1010 , memory 1020 , input / output interface 1030 , and communication interface 1040 );
[0213] The processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 are connected to each other in communication within the device via a bus 1050 .
[0214] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processing method of the above-mentioned data standard is implemented.
[0215] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0216] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0217] The data standard processing method and device, electronic device and storage medium provided in the embodiments of the present application automatically collect or manually upload various structured and unstructured data standards, automatically process the data standards using algorithms such as natural language and large models, and perform labeling and classification, and then optimize the subject framework to achieve update management of the data standards, thereby forming a complete and real-time data standard framework, and then the data standards can be retrieved and managed through the subject framework, effectively improving the accuracy and efficiency of the data standards.
[0218] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0219] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0220] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0221] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0222] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0223] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0224] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0225] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0226] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0227] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0228] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A method for processing data standards, characterized in that: The method comprises the following steps: Acquire a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types; Performing standardization processing on the first data standard to be processed to obtain a second data standard to be processed, wherein the data structures of a plurality of the second data standards to be processed are the same; Classifying all the second data standards to be processed to obtain data standard sets of various types; Constructing a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards; Optimizing the first subject framework according to preset information to obtain a second subject framework, wherein the preset information includes expert review information and target object behavior information; Collecting a third data standard to be processed according to the source of the first data standard to be processed; The second subject framework is updated according to the third data standard to be processed.
2. The method according to claim 1, characterized in that The obtaining of a plurality of first data standards to be processed includes: crawling a third data standard to be processed at a preset time interval by a crawler algorithm, wherein the third data standard to be processed includes a data standard chart or a data standard document; Evaluate the authority and reliability of the third data standard to be processed according to the source of the third data standard to be processed to obtain an evaluation result; The third data standard to be processed is preprocessed according to the evaluation result to obtain the first data standard to be processed.
3. The method according to claim 1, characterized in that The step of performing standardization processing on the first data standard to be processed to obtain a second data standard to be processed includes: Parsing the first data standard to be processed by a preset algorithm to obtain key information; The key information is standardized to obtain the second data standard to be processed.
4. The method according to claim 1, characterized in that The step of classifying all the second data standards to be processed to obtain data standard sets of various types includes: Classifying the second to-be-processed data standard into predefined types to obtain a classification result; generating a data standard entry of the second data standard to be processed according to the classification result; Generate data labels for the data standard items through text classification and word vector algorithms; Associating the data standard entry with the data tag to obtain a first association relationship; Generate data standard sets of various types according to the classification result and the first association relationship.
5. The method according to claim 1, characterized in that The step of constructing a first subject framework according to the data standard set includes: Extract the core theme of each data standard in the data standard set through natural language processing and knowledge graph algorithm; Associating the core topic with the business information to obtain a second association relationship; Constructing a third theme framework according to the second association relationship; The third subject framework is reviewed and optimized to obtain the first subject framework, and each data standard on the first subject framework corresponds to a preset code.
6. The method according to claim 1, characterized in that The step of optimizing the first theme framework according to preset information to obtain a second theme framework includes: Obtain expert review opinions on the data standards in the first subject framework; The data standards in the first subject framework are optimized according to the expert review opinions to obtain the second subject framework.
7. The method according to claim 1, characterized in that The updating of the second subject framework according to the third to-be-processed data standard includes: Calculate a weighted evaluation value of the third data standard to be processed according to the standard mapping frequency, timeliness and importance; Determining the priority of the third to-be-processed data standard according to the weighted evaluation value; The second subject framework is updated according to the priority of the third data standard to be processed.
8. A data standard processing device, characterized in that: The device comprises: A first module is used to obtain a plurality of first data standards to be processed, wherein the plurality of first data standards to be processed include a plurality of data structure types; A second module is used to perform standardization processing on the first data standard to be processed to obtain a second data standard to be processed, and the data structures of multiple second data standards to be processed are the same; A third module is used to classify all the second data standards to be processed to obtain data standard sets of various types; A fourth module is used to construct a first subject framework according to the data standard set, wherein the first subject framework includes the affiliation and priority of multiple data standards; A fifth module is used to optimize the first subject framework according to preset information to obtain a second subject framework, wherein the preset information includes expert review information and target object behavior information; A sixth module is used to collect a third data standard to be processed according to the source of the first data standard to be processed; The seventh module is used to update the second subject framework according to the third standard of data to be processed.
9. An electronic device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
