Text modal data management method and device, electronic equipment and storage medium
By building a text modal master data extraction model and natural language processing system, combining named entity recognition and data cleaning modules, the problem of changing structure hierarchical information and formats in text modal data is solved, and high-quality data extraction and Management avoids the phenomenon of large-scale illusions and improves extraction efficiency and accuracy.
Patent Information
- Application Number
- CN202411116783.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-22
AI Technical Summary
When processing text modal data, it is difficult to extract repetitive structure hierarchical information at high quality, especially in hierarchical structures and few sample learning scenarios, and the format recognition rate of text data with variable formats is not high, which is easy to appear. The hallucination phenomenon of the big model affects the accuracy of the extraction results.
By building a text modal master data extraction model, obtaining the master data prompt words for master data extraction, and using a natural language processing system to generate a master data directory, combining named entity recognition and data cleaning integration modules to realize intelligent management of text modal data , improve extraction efficiency and accuracy.
It realizes high-quality extraction of text modal data, improves management efficiency and accuracy, solves the problems of repeated information processing, format recognition and hallucination phenomena, and improves the quality and efficiency of data extraction.
Smart Images

Figure CN119005192A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data management, and in particular to a text modality data management method, device, electronic device and storage medium. Background Art
[0002] Currently, in the field of industrial data management, the processing methods for text modal data have the following shortcomings:
[0003] 1) Insufficient processing of repeated and structural information: Existing methods have difficulty in extracting and locating repeated and structural information with high quality, especially hierarchical structures;
[0004] 2) Low extraction quality in few-shot or zero-shot scenarios: In the absence of a large amount of labeled data, existing methods are difficult to achieve high-quality extraction;
[0005] 3) Format limitation of text modal data: When extracting text modal data with variable formats, the format recognition rate of text modal data is not high;
[0006] 4) Large model hallucination phenomenon: Although restrictions have been placed on large model hallucinations, hallucinations may still occur, affecting the accuracy of text modal data extraction results. Summary of the invention
[0007] The main purpose of the embodiments of the present application is to propose a text modal data management method, device, electronic device and storage medium, which can realize intelligent management of text modal data, improve the efficiency of text modal data management, realize text modal master data extraction, and improve the accuracy and efficiency of text modal data extraction.
[0008] On the one hand, an embodiment of the present application proposes a text modality data management method, the method comprising the following steps:
[0009] Get multiple text modal data;
[0010] Constructing a main data prompt word corresponding to each of the text modal data through a text modal main data extraction model, and performing main data extraction on each of the text modal data according to each of the main data prompt words to obtain text modal main data corresponding to each of the text modal data;
[0011] A natural language processing system is used to extract directories based on the main data of each text modality, generate corresponding main data directories, and use the main data directories to manage data of each text modality main data.
[0012] In some embodiments, the method further comprises:
[0013] Get a text modality sample dataset;
[0014] The text modality main data extraction model is pre-trained using the text modality sample data set.
[0015] In some embodiments, the natural language processing system includes a data cleaning and integration module and a named entity recognition module, and the natural language processing system extracts a directory according to each of the text modality master data to generate a corresponding master data directory, and uses the master data directory to manage each of the text modality master data, specifically including:
[0016] Performing named entity recognition on each of the text modality main data by means of the named entity recognition module, and extracting entity information corresponding to each of the text modality main data;
[0017] The data cleaning and integration module cleans and integrates the entity information corresponding to each of the text modality master data to obtain the structured entity information corresponding to each of the text modality master data;
[0018] The data cleaning and integration module divides the data according to the structured entity information corresponding to each of the text modality master data, and determines a plurality of final-level categories and a plurality of target text modality master data stored under each of the final-level categories;
[0019] The master data directory is generated according to each of the final-level categories and a plurality of the target text modal master data stored under each of the final-level categories.
[0020] In some embodiments, the text modality sample data set includes a plurality of text layout associated data, and the pre-training of the text modality main data extraction model using the text modality sample data set specifically includes:
[0021] Input each of the text layout associated data into the text modality master data extraction model, use the text modality master data extraction model to capture the association between text and layout, learn the corresponding text and layout association parameters and output the text classification information, layout structure information, text position information and layout position information corresponding to each of the text layout associated data.
[0022] In some embodiments, constructing a main data prompt word corresponding to each of the text modal data through a text modal main data extraction model, performing main data extraction on each of the text modal data according to each of the main data prompt words, and obtaining text modal main data corresponding to each of the text modal data specifically includes:
[0023] Performing preliminary master data extraction on the text modality data by using an automated tool to obtain preliminary text modality master data corresponding to the text modality data;
[0024] Acquire a preset query text, and construct a master data to be extracted information group corresponding to the text modal data according to the text modal data, the preliminary text modal master data corresponding to the text modal data, and the query text;
[0025] The main data information group to be extracted is input into the text modality main data extraction model, and the main data extraction prompt words are generated by the text modality main data extraction model according to the main data information group to be extracted. According to the main data extraction prompt words, the text modality data contained in the main data information group to be extracted is subjected to main data structured extraction, and the corresponding text modality main data is output.
[0026] In some embodiments, the named entity recognition module includes a bidirectional long short-term memory neural network unit and a conditional random field unit, and the named entity recognition is performed on each of the text modality main data by the named entity recognition module to extract entity information corresponding to each of the text modality main data, specifically including:
[0027] Extracting entity features corresponding to each of the text modality main data through the bidirectional long short-term memory neural network unit;
[0028] The conditional random field unit performs modeling processing according to the entity features corresponding to each of the text modality main data to obtain the entity information corresponding to each of the text modality main data.
[0029] In some embodiments, the data cleaning and integration module performs data division according to the structured entity information corresponding to each of the text modality master data, and determines a plurality of final-level categories and a plurality of target text modality master data stored under each of the final-level categories, specifically including:
[0030] Extracting and identifying key parameters of the structured entity information corresponding to each of the text modality master data through the data cleaning and integration module to obtain key information corresponding to each of the structured entity information;
[0031] The data cleaning and integration module classifies and divides the data according to the key information corresponding to each structured entity information, determines a plurality of final-level classifications, and determines a plurality of target text modality master data corresponding to each final-level classification from a plurality of text modality master data.
[0032] On the other hand, an embodiment of the present application provides a text modality data management device, the device comprising:
[0033] The first module is used to obtain multiple text modal data;
[0034] The second module is used to construct a main data prompt word corresponding to each of the text modal data through a text modal main data extraction model, and perform main data extraction on each of the text modal data according to each of the main data prompt words to obtain the text modal main data corresponding to each of the text modal data;
[0035] The third module is used to perform directory extraction based on each of the text modality master data through a natural language processing system, generate a corresponding master data directory, and use the master data directory to perform data management on each of the text modality master data.
[0036] On the other hand, an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the text modal data management method described above when executing the computer program.
[0037] On the other hand, an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the text modal data management method described above.
[0038] The embodiments of the present application include at least the following beneficial effects: the present application provides a text modality data management method, device, electronic device and storage medium, which constructs the master data prompt words corresponding to each text modality data through the text modality master data extraction model, extracts the master data of each text modality data according to each master data prompt word, obtains the text modality master data corresponding to each text modality data, extracts the directory according to each text modality master data through the natural language processing system, generates the corresponding master data directory, and manages the data of each text modality master data using the master data directory. The present application can realize the intelligent management of text modality data, improve the efficiency of text modality data management, realize the extraction of text modality master data, and improve the accuracy and efficiency of text modality data extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 is a flowchart of a text modality data management method provided by an embodiment of the present application;
[0040] Figure 2 is a flowchart of step S102 in an embodiment of the present application;
[0041] Figure 3 is a schematic diagram of extracting text modal main data by a text modal main data extraction model in an embodiment of the present application;
[0042] Figure 4 is a flowchart of step S103 in an embodiment of the present application;
[0043] Figure 5is a flowchart of step S301 in an embodiment of the present application;
[0044] Figure 6 is a flowchart of step S303 in an embodiment of the present application;
[0045] Figure 7 is a schematic diagram of the master data directory in an embodiment of the present application;
[0046] Figure 8 is another flow chart of a text modality data management method provided by an embodiment of the present application;
[0047] Fig. 9 It is a structural schematic diagram of a text modality data management device provided in an embodiment of the present application;
[0048] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0049] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.
[0050] It is understood that the terms "first", "second", etc. used in this application can be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another concept. For example, without departing from the scope of the embodiment of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination".
[0051] The terms "at least one", "multiple", "each", "any", etc. used in this application, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.
[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0053] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.
[0054] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following explanations.
[0055] 1) Master Data: Master data refers to the basic information of an organization that meets the needs of cross-departmental business collaboration and reflects the status attributes of core business entities. Compared with transaction data, its attributes are more stable, accuracy requirements are higher, and it has unique identification. As the most important data asset of an enterprise, master data is not designed for business systems, but remains relatively independent, serving business systems but higher than business systems that use master data; generally speaking, master data is data with high business value that can be reused across various business departments within the enterprise and exists in multiple heterogeneous application systems.
[0056] 2) Text modal data: refers to data in the form of text, which has the following characteristics: Semi-structured: Text data is neither completely unstructured nor completely structured. It may contain structural fields, such as title, author, publication date, length, classification, etc., and may also contain a large amount of unstructured data, such as abstracts and content; High dimensionality: The dimension of text vectors can generally be as high as tens of thousands of dimensions, which makes general data mining and data retrieval methods impractical due to excessive computational effort or high cost; High data volume: There are at least thousands of text samples in a general text library, and the workload of preprocessing, encoding, mining, etc. of these texts is very large, so manual methods are generally not feasible; Semanticity: There are cases where one word has multiple meanings, multiple words have one meaning, and there are cases where the context is related in time and space.
[0057] 3) OCR (Optical Character Recognition): refers to the process in which an electronic device (such as a scanner or digital camera) examines characters printed on paper, determines their shapes by detecting dark and light patterns, and then uses character recognition methods to translate the shapes into computer text; that is, for printed characters, an optical method is used to convert the text in a paper document into a black and white dot matrix image file, and the recognition software is used to convert the text in the image into a text format for further editing and processing by word processing software.
[0058] 4) Self-supervised learning: Self-supervised learning is a type of unsupervised learning, also known as (pretext task). Self-supervised learning mainly uses auxiliary tasks (pretext) to mine its own supervisory information from large-scale unsupervised data, and trains the network through this constructed supervisory information, so that it can learn representations that are valuable for downstream tasks. Self-supervised learning can complete training on unlabeled data.
[0059] 5) Query text: It is a request entered by the user into a search engine or database, aiming to find information about a specific file, website, record or a series of records. This request can be a paragraph of text, a keyword, or a very short text, and its purpose is to obtain the required information. When processing these requests, the system will perform query understanding, that is, accurately understand and process the query entered by the user to improve the effect of the search service. The query understanding process includes segmenting the query text, understanding its intent, and dealing with possible spelling errors, ambiguities or inaccurate expressions. This process is usually handled by the QP (Query Processor) module, and aims to improve the accuracy and efficiency of the search service.
[0060] 6) NLP: Natural language processing technology is a discipline that studies language issues in human-computer interaction. Natural language processing (NLP) involves a branch of language information processing and is also one of the core topics of artificial intelligence (AI). The key to NLP technology is to enable computers to "understand" natural language, so it is also called natural language understanding (NLU) or computational linguistics. This technology imitates the language and thinking patterns of successful people, decodes their thinking patterns, discovers the laws behind human thoughts, emotions and behaviors, and summarizes them into a set of replicable and imitable programs.
[0061] With the continuous development of information technology, enterprises and organizations are increasingly facing the challenge of managing large-scale and diverse data. As the core of information management, master data management is crucial to improving the consistency, accuracy and integrity of data. However, traditional master data management methods can only process structured data, and have limited processing and management capabilities for unstructured and semi-structured data such as documents, images, audio, video, etc.
[0062] Currently, the processing of text modal data faces the following major pain points: large layout differences: different types of documents and different formats of the same type of documents lead to huge layout differences, which brings difficulties to information extraction and positioning; complex key-value correspondence: the key-value correspondence depends on context, spatial alignment, table arrangement and other information, making it difficult for pure text processing logic to meet general situations; duplicate information and positioning errors: duplicate information in form drawings may result in successful information extraction but incorrect positioning of the target information, affecting subsequent processes.
[0063] At present, there are many research directions and methods for processing text modal data, including OCR technology, multimodal models, etc. However, these methods still have the following problems in practical applications:
[0064] 1) Insufficient processing of repeated and structural information: Existing methods have difficulty in extracting and locating repeated and structural information with high quality, especially hierarchical structures;
[0065] 2) Low extraction quality in few-shot or zero-shot scenarios: In the absence of a large amount of labeled data, existing methods are difficult to achieve high-quality extraction;
[0066] 3) Format limitation of text modal data: When extracting text modal data with variable formats, the format recognition rate of text modal data is not high;
[0067] 4) Large model hallucination phenomenon: Although restrictions have been placed on large model hallucinations, hallucinations may still occur, affecting the accuracy of text modal data extraction results.
[0068] Based on this, the embodiments of the present application provide a text modal data management method, device, electronic device and storage medium, which utilizes a text modal master data extraction model to achieve higher-quality master data extraction, improve the accuracy and efficiency of master data extraction, complete accurate directory extraction through a natural language processing system, realize intelligent management of text modal data, and provide strong support for text modal master data management.
[0069] Reference Figure 1 , Figure 1This is an optional flowchart of a text modality data management method provided by an embodiment of the present application. The method may include but is not limited to steps S101 to S103:
[0070] Step S101, obtaining a plurality of text modal data;
[0071] Step S102, constructing a main data prompt word corresponding to each text modal data through a text modal main data extraction model, performing main data extraction on each text modal data according to each main data prompt word, and obtaining text modal main data corresponding to each text modal data;
[0072] Step S103, extracting a directory based on each text modality master data through a natural language processing system, generating a corresponding master data directory, and using the master data directory to manage the data of each text modality master data.
[0073] In step S101 of some embodiments, various text modality data are collected by a text modality data collection module. Optionally, there are many different document types of data in the industrial field, such as engineering drawings, simulation data, component information drawings, etc. The document types include but are not limited to word documents, pdf documents, pictures, and Excel documents, etc. For paper document data, a scanner or camera or other device can be used to convert it into an electronic document.
[0074] The text modality data acquisition module supports access to multiple data sources. Based on a distributed architecture and using multi-threaded parallel technology, it can simultaneously process multi-source unstructured data and achieve connection with different data sources and data capture.
[0075] The text modal data storage module is used to store various text modal data captured by the text modal data acquisition module. For the text modal data, the corresponding storage method is selected according to its characteristics and application scenarios. A distributed storage architecture is used to store unstructured data on multiple nodes in a scalable manner, thereby improving the security and reliability of the data and facilitating the text modal master data extraction model to access and read the data.
[0076] In step S102 of some embodiments, the above-mentioned text modality main data extraction model is constructed based on LayoutLMv3 as the basic model, the concept of information group is introduced as the main data prompt (Prompt) project, and the main data extraction of text modality data is performed through the text modality main data extraction model.
[0077] LayoutLMv3 is a model based on natural language processing (NLP) that aims to parse and understand the layout structure in documents. Its goal is to parse and understand the layout structure in documents. It combines traditional text classification and layout analysis techniques to jointly model and predict text and layout information in images. The model learns cross-modal representations through a unified text-image multimodal Transformer, achieves information conversion through multi-head self-attention and feedforward neural network layers, and combines residual connections and normalization to improve training effects.
[0078] In some embodiments, reference Figure 2 , Figure 2 This is an optional flowchart of step S102 in the embodiment of the present application. Step S102 may include but is not limited to steps S201 to S203:
[0079] Step S201, performing preliminary extraction of master data on the text modality data by an automated tool to obtain preliminary text modality master data corresponding to the text modality data;
[0080] Step S202, obtaining a preset query text, and constructing a master data to be extracted information group corresponding to the text modality data according to the text modality data, the preliminary text modality master data corresponding to the text modality data, and the query text;
[0081] Step S203, input the main data information group to be extracted into the text modality main data extraction model, generate main data extraction prompt words according to the main data information group to be extracted through the text modality main data extraction model, perform main data structured extraction on the text modality data contained in the main data information group to be extracted according to the main data extraction prompt words, and output the corresponding text modality main data.
[0082] In step S201 of some embodiments, the automation tools include but are not limited to text summarization extraction tools, keyword extraction tools, named entity recognition tools, topic model analysis tools, etc., and the text modality data is automatically processed by the automation tools to output corresponding preliminary text modality master data.
[0083] In step S202 of some embodiments, text modal data, preliminary text modal main data corresponding to the text modal data, and query text are all used as target text content, and a corresponding main data information group to be extracted is constructed, wherein the query text is a preset Query text.
[0084] For example, define Among them, S is the input and output text extraction instance set, x i is the i-th text modal data, y iis the preliminary text modality master data corresponding to the i-th text modality data, and n is the number of text modality data.
[0085] Given a query text x q , query text x q For the preset Query text, define a function f, which combines the sample set S and the query text x q Mapped into new text image and mask: [x vp ,m]=f(S,x q ). Among them, x vp is the main data extraction prompt word, and the mask m defines the text modality main data extraction model in the text modality data x i The goal of the text modality main data extraction model is to extract the prompt word x based on the main data. vp , outputs y without performing any additional training vp , and extract the masks m and y vp The corresponding content gets y q , where y vp For the text modal data x i The metadata information corresponding to the extracted text modality master data, y q Metadata information y vp Corresponding to the text modality master data, optionally, the text modality master data extraction model outputs structured metadata information of the text modality master data, including the master data name, category, model, attributes and other information.
[0086] In some embodiments, reference Figure 3 , Figure 3 is an optional schematic diagram of extracting text modal master data by a text modal master data extraction model in an embodiment of the present application, wherein the query text x q , text modality data x i and the corresponding preliminary text modality master data y i The network information group (such as the main data to be extracted information group) is formed, and the network information group is input into the text modality main data extraction model. The text modality main data extraction model LayoutLM generates the main data extraction prompt word x vp , and extract the prompt word x according to the main data vp Generate y vp , and extract the masks m and y vp The corresponding information is obtained q , y q Filled into the mesh information group, the mesh information group includes four parts of information, namely text modal data x i , preliminary text modality master data y i 、Query text x qand text modal master data y q .
[0087] In some embodiments, the natural language processing system includes a data cleaning and integration module and a named entity recognition module. Figure 4 , Figure 4 This is an optional flowchart of step S103 in the embodiment of the present application. Step S103 may include but is not limited to steps S301 to S304:
[0088] Step S301, performing named entity recognition on each text modality main data through a named entity recognition module, and extracting entity information corresponding to each text modality main data;
[0089] Step S302, performing data cleaning and integration on the entity information corresponding to each text modality master data through a data cleaning and integration module to obtain structured entity information corresponding to each text modality master data;
[0090] Step S303, dividing the data according to the structured entity information corresponding to each text modality master data through the data cleaning and integration module, and determining a plurality of final-level categories and a plurality of target text modality master data stored under each final-level category;
[0091] Step S304, generating a master data directory according to each final level classification and a number of target text modal master data stored under each final level classification.
[0092] In some embodiments, in order to avoid problems such as inconsistent data standards, inconsistent descriptions, and low overall quality among the extracted multiple text modal master data, such as incorrect or overlapping material classifications; non-standard material descriptions, incorrect attribute filling or missing key attributes; material descriptions containing special characters, etc., therefore, before catalog extraction, the text modal master data is cleaned using the above-mentioned data cleaning and integration module.
[0093] Optionally, the data cleaning and integration module includes an intelligent data management tool developed using NLP technology, which implements the following functions: ① Intelligent identification of classification: Automatically identify classification affiliation through algorithm catalog extraction function, and manually supplement classifications that are not recommended; ② Intelligent identification of key parameters: Identify key parameters of material descriptions through algorithms to form structured data.
[0094] The above-mentioned intelligent data management tool is used to clean the text modal master data, classify the text modal master data, identify and process the key parameters of the material description in the text modal master data, and form structured text modal master data, which solves the problem of multiple codes for one object caused by the order of precedence, special characters, and irregular descriptions.
[0095] In step S301 of some embodiments, NLP technology is used to perform catalog extraction, and named entity recognition is performed on structured text modality main data through a named entity recognition module to extract entity information corresponding to each text modality main data.
[0096] In some embodiments, the named entity recognition module includes a bidirectional long short-term memory neural network unit and a conditional random field unit. Figure 5 , Figure 5 This is an optional flowchart of step S301 in the embodiment of the present application. Step S301 may include but is not limited to steps S401 to S402:
[0097] Step S401, extracting entity features corresponding to each text modality main data through a bidirectional long short-term memory neural network unit;
[0098] Step S402, modeling is performed according to entity features corresponding to each text modality main data by a conditional random field unit to obtain entity information corresponding to each text modality main data.
[0099] In some embodiments, a bidirectional long short-term memory neural network (BiLSTM) unit and a conditional random field (CRF) unit are used to implement named entity recognition. The bidirectional long short-term memory neural network unit learns the dependency between observation sequences through a bidirectional setting, and automatically extracts entity features corresponding to the text modality main data. The conditional random field unit is used to model the entity features extracted by the bidirectional long short-term memory neural network unit, process the transfer relationship between the features, and output the entity information corresponding to each text modality main data, thereby improving the accuracy of named entity recognition.
[0100] In step S302 of some embodiments, the data cleaning and integration module cleans and integrates the entity information corresponding to each identified text modal master data, such as material name, model, specification and other information, removes the modifying content therein, such as redundant descriptive text, symbols, etc., to obtain structured entity information.
[0101] In some embodiments, reference Figure 6 , Figure 6 This is an optional flowchart of step S303 in the embodiment of the present application. Step S303 may include but is not limited to steps S501 to S502:
[0102] Step S501, extracting and identifying key parameters of structured entity information corresponding to each text modality master data through a data cleaning and integration module, and obtaining key information corresponding to each structured entity information;
[0103] Step S502, the data cleaning and integration module classifies the data according to the key information corresponding to each structured entity information, determines multiple final classifications and determines a number of target text modality master data corresponding to each final classification from multiple text modality master data.
[0104] In some embodiments, the data cleaning and integration module performs final classification based on structured entity information, which is also the most fine-grained classification of text modality master data, so that each text modality master data has a base name for subsequent management and query of the master data.
[0105] In step S304 of some embodiments, a master data directory is generated according to multiple final classifications. For example, referring to Figure 7 , Figure 7 It is an optional schematic diagram of the master data directory in the embodiment of the present application, wherein it is assumed that the generated master data directory is a three-level classification directory, including final-level classifications 1 to 8, and a number of target text modal master data are stored under each final-level classification. Final-level classifications 1 and 2 belong to secondary classification 1, final-level classification 3 belongs to secondary classification 2, final-level classifications 4, 5 and 6 belong to secondary classification 3, and final-level classifications 7 and 8 belong to secondary classification 4. At the same time, secondary classifications 1 and 2 belong to primary classification A, and secondary classifications 3 and 4 belong to primary classification B. Based on the above-mentioned attribution relationship between the final-level classification and the secondary classification and the attribution relationship between the secondary classification and the primary classification, a master data directory is generated.
[0106] In some embodiments, reference Figure 8 , Figure 8 is another optional flow chart of a text modality data management method provided in an embodiment of the present application, wherein the text modality data management method further includes steps S601 to S602:
[0107] Step S601, obtaining a text modality sample data set;
[0108] Step S602: pre-training the text modality main data extraction model using the text modality sample data set.
[0109] In some embodiments, the preparation phase of the text modality primary data extraction model built on the basis of LayoutLMv3 can be divided into two main steps: pre-training and fine-tuning. In the pre-training phase, the text modality primary data extraction model is trained with large-scale unlabeled text and layout data (such as the above-mentioned text layout association data) to learn the association between text and layout. The text modality primary data extraction model adopts a multi-task learning framework to simultaneously predict text classification, layout structure, and position information to capture the complex relationship between text and layout.
[0110] In step S602 of some embodiments, the text modality sample data set includes multiple text layout associated data. Specifically, each text layout associated data is input into the text modality main data extraction model, and the text modality main data extraction model is used to capture the association relationship between text and layout, learn the corresponding text and layout association parameters, and output the text classification information, layout structure information, text position information and layout position information corresponding to each text layout associated data.
[0111] Reference Fig. 9 , Fig. 9 is an optional structural diagram of a text modality data management device provided in an embodiment of the present application, the device is used to implement the above-mentioned text modality data management method, and the device may include:
[0112] The first module is used to obtain multiple text modal data;
[0113] The second module is used to construct the main data prompt words corresponding to each text modal data through the text modal main data extraction model, and perform main data extraction on each text modal data according to each main data prompt word to obtain the text modal main data corresponding to each text modal data;
[0114] The third module is used to extract directories based on the main data of each text modality through a natural language processing system, generate corresponding main data directories, and use the main data directories to manage the main data of each text modality.
[0115] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0116] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned text modal data management method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.
[0117] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0118] See also Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:
[0119] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;
[0120] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the text modal data management method of the embodiment of this application;
[0121] Input / output interface 903, used to implement information input and output;
[0122] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);
[0123] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);
[0124] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .
[0125] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned text modal data management method is implemented.
[0126] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0127] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0128] The embodiments of the present application provide a text modality data management method, device, electronic device and storage medium, which constructs master data prompt words corresponding to each text modality data through a text modality master data extraction model, performs master data extraction on each text modality data according to each master data prompt word, obtains text modality master data corresponding to each text modality data, performs directory extraction according to each text modality master data through a natural language processing system, generates a corresponding master data directory, and uses the master data directory to perform data management on each text modality master data. The embodiments of the present application can realize intelligent management of text modality data, improve the efficiency of text modality data management, realize text modality master data extraction, and improve the accuracy and efficiency of text modality data extraction.
[0129] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0130] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.
[0131] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0132] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.
[0133] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0134] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.
[0135] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0136] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0138] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.
[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.
[0140] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.
Claims
1. A text modality data management method, It is characterized in that The method comprises the following steps: Get multiple text modal data; Constructing a main data prompt word corresponding to each of the text modal data through a text modal main data extraction model, and performing main data extraction on each of the text modal data according to each of the main data prompt words to obtain text modal main data corresponding to each of the text modal data; A natural language processing system is used to extract directories based on the main data of each text modality, generate corresponding main data directories, and use the main data directories to manage data of each text modality main data.
2. The text modality data management method according to claim 1, It is characterized in that The method further comprises: Get a text modality sample dataset; The text modality main data extraction model is pre-trained using the text modality sample data set.
3. The text modality data management method according to claim 1, It is characterized in that The natural language processing system includes a data cleaning and integration module and a named entity recognition module. The natural language processing system extracts a directory according to each of the text modality master data, generates a corresponding master data directory, and uses the master data directory to manage each of the text modality master data, specifically including: Performing named entity recognition on each of the text modality main data by means of the named entity recognition module, and extracting entity information corresponding to each of the text modality main data; The data cleaning and integration module cleans and integrates the entity information corresponding to each of the text modality master data to obtain the structured entity information corresponding to each of the text modality master data; The data cleaning and integration module divides the data according to the structured entity information corresponding to each of the text modality master data, and determines a plurality of final-level categories and a plurality of target text modality master data stored under each of the final-level categories; The master data directory is generated according to each of the final-level categories and a plurality of the target text modal master data stored under each of the final-level categories.
4. The text modality data management method according to claim 2, It is characterized in that The text modality sample data set includes a plurality of text layout associated data, and the pre-training of the text modality main data extraction model using the text modality sample data set specifically includes: Input each of the text layout associated data into the text modality master data extraction model, use the text modality master data extraction model to capture the association between text and layout, learn the corresponding text and layout association parameters and output the text classification information, layout structure information, text position information and layout position information corresponding to each of the text layout associated data.
5. The text modality data management method according to claim 1, It is characterized in that The method of constructing a main data prompt word corresponding to each of the text modal data by using a text modal main data extraction model, performing main data extraction on each of the text modal data according to each of the main data prompt words, and obtaining text modal main data corresponding to each of the text modal data specifically includes: Performing preliminary master data extraction on the text modality data by using an automated tool to obtain preliminary text modality master data corresponding to the text modality data; Acquire a preset query text, and construct a master data to be extracted information group corresponding to the text modal data according to the text modal data, the preliminary text modal master data corresponding to the text modal data, and the query text; The main data information group to be extracted is input into the text modality main data extraction model, and the main data extraction prompt words are generated by the text modality main data extraction model according to the main data information group to be extracted. According to the main data extraction prompt words, main data structured extraction is performed on the text modality data contained in the main data information group to be extracted, and the corresponding text modality main data is output.
6. The text modality data management method according to claim 3, It is characterized in that The named entity recognition module includes a bidirectional long short-term memory neural network unit and a conditional random field unit. The named entity recognition module is used to perform named entity recognition on each of the text modality main data to extract entity information corresponding to each of the text modality main data, specifically including: Extracting entity features corresponding to each of the text modality main data through the bidirectional long short-term memory neural network unit; The conditional random field unit performs modeling processing according to the entity features corresponding to each of the text modality main data to obtain the entity information corresponding to each of the text modality main data.
7. The text modality data management method according to claim 3, It is characterized in that The data cleaning and integration module divides the data according to the structured entity information corresponding to each of the text modal master data, determines a plurality of final-level categories and a plurality of target text modal master data stored under each of the final-level categories, and specifically includes: Extracting and identifying key parameters of the structured entity information corresponding to each of the text modality master data through the data cleaning and integration module to obtain key information corresponding to each of the structured entity information; The data cleaning and integration module classifies and divides the data according to the key information corresponding to each structured entity information, determines a plurality of final-level classifications, and determines a plurality of target text modality master data corresponding to each final-level classification from a plurality of text modality master data.
8. A text modality data management device, It is characterized in that The device comprises: The first module is used to obtain multiple text modal data; The second module is used to construct a main data prompt word corresponding to each of the text modal data through a text modal main data extraction model, and perform main data extraction on each of the text modal data according to each of the main data prompt words to obtain the text modal main data corresponding to each of the text modal data; The third module is used to perform directory extraction based on each of the text modality master data through a natural language processing system, generate a corresponding master data directory, and use the master data directory to perform data management on each of the text modality master data.
9. An electronic device, It is characterized in that The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the text modality data management method described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the text modality data management method described in any one of claims 1 to 7 is implemented.
