Digital document acquisition method and device, computer device and storage medium

Abstract

The application relates to the technical field of file digitization processing, in particular to a digital document acquisition method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an initial guide file associated with business processing in a preset field; performing image processing on the initial guide file to obtain a first processing result; performing image-text recognition on the first processing result to obtain a text format file; extracting a processing rule corresponding to the text format file; and performing information screening on the text format file according to the processing rule to obtain a target guide file. The method can solve the problems of incoherent information, differences in supervision interpretation and non-uniform institutional implementation standards.

Description

Technical Field

[0001] This application relates to the field of document digitization technology, and in particular to a method, apparatus, computer device and storage medium for acquiring digital documents. Background Technology

[0002] With the continuous development of the financial industry and the increasing regulatory requirements, various management documents are issued on a daily basis to provide timely and effective policy guidance for financial institutions in business operations. Under the current regulatory system, regulatory agencies generally use paper documents to issue regulatory rules.

[0003] Relevant personnel need to extract and understand the regulatory standards for the financial industry based on paper management documents. However, the paper documents are not clear or accurate enough in terms of the connotation and extension of the regulatory rules. Multiple regulatory agencies have different definitions of the same rules, or the same rules with different names, which makes the regulatory standards of regulatory agencies at different levels inconsistent. Therefore, there are problems such as poor information flow, different interpretations of regulations, and inconsistent implementation standards among institutions, which brings great trouble to the compliance operation of financial institutions.

[0004] To address the aforementioned technical problems, there is an urgent need for a method, apparatus, computer equipment, and storage medium for acquiring digital documents. Summary of the Invention

[0005] Therefore, it is necessary to provide a digital document acquisition method, device, computer equipment, computer-readable storage medium, and computer program product that can solve the problems of information gaps, differing regulatory interpretations, and inconsistent institutional implementation standards, in order to address the aforementioned technical issues.

[0006] Firstly, this application provides a method for acquiring digital documents. The method includes:

[0007] Obtain the initial guidance document associated with the business processing in the preset domain;

[0008] The initial guidance file is image-processed to obtain a first processing result;

[0009] The first processing result is subjected to image and text recognition to obtain a text-formatted file;

[0010] Extract the corresponding processing rules for the text format file;

[0011] According to the processing rules, the text-formatted file is filtered to obtain the target guidance file.

[0012] In one embodiment, the processing rules for extracting the text format file include:

[0013] If processing rules exist in the text file, obtain the location information of the processing rules in the text file;

[0014] Based on the location information, extract the corresponding processing rules.

[0015] In one embodiment, extracting the corresponding processing rules based on the location information includes:

[0016] Based on the location information, obtain indicator text that meets preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions, and numerical results.

[0017] Entity disambiguation and semantic role labeling are performed on the indicator text respectively to obtain the second processing result;

[0018] Based on the second processing result, obtain the processing rules.

[0019] In one embodiment, entity disambiguation of the indicator text includes:

[0020] Entity recognition is performed on the indicator text to obtain the entity name of the indicator text;

[0021] Obtain the context information of the entity name, including the paragraph or document in which the entity name is located;

[0022] Based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies.

[0023] The entity features are disambiguated using a preset model to obtain the disambiguation result, and the disambiguation result is then normalized.

[0024] In one embodiment, semantic role labeling of the indicator text includes:

[0025] Obtain a corpus containing semantic role labels, wherein the corpus includes at least one text and corresponding semantic role labeling information;

[0026] Extract features from each word in each text, including part of speech, word meaning, and grammatical dependency relationships;

[0027] A long short-term memory neural network model is trained using the features of each word, and a conditional random field model is trained using the semantic role labeling information.

[0028] Based on the trained Long Short-Term Memory Neural Network model and Conditional Random Field model, the index text is predicted to obtain the semantic role labeling results for each word.

[0029] In one embodiment, training the long short-term memory neural network model using the features of the individual words includes:

[0030] The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set.

[0031] The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that is adapted to the convolutional layer.

[0032] The features of the individual words are merged using a non-linear function to obtain features of the same dimension.

[0033] Based on the features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

[0034] Secondly, this application also provides a digital document acquisition device. The device includes:

[0035] The acquisition module is used to acquire initial guidance files associated with business processes in a preset domain;

[0036] The processing module is used to perform image processing on the initial guidance file to obtain a first processing result;

[0037] The processing module is also used to perform image and text recognition on the first processing result to obtain a text-formatted file;

[0038] The processing module is also used to extract the corresponding processing rules for the text format file;

[0039] The processing module is also used to filter information from the text-formatted file according to the processing rules to obtain the target guidance file.

[0040] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0041] Obtain the initial guidance document associated with the business processing in the preset domain;

[0042] The initial guidance file is image-processed to obtain a first processing result;

[0043] The first processing result is subjected to image and text recognition to obtain a text-formatted file;

[0044] Extract the corresponding processing rules for the text format file;

[0045] According to the processing rules, the text-formatted file is filtered to obtain the target guidance file.

[0046] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0047] Obtain the initial guidance document associated with the business processing in the preset domain;

[0048] The initial guidance file is image-processed to obtain a first processing result;

[0049] The first processing result is subjected to image and text recognition to obtain a text-formatted file;

[0050] Extract the corresponding processing rules for the text format file;

[0051] According to the processing rules, the text-formatted file is filtered to obtain the target guidance file.

[0052] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0053] Obtain the initial guidance document associated with the business processing in the preset domain;

[0054] The initial guidance file is image-processed to obtain a first processing result;

[0055] The first processing result is subjected to image and text recognition to obtain a text-formatted file;

[0056] Extract the corresponding processing rules for the text format file;

[0057] According to the processing rules, the text-formatted file is filtered to obtain the target guidance file.

[0058] The aforementioned digital document acquisition methods, devices, computer equipment, storage media, and computer program products, through digitization processing and recognition technologies, convert paper management documents into digital documents, thereby solving the problems of unclear and inaccurate paper documents. This method can help relevant personnel understand regulatory standards more accurately and avoid inconsistencies in implementation standards caused by differences in the definitions of regulatory rules among different institutions. At the same time, digitization processing and recognition technologies also improve the flow of information and reduce the difficulties of compliance operations for institutions. Attached Figure Description

[0059] Figure 1 This is an application environment diagram of a digital document acquisition method in one embodiment;

[0060] Figure 2 This is a flowchart illustrating a digital document acquisition method in one embodiment;

[0061] Figure 3 This is a flowchart illustrating a digital document acquisition method in another embodiment;

[0062] Figure 4 This is a structural block diagram of a digital document acquisition device in one embodiment;

[0063] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] Under the current regulatory framework, regulatory agencies lack a real-time updated and accessible electronic rule base, failing to provide a complete view of rules and precise online search capabilities, and also lacking established correlations between rules. This results in financial institutions lacking timely, effective, complete, and accurate policy guidance in various business scenarios. Simultaneously, the maintenance and management of rules within regulatory agencies are inadequate, easily leading to regulatory gaps and conflicts. The definitions of regulatory rules are unclear and inaccurate, with confusion and discrepancies among different regulatory agencies, resulting in inconsistent regulatory standards at various levels and hindering the compliant operation of financial institutions. Furthermore, the analysis, interpretation, implementation, and enforcement of regulatory rules are hampered by issues such as poor information flow, differing interpretations, inconsistent enforcement standards, and outdated regulatory systems.

[0066] The digital document acquisition method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. The data storage system can store the first processing result, text-formatted files, and managed digital documents that server 104 needs to process. The data storage system can be integrated onto server 104 or located in the cloud or on another network server. Specifically, server 104 obtains an initial guidance file associated with a preset domain, performs image processing on the initial guidance file to obtain a first processing result, performs image and text recognition on the first processing result to obtain a text-formatted file, extracts the corresponding processing rules from the text-formatted file, and obtains the managed digital documents associated with the preset domain according to the processing rules.

[0067] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0068] In one embodiment, such as Figure 2 As shown, a method for acquiring digital documents is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0069] Step S202: Obtain the initial guidance document associated with the business processing in the preset domain.

[0070] In this plan, the presupposed field is the financial field. Initial guidance documents for the financial field typically include the following aspects:

[0071] Laws and regulations: These include financial-related laws and regulations formulated by the national and local governments. Regulatory provisions: These are relevant provisions issued by financial regulatory agencies, such as various regulations and guidelines issued by the China Banking and Insurance Regulatory Commission (CBIRC), the China Securities Regulatory Commission (CSRC), and other regulatory bodies.

[0072] Regulatory documents, broadly defined, encompass various file formats such as Word, Excel, PDF, and JPG. Based on the specific information stored within these files, regulatory documents can be categorized into three main types: text-based information, image-based information, and tabular information.

[0073] Textual information refers to the content presented in text form in regulatory documents, typically including policy descriptions, regulations, and guidelines. This textual information can be obtained and processed by extracting the text content from the documents.

[0074] Image-based information refers to content presented in the form of images in regulatory documents, such as flowcharts, organizational charts, and diagrams. This type of information can be obtained by scanning paper documents or converting documents of other formats into image files. The obtained image files can then be analyzed, recognized, and understood using image processing technology to extract the information within them.

[0075] Tabular information refers to content presented in tabular form in regulatory documents, such as statistical data, survey results, and report tables. This type of information is usually stored in Excel or PDF format. For tabular information, the table structure in the file can be parsed, and the data content extracted for analysis and processing.

[0076] Step S204: Visualize the initial guidance file to obtain the first processing result.

[0077] Specifically, digitizing the initial guidance document involves converting the paper document into a digital image file for easy storage, retrieval, and sharing. First, the paper initial guidance document is scanned using a scanner, converting it into a digital image. Then, a series of image processing operations are performed on the scanned image, including noise removal, brightness and contrast adjustment, and edge cropping, to improve image quality and clarity.

[0078] Step S206: Perform image and text recognition on the first processing result to obtain a text-formatted file.

[0079] Specifically, text-based information, where the carrier is raw text (such as text in Word), can be extracted directly from the source file. Extracting text from images requires OCR technology, i.e., text recognition based on images. If the image contains only plain text, OCR technology can efficiently extract the text. If the image contains non-plain text information, such as tables, it requires the ability to extract text from pre-defined, uniformly formatted documents, tables with identical row and column structures, or flowcharts following the same standards and symbol formats. In these cases, the corresponding text recognition and extraction can also achieve high accuracy.

[0080] Optical Character Recognition (OCR) technology is used to recognize text in images, converting the text content into editable and searchable text. OCR technology can recognize text in different languages ​​and fonts and convert it into electronic text. The recognized text is then organized and indexed, categorized and sorted according to file attributes (such as filename, date, author, etc.) for easy retrieval and management. Next, the imaged files are stored in a digital document management system, which can choose cloud storage or local server storage. The document management system allows for version control, access control, backup, and recovery of files. Furthermore, the search function provided by the document management system allows for quick retrieval and location of desired files based on keywords and file attributes. Simultaneously, access control enables file sharing and collaboration, improving work efficiency.

[0081] Through the above image processing workflow, the initial guidance document can be transformed from a paper format into a digital image file, enabling text recognition, document management, retrieval, and sharing functions. This improves the accessibility and usability of the initial guidance document, allowing users to quickly find and use relevant regulations and guidelines.

[0082] Step S208: Extract the corresponding processing rules for text format files.

[0083] Step S210: According to the processing rules, the text format file is filtered to obtain the target guidance file.

[0084] Specifically, extracting text-formatted file processing rules refers to transforming the content of text files into structured data for analysis, processing, and application. These rules can be extracted from text automatically for further data analysis, machine learning, and other applications. Below are some examples of common text processing rules:

[0085] Keyword extraction: Extract frequently occurring keywords or phrases from the text using word frequency statistics or natural language processing methods, and use them as features or tags of the text.

[0086] Sentiment analysis: Using natural language processing technology, text is analyzed to quantify its sentiment tendency as positive or negative values ​​or degree values, which are used for applications such as sentiment analysis and public opinion monitoring.

[0087] Text classification: Using machine learning algorithms or deep learning models, text is classified according to predefined categories or labels to achieve automatic text classification and categorization.

[0088] Named entity recognition: Using natural language processing technology, specific entities such as names of people, places, and organizations are identified from text and transformed into structured data for applications such as information extraction and knowledge graph construction.

[0089] Time series analysis: Processing text containing time information to extract time series data and performing time series analysis, such as trend forecasting and seasonality analysis.

[0090] By extracting the corresponding processing rules from text-formatted files, unstructured text data can be transformed into structured data, facilitating subsequent data analysis, modeling, and applications. This allows for better utilization of the information within text data, enabling the discovery of patterns, the extraction of knowledge, and support for decision-making and innovation.

[0091] The aforementioned methods for acquiring digital documents utilize digitization and recognition technologies to convert paper management documents into digital documents, thereby resolving the issues of unclear and inaccurate paper documents. This method helps relevant personnel understand regulatory standards more accurately, avoiding inconsistencies in implementation standards caused by differences in the definitions of regulatory rules among different institutions. Simultaneously, digitization and recognition technologies also improve information flow and reduce the challenges of compliance operations for organizations.

[0092] In one embodiment, the processing rules for extracting text-formatted files include:

[0093] If processing rules exist in a text file, obtain the location information of the processing rules in the text file; based on the location information, extract the corresponding processing rules.

[0094] Natural Language Processing (NLP) techniques can be used to segment digital documents and perform part-of-speech tagging, aiming to divide text into individual words (or phrases) and determine the part of speech of each word, thereby forming an effective vocabulary. This involves dividing text into the smallest semantic units, typically words or phrases. Segmentation can be based on spaces, punctuation marks, or more complex rules, or it can utilize machine learning models, including statistical methods (such as n-gram models) and deep learning-based models (such as recurrent neural networks or Transformer models).

[0095] Specifically, each segmented word is labeled with its part of speech, which determines the part of speech of the word in the context. Common parts of speech include nouns, verbs, adjectives, and adverbs. Part-of-speech tagging can be performed using rule-based methods or supervised learning models (such as Hidden Markov Models or Conditional Random Fields).

[0096] Then, stemming or lemmatization is performed on the segmented words to group words of different forms into their common root form. Stemming is usually achieved through rule-based or heuristic algorithms, while lemmatization uses dictionaries or machine learning-based methods.

[0097] In the aforementioned digital document acquisition methods, word segmentation and part-of-speech analysis transform the text into structured lexical information, facilitating subsequent tasks such as text analysis, information extraction, and language model building. This effective vocabulary can be used in natural language processing applications such as text classification, sentiment analysis, machine translation, and question-answering systems.

[0098] In one embodiment, based on location information, corresponding processing rules are extracted, and referenced... Figure 3 ,include:

[0099] Step S302: Based on the location information, obtain the indicator text that meets the preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions, and numerical results.

[0100] Specifically, the algorithms required are end-to-end algorithm models such as Text-CNN, BERT, and XLNet.

[0101] Quantifiable metrics are the specific names or descriptions of the metrics, such as sales revenue, profit margin, and user growth rate. This part of the text describes the specific metrics that need to be quantified. Metrics with constraints may contain constraints such as time range, geographic location, and product category. These conditions are used to further constrain and define the metrics. Numerical results metrics typically contain specific numerical results or ranges, representing the specific values ​​of the metrics.

[0102] Step S304: Perform entity disambiguation and semantic role labeling on the indicator text to obtain the second processing result.

[0103] Specifically, entity disambiguation refers to disambiguating entities in indicator text, that is, determining the specific meaning of the entity in a particular context. For example, for the term "sales revenue" in indicator text, it is necessary to determine whether it refers to the sales revenue of a specific company or the sales revenue of the entire market.

[0104] Semantic role labeling refers to labeling the various semantic roles in indicator text, such as subject, predicate, and object. This helps to further understand the semantic relationships between different parts of the indicator text.

[0105] By performing entity disambiguation and semantic role labeling on the indicator text, a second processing result can be obtained, namely, further semantic parsing and understanding of the indicator text, thereby providing more accurate and richer information for subsequent processing.

[0106] Step S306: Obtain the processing rules based on the second processing result.

[0107] Specifically, based on the semantic role labeling results, the relationship between quantitative indicators, limiting conditions, and numerical results is determined. For example, by identifying the subject and object, it is determined which attribute of which entity the quantitative indicator refers to, and what the limiting conditions are.

[0108] Based on the entity disambiguation results, determine the specific entity to which the quantitative indicators and numerical results belong. For example, determine whether sales revenue refers to the sales revenue of a specific company or the sales revenue of the entire market.

[0109] Based on predefined rules or templates, quantitative information in indicator text is extracted and transformed into structured data, such as numerical values ​​or numerical ranges.

[0110] The aforementioned digital document acquisition methods can transform indicator information from text into quantifiable rules, facilitating subsequent data analysis, modeling, and applications. These processing rules can be used to automate the processing of indicator text and support various data analysis and decision-making applications.

[0111] In one embodiment, entity disambiguation of indicator text includes:

[0112] Entity recognition is performed on the indicator text to obtain the entity names; the context information of the entity names is obtained, including the paragraph or document in which the entity names are located; based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies; the entity features are normalized using a pre-defined model; and entity disambiguation is performed based on the result of the normalization process.

[0113] Specifically, the results of entity disambiguation are uniformly represented as normalized entity identifiers. Entity identifiers can be unique IDs or links pointing to the corresponding entities in the knowledge base. Through entity disambiguation and normalization, entity information in indicator text can be obtained more accurately, providing accurate entity identification for subsequent data analysis and applications. The specific steps are as follows:

[0114] Entity recognition: Using natural language processing techniques, such as named entity recognition algorithms, entity recognition is performed on indicator text to obtain entity names.

[0115] Contextual information extraction: Based on the entity recognition results, the contextual information of the entity name is obtained, including the paragraph or document in which it appears. This helps in understanding the entity's context and background information.

[0116] Entity feature extraction: Based on contextual information, extract entity features, including word vectors, entity types, and syntactic dependencies. Word vectors represent the entity's position in the semantic space, entity types help further distinguish entity categories, and syntactic dependencies reflect the syntactic relationships between the entity and other words.

[0117] Entity disambiguation: Using a pre-defined entity disambiguation model, entity features are processed to determine the unique identifier of each entity in the knowledge base. The purpose of entity disambiguation is to distinguish entities with the same name, ensuring that the obtained entity information is accurate.

[0118] Entity normalization: This involves normalizing the results of entity disambiguation, representing entity identifiers in a standardized form. This ensures consistency of entity information when interacting and sharing across different systems and applications.

[0119] The aforementioned digital document acquisition method can extract entity names from indicator text and obtain their contextual information and features. Through entity disambiguation and normalization, accurate entity identifiers can be obtained, providing reliable entity information for subsequent data analysis and applications.

[0120] In one embodiment, semantic role labeling of indicator text includes:

[0121] Obtain a corpus containing semantic role labels, which includes at least one text and its corresponding semantic role annotation information; extract features of each word in each text, including part-of-speech, word meaning, and grammatical dependency relationships; train a long short-term memory neural network model using the features of each word, and train a conditional random field model using the semantic role annotation information; predict the index text based on the trained long short-term memory neural network model and conditional random field model to obtain the semantic role annotation results of each word.

[0122] Specifically, obtaining a corpus containing semantic role labels means finding a corpus that contains at least one text and its corresponding semantic role annotation information. The corpus can come from existing corpus resources or be created through manual annotation.

[0123] Then, for each text, natural language processing techniques, such as word segmentation, part-of-speech tagging, semantic parsing, and dependency parsing, are used to extract the features of each word. Part-of-speech tags indicate the word class in a sentence, semantic tags indicate the semantic information of a word, and dependency relations indicate the syntactic relationships between words.

[0124] Training a Long Short-Term Memory (LSTM) neural network model: Using extracted word features as input, a LSTM neural network model is trained to learn the relationship between words and their semantic roles. The LSTM model can capture long-distance dependencies and is suitable for sequence labeling tasks.

[0125] Training a Conditional Random Field (CRF) model: Using semantic role annotation information as training data, a CRF model is trained to learn the dependencies between label sequences. The CRF model can consider contextual information between labels, improving annotation accuracy.

[0126] Predicting the indicator text: Using a trained LSTM and CRF model, the indicator text is predicted. The extracted word features are input into the LSTM model to obtain the context representation of each word. Then, the context representation is combined with the CRF model to predict the sequence of labels, obtaining the semantic role annotation results for each word.

[0127] The aforementioned digital document acquisition method can automatically identify and label the semantic role of each word in the text. Through a trained model, it can predict the semantic role of each word in new text. This provides more accurate semantic information for natural language processing tasks, improving the precision and effectiveness of text understanding and analysis.

[0128] In one embodiment, the method further includes:

[0129] The long short-term memory neural network model is trained using the features of each word, including:

[0130] The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set.

[0131] The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that fits the convolutional layer.

[0132] A non-linear function is used to merge the features of each word to obtain features of the same dimension.

[0133] Based on features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

[0134] Specifically, the features of each word are first preprocessed to construct word vectors. This can be achieved by using a pre-trained word vector model (such as Word2Vec, GloVe, etc.) or by training a word vector model yourself. Then, embedding and convolutional layers are set in the LSTM model. The constructed word vectors are embedded, and the output dimension of the embedding layer is convolved to adapt it to the input dimension of the convolutional layer. Next, a non-linear function is used to merge the features of each word to obtain feature representations with the same dimension. This can be achieved by using activation functions (such as ReLU, Sigmoid, etc.). Finally, based on features with the same dimension, hyperparameters (such as learning rate, batch size, number of iterations, etc.) are set, the LSTM model is instantiated, and an optimizer (such as Adam, SGD, etc.) is defined. Then, the LSTM model is trained by inputting training data and labels. During training, the error between the model output and the true labels is calculated, and the optimizer is used to minimize the error and update the model parameters. Through multiple iterations, the model is continuously optimized until a preset stopping condition is reached.

[0135] In this embodiment, the LSTM model can effectively capture and learn contextual information in text and extract semantic features of words. Through training, the model can learn the relationships between words, thereby better representing the meaning of words. By training the LSTM model, text data can be mapped to specific categories or sentiment labels, achieving text classification and sentiment analysis tasks. The model can automatically determine the category or sentiment tendency of text based on word features.

[0136] In one embodiment, the method further includes:

[0137] Based on the managed digital documents, establish a management database associated with the preset domain; obtain the usage frequency of historical query operations and filtering conditions; set field indexes for the management database based on the usage frequency, and optimize the management database through partitioning, table partitioning, caching, and load balancing.

[0138] Specifically, this refers to establishing a management database associated with a predefined domain. This involves designing the database's table structure and fields according to the needs of the predefined domain, ensuring the database meets the data management requirements of that domain. Digital documents are then stored according to the predefined domain's classification, creating corresponding tables or collections.

[0139] Data storage formats in databases are typically categorized into conditions, main indicators, symbols, and specific quantitative figures and dimensions. For example, if the text is "A microfinance company registered in City A shall have a registered capital of no less than 5 million yuan", its corresponding storage format should be: condition "a microfinance company registered in City A", indicator "registered capital", symbol ">", quantitative figure "5,000,000", and dimension "RMB".

[0140] For the completed management database, the following optimization methods can be used:

[0141] Obtaining the usage frequency of historical query operations and filter conditions involves recording and analyzing the usage of historical query operations and filter conditions, including the fields queried, query conditions, sorting methods, etc., and statistically analyzing the usage frequency of each operation and filter condition. This information can be obtained through database query logs or monitoring tools.

[0142] Setting up indexes for database fields involves creating indexes on frequently used fields based on the frequency of query operations and filtering conditions. Indexes can speed up queries and improve database performance. Depending on the specific needs, you can set up single-column indexes, composite indexes, or full-text indexes to meet different query requirements.

[0143] Partitioning and table partitioning are methods used to divide and store large amounts of data in a database. Partitioning data based on a specific field reduces the amount of data retrieved and improves query efficiency. Partitioning tables according to certain rules can balance the load and improve the performance of concurrent access.

[0144] Caching involves using caching techniques to handle frequently queried data. Query results are cached in memory, allowing them to be retrieved directly from the cache on subsequent queries, reducing database read pressure and improving query speed.

[0145] Load balancing, or load distribution, is used to distribute the load on a database with high access volume. By distributing requests across multiple database servers, system availability and performance can be improved.

[0146] The above-mentioned methods for obtaining digital documents, through optimization measures, can improve the query performance and response speed of the management database, meet the needs of digital document management, and enhance the overall efficiency of the system.

[0147] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0148] Based on the same inventive concept, this application also provides a digital document acquisition device for implementing the digital document acquisition method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the digital document acquisition device provided below can be found in the limitations of the digital document acquisition method described above, and will not be repeated here.

[0149] In one embodiment, such as Figure 4 As shown, a digital document acquisition device is provided, including: an acquisition module 402, used to acquire an initial guidance document associated with business processing in a preset domain;

[0150] Processing module 404 is used to perform image processing on the initial guidance file to obtain the first processing result;

[0151] The processing module 404 is also used to perform image and text recognition on the first processing result to obtain a text format file;

[0152] Processing module 404 is also used to extract the corresponding processing rules for text format files;

[0153] The processing module 404 is also used to filter information from the text format file according to processing rules to obtain the target guidance file.

[0154] In one embodiment, the acquisition module 402 is further configured to acquire the position information of the processing rules in the text file if processing rules exist in the text file; and extract the corresponding processing rules based on the position information.

[0155] In one embodiment, the processing module 404 is further configured to obtain, based on the location information, an indicator text that meets preset conditions from a text-formatted file. The indicator text includes quantitative indicators, limiting conditions, and numerical results.

[0156] Processing module 404 is also used to perform entity disambiguation and semantic role annotation on the indicator text to obtain a second processing result;

[0157] The processing module 404 is also used to obtain processing rules based on the second processing result.

[0158] In one embodiment, the processing module 404 is further configured to perform entity recognition on the indicator text to obtain the entity name of the indicator text.

[0159] The acquisition module 402 is also used to acquire the context information of the entity name, including the paragraph or document in which the entity name is located;

[0160] The processing module 404 is also used to obtain entity features based on context information. The entity features include word vectors, entity types, and syntactic dependencies.

[0161] The processing module 404 is also used to normalize entity features using a preset model; and to perform entity disambiguation based on the result of the normalization process.

[0162] In one embodiment, the acquisition module 402 is further configured to acquire a corpus containing semantic role labels, the corpus including at least one text and corresponding semantic role labeling information.

[0163] Processing module 404 is also used to extract features of each word in each text, including part of speech, word meaning and grammatical dependency relationship;

[0164] Processing module 404 is also used to train a long short-term memory neural network model using the features of each word and to train a conditional random field model using semantic role labeling information.

[0165] The processing module 404 is also used to predict the index text based on the trained long short-term memory neural network model and conditional random field model, and obtain the semantic role labeling results of each word.

[0166] In one embodiment, the processing module 404 is further configured to preprocess the features of each word, construct word vectors, and set the embedding layer and convolutional layer of the long short-term memory neural network model.

[0167] Processing module 404 is also used to perform word vector embedding operations, and to perform convolution operations on the output dimension of the embedding layer to transform it into the input dimension that is adapted to the convolution layer.

[0168] Processing module 404 is also used to merge the features of each word using a non-linear function to obtain features of the same dimension;

[0169] The processing module 404 is also used to train a long short-term memory neural network model based on features of the same dimension by setting hyperparameters, instantiating the model, and defining an optimizer.

[0170] In one embodiment, the processing module 404 is further configured to establish a management database associated with a preset domain based on the managed digital documents;

[0171] The acquisition module 402 is also used to acquire the frequency of use of historical query operations and filter conditions;

[0172] The processing module 404 is also used to set field indexes for the management database based on usage frequency, and to optimize the management database through partitioning, table partitioning, caching, and load balancing.

[0173] Each module in the aforementioned digital document acquisition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0174] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a digital document retrieval method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0175] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0176] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0177] Step S202: Obtain the initial guidance document associated with the business processing in the preset domain.

[0178] Step S204: Visualize the initial guidance file to obtain the first processing result.

[0179] Step S206: Perform image and text recognition on the first processing result to obtain a text-formatted file.

[0180] Step S208: Extract the corresponding processing rules for text format files.

[0181] Step S210: According to the processing rules, the text format file is filtered to obtain the target guidance file.

[0182] In one embodiment, the processing rules for extracting text-formatted files include:

[0183] If processing rules exist in a text file, obtain the location information of the processing rules in the text file;

[0184] Extract the corresponding processing rules based on the location information.

[0185] In one embodiment, based on location information, corresponding processing rules are extracted, including:

[0186] Based on the location information, obtain the indicator text that meets the preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions and numerical results.

[0187] Entity disambiguation and semantic role labeling are performed on the indicator text to obtain the second processing result;

[0188] Based on the second processing result, obtain the processing rules.

[0189] In one embodiment, entity disambiguation of indicator text includes:

[0190] Perform entity recognition on the indicator text to obtain the entity names of the indicator text;

[0191] Obtain the context information of the entity name, including the paragraph or document in which the entity name is located;

[0192] Based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies.

[0193] The entity features are normalized using a pre-defined model; based on the results of the normalization, entity disambiguation is performed.

[0194] In one embodiment, semantic role labeling of indicator text includes:

[0195] Obtain a corpus containing semantic role labels. The corpus includes at least one text and its corresponding semantic role labeling information.

[0196] Extract features from each word in each text, including part of speech, word meaning, and grammatical dependency relationships;

[0197] A long short-term memory neural network model is trained using the features of each word, and a conditional random field model is trained using semantic role labeling information.

[0198] Based on the trained Long Short-Term Memory Neural Network model and Conditional Random Field model, the indicator text is predicted to obtain the semantic role labeling results for each word.

[0199] In one embodiment, a long short-term memory neural network model is trained using the features of each word, including:

[0200] The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set.

[0201] The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that fits the convolutional layer.

[0202] A non-linear function is used to merge the features of each word to obtain features of the same dimension.

[0203] Based on features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

[0204] In one embodiment, the method further includes:

[0205] Based on the management of digital documents, establish a management database associated with the preset domain;

[0206] Get the frequency of use of historical query operations and filter conditions;

[0207] Based on usage frequency, set field indexes for the management database and optimize the management database through partitioning, table splitting, caching, and load balancing.

[0208] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0209] Step S202: Obtain the initial guidance document associated with the business processing in the preset domain.

[0210] Step S204: Visualize the initial guidance file to obtain the first processing result.

[0211] Step S206: Perform image and text recognition on the first processing result to obtain a text-formatted file.

[0212] Step S208: Extract the corresponding processing rules for text format files.

[0213] Step S210: According to the processing rules, the text format file is filtered to obtain the target guidance file.

[0214] In one embodiment, the processing rules for extracting text-formatted files include:

[0215] If processing rules exist in a text file, obtain the location information of the processing rules in the text file;

[0216] Extract the corresponding processing rules based on the location information.

[0217] In one embodiment, based on location information, corresponding processing rules are extracted, including:

[0218] Based on the location information, obtain the indicator text that meets the preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions and numerical results.

[0219] Entity disambiguation and semantic role labeling are performed on the indicator text to obtain the second processing result;

[0220] Based on the second processing result, obtain the processing rules.

[0221] In one embodiment, entity disambiguation of indicator text includes:

[0222] Perform entity recognition on the indicator text to obtain the entity names of the indicator text;

[0223] Obtain the context information of the entity name, including the paragraph or document in which the entity name is located;

[0224] Based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies.

[0225] The entity features are normalized using a pre-defined model; based on the results of the normalization, entity disambiguation is performed.

[0226] In one embodiment, semantic role labeling of indicator text includes:

[0227] Obtain a corpus containing semantic role labels. The corpus includes at least one text and its corresponding semantic role labeling information.

[0228] Extract features from each word in each text, including part of speech, word meaning, and grammatical dependency relationships;

[0229] A long short-term memory neural network model is trained using the features of each word, and a conditional random field model is trained using semantic role labeling information.

[0230] Based on the trained Long Short-Term Memory Neural Network model and Conditional Random Field model, the indicator text is predicted to obtain the semantic role labeling results for each word.

[0231] In one embodiment, a long short-term memory neural network model is trained using the features of each word, including:

[0232] The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set.

[0233] The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that fits the convolutional layer.

[0234] A non-linear function is used to merge the features of each word to obtain features of the same dimension.

[0235] Based on features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

[0236] In one embodiment, the method further includes:

[0237] Based on the management of digital documents, establish a management database associated with the preset domain;

[0238] Get the frequency of use of historical query operations and filter conditions;

[0239] Based on usage frequency, set field indexes for the management database and optimize the management database through partitioning, table splitting, caching, and load balancing.

[0240] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0241] Step S202: Obtain the initial guidance document associated with the business processing in the preset domain.

[0242] Step S204: Visualize the initial guidance file to obtain the first processing result.

[0243] Step S206: Perform image and text recognition on the first processing result to obtain a text-formatted file.

[0244] Step S208: Extract the corresponding processing rules for text format files.

[0245] Step S210: According to the processing rules, the text format file is filtered to obtain the target guidance file.

[0246] In one embodiment, the processing rules for extracting text-formatted files include:

[0247] If processing rules exist in a text file, obtain the location information of the processing rules in the text file;

[0248] Extract the corresponding processing rules based on the location information.

[0249] In one embodiment, based on location information, corresponding processing rules are extracted, including:

[0250] Based on the location information, obtain the indicator text that meets the preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions and numerical results.

[0251] Entity disambiguation and semantic role labeling are performed on the indicator text to obtain the second processing result;

[0252] Based on the second processing result, obtain the processing rules.

[0253] In one embodiment, entity disambiguation of indicator text includes:

[0254] Perform entity recognition on the indicator text to obtain the entity names of the indicator text;

[0255] Obtain the context information of the entity name, including the paragraph or document in which the entity name is located;

[0256] Based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies.

[0257] The entity features are normalized using a pre-defined model; based on the results of the normalization, entity disambiguation is performed.

[0258] In one embodiment, semantic role labeling of indicator text includes:

[0259] Obtain a corpus containing semantic role labels. The corpus includes at least one text and its corresponding semantic role labeling information.

[0260] Extract features from each word in each text, including part of speech, word meaning, and grammatical dependency relationships;

[0261] A long short-term memory neural network model is trained using the features of each word, and a conditional random field model is trained using semantic role labeling information.

[0262] Based on the trained Long Short-Term Memory Neural Network model and Conditional Random Field model, the indicator text is predicted to obtain the semantic role labeling results for each word.

[0263] In one embodiment, a long short-term memory neural network model is trained using the features of each word, including:

[0264] The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set.

[0265] The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that fits the convolutional layer.

[0266] A non-linear function is used to merge the features of each word to obtain features of the same dimension.

[0267] Based on features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

[0268] In one embodiment, the method further includes:

[0269] Based on the management of digital documents, establish a management database associated with the preset domain;

[0270] Get the frequency of use of historical query operations and filter conditions;

[0271] Based on usage frequency, set field indexes for the management database, and optimize the management database through partitioning, table splitting, caching, and load balancing.

[0272] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0273] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0274] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0275] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for acquiring digital documents, characterized in that, The method includes: Obtain the initial guidance document associated with the business processing in the preset domain; The initial guidance file is image-processed to obtain a first processing result; The first processing result is subjected to image and text recognition to obtain a text-formatted file; Extract the corresponding processing rules for the text format file; According to the processing rules, the text-formatted file is filtered to obtain the target guidance file; The processing rules for extracting the corresponding text format files include: If processing rules exist in the text file, obtain the location information of the processing rules in the text file; Based on the location information, obtain indicator text that meets preset conditions from the text format file. The indicator text includes quantitative indicators, limiting conditions, and numerical results. Entity disambiguation and semantic role labeling are performed on the indicator text respectively to obtain the second processing result; Based on the second processing result, the processing rules are obtained; The method further includes determining the relationship between quantitative indicators, limiting conditions, and numerical results based on the semantic role labeling results.

2. The method according to claim 1, characterized in that, Entity disambiguation of the indicator text includes: Entity recognition is performed on the indicator text to obtain the entity name of the indicator text; Obtain the context information of the entity name, including the paragraph or document in which the entity name is located; Based on the context information, entity features are obtained, including word vectors, entity types, and syntactic dependencies. The entity features are normalized using a preset model; Based on the results of the normalization process, entity disambiguation is performed.

3. The method according to claim 1, characterized in that, Semantic role labeling of the indicator text includes: Obtain a corpus containing semantic role labels, wherein the corpus includes at least one text and corresponding semantic role labeling information; Extract features from each word in each text, including part of speech, word meaning, and grammatical dependency relationships; A long short-term memory neural network model is trained using the features of each word, and a conditional random field model is trained using the semantic role labeling information. Based on the trained Long Short-Term Memory Neural Network model and Conditional Random Field model, the index text is predicted to obtain the semantic role labeling results for each word.

4. The method according to claim 3, characterized in that, The step of training a long short-term memory neural network model using the features of each word includes: The features of each word are preprocessed to construct word vectors, and the embedding layer and convolutional layer of the long short-term memory neural network model are set. The word vectors are embedded, and the output dimension of the embedding layer is convolved to transform it into the input dimension that adapts to the convolutional layer. A non-linear function is used to merge the features of each word to obtain features of the same dimension. Based on the features of the same dimension, a long short-term memory neural network model is trained by setting hyperparameters, instantiating the model, and defining an optimizer.

5. A digital document acquisition device, characterized in that, The device includes: The acquisition module is used to acquire initial guidance files associated with business processes in a preset domain; The processing module is used to perform image processing on the initial guidance file to obtain a first processing result; The processing module is also used to perform image and text recognition on the first processing result to obtain a text-formatted file; The processing module is also used to extract the corresponding processing rules for the text format file; The processing module is also used to filter information from the text-formatted file according to the processing rules to obtain the target guidance file; The acquisition module is also used to obtain the location information of the processing rules in the text file when the processing rules exist in the text file; and to extract the corresponding processing rules based on the location information. The processing module is also used to obtain indicator text that meets preset conditions from a text-formatted file based on location information. The indicator text includes quantitative indicators, limiting conditions, and numerical results. The processing module is also used to perform entity disambiguation and semantic role labeling on the indicator text to obtain a second processing result. The processing module is also used to obtain processing rules based on the second processing result. The device is also used to determine the relationship between quantitative indicators, limiting conditions, and numerical results based on the semantic role labeling results.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images