File task processing method, device and equipment and readable storage medium

Abstract

The invention relates to the technical field of archive management, and particularly provides an archive task processing method, device and equipment and a readable storage medium, and the method comprises the steps: carrying out the primary processing of a to-be-processed archive through a preset archive processing large model, and obtaining the global information of the to-be-processed archive, the primary processing comprises at least one of global understanding, context analysis, key information extraction, semantic relation extraction and cross-archive information integration; the global information is subjected to final processing through a preset file processing small model, a processing result is obtained, and final processing comprises at least one of keyword extraction, file classification and file abstract generation; and processing a target archive task according to the affair processing result, the target archive task including archive management, information query or decision support of the to-be-processed archive. Through the method, the effect of efficiently and accurately processing the file task can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of archive management, and in particular to a method, apparatus, device and readable storage medium for processing archive tasks. Background Art

[0002] In the field of archival management and information processing, with the rapid development of information technology and the dramatic increase in the volume of archival data, efficient and accurate processing and analysis of this data has become a pressing issue. Traditional archival processing methods often rely on manual operations or simple automated tools, which are insufficient for handling large-scale archival tasks with complex semantic relationships. Manual processing is time-consuming and labor-intensive, and is easily influenced by personal experience and subjective judgment. Simple automated tools are often only capable of basic tasks such as classification and retrieval, and are unable to handle complex tasks involving deep semantic understanding and cross-archival information integration.

[0003] The rapid development of artificial intelligence (AI) technology has provided new insights into archival processing. In particular, advancements in natural language processing (NLP) and machine learning (ML) technologies have enabled computers to gain a deeper understanding and analysis of textual data. However, single models still face numerous challenges in handling complex archival tasks, and their analysis remains inaccurate.

[0004] Therefore, how to handle archival tasks efficiently and accurately is a technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method for processing archive tasks. Through the technical solutions of the embodiments of the present application, the effect of processing archive tasks efficiently and accurately can be achieved.

[0006] In a first aspect, an embodiment of the present application provides a method for processing archival tasks, comprising: performing preliminary processing on the archives to be processed through a preset archival processing large model to obtain global information of the archives to be processed, wherein the preliminary processing includes: global understanding, context analysis, extraction of key information, extraction of semantic relationships and cross-archival information integration, and the archival processing large model is trained by combining unsupervised learning and supervised learning; performing final processing on the global information through a preset archival processing small model to obtain a processing result, wherein the final processing includes: extracting keywords, classifying archives and generating at least one of archive summaries, and the archival processing small model is trained by supervised learning; processing the target archival task through the task processing result, wherein the target archival task includes archival management, information query or decision support of the archives to be processed.

[0007] In the above-mentioned embodiments of this application, a pre-built large archive processing model understands and processes global archival information, and a small archive processing model is then constructed to perform specialized optimization of global information for specific archival tasks. This method combines the global understanding of the large model with the specialized processing capabilities of the small model to hierarchically process complex archival tasks: the large model first provides global information and contextual understanding, and then the small model efficiently executes specific tasks and outputs results. This achieves the effect of efficiently and accurately processing archival tasks.

[0008] In some embodiments, the global information includes: at least one of the global definition, contextual connection, key text and semantic relationship of the archive to be processed; the processing result includes: at least one of the keywords, archive type and archive summary of the archive to be processed.

[0009] In the above-mentioned embodiment of the present application, by acquiring global information and processing results, a data basis can be provided for the processing of subsequent target archive tasks.

[0010] In some embodiments, before performing preliminary processing on the archives to be processed through a preset archive processing large model to obtain global information of the archives to be processed, it also includes: constructing an archive set of various types of historical archives; preprocessing the archive set to obtain training samples, wherein the preprocessing includes: at least one of word segmentation, removal of stop words and stem extraction; using the masking method in unsupervised learning, using the training samples to train the basic neural network architecture to obtain the initial archive processing large model; labeling the archive set to obtain the labeled archive set; using the supervised learning method, using the labeled archive set to adjust the parameters of the initial archive processing large model to obtain the archive processing large model.

[0011] In the above embodiment of the present application, after constructing an archive set, an unsupervised masked training method is used to train the initial archive processing large model, and then a supervised learning method is used to fine-tune the initial archive processing large model to obtain the archive processing large model. The dual-supervised learning method is used to train the archive processing large model, which can quickly and accurately identify the archives to be processed and obtain global information.

[0012] In some embodiments, before performing preliminary processing on the archives to be processed through a preset archive processing large model to obtain global information of the archives to be processed, it also includes: processing the archives in the archive set through the archive processing large model to obtain multiple global information of the archive set; marking the archive type, summary and keywords in the multiple global information to obtain a second training sample; using a supervised learning method, using the second training sample to train the basic convolutional neural network to obtain a small archive processing model.

[0013] In the above embodiments of the present application, by labeling global information, it can be used as a training sample for supervised learning of a case processing small model. The obtained case processing small model can process global information and accurately obtain processing results.

[0014] In some embodiments, the global information is finally processed by a preset archive processing small model to obtain a processing result, including: obtaining a correlation score between the archive processing large model and the archive processing small model; performing weighted summation of the information in the global information according to the correlation score through the attention mechanism or gating mechanism of the archive processing small model to obtain partial global information; and finally processing the partial global information through the archive processing small model to obtain a processing result.

[0015] In the above embodiments of the present application, the attention mechanism or gating mechanism of the archive processing small model can be used to filter the information in the global information, and the most important part of the global information can be selected as the data basis for the final processing, so as to obtain more accurate processing results.

[0016] In some embodiments, the preliminary processing is cross-archival information integration, and the archives to be processed are preliminarily processed through a preset archive processing big model to obtain global information of the archives to be processed, including: constructing an initial relationship map between the archives to be processed and the associated archives; optimizing the information in the initial relationship map through the archive processing big model to obtain a relationship map, and the global information includes the relationship map.

[0017] In the above embodiment of the present application, when processing cross-archival information integration, the relationship map between the archives to be processed and the associated archives can be prioritized as global information to provide a data basis for subsequent archive processing small model processing.

[0018] In some embodiments, the target archive task is processed through the processing results, including: constructing a task list of archives to be processed through the processing results, wherein the task list includes archive management, information query and decision support; matching the target archive task in the task list, and automatically executing the target archive task.

[0019] In the above embodiment of the present application, an accurate task list can be prepared in advance, target archive tasks can be quickly acquired through task matching, and target archive tasks can be automatically executed.

[0020] In a second aspect, an embodiment of the present application provides a device for processing an archive task, comprising: A first processing module is configured to perform preliminary processing on the archive to be processed using a preset archive processing model to obtain global information about the archive to be processed, wherein the preliminary processing includes at least one of: global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration; the archive processing model is trained by combining unsupervised learning and supervised learning; a second processing module, configured to perform final processing on the global information using a preset archive processing model to obtain a processing result, wherein the final processing includes at least one of extracting keywords, classifying archives, and generating archive summaries, and the archive processing model is trained using supervised learning; The third processing module is used to process target archive tasks based on the processing results, wherein the target archive tasks include archive management, information query or decision support of the archives to be processed.

[0021] Optional, global information, including: at least one of a global definition, contextual connection, key text, and semantic relationship of the archive to be processed; The processing result includes: at least one of the keywords, file type and file summary of the file to be processed.

[0022] Optionally, the device further includes: A first training module is used for constructing an archive set of various types of historical archives before the first processing module performs preliminary processing on the archives to be processed using a preset archive processing model to obtain global information of the archives to be processed; Preprocessing the archive set to obtain training samples, wherein the preprocessing includes: at least one of: word segmentation, stop word removal, and stemming; Adopting the mask method in unsupervised learning, the basic neural network architecture is trained with training samples to obtain the initial archive processing model; Annotate the file set to obtain an annotated file set; The supervised learning method is adopted to adjust the parameters of the initial archive processing model using the labeled archive set to obtain the archive processing model.

[0023] Optionally, the device further includes: A second training module is used for processing the archives in the archive set using the archive processing large model to obtain multiple global information of the archive set before the second processing module performs preliminary processing on the archive to be processed using the preset archive processing large model to obtain global information of the archive to be processed; Annotating the file type, summary and keywords in the multiple global information to obtain a second training sample; The supervised learning method is adopted to train the basic convolutional neural network with the second training sample to obtain a small file processing model.

[0024] Optionally, the second processing module is specifically configured to: Obtaining the correlation score between the large archive processing model and the small archive processing model; Through the attention mechanism or gating mechanism of the archive processing model, the information in the global information is weighted summed according to the relevance score to obtain partial global information; The final processing of some global information is performed through the small archive processing model to obtain the processing results.

[0025] Optionally, the preliminary processing is cross-archival information integration, and the first processing module is specifically used to: Constructing an initial relationship map between the files to be processed and the associated files; The information in the initial relationship map is optimized through the archive processing model to obtain the relationship map. The global information includes the relationship map.

[0026] Optionally, the third processing module is specifically configured to: Based on the processing results, a task list of files to be processed is constructed, wherein the task list includes file management, information query and decision support; Match the target archive tasks in the task list and automatically execute the target archive tasks.

[0027] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect above are executed.

[0028] In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the method provided in the first aspect are executed.

[0029] Other features and advantages of the present application will be described in the subsequent description, and in part will become apparent from the description, or will be understood by practicing the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A flowchart of a method for processing archive tasks provided in an embodiment of the present application; Figure 2 A flowchart of an implementation method for processing archive tasks provided in an embodiment of the present application; Figure 3 A schematic block diagram of a device for processing archive tasks provided in an embodiment of the present application; Figure 4 A schematic block diagram of the structure of an apparatus for processing archive tasks provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0034] First, some of the terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0035] The Transformer model architecture uses a self-attention structure instead of the RNN structure commonly used in NLP tasks. Its biggest advantage over the RNN structure is its parallel computing capabilities.

[0036] NLP stands for Neuro-Linguistic Programming. In Hong Kong, it's also translated as "Psycho-Grammar Programming." N (Neuro) refers to the nervous system, including the brain and thought processes; L (Linguistic) refers to language, or more precisely, the process from sensory input to the formation of meaning; and P (Programming) refers to the specific set of instructions to be executed to produce a certain outcome. This refers to our thinking and behavioral habits, which, like computer programs, can be changed through software updates. Therefore, NLP is interpreted as the study of how our brains work. For this reason, NLP is translated as "Psycho-Grammar Programming" or "Neuro-Linguistic Programming."

[0037] The Sequence to Sequence model is a deep learning model that is mainly used to process sequence data. Its core idea is to convert the input sequence into the output sequence through an encoder and a decoder.

[0038] An ML model (machine learning model) uses algorithms to enable computers to learn from data and make predictions or decisions. Machine learning is a subfield of artificial intelligence. Its core concept is to use algorithms to model and learn from data, thereby enabling predictions or classifications of new data. The basic steps of machine learning include: data collection, data preparation, model selection, training, evaluation, and parameter tuning.

[0039] This application is applied to the scenario of archive processing. The specific scenario is to identify the global information in the archive through a large model, process the global information through a small model, obtain the processing results, and finally achieve the task of the target archive based on the processing results.

[0040] In the field of archival management and information processing, with the rapid development of information technology and the dramatic increase in the volume of archival data, efficient and accurate processing and analysis of this data has become a pressing challenge. Traditional archival processing methods often rely on manual operations or simple automated tools, which are insufficient for handling large-scale archival tasks with complex semantic relationships. Manual processing is time-consuming and labor-intensive, and is easily influenced by personal experience and subjective judgment. Simple automated tools are often only capable of basic tasks such as classification and retrieval, and struggle to handle complex tasks involving deep semantic understanding and cross-archival information integration. The rapid development of artificial intelligence (AI) technology has provided new insights into archival processing. In particular, continuous advances in natural language processing (NLP) and machine learning (ML) technologies have enabled computers to gain a deeper understanding and analysis of textual data. However, single models still face numerous challenges in handling complex archival tasks, resulting in limited analytical accuracy.

[0041] To this end, this application uses a preset large archive processing model to perform preliminary processing on the archives to be processed to obtain global information about the archives to be processed. The preliminary processing includes at least one of global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration. The large archive processing model is trained through a combination of unsupervised and supervised learning. The global information is finally processed using a preset small archive processing model to obtain a processing result. The final processing includes at least one of keyword extraction, archive classification, and generation of archive summaries. The small archive processing model is trained through supervised learning. The processing result is used to process the target archive task, wherein the target archive task includes archive management, information query, or decision support for the archives to be processed. This method can achieve the effect of efficiently and accurately processing archive tasks.

[0042] In the embodiment of the present application, the execution entity may be an archive task processing device in the archive task processing system. In actual applications, the archive task processing device may be an electronic device such as a terminal device and a server, and no limitation is made here.

[0043] The following combination Figure 1 The method for processing archive tasks in an embodiment of the present application is described in detail.

[0044] Please see Figure 1 , Figure 1 A flowchart of a method for processing archive tasks provided in an embodiment of the present application is shown as follows: Figure 1 The methods shown for processing archive tasks include: Step 110: Perform preliminary processing on the files to be processed using a preset file processing model to obtain global information of the files to be processed.

[0045] Among them, the preliminary processing includes: at least one of global understanding, context analysis, extraction of key information, extraction of semantic relationships and cross-archival information integration. It can also include processes such as analysis and summary of the overall meaning of the archive. Global understanding includes understanding of global meaning and effect, and can also include understanding of the layout of the global structure. Context analysis includes context association analysis, meaning analysis and specific context content analysis. Extracting key information includes extracting key texts, phrases or words. Semantic relationships can be semantic relationships of context, or semantic relationships between different documents. Cross-archival information integration includes information integration of documents to be processed and documents uploaded by other systems or other departments. The archives to be processed can be archives of various departments and units, such as school student archives, unit employee archives or documents of relevant departments, etc. The present application is not limited to this. The large archive processing model is trained by combining unsupervised learning and supervised learning.

[0046] In some embodiments of the present application, the global information includes at least one of the global definition of the file to be processed, context connection, key text, and semantic relationship; the processing result includes at least one of the keyword, file type, and file summary of the file to be processed.

[0047] In the above process of the present application, by obtaining the global information and the processing result, it can provide a data basis for the subsequent processing of the target file task.

[0048] Among them, the global information may further include at least one of the context connection, key text, and semantic relationship of other files during the cross-file information integration, the name and address of other files, etc. The processing result may further include results such as the file directory, the unit to which the file belongs, and the file usage.

[0049] In some embodiments of the present application, before obtaining the global information of the file to be processed by preliminarily processing the file to be processed through a preset file processing large model, Figure 1 the method shown further includes: constructing a file set of multiple types of historical files; preprocessing the file set to obtain training samples, where the preprocessing includes at least one of word segmentation, stop word removal, and stemming; adopting the masking method in unsupervised learning, using the training samples to train a basic neural network architecture to obtain an initial file processing large model; annotating the file set to obtain an annotated file set; adopting the supervised learning method, using the annotated file set to adjust the parameters of the initial file processing large model to obtain a file processing large model.

[0050] In the above process of the present application, after constructing the file set and training the initial file processing large model by using the unsupervised masking method, and then fine-tuning the initial file processing large model by using the supervised learning method to obtain the file processing large model, training the file processing large model by using the double-supervised learning method can quickly and accurately identify the file to be processed and obtain the global information.

[0051] Among them, the preprocessing may further include processing such as adding, deleting, modifying, querying, truncating, and merging the data in the file. The training samples include multiple files. Annotating the file set may include annotating the words, file types, or other key information of the files in the file set. Word segmentation can use a professional word segmentation tool or model to segment the text and split the sentence into words or phrases. Stop word removal includes removing the stop words in the text, such as "of", "is", "in", etc., to reduce noise. Stemming can perform stemming on English text and restore the word to its basic form. The preprocessed text will be converted into an input form understandable by the model, such as a word embedding vector.

[0052] Optionally, the masking method in unsupervised learning is adopted, and the basic neural network architecture is trained with training samples to obtain an initial archive processing large model. The keywords, fields or texts in the archive set can be masked by random masking. The masked archive set is input into the basic neural network architecture for training, and the masked output keywords, fields or texts are compared with the masked keywords, fields or texts, and the basic neural network architecture is continuously adjusted so that the final output is the pre-masked content to obtain the initial archive processing large model.

[0053] The basic neural network architecture can adopt a Transformer architecture or BERT, using a self-attention mechanism to capture long-range dependencies in archival text and a multi-layer encoder-decoder structure to achieve complex semantic understanding and generation. Large-scale archival datasets are collected for archival processing, including various types of archival documents, historical records, and reports, to ensure data diversity and comprehensiveness. The collected datasets are used to pre-train the large archival processing model, enabling it to learn the representation and understanding capabilities of archival knowledge, including vocabulary, syntax, semantics, and contextual relationships.

[0054] In some embodiments of the present application, before performing preliminary processing on the to-be-processed archive using a preset archive processing model to obtain global information of the to-be-processed archive, Figure 1 The method shown also includes: processing the archives in the archive set through the large archive processing model to obtain multiple global information of the archive set; marking the archive type, summary and keywords in the multiple global information to obtain a second training sample; using a supervised learning method, using the second training sample to train the basic convolutional neural network to obtain a small archive processing model.

[0055] In the above process, the present application can use the annotation of global information as a training sample for supervised learning of the case processing small model, and the obtained case processing small model can process global information and accurately obtain processing results.

[0056] Among them, the archive type, summary, and keywords in multiple global information are labeled. The archive type, summary, and keywords of each global information can be labeled separately, and the labeled global information can be used as the second training data. In addition, the labeled information can also include keywords, text, or archive summaries. The small archive processing model selects a convolutional neural network (CNN) for text classification and a sequence-to-sequence (Seq2Seq) model for summary generation based on the needs of the specific task, including the specific task execution scenario. For each specific archive task, task-related datasets are collected and prepared. These datasets should contain the specific information required for the task, such as keyword annotations, classification labels, or summary samples. The small archive processing model is specifically trained using task-related datasets so that it can complete specific tasks efficiently and accurately based on the global information provided by the large archive processing model. During the training process, transfer learning or fine-tuning techniques can be used to transfer part of the knowledge of the large archive processing model to the small model to improve the performance of the small model. The specific partial knowledge can be set according to needs.

[0057] Optionally, a supervised learning method is used to train the basic convolutional neural network using the second training sample to obtain a small archive processing model, including: inputting the annotated global information as the second training sample into the basic convolutional neural network, comparing the output content with the annotated content, and adjusting the model parameters of the basic convolutional neural network according to the comparison results until the cross entropy storage calculated from the output content and the annotated content meets the expected requirements, and then the small archive processing model is trained.

[0058] Among them, a large archive processing model is constructed for understanding and processing global archival information, and a small archive processing model is constructed for rapid response to specific archival tasks; among them, the large archive processing model is responsible for processing the complex semantic relationships, contextual understanding and cross-archival information integration of archives, and the small archive processing model is used for special optimization for specific tasks, including keyword extraction, classification and summary generation; the large archive processing model is pre-trained using large-scale archival data sets to enable it to have archival knowledge understanding and analysis capabilities; for specific archival tasks, task-related data sets are collected and prepared, and the small archive processing model is specially trained to enable it to complete specific tasks based on the global information provided by the large model; combining the global understanding ability of the large archive processing model and the special processing ability of the small archive processing model, the input complex archival tasks are processed in a hierarchical manner, firstly, the large archive processing model provides global information and contextual understanding, and then the small archive processing model executes specific tasks and outputs results.

[0059] In some embodiments of the present application, the preliminary processing is cross-archival information integration, and the archives to be processed are preliminarily processed through a preset archive processing model to obtain global information of the archives to be processed, including: constructing an initial relationship graph between the archives to be processed and the associated archives; optimizing the information in the initial relationship graph through the archive processing model to obtain a relationship graph, and the global information includes the relationship graph.

[0060] In the above process, when processing cross-archival information integration, the present application can prioritize the relationship map between the archives to be processed and the associated archives as global information to provide a data basis for subsequent archive processing small model processing.

[0061] Among them, the initial relationship map includes the relationship between the files to be processed and the associated files, including directory relationships, content relationships, and type relationships.

[0062] Optionally, the information in the initial relationship map is optimized through the archive processing big model, including deleting useless information in the initial relationship map, such as auxiliary words and page numbers, and extracting the main relationship content in the initial relationship map to obtain a relationship map.

[0063] The integration of cross-archival information and the construction of a correlation graph can be achieved in the following way: For scenarios where multiple archival information needs to be integrated to complete a task, this application uses the memory network or graph neural network capabilities of a large model to construct a correlation graph between archives. The specific steps are as follows: Entity and Relation Extraction: Extract entities (such as names of people, places, and events) and their relationships (such as "A is B's father," "C had event D," etc.) from each archival text. This can be achieved through named entity recognition (NER) and relation extraction techniques.

[0064] Preliminary Entity Relationship Diagram Construction: Represent the extracted entities and relationships as nodes and edges, and construct a preliminary entity relationship diagram. Nodes represent entities, and edges represent relationships between entities. At this point, the diagram may contain some incorrect or redundant relationships.

[0065] Optimizing the association graph: Leveraging the semantic understanding capabilities of the archival processing model, we refine and optimize the initial entity-relationship diagram. The archival processing model can identify implicit relationships (such as those inferred from context) and remove erroneous relationships (such as those incorrectly extracted due to text misunderstandings). The optimized association graph more accurately reflects the connections between archives.

[0066] Application of the association graph: The optimized association graph is used as one of the inputs to the archive processing model. When handling specific tasks, the archive processing model can utilize the information in the association graph, such as the associations between entities and the temporal relationships between events, to assist in completing the task. For example, in the archive summary generation task, the archive processing model can generate a more accurate and coherent summary based on the key entities and relationships in the association graph.

[0067] Taking archive classification and summary generation as an example, the comprehensive application of the combination of large and small models, the attention mechanism, and the association graph in the present invention is explained. The large archive processing model processes the archive text, outputs a global information representation, and constructs an association graph between archives. When processing the classification task, the small archive processing model uses the attention mechanism to selectively integrate the global information of the large model to improve the accuracy of classification. When processing the summary generation task, the small model uses the key entities and relationships in the association graph to generate a more accurate and coherent summary. In this way, the present invention achieves efficient processing of complex archive tasks.

[0068] Optionally, during the use of the large and small archive processing models, you can continuously collect user feedback and task execution results to continuously optimize and iterate the large and small archive processing models. This can be achieved by introducing new datasets, adjusting model structure or parameters, and other methods to improve model performance and accuracy. Regularly evaluate the synergy between the large and small models to ensure optimal information flow and task allocation between them.

[0069] Step 120: Perform final processing on the global information through the preset file processing model to obtain the processing result.

[0070] The final processing includes at least one of extracting keywords, classifying archives, and generating archive summaries. It may also include generating a directory and a summary. The archive processing model is trained through supervised learning.

[0071] In some embodiments of the present application, the global information is finally processed by a preset archive processing small model to obtain a processing result, including: obtaining a correlation score between the archive processing large model and the archive processing small model; performing weighted summation of the information in the global information according to the correlation score through the attention mechanism or gating mechanism of the archive processing small model to obtain partial global information; and finally processing the partial global information through the archive processing small model to obtain a processing result.

[0072] In the above process, the present application can filter the information in the global information through the attention mechanism or gating mechanism of the archive processing small model, select the most important part of the global information as the data basis for the final processing, and obtain more accurate processing results.

[0073] Specifically, the method for obtaining global information and processing results can be obtained through the following steps: Assume that after the archive processing model encodes the input archive text, the output global information is represented as ,in, is the hidden state vector at position i, i is a positive integer, and n is the length of the output sequence; S2: When the file processing model is processing a specific task, the current state is represented as , where t represents the number of time steps or layers in the processing of the small model; S3: Use the dot product attention mechanism to calculate the current state of the small file processing model Handle large models with archives per output location The correlation score between , the calculation formula is: ; in, express The transpose of is the output vector of the large model for archival processing; S4: Relevance score Apply the softmax function for normalization to obtain the attention weight , and the sum of the weights of all positions is 1; = et; S5: According to attention weight , perform weighted summation on the output H of the large model to obtain the context information required by the small model for archival processing ; = ; S6: context information Current state of the small model with archive processing Perform fusion, update the state of the archive processing small model, and obtain the new state ; S7: Repeat steps S3 to S6 until the archive processing model completes the processing of the specific task and outputs the final processing result.

[0074] Optionally, the attention mechanism or gating mechanism of the archive processing model is used to perform weighted summation of the global information according to the relevance score to obtain partial global information. The archive processing model is used to perform final processing on the partial global information to obtain the processing result, which can be achieved in the following way: Calculate a relevance score: For each current state of the small model (such as the hidden state in an RNN or the query vector in a Transformer) when processing the archival text, calculate a relevance score between it and the output of the large model (such as the encoder's output representation). This is typically implemented using dot products, bilinear functions, or trainable feedforward neural networks. A higher score indicates a greater relevance between the large model's output and the current state of the small model.

[0075] Weighted summation: Based on the calculated relevance scores, the outputs of the large model are weighted and summed. The weights are obtained by normalizing the relevance scores with a softmax function. This allows the small model to dynamically select and integrate global information based on its relevance to the large model output, forming contextual information.

[0076] Application of contextual information: The contextual information obtained through weighted summation is used as additional input to the small model, or is concatenated or added to the small model's state to assist the small model in completing specific archival tasks. For example, in text classification tasks, contextual information can help the small model more accurately understand the semantics of the text, thereby improving classification accuracy.

[0077] Step 130: Process the target file task based on the task processing result.

[0078] Target archive tasks include archive management, information query, or decision support for pending archives. Other tasks may also include catalog query and archive application. Archive management can include archive classification or storage. Information query can include searching for key information or archive types within pending archives. Decision support includes application, retrieval, or linking of target archives with other archives.

[0079] In some embodiments of the present application, target archive tasks are processed through the processing results, including: constructing a task list of archives to be processed through the processing results, wherein the task list includes archive management, information query and decision support; matching the target archive tasks in the task list, and automatically executing the target archive tasks.

[0080] In the above process, this application can accurately list tasks in advance, quickly obtain target archive tasks through task matching, and automatically execute target archive tasks.

[0081] The task list may also include other tasks such as directory query and archive application.

[0082] Optionally, the target archive tasks in the task list are matched and automatically executed by matching the target archive tasks by keyword similarity matching after keyword extraction, or directly obtaining the target archive tasks by similarity matching based on the task name or text.

[0083] In the above Figure 1 In the process shown, the present application performs preliminary processing on the pending archives using a preset large archive processing model to obtain global information about the pending archives, wherein the preliminary processing includes: global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration. The large archive processing model is trained through a combination of unsupervised learning and supervised learning. The global information is finally processed using a preset small archive processing model to obtain a processing result. The final processing includes: extracting keywords, classifying archives, and generating archive summaries. The small archive processing model is trained through supervised learning. The target archive task is processed based on the processing result, wherein the target archive task includes archive management, information query, or decision support for the pending archives. This method can achieve the effect of efficiently and accurately processing archive tasks.

[0084] The following combination Figure 2 The implementation method of processing archive tasks in an embodiment of the present application is described in detail.

[0085] Please see Figure 2 , Figure 2 A flowchart of an implementation method for processing archive tasks provided in an embodiment of the present application is shown in FIG. Figure 2 The implementation method of the processing archive task shown includes: Step 210: Construct an initial large model.

[0086] Specifically: Adopting the mask method in unsupervised learning, the basic neural network architecture is trained with training samples to obtain the initial file processing model.

[0087] Step 220: Train the initial large model using data to obtain a large model.

[0088] Specifically: A supervised learning approach is adopted to adjust the parameters of the initial archive processing model using the labeled archive set to obtain the archive processing model.

[0089] Step 230: Process the files to be processed through the large model to obtain global information.

[0090] Specifically: The archives in the archive set are processed through the archive processing large model to obtain multiple global information of the archive set.

[0091] Step 240: Construct an initial small model.

[0092] Specifically: Use the preset convolutional neural network structure as the initial small model.

[0093] Step 250: Train the initial small model using data to obtain a small model.

[0094] Specifically: The archives in the archive set are processed through the large archive processing model to obtain multiple global information of the archive set; the archive type, summary and keywords in the multiple global information are marked to obtain the second training sample; using a supervised learning method, the basic convolutional neural network is trained with the second training sample to obtain the small archive processing model.

[0095] Step 260: Process the global information through the small model to obtain a processing result.

[0096] Specifically: obtain the correlation score between the large archive processing model and the small archive processing model; use the attention mechanism or gating mechanism of the small archive processing model to perform weighted summation of the global information according to the correlation score to obtain partial global information; use the small archive processing model to perform final processing on the partial global information to obtain the processing result.

[0097] Step 270: Execute the file task according to the processing result.

[0098] Specifically: Match the target archive tasks in the task list and automatically execute the target archive tasks.

[0099] also, Figure 2 The specific methods and steps shown can be found in Figure 1 The method shown here will not be described in detail.

[0100] Previous article passed Figure 1-Figure 2 Describes the method of processing archive tasks, the following is combined with Figure 3-Figure 4 Describes the device that processes archive tasks.

[0101] Please refer to Figure 3 , is a schematic block diagram of a device 300 for processing archive tasks provided in an embodiment of the present application. The device 300 may be a module, program segment or code on an electronic device. The device 300 is similar to the above Figure 1 The method embodiment corresponds to the embodiment that can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device 300 can be found in the description below. To avoid repetition, detailed description is appropriately omitted here.

[0102] Optionally, the device 300 includes: A first processing module 310 is configured to perform preliminary processing on the archive to be processed using a preset archive processing model to obtain global information about the archive to be processed. The preliminary processing includes at least one of global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration. The archive processing model is trained using a combination of unsupervised and supervised learning. A second processing module 320 is configured to perform final processing on the global information using a preset archive processing model to obtain a processing result, wherein the final processing includes at least one of extracting keywords, classifying archives, and generating archive summaries. The archive processing model is trained using supervised learning. The third processing module 330 is used to process target archive tasks based on the processing results, wherein the target archive tasks include archive management, information query or decision support of the archives to be processed.

[0103] Optionally, the global information includes: at least one of the global definition, contextual connection, key text and semantic relationship of the file to be processed; the processing result includes: at least one of the keywords, file type and file summary of the file to be processed.

[0104] Optionally, the device further includes: The first training module is used for constructing an archive set of various types of historical archives before the first processing module performs preliminary processing on the archives to be processed through a preset archive processing large model to obtain global information of the archives to be processed; preprocessing the archive set to obtain training samples, wherein the preprocessing includes: at least one of word segmentation, removal of stop words and stem extraction; using the masking method in unsupervised learning, using the training samples to train the basic neural network architecture to obtain the initial archive processing large model; labeling the archive set to obtain the labeled archive set; using the supervised learning method, using the labeled archive set to adjust the parameters of the initial archive processing large model to obtain the archive processing large model.

[0105] Optionally, the device further includes: The second training module is used for the second processing module to process the archives in the archive set through the preset archive processing large model before performing preliminary processing on the archives to be processed to obtain global information of the archives to be processed, thereby obtaining multiple global information of the archive set; marking the archive type, summary and keywords in the multiple global information to obtain a second training sample; and using the second training sample to train the basic convolutional neural network in a supervised learning manner to obtain a small archive processing model.

[0106] Optionally, the second processing module is specifically configured to: Obtain the correlation score between the large archive processing model and the small archive processing model; perform weighted summation of the information in the global information according to the correlation score through the attention mechanism or gating mechanism of the small archive processing model to obtain partial global information; perform final processing on the partial global information through the small archive processing model to obtain the processing result.

[0107] Optionally, the preliminary processing is cross-archival information integration, and the first processing module is specifically used to: Construct an initial relationship map between the files to be processed and the associated files; optimize the information in the initial relationship map through the file processing model to obtain the relationship map, and the global information includes the relationship map.

[0108] Optionally, the third processing module is specifically configured to: Based on the processing results, a task list of archives to be processed is constructed, where the task list includes archive management, information query and decision support; the target archive tasks in the task list are matched and automatically executed.

[0109] Please refer to Figure 4 This is a schematic block diagram of a device for processing archive tasks provided in an embodiment of the present application. The device may include a memory 410 and a processor 420. Optionally, the device may also include: a communication interface 430 and a communication bus 440. The device is similar to the above-mentioned Figure 1 The method embodiment corresponds to the embodiment that can be executed Figure 1 The various steps involved in the method embodiment and the specific functions of the device can be found in the description below.

[0110] Specifically, the memory 410 is used to store computer-readable instructions.

[0111] Processor 420 is used to process the readable instructions stored in the memory and can execute Figure 1 The steps in the method.

[0112] The communication interface 430 is used for signaling or data communication with other node devices, for example, for communication with a server or terminal, or for communication with other device nodes, but the embodiments of the present application are not limited thereto.

[0113] The communication bus 440 is used to realize direct connection and communication among the above components.

[0114] Among them, the communication interface 430 of the device in the embodiment of the present application is used to communicate signaling or data with other node devices. The memory 410 can be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 410 can also be at least one storage device located away from the aforementioned processor. The memory 410 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 420, the electronic device executes the above-mentioned Figure 1 The method process shown. The processor 420 can be used on the device 300 and is used to perform the functions of the present application. For example, the above-mentioned processor 420 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, but the embodiments of the present application are not limited thereto.

[0115] The embodiment of the present application further provides a readable storage medium, wherein when the computer program is executed by a processor, Figure 1 The method process in the illustrated method embodiment is performed by the electronic device.

[0116] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0117] In summary, the embodiments of the present application provide a method, apparatus, device, and readable storage medium for processing archival tasks. The method includes performing preliminary processing on the archives to be processed using a preset archival processing large model to obtain global information of the archives to be processed, wherein the preliminary processing includes: global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration. The archival processing large model is trained by combining unsupervised learning and supervised learning; performing final processing on the global information using a preset archival processing small model to obtain a processing result. The final processing includes: extracting keywords, classifying archives, and generating at least one of archive summaries. The archival processing small model is trained by supervised learning; and processing the target archival task based on the processing result. The target archival task includes archival management, information query, or decision support for the archives to be processed. This method can achieve the effect of efficiently and accurately processing archival tasks.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0119] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0120] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0121] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0122] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0123] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for processing an archive task, characterized in that: include: Performing preliminary processing on the archives to be processed using a preset archive processing model to obtain global information about the archives to be processed, wherein the preliminary processing includes at least one of: global understanding, context analysis, extraction of key information, extraction of semantic relationships, cross-archival information integration, and analysis and summary of the overall meaning of the archives. The archive processing model is trained by combining unsupervised learning and supervised learning. The global information is finally processed by a preset archive processing model to obtain a processing result, wherein the final processing includes: extracting keywords, classifying archives, generating archive summaries, generating a directory, and generating a summary, and the archive processing model is trained by supervised learning; The target archive tasks are processed based on the processing results, wherein the target archive tasks include archive management, information query, decision support, directory query and archive application of the archives to be processed.

2. The method according to claim 1, characterized in that The global information includes: At least one of a global definition, contextual connection, key text, and semantic relationship of the to-be-processed archive; The processing result includes: at least one of the keywords, file type and file summary of the file to be processed.

3. The method according to claim 2, characterized in that Before performing preliminary processing on the to-be-processed archives by using the preset archive processing model to obtain global information of the to-be-processed archives, the method further includes: Construct archive collections of various types of historical archives; Preprocessing the archive set to obtain training samples, wherein the preprocessing includes: at least one of word segmentation, stop word removal, and stem extraction; Adopting the mask method in unsupervised learning, the basic neural network architecture is trained using the training samples to obtain an initial file processing model; Annotating the file set to obtain an annotated file set; The parameters of the initial archive processing large model are adjusted using the labeled archive set in a supervised learning manner to obtain the archive processing large model.

4. The method according to claim 3, characterized in that Before performing preliminary processing on the to-be-processed archives by using the preset archive processing model to obtain global information of the to-be-processed archives, the method further includes: Processing the archives in the archive set by using the archive processing macro model to obtain a plurality of global information of the archive set; Marking the file type, summary and keywords in the plurality of global information to obtain a second training sample; The basic convolutional neural network is trained using the second training sample in a supervised learning manner to obtain the small file processing model.

5. The method according to any one of claims 1 to 4, characterized in that The global information is finally processed by the preset archive processing model to obtain a processing result, including: Obtaining a correlation score between the large archive processing model and the small archive processing model; Performing weighted summation of the information in the global information according to the relevance score through the attention mechanism or gating mechanism of the archive processing mini-model to obtain partial global information; The part of the global information is finally processed by the archive processing small model to obtain the processing result.

6. The method according to any one of claims 1 to 4, characterized in that The preliminary processing is the cross-archival information integration, and the preliminary processing of the archives to be processed is performed using a preset archive processing model to obtain global information of the archives to be processed, including: Constructing an initial relationship graph between the files to be processed and the associated files; The information in the initial relationship map is optimized by the archive processing macro model to obtain a relationship map, and the global information includes the relationship map.

7. The method according to any one of claims 1 to 4, characterized in that The target file task is processed based on the task processing results, including: Constructing a task list of the files to be processed based on the processing results, wherein the task list includes file management, information query and decision support; Match the target archive task in the task list and automatically execute the target archive task.

8. A device for processing archive tasks, characterized in that: include: A first processing module is configured to perform preliminary processing on the archive to be processed using a preset archive processing model to obtain global information about the archive to be processed, wherein the preliminary processing includes at least one of: global understanding, context analysis, extraction of key information, extraction of semantic relationships, and cross-archival information integration; the archive processing model is trained by combining unsupervised learning and supervised learning; a second processing module, configured to perform final processing on the global information using a preset archive processing model to obtain a processing result, wherein the final processing includes at least one of extracting keywords, classifying archives, and generating archive summaries, and the archive processing model is trained using a supervised learning method; The third processing module is used to process the target archive task based on the processing results, wherein the target archive task includes archive management, information query or decision support of the archive to be processed.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method according to any one of claims 1 to 7 are executed.

10. A computer-readable storage medium, characterized in that include: A computer program, when running on a computer, causes the computer to perform the method according to any one of claims 1 to 7.

Images