A multi-format file structuring method and system for nuclear power plants based on deep learning

Through deep learning technology, the multi-format files of nuclear power plants are converted into structured XML files, which solves the problem of file management and retrieval difficulties, reduces labor costs, and meets confidentiality requirements, and realizes rapid retrieval and management of files.

CN115048909BActive Publication Date: 2024-11-12CHINA NUCLEAR POWER OPERATION TECH CORP +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110256994.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-09
Publication Date
2024-11-12
Estimated Expiration
2041-03-09

AI Technical Summary

Technical Problem

Multi-format file management and retrieval difficulties arising from nuclear power plants, and existing methods rely on manual processing, consume labor costs and cannot meet confidentiality requirements.

Method used

Using deep learning technology, multi-format files are converted into structured xml files through file acquisition, content extraction, segmentation, word vector generation and content classification modules, and the textCNN algorithm is used to determine the classification of file content and build the xml format structure. .

Benefits of technology

It realizes rapid retrieval and management of files, reduces labor costs, and provides guarantees for confidentiality of files.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115048909B_ABST
    Figure CN115048909B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for structural processing of multi-format files of a nuclear power plant, comprising the following steps: obtaining files to be structured from a server storing files of the nuclear power plant to form a file set D; utilizing a file model content extraction module to traverse all files in the set D, selecting a suitable processing method according to the file suffix, extracting the content of the file, and converting it into a text file format, while maintaining the relationship between paragraphs, and replacing pictures and table data therein with corresponding titles, thereby forming a text file set D′; the beneficial effect is that utilizing computer technology and deep learning algorithms to perform structural processing on a large number of nuclear power plant files, not only facilitating the retrieval and utilization of files and reducing labor costs, but also ensuring the confidentiality of files.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of deep learning in artificial intelligence, and specifically relates to a multi-format file structuring method and system for a nuclear power plant based on deep learning. Background Art

[0002] During the operation and management of nuclear power plants, a large number of files in different formats will be generated, including various technical files and business files. The diversity of file types and file contents makes the management, retrieval and use of files very inconvenient. In order to solve this problem, the existing method is to use a template to construct a document in XML file format. On the one hand, it realizes the structuring of file content, making file retrieval faster and reading easier; on the other hand, it realizes the standardization of file formats and frees the reading of multi-format files from software limitations. However, this method is mainly based on manual processing in practice, which consumes a lot of manpower costs. Nowadays, artificial intelligence technology is developing rapidly, and deep learning technology has achieved achievements in replacing humans in many fields. Therefore, a method based on deep learning can be used to structure multi-format files to replace manual processing.

[0003] During the operation and management of nuclear power plants, a large number of documents related to technology or business are generated. These documents differ in file type and content. A large number of different documents is a difficult task for document management. At the same time, it is also difficult to quickly retrieve the required documents when reference is needed. In the existing methods, the file content can be converted into an XML file, and the content can be saved in the XML file according to the hierarchical structure, so that it is convenient to search by keywords in the file content. However, the existing methods for structuring files are mainly manual processing, which is time-consuming and labor-intensive, and there are certain confidentiality requirements for the files in nuclear power plants. Summary of the invention

[0004] The purpose of the present invention is to propose a method and system for structured processing of multi-format files of a nuclear power plant by combining relevant knowledge and techniques of deep learning.

[0005] The technical solution of the present invention is as follows: A method for structured processing of multi-format files of a nuclear power plant comprises the following implementation steps:

[0006] Step 001: Obtain files that need to be structured from a server storing files in a nuclear power plant to form a file set D;

[0007] Step 002: traverse all the files in set D using the file content extraction module, select a suitable processing method according to the file extension, extract the content in the file, and convert it into a text file format, while maintaining the relationship between the paragraphs, and replace the pictures and table data with corresponding titles, thereby forming a text file set D′;

[0008] Step 003: Use the file content segmentation module to traverse each text file in the set D′, divide the title and paragraph into an independent text data unit respectively, and save it into a text file to obtain a set T. At the same time, the order of paragraphs and titles in each file is recorded in the corresponding CSV file to obtain a set T′;

[0009] Step 004: According to the records in set T, each independent text data unit in set T is converted into a corresponding fixed-length word vector using the trained word2vetor model, and according to the arrangement order of each text data unit in the corresponding file in set T′, the word vector is stored in the corresponding CSV file to obtain set V;

[0010] Step 005: Take the word vectors in set V as input, use the deep learning algorithm TextCNN to classify the input word vectors, and determine whether the corresponding text data belongs to the title, paragraph, figure, or table. The process is as follows:

[0011] (1) Select a CSV file from V and the corresponding CSV file from the set T′, and sequentially convert a set of n word vectors X = {x 1 ,x 2 ,…,x n} is passed into the TextCNN algorithm, where x i is a numeric vector of length k;

[0012] (2) TextCNN first uses convolution to calculate a set of numerical word vectors to obtain a set of feature graphs F = {f 1 ,f 2 ,…,f i ,…,f n-h+1}, where h represents the longitudinal length of the convolution kernel, and f i Represents a feature map;

[0013] (3) Perform global maximum pooling calculation on each feature map to obtain a feature vector where c i represents a feature value;

[0014] (4) Using the fully connected layer and the softmax layer, calculate the classification with the largest classification probability value corresponding to the word vector, and output the category prediction results of each word vector;

[0015] Step 006: According to the calculation results of the TextCNN algorithm, determine the classification of the corresponding word vector, and then perform the same operation on all word vectors in the same file and record them in the corresponding CSV file in order;

[0016] Step 007: Perform the above steps for each file in the set V until the word vectors in all files in the set V are classified, thereby obtaining a CSV file set R recording the classification results;

[0017] Step 008: Based on the syntax format of the set R and the XML file, use the structured file generation module to construct a structured file in XML format for the files in the set D.

[0018] A system for structured processing of multi-format files of a nuclear power plant comprises a file acquisition module, a file content extraction module, a file content segmentation module, a word vector generation module, a content classification module and a structured file generation module.

[0019] The file acquisition module acquires the files that need to be structured from the server where the nuclear power plant files are stored, and transmits the acquired files to the file content extraction module.

[0020] The file content extraction module: according to the file data transmitted by the file acquisition module, the text content in the acquired file is extracted using computer technology corresponding to the transmitted file format, and converted into a text file, in which only the corresponding titles of pictures and tables are saved, and then the text file is transmitted to the file content segmentation module.

[0021] The file content segmentation module: based on the text file obtained after processing by the file content extraction module, the content therein is segmented according to paragraphs and titles, each paragraph and title is used as a separate data unit, in which pictures and tables are represented by titles, and the arrangement order of each segment in the file is recorded at the same time. After that, the processed text data unit and segment arrangement order information are passed to the word vector generation module.

[0022] The word vector generation module uses the trained word2vector model to process the text data units passed in by the file content segmentation module, converts the data units obtained after the file content segmentation processing of a single file, obtains a set of word vectors with a fixed length and arranges them according to the segment arrangement order information, and then passes the arranged word vector file data into the content classification module.

[0023] The content classification module uses the trained deep learning algorithm TextCNN to pass the word vectors generated by the word vector generation module into the TextCNN algorithm one by one for calculation according to their arrangement order in the original file, so as to obtain the content classification of the word vector, and summarize the classification results of each file in order and pass them into the structured file generation module.

[0024] The structured file generation module: constructs a structured file corresponding to the file according to the grammatical format of the XML file, combined with the classification results of the word vectors transmitted by the content classification module, the segmented content corresponding to the word vectors and the arrangement order of the segmented content.

[0025] The beneficial effect of the present invention is that a large number of nuclear power plant documents are processed in a structured manner using computer technology and deep learning algorithms, which not only facilitates the retrieval and utilization of documents and reduces labor costs, but also ensures the confidentiality of the documents. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of a multi-format file structuring method for a nuclear power plant based on deep learning according to the present invention. DETAILED DESCRIPTION

[0027] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] A system for structured processing of multi-format files of a nuclear power plant comprises: a file acquisition module, a file content extraction module, a file content segmentation module, a word vector generation module, a content classification module, and a structured file generation module.

[0029] File acquisition module: obtains the files that need to be structured from the server where the nuclear power plant files are stored, and transmits the obtained files to the file content extraction module.

[0030] File content extraction module: Based on the file data transmitted by the file acquisition module, the text content in the acquired file is extracted using computer technology corresponding to the transmitted file format and converted into a text file, in which only the corresponding titles of pictures and tables are saved. After that, the text file is passed to the file content segmentation module.

[0031] File content segmentation module: Based on the text file obtained after processing by the file content extraction module, the content is segmented according to paragraphs and titles. Each paragraph and title is a separate data unit, where pictures and tables are represented by titles, and the order of each segment in the file is recorded. After that, the processed text data unit and segment order information are passed to the word vector generation module.

[0032] Word vector generation module: Use the trained word2vector model to process the text data units passed in by the file content segmentation module, convert the data units obtained after the file content segmentation processing of a single file, obtain a set of word vectors with a fixed length and arrange them according to the segment arrangement order information, and then pass the arranged word vector file data into the content classification module.

[0033] Content classification module: Using the trained deep learning algorithm TextCNN, the word vectors generated by the word vector generation module are passed into the TextCNN algorithm one by one according to their arrangement order in the original file for calculation to obtain the content classification of the word vector. The classification results of each file are summarized in order and passed to the structured file generation module.

[0034] Structured file generation module: According to the grammatical format of the XML file, combined with the classification results of the word vectors transmitted by the content classification module, the segmented content corresponding to the word vectors and the arrangement order of the segmented content, the structured file corresponding to the file is constructed.

[0035] like Figure 1 As shown, a method for structured processing of multi-format files of a nuclear power plant includes the following implementation steps:

[0036] Step 001: Obtain files that need to be structured from a server storing files in a nuclear power plant to form a file set D.

[0037] Step 002: Use the file model content extraction module to traverse all files in the set D, select the appropriate processing method according to the file extension, extract the content of the file, and convert it into a text file format while maintaining the relationship between the paragraphs. The pictures and tables are replaced with the corresponding titles, thus forming a text file set D′.

[0038] Step 003: Use the file content segmentation module to traverse each text file in the set D', divide the title and paragraph into an independent text data unit, and save it into a text file to obtain a set T. At the same time, the order between paragraphs and titles in each file is recorded in the corresponding CSV file to obtain a set T'

[0039] Step 004: According to the records in set T, use the trained word2vetor model to convert each independent text data unit in set T into a corresponding fixed-length word vector, and store the word vector in the corresponding CSV file according to the arrangement order of each text data unit in the corresponding file in set T′ to obtain set V.

[0040] Step 005: Take the word vectors in set V as input, and use the deep learning algorithm TextCNN to classify the input word vectors to determine whether the corresponding text data belongs to the title, paragraph, figure, table, etc.

[0041] (1) Select a CSV file from V and the corresponding CSV file from the set T′, and sequentially convert a set of n word vectors X = {x 1 ,x 2 ,…,x n} is passed into the TextCNN algorithm, where x i is a numeric vector of length k.

[0042] (2) TextCNN first uses convolution to calculate a set of numerical word vectors to obtain a set of feature graphs F = {f 1 ,f 2 ,…,f i ,…,f n-h+1}, where h represents the longitudinal length of the convolution kernel, and f i Represents a feature map.

[0043] (3) Perform global maximum pooling calculation on each feature map to obtain a feature vector where c i Represents a feature value

[0044] (4) Use the fully connected layer and the softmax layer to calculate the classification with the largest classification probability value corresponding to the word vector, and output the category prediction results of each word vector.

[0045] Step 004: Determine the classification of the corresponding word vector according to the calculation results of the TextCNN algorithm. Then all word vectors in the same file are subjected to the same operation and recorded in the corresponding CSV file in order.

[0046] Step 005: Perform the above steps for each file in the set V until the word vectors in all files in the set V are classified, thereby obtaining a CSV file set R that records the classification results.

[0047] Step 006: Based on the syntax format of the set R and the XML file, use the structured file generation module to construct a structured file in XML format for the files in the set D.

Claims

1. A method for structured processing of multi-format files of a nuclear power plant, characterized in that: The implementation steps include: Step 001: Obtain files that need to be structured from a server storing files in a nuclear power plant to form a file set D; Step 002: traverse all the files in set D using the file content extraction module, select a suitable processing method according to the file extension, extract the content in the file, and convert it into a text file format, while maintaining the relationship between the paragraphs, and replace the pictures and table data with corresponding titles, thereby forming a text file set D′; Step 003: Use the file content segmentation module to traverse each text file in the set D′, divide the title and paragraph into an independent text data unit respectively, and save it into a text file to obtain a set T. At the same time, the order of paragraphs and titles in each file is recorded in the corresponding CSV file to obtain a set T′; Step 004: According to the records in set T, each independent text data unit in set T is converted into a corresponding fixed-length word vector using the trained word2vetor model, and according to the arrangement order of each text data unit in the corresponding file in set T′, the word vector is stored in the corresponding CSV file to obtain set V; Step 005: Take the word vectors in set V as input, use the deep learning algorithm TextCNN to classify the input word vectors, and determine whether the corresponding text data belongs to the title, paragraph, figure, or table. The process is as follows: (1) Select a CSV file from V and the corresponding CSV file from the set T′, and sequentially convert a set of n word vectors X = {x1, x2, ..., x n } is passed into the TextCNN algorithm, where x i is a numeric vector of length k; (2) TextCNN first uses convolution to calculate a set of numerical word vectors to obtain a set of feature graphs F = {f1, f2, ..., f i , ..., f n-h+1 }, where h represents the longitudinal length of the convolution kernel, and f i Represents a feature map; (3) Perform global maximum pooling calculation on each feature map to obtain a feature vector where c i represents a feature value; (4) Using the fully connected layer and the softmax layer, calculate the classification with the largest classification probability value corresponding to the word vector, and output the category prediction results of each word vector; Step 006: According to the calculation results of the TextCNN algorithm, determine the classification of the corresponding word vector, and then perform the same operation on all word vectors in the same file and record them in the corresponding CSV file in order; Step 007: Set v Repeat the above steps for each file in the collection until the collection v Until the word vectors in all files in the file are determined to be classified, a set of CSV files recording the classification results is obtained. R ; Step 008: According to the collection R and the syntax format of XML files, using the structured file generation module to generate the collection D The files in the xml format are used to construct structured files.

2. A system applied to the method for structured processing of multi-format files of a nuclear power plant as claimed in claim 1, characterized in that: It includes file acquisition module, file content extraction module, file content segmentation module, word vector generation module, content classification module, and structured file generation module; The file acquisition module: acquires the files that need to be structured from the server where the nuclear power plant files are stored, and transmits the acquired files to the file content extraction module; The file content extraction module: according to the file data transmitted by the file acquisition module, the text content in the acquired file is extracted by using the computer technology corresponding to the transmitted file format, and converted into a text file, in which only the corresponding titles of the pictures and tables are saved, and then the text file is transmitted to the file content segmentation module; The file content segmentation module: according to the text file obtained after processing by the file content extraction module, the content is segmented according to paragraphs and titles, each paragraph and title is used as a separate data unit, wherein pictures and tables are represented by titles, and the arrangement order of each segment in the file is recorded at the same time, and then the processed text data unit and segment arrangement order information are transmitted to the word vector generation module; The word vector generation module: uses the trained word2vector model to process the text data units passed in by the file content segmentation module, converts the data units obtained after the file content segmentation processing of a single file, obtains a set of word vectors with a fixed length and arranges them according to the segment arrangement order information, and then passes the arranged word vector file data to the content classification module; The content classification module: uses the trained deep learning algorithm TextCNN to pass the word vectors generated by the word vector generation module into the TextCNN algorithm one by one according to their arrangement order in the original file for calculation, obtains the content classification of the word vector, and summarizes the classification results of each file in order and passes them into the structured file generation module; The structured file generation module: constructs a structured file corresponding to the file according to the grammatical format of the XML file, combined with the classification results of the word vectors transmitted by the content classification module, the segmented content corresponding to the word vectors and the arrangement order of the segmented content.