Archive information classification management system based on data mining

Abstract

The invention relates to the technical field of archive information management, and particularly discloses an archive information classification management system based on data mining, which is characterized in that an original supervision file is regularly acquired from a specific website through a timed crawler and an API (Application Program Interface), and the original supervision file is segmented; carrying out word segmentation and stop word removal to obtain keywords of each first paragraph, recording the keywords as a second paragraph set, constructing a global keyword overlap analysis model based on the second paragraph set, carrying out global overlap analysis on the keywords of the supervision file a and the supervision file b, and preliminarily screening whether the supervision file a and the supervision file b contain overlap contents or not; a local keyword overlapping analysis model is constructed according to the keyword set, keyword overlapping indexes in paragraphs in the supervision file a and the supervision file b are analyzed, the overlapping content in the supervision file a and the supervision file b is determined through secondary screening, the problem of overlapping of supervision files of multiple departments can be effectively solved, and information circulation and resource configuration are optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of archive information management, and more specifically, to an archive information classification management system based on data mining. Background Art

[0002] With the vigorous promotion of environmental protection, more stringent and detailed requirements are now put forward for sewage treatment enterprises. The relevant documents are usually complex in content and structure, involving process flow, emission standards, monitoring methods and other aspects. Due to the complex content of the documents, many professional terms and large differences in structure, the traditional manual review and induction methods have obvious shortcomings in processing massive documents. On the one hand, manual interpretation consumes a lot of manpower and time. On the other hand, due to the limited human resources and differences in subjective understanding, it is easy to cause information omissions and understanding deviations, which in turn affects the timeliness of corporate decision-making and environmental governance. To this end, sewage treatment enterprises urgently need to use existing digital technologies to automatically extract information related to their own production and emission management from massive environmental protection standard documents, so as to ensure that the company's operations meet the latest policy requirements and make timely adjustments and optimizations in technology. The existing literature (Zheng Hengyi. Research on text information extraction methods based on deep learning [D]. Shenzhen University, 2022. DOI: 10.27321 / d.cnki.gszdu.2022.002442.) proposes a method for extracting information from unstructured text using deep learning technology.

[0003] Data mining has achieved remarkable results in the fields of text analysis, automatic classification, topic modeling, etc., especially in the processing of unstructured and semi-structured data. Models based on deep learning (such as BERT, GPT and other pre-trained models) can automatically extract key information from text, identify professional terms and implicit semantic relationships, and provide strong technical support for document classification. The existing regulatory departments for sewage treatment include environmental protection, water conservancy, industry and other departments. The management methods and supervision measures of multiple departments are prone to overlap in regulatory documents if there is a lack of coordination, causing sewage treatment companies to repeatedly provide the same information and data to multiple departments, which not only wastes the company's internal resources, but also increases the complexity of information processing. In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides an archive information classification management system based on data mining, which determines the overlapping content in the regulatory file a and the regulatory file b through secondary screening to solve the problem of overlapping existing regulatory documents, which causes sewage treatment enterprises to repeatedly provide the same information and data to multiple departments. It can effectively solve the problem of overlapping regulatory documents of multiple departments, optimize information flow and resource allocation, so as to solve the problems raised in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A file information classification management system based on data mining includes a document collection and processing module, a text feature extraction and representation module, a global overlap analysis module, a local overlap analysis module and a file feedback and update module; the document collection and processing module periodically obtains original regulatory documents from a specific website through a timed crawler and an API interface, and performs segmentation processing on the original regulatory documents to obtain the first paragraph set of regulatory documents a, and then uses a word segmentation tool to extract and represent the first paragraph set D. a Perform word segmentation and remove stop words to obtain the keywords of each first paragraph, which are recorded as the second paragraph set; the text feature extraction and representation module is used to obtain the second paragraph sets of regulatory documents a and regulatory documents b respectively, and obtain the keyword sets of each paragraph in the second paragraph sets of regulatory documents a and regulatory documents b respectively; the global overlap analysis module constructs a global keyword overlap analysis model based on the second paragraph set, performs a global overlap analysis on the keywords of regulatory documents a and regulatory documents b, and preliminarily screens whether regulatory documents a and regulatory documents b contain overlapping content; the local overlap analysis module is used to construct a local keyword overlap analysis model based on the keyword set when it is preliminarily determined that regulatory documents a and regulatory documents b contain overlapping content, analyze the keyword overlap indicators in each paragraph in regulatory documents a and regulatory documents b, and perform a secondary screening to determine the overlapping content in regulatory documents a and regulatory documents b.

[0007] As a further solution of the present invention, the first paragraph set of the supervision document a in the document collection and processing module in, is the first paragraph of the regulatory document a, n is the total number of first paragraphs of the regulatory document a, and then the first paragraph set D is segmented by the word segmentation tool. a Perform word segmentation and remove stop words to obtain the keywords of each first paragraph, record them as the second paragraph set, and get the second paragraph set is the keyword set of the second paragraph of the i-th regulatory document a, and the i-th second paragraph corresponds to the first paragraph of the regulatory document a

[0008] As a further solution of the present invention, the text feature extraction and representation module is used to obtain the second paragraph set of the regulatory document a and the regulatory document b, and the second paragraph set of the regulatory document b. The first paragraph of the regulatory document b The keyword set extracted from the regulatory document a is obtained, m is the number of the first paragraph of the regulatory document b, and the keyword set of each paragraph in the second paragraph set of the regulatory document a and the regulatory document b is obtained respectively. For the keyword set of the regulatory document a Among them, is the b-th keyword in the i-th second paragraph, and B is the total number of keywords in the i-th second paragraph; for the keyword set of regulatory document b Among them, is the d-th keyword in the j-th second paragraph, and D is the total number of keywords in the j-th second paragraph.

[0009] As a further solution of the present invention, the construction steps of the global keyword overlap analysis model in the global overlap analysis module are as follows: Extract the keyword sets and Calculate and The intersection of the two sets, that is, the number of keywords that appear in both regulatory document a and regulatory document b, is denoted as Then calculate the union of the two sets, that is, the total number of all keywords that appear at least once in regulatory document a and regulatory document b, denoted as Divide the number of keywords in the intersection by the number of keywords in the union to obtain the global keyword overlap index S1;

[0010] The formula of the global keyword overlap analysis model is:

[0011]

[0012] In the formula: S1 is the global keyword overlap index, is the set of second paragraphs of regulatory document a, is the set of second paragraphs of regulatory document b, is the number of keywords in the intersection of the set of second paragraphs of regulatory document a and the set of second paragraphs of regulatory document b, is the number of keywords in the union of the set of second paragraphs of regulatory document a and the set of second paragraphs of regulatory document b;

[0013] Compare the global keyword overlap index with the preset global keyword overlap index threshold. If the global keyword overlap index is greater than or equal to the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b contain overlapping content; if the global keyword overlap index is less than the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b do not contain overlapping content.

[0014] As a further solution of the present invention, the specific construction steps of the local keyword overlap analysis model in the local overlap analysis module are as follows: for each pair of second paragraphs in regulatory document a and regulatory document b, calculate the ratio of the intersection to the union of the respective keyword sets in this pair of paragraphs, that is, obtain the local keyword similarity. Add up the local keyword similarities of all second paragraphs, and then divide by the product of the total number of second paragraphs in regulatory document a and regulatory document b to obtain the local keyword overlap index S2;

[0015] The formula of the local keyword overlap analysis model is:

[0016]

[0017] In the formula: S2 is the local keyword overlap index, is the keyword set of the i-th second paragraph of regulatory document a, is the keyword set of the j-th second paragraph of regulatory document b, n is the total number of first paragraphs of regulatory document a, and m is the total number of first paragraphs of regulatory document b.

[0018] As a further solution of the present invention, the document feedback update module constructs an overlap degree analysis model based on the global keyword overlap index and the local keyword overlap index, quantitatively analyzes the overlap degree of keywords between regulatory document a and regulatory document b, and obtains the overlap degree K of keywords between regulatory document a and regulatory document b by weighted fusion of the global keyword overlap index S1 and the local keyword overlap index S2 between regulatory document a and regulatory document b o , where the influence of the global keyword overlap index and the local keyword overlap index is balanced by the weight coefficient γ.

[0019] The formula of the overlap degree analysis model is:

[0020] K o =γ·S1+(1 - γ)·S2;

[0021] In the formula: K o is the overlap degree of keywords between regulatory document a and regulatory document b, γ is the weight for balancing the global keyword overlap index and the local keyword overlap index, S1 is the global keyword overlap index, and S2 is the local keyword overlap index.

[0022] It should be noted that the value range of γ is between 0 and 1. When γ is higher, the influence of the global index is greater; when γ is lower, the influence of the local index is greater.

[0023] As a further solution of the present invention, the overlapping degree of keywords between regulatory document a and regulatory document b is compared with a preset document overlapping degree threshold. If the overlapping degree of keywords between regulatory document a and regulatory document b is greater than or equal to the preset document overlapping degree threshold, there is an overlap between regulatory document a and regulatory document b; if the overlapping degree of keywords between regulatory document a and regulatory document b is less than the preset document overlapping degree threshold, there is no overlap between regulatory document a and regulatory document b.

[0024] Technical effects and advantages of an archive information classification and management system based on data mining according to the present invention: The present invention regularly obtains original regulatory documents from specific websites through timed crawlers and API interfaces, segments the original regulatory documents, and then obtains the keywords of each first paragraph by word segmentation and removing stop words, which are recorded as the second paragraph set. A global keyword overlapping analysis model is constructed based on the second paragraph set to conduct a global overlapping analysis of the keywords of regulatory document a and regulatory document b, and preliminarily screen whether regulatory document a and regulatory document b contain overlapping content; a local keyword overlapping analysis model is constructed according to the keyword set to analyze the keyword overlapping indicators in each paragraph of regulatory document a and regulatory document b, and secondly screen and determine the overlapping content in regulatory document a and regulatory document b, and conduct a detailed comparison of each paragraph, so as to realize the secondary screening of the overlapping content between documents, reduce the risks of misjudgment and missed judgment, effectively solve the problem of overlapping of regulatory documents of multiple departments, and optimize information circulation and resource allocation; it can identify the overlapping parts between regulatory documents, effectively avoid enterprises from repeatedly submitting the same information to different departments, thus saving internal resources of enterprises and reducing the complexity of information processing. Brief Description of the Drawings

[0025] Figure 1 It is a monthly trend chart of the overlapping degree of regulatory documents provided by the present invention;

[0026] Figure 2 It is a circular chart of the distribution of document types provided by the present invention;

[0027] Figure 3 It is a schematic diagram of the list of recently processed documents provided by the present invention;

[0028] Figure 4 It is a schematic structural diagram of an archive information classification and management system based on data mining provided by the present invention. Detailed Embodiments

[0029] Next, the technical solutions in the present invention will be clearly and completely described in conjunction with the drawings in the present invention. Obviously, the described technical solutions are only a part of the present invention, rather than all of them. All other technical solutions obtained by those of ordinary skill in the art based on the technical solutions in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0030] Example 1

[0031] Figure 4 The structural schematic diagram of an archive information classification and management system based on data mining provided by the present invention is shown in the figure. An archive information classification and management system based on data mining includes a regional medical inspection data management system based on blockchain, including a document collection and processing module, a text feature extraction and representation module, a global overlap analysis module, a local overlap analysis module, and a file feedback and update module; the document collection and processing module is connected to the text feature extraction and representation module, the text feature extraction and representation module is respectively connected to the global overlap analysis module and the local overlap analysis module, and the global overlap analysis module and the local overlap analysis module are respectively connected to the file feedback and update module.

[0032] The document collection and processing module regularly obtains original regulatory documents from specific websites through a timed crawler and an API interface, and segments the original regulatory documents to obtain the first paragraph set of regulatory document a Among them, is the i-th first paragraph of regulatory document a, n is the total number of the first paragraphs of regulatory document a, and then the first paragraph set D is segmented by a word segmentation tool a and stop words are removed to obtain the keywords of each first paragraph, which is recorded as the second paragraph set, and the second paragraph set is obtained is the keyword set of the i-th second paragraph of regulatory document a, and the i-th second paragraph corresponds to the first paragraph of regulatory document a

[0033] The text feature extraction and representation module is used to respectively obtain the second paragraph sets of regulatory document a and regulatory document b. The second paragraph set of regulatory document b is the keyword set extracted from the first paragraph of regulatory document b where m is the number of the first paragraphs of regulatory document b. The keyword sets of each paragraph in the second paragraph sets of regulatory document a and regulatory document b are respectively obtained. For the keyword set of regulatory document a Among them, is the b-th keyword in the i-th second paragraph, and B is the total number of keywords in the i-th second paragraph; for the keyword set of regulatory document b Among them, is the d-th keyword in the j-th second paragraph, and D is the total number of keywords in the j-th second paragraph.

[0034] The global overlap analysis module constructs a global keyword overlap analysis model based on the second paragraph sets, conducts a global overlap analysis on the keywords in regulatory document a and regulatory document b, and preliminarily screens whether regulatory document a and regulatory document b contain overlapping content. The formula for the global keyword overlap analysis model is:

[0035]

[0036] In the formula: S1 is the global keyword overlap index, is the second paragraph set of regulatory document a, is the second paragraph set of regulatory document b, is the number of keywords in the intersection of the second paragraph set of regulatory document a and the second paragraph set of regulatory document b, is the number of keywords in the union of the second paragraph set of regulatory document a and the second paragraph set of regulatory document b;

[0037] Extract the keyword sets from the second paragraph sets of regulatory document a and regulatory document b and calculate The intersection of the two sets is the number of keywords that appear in both regulatory document a and regulatory document b, denoted as Then calculate the union of the two sets, which is the total number of all keywords that appear at least once in regulatory document a and regulatory document b, denoted as Divide the number of keywords in the intersection by the number of keywords in the union to obtain the global keyword overlap index S1;

[0038] Compare the global keyword overlap index with the preset global keyword overlap index threshold. If the global keyword overlap index is greater than or equal to the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b contain overlapping content; if the global keyword overlap index is less than the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b do not contain overlapping content.

[0039] The local overlap analysis module is used to, when it is preliminarily determined that regulatory document a and regulatory document b contain overlapping content, construct a local keyword overlap analysis model based on the keyword sets, analyze the keyword overlap indexes in each paragraph of regulatory document a and regulatory document b, and secondarily screen to determine the overlapping content in regulatory document a and regulatory document b. The formula for the local keyword overlap analysis model is:

[0040]

[0041] In the formula: S2 is the local keyword overlap index, It is the keyword set of the i-th second paragraph of regulatory document a. It is the keyword set of the j-th second paragraph of regulatory document b. n is the total number of the first paragraphs of regulatory document a, and m is the total number of the first paragraphs of regulatory document b.

[0042] For each pair of second paragraphs in regulatory document a and regulatory document b, calculate the ratio of the intersection to the union of the respective keyword sets in this pair of paragraphs, that is, obtain the local keyword similarity. Add up the local keyword similarities of all second paragraphs, and then divide by the product of the total number of second paragraphs in regulatory document a and regulatory document b to get the local keyword overlap index S2.

[0043] The document feedback update module constructs an overlap degree analysis model based on the global keyword overlap index and the local keyword overlap index, and quantitatively analyzes the overlap degree of keywords between regulatory document a and regulatory document b. The formula of the overlap degree analysis model is:

[0044] K o =γ·S1+(1 - γ)·S2;

[0045] In the formula: K o is the overlap degree of keywords between regulatory document a and regulatory document b, γ is the weight balancing the global keyword overlap index and the local keyword overlap index, S1 is the global keyword overlap index, and S2 is the local keyword overlap index.

[0046] By weighted fusion of the global keyword overlap index S1 and the local keyword overlap index S2 between regulatory document a and regulatory document b, the overlap degree K of keywords between regulatory document a and regulatory document b is obtained o , where the influence of the global keyword overlap index and the local keyword overlap index is balanced by the weight coefficient γ.

[0047] It should be noted that the value range of γ is between 0 and 1. When γ is higher, the influence of the global index is greater; when γ is lower, the influence of the local index is greater.

[0048] Compare the overlap degree of keywords between regulatory document a and regulatory document b with the preset document overlap degree threshold. If the overlap degree of keywords between regulatory document a and regulatory document b is greater than or equal to the preset document overlap degree threshold, then there is an overlap between regulatory document a and regulatory document b; if the overlap degree of keywords between regulatory document a and regulatory document b is less than the preset document overlap degree threshold, then there is no overlap between regulatory document a and regulatory document b.

[0049] Figure 1Monthly trend chart of the overlap degree of regulatory documents provided by the present invention; the vertical axis is "percentage", representing the proportion of document overlap; the horizontal axis is "month", showing the changes in the document overlap degree for each month. For example, the overlap degree in September is 62, which can intuitively understand the changing trend of the similarity or repetition degree of document content in different periods, providing a reference for subsequent analysis or management.

[0050] Figure 2 Pie chart of document type distribution provided by the present invention; different colors are used in the pie chart to distinguish various document types, such as "monitoring report", "analysis report", "operation record", "emission data", "others", etc. The central text shows the currently selected document type (such as "monitoring report"), and the corresponding document quantity (such as 1,048 copies) can be viewed in the tooltip, which can help managers quickly understand the proportion of each type of document in the system, facilitating subsequent classified management or key attention for different types of documents.

[0051] Figure 3 Schematic diagram of the list of recently processed documents provided by the present invention; the table lists the information of the documents recently processed or uploaded in the system, including document name, type, overlap degree, last modification time, processing status, and available operations. For example, the type of "Sewage Treatment Monitoring Report for the Third Quarter of 2023.pdf" is "Emission Monitoring" or "Others", the overlap degree is 87%, the last modification time is 2023-10-15 14:30, and the status is "completed"; the type of "Sewage Treatment Operation Analysis Report - 2023.docx" is "Analysis Report", the overlap degree is 65%, the last modification time is 2023-10-14 09:15, and the status is "processing" or "pending review". Through this list, users can intuitively view the basic attributes and overlap degree of documents and perform operations such as viewing, editing, reviewing, or deleting.

[0052] It should be noted that the preset threshold of the document overlap degree can be set artificially according to the actual situation. When it is necessary to screen for duplicate documents, the preset threshold of the document overlap degree is increased. At this time, if the overlap degree of keywords between regulatory document a and regulatory document b is greater than or equal to the preset threshold of the document overlap degree, then regulatory document a and regulatory document b are duplicate documents; if the overlap degree of keywords between regulatory document a and regulatory document b is less than the preset threshold of the document overlap degree, then regulatory document a and regulatory document b are not duplicate documents.

[0053] It should be noted that specific websites include the official websites of multiple departments such as environmental protection, water conservancy, and industry;

[0054] The document collection and processing module also includes preprocessing the original regulatory documents, and the preprocessing includes text cleaning, encoding conversion, word segmentation and part-of-speech tagging, stop word removal, and format standardization;

[0055] Text clarity involves removing HTML tags, special symbols, redundant spaces and other interfering information; encoding conversion involves unifying the encoding formats of various types of text (such as UTF-8) to avoid character errors caused by encoding problems; word segmentation and part-of-speech tagging involves using Chinese word segmentation tools (such as jieba, THULAC, etc.) to segment the text and tag each word with the part of speech; custom thesaurus can be used for optimization of professional terminology; stop word removal involves building a stop word list based on the business field, eliminating meaningless words, and improving the accuracy of subsequent feature extraction; format standardization involves template parsing and structured tagging of semi-structured documents (such as files with clause numbers, titles, and paragraph structures) to ensure a clear internal structure of the document.

[0056] The specific process of segmentation is to segment the text according to chapters, clauses or paragraph marks, and each paragraph is regarded as the basic comparison unit.

[0057] Example 2

[0058] The above-mentioned blockchain-based regional medical inspection data management system is used to collect documents and extract keywords from two regulatory documents, and the first paragraph set is extracted from regulatory document a. in Corresponding keyword set Corresponding keyword set Corresponding keyword set Extract the first paragraph set from regulatory document b in Corresponding keyword set Corresponding keyword set Corresponding keyword set

[0059] Globally integrate the data of all the second paragraphs (i.e., the keyword sets) in the two documents respectively: The global keyword set of Document A: {"sewage", "treatment", "standards", "discharge", "requirements", "process", "operation", "flow", "monitoring", "environmental protection", "measures"}; The global keyword set of Document B: {"sewage", "discharge", "standards", "treatment", "requirements", "process", "flow", "equipment", "specifications", "monitoring", "environment", "audit"}.

[0060] Calculate the intersection and union of the two global sets: The intersection is {"sewage", "treatment", "standards", "discharge", "requirements", "process", "flow", "monitoring"} (a total of 8 keywords); The union: {"sewage", "treatment", "standards", "discharge", "requirements", "process", "operation", "flow", "monitoring", "environmental protection", "measures", "equipment", "specifications", "environment", "audit"}; Based on this, calculate the global keyword overlap index to be 0.33.

[0061] When the global determination deems that there is an overlap, the system pairwise compares the keyword sets of each paragraph in the two documents and calculates the local keyword overlap index, as shown in Table 1 below:

[0062]

[0063]

[0064] Based on the above table, the calculated local keyword overlap index is 0.2577.

[0065] In the embodiment of the present invention, the original regulatory documents are obtained from specific websites regularly through a timed crawler and an API interface, and the original regulatory documents are segmented. Then, by word segmentation and removing stop words, the keywords of each first paragraph are obtained, denoted as the second paragraph set. Based on the second paragraph set, a global keyword overlap analysis model is constructed to perform global overlap analysis on the keywords of Regulatory Document A and Regulatory Document B, and preliminarily screen whether Regulatory Document A and Regulatory Document B contain overlapping content; According to the keyword set, a local keyword overlap analysis model is constructed to analyze the keyword overlap index in each paragraph of Regulatory Document A and Regulatory Document B, and the overlapping content in Regulatory Document A and Regulatory Document B is determined through secondary screening, and each paragraph is carefully compared, so as to achieve secondary screening of the overlapping content between documents, reduce the risk of misjudgment and missed judgment, and can effectively solve the problem of overlapping regulatory documents of multiple departments, optimize information circulation and resource allocation; It can identify the overlapping parts between regulatory documents, effectively avoid enterprises from submitting the same information repeatedly to different departments, thus saving internal resources of enterprises and reducing the complexity of information processing.

[0066] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims described above.

[0067] Finally: The above is only the preferred solution of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An archive information classification and management system based on data mining, comprising a document collection and processing module, a text feature extraction and representation module, a global overlap analysis module, a local overlap analysis module, and a file feedback and update module; characterized in that, The document collection and processing module regularly obtains the original regulatory documents from specific websites through a timed crawler and an API interface, and segments the original regulatory documents to obtain the first paragraph set of regulatory document a. Then, it performs word segmentation on the first paragraph set D a to perform word segmentation and remove stop words to obtain the keywords of each first paragraph, which are recorded as the second paragraph set. The text feature extraction and representation module is used to obtain the second paragraph sets of regulatory document a and regulatory document b respectively, and obtain the keyword sets of each paragraph in the second paragraph sets of regulatory document a and regulatory document b respectively. The global overlap analysis module constructs a global keyword overlap analysis model based on the second paragraph set, performs global overlap analysis on the keywords of regulatory document a and regulatory document b, and preliminarily screens whether regulatory document a and regulatory document b contain overlapping content. The local overlap analysis module is used to, when it is preliminarily determined that regulatory document a and regulatory document b contain overlapping content, construct a local keyword overlap analysis model according to the keyword set, analyze the keyword overlap indicators in each paragraph of regulatory document a and regulatory document b, and secondarily screen to determine the overlapping content in regulatory document a and regulatory document b.

2. The archive information classification and management system based on data mining according to claim 1, wherein The first paragraph set of regulatory document a in the document collection and processing module Among them, is the i-th first paragraph of regulatory document a, n is the total number of the first paragraphs of regulatory document a, and then the first paragraph set D is segmented by a word segmentation tool a to segment words and remove stop words to obtain the keywords of each first paragraph, which is recorded as the second paragraph set, and the second paragraph set is obtained is the keyword set of the i-th second paragraph of regulatory document a, and the i-th second paragraph corresponds to the first paragraph of regulatory document a 3. The archive information classification and management system based on data mining according to claim 1, characterized in that, The text feature extraction and representation module is used to obtain the second paragraph sets of regulatory document a and regulatory document b respectively, and the second paragraph set of regulatory document b is the first paragraph of regulatory document b is the keyword set extracted from the first paragraph of regulatory document b, m is the number of the first paragraphs of regulatory document b, and the keyword sets of each paragraph in the second paragraph sets of regulatory document a and regulatory document b are obtained respectively. For the keyword set of regulatory document a where is the b-th keyword in the i-th second paragraph, and B is the total number of keywords in the i-th second paragraph; for the keyword set of regulatory document b where is the d-th keyword in the j-th second paragraph, and D is the total number of keywords in the j-th second paragraph.

4. The archive information classification and management system based on data mining according to claim 1, characterized in that, The construction steps of the global keyword overlap analysis model in the global overlap analysis module are as follows: Extract the keyword set from the second paragraph sets of regulatory document a and regulatory document b and calculate and the intersection of the two sets, which is the number of keywords that appear in both regulatory document a and regulatory document b, denoted as Then calculate the union of the two sets, which is the total number of all keywords that appear at least once in regulatory document a and regulatory document b, denoted as Divide the number of keywords in the intersection by the number of keywords in the union to obtain the global keyword overlap index S1; Compare the global keyword overlap index with the preset global keyword overlap index threshold. If the global keyword overlap index is greater than or equal to the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b contain overlapping content; if the global keyword overlap index is less than the preset global keyword overlap index threshold, it is preliminarily determined that regulatory document a and regulatory document b do not contain overlapping content.

5. The archival information classification and management system based on data mining according to claim 1, characterized in that, The specific construction steps of the local keyword overlap analysis model in the local overlap analysis module are as follows: For each pair of second paragraphs in regulatory document a and regulatory document b, calculate the ratio of the intersection to the union of the respective keyword sets in this pair of paragraphs, that is, obtain the local keyword similarity. Add up the local keyword similarities of all second paragraphs, and then divide by the product of the total number of second paragraphs in regulatory document a and regulatory document b to obtain the local keyword overlap index S2.

6. The archival information classification and management system based on data mining according to claim 1, characterized in that The document feedback update module constructs an overlap degree analysis model based on the global keyword overlap index and the local keyword overlap index, and quantitatively analyzes the overlap degree of keywords between regulatory document a and regulatory document b. By weighted fusion of the global keyword overlap index S1 and the local keyword overlap index S2 between regulatory document a and regulatory document b, the overlap degree K of keywords between regulatory document a and regulatory document b is obtained o , where the influence of the global keyword overlap index and the local keyword overlap index is balanced by the weight coefficient γ.

7. The archive information classification and management system based on data mining according to claim 6, characterized in that, Compare the overlap degree of keywords between regulatory document a and regulatory document b with the preset document overlap degree threshold. If the overlap degree of keywords between regulatory document a and regulatory document b is greater than or equal to the preset document overlap degree threshold, there is an overlap between regulatory document a and regulatory document b; If the overlap degree of keywords between regulatory document a and regulatory document b is less than the preset document overlap degree threshold, there is no overlap between regulatory document a and regulatory document b.

Images