An intelligent batch document management system and method based on edge algorithm

Abstract

The present invention relates to the field of data processing technology, and discloses a batch document intelligent management system and method based on edge algorithm, the system comprising: a document collection module counts all documents uploaded in the edge computing data layer, performs cluster analysis on all uploaded documents according to key features, a document analysis module determines the influence coefficient of each document according to the document size and word proportion, a document judgment module compares the average influence coefficient with the average influence coefficient threshold, determines the document storage set of each document in the document series according to the comparison result, compares the document keywords of each document in the document storage set, determines whether to store it in the document storage subset according to the comparison result, and a document management module determines the management score of the document storage subset according to the storage status, and determines the document management level of the document storage subset according to the management score. The present invention realizes intelligent management of documents by utilizing edge algorithms and performing data processing on them.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to an intelligent batch document management system and method based on an edge algorithm. Background Art

[0002] With the development of the information age, massive amounts of literature data continue to appear in various academic resource libraries. How to efficiently manage these documents has become a problem to be solved in the field of academic research and academic management. Traditional literature management usually relies on centralized cloud servers and is managed through static classification. However, with the surge in the amount of literature data, traditional management methods have the limitations of frequent data exchange, processing delays, and bandwidth bottlenecks. In addition, due to the huge amount of literature data, the processing of literature still relies on manual classification and management, and lacks understanding and processing of literature content, resulting in deficiencies in literature management.

[0003] Therefore, it is necessary to design an intelligent batch document management system and method based on edge algorithm to solve the problems existing in current technology. Summary of the invention

[0004] In view of this, the present invention proposes an intelligent batch document management system and method based on edge algorithm, aiming to solve the limitations of traditional management methods such as frequent data exchange, processing delays and bandwidth bottlenecks, and manual classification and management lack of understanding and processing of document content, resulting in insufficient management of documents.

[0005] In one aspect, the present invention proposes an intelligent batch document management system based on an edge algorithm, comprising: The document collection module is configured to count all documents uploaded in the edge computing data layer, extract key features of each document, perform cluster analysis on all uploaded documents according to the key features, and divide the document series according to the clustering results; A document analysis module is configured to determine the influence coefficient of each document according to the document size and the word proportion, and determine the average influence coefficient of each series based on the influence coefficients in each series; A document judgment module is configured to compare the average influence coefficient with the average influence coefficient threshold, determine the document storage set of each document in the document series according to the comparison result, compare the document keywords of each document in the document storage set, and determine whether to store it in the document storage subset according to the comparison result; The document management module is configured to collect the storage status of the document storage subset, the storage status including the total amount of documents and the change per unit time, determine the management score of the document storage subset according to the storage status, and determine the document management level of the document storage subset according to the management score.

[0006] Furthermore, when extracting the key features of each document, we include: The document collection module pre-processes all uploaded documents, and the pre-processing includes unifying the document format, adjusting the document image resolution, and correcting punctuation marks; Analyze the preprocessing results based on NLP, perform word segmentation, remove stop words and extract stems on all preprocessed documents, use dependency syntactic analysis to parse the relationship between words and extract noun phrases, and use the noun phrases that conform to the syntactic structure as document keywords; The ratio of the keyword of each document to the word count of the document is determined as the word ratio, and the word ratio and document size of each document are used as the key features.

[0007] Furthermore, when clustering all uploaded documents according to the key features and dividing the document series according to the clustering results, it includes: The document collection module normalizes each parameter in the key feature to form a document feature vector, constructs the document feature vectors of all pre-processed documents into a document matrix, sets the number of clusters k, and calculates the cluster distance from each document feature vector to each cluster center; The cluster distance is given by the following formula: ; in, represents the clustering distance, Indicates the literature No. eigenvalues, Represents the cluster center No. eigenvalues; Divide each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance, redetermine the position of each cluster center, and continue to redefine each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance and the redetermined cluster center until the position of the cluster center no longer changes or reaches the preset number of iterations, then divide all preprocessed documents into k document sequences.

[0008] Furthermore, when determining the impact coefficient of each document based on the document size and word proportion, it includes: The document analysis module obtains the impact coefficient of each document according to the following formula: ; in, represents the impact coefficient of the i-th document, Indicates the number of times the keyword of the i-th document appears. represents the document size of the i-th document, represents the word proportion of the i-th document, and represents the weight coefficient, and .

[0009] Furthermore, when comparing the average impact coefficient with the average impact coefficient threshold and determining the document storage set of each document in the document series according to the comparison result, it includes: The document judgment module pre-sets an average impact coefficient threshold, and the document storage set includes a first document storage set and a second document storage set; The average impact coefficient is the sum of the impact coefficients of all documents in each document series; When the average influence coefficient is greater than or equal to the average influence coefficient threshold, the document judgment module stores all documents in the document sequence into the first document storage set; When the average impact coefficient is less than the average impact coefficient threshold, the document judgment module stores all documents in the sequence into the second document storage set; The storage management time of the first storage set of documents is greater than the storage management time of the second storage set of documents.

[0010] Furthermore, when comparing the document keywords of each document in the document storage set and determining whether to store the document in the document storage subset according to the comparison result, the method includes: When there are documents with the same number of document keywords and the same document keywords in the first document storage set, the document judgment module determines the document as a document to be moved; The document judgment module counts the number of document keywords of each document to be moved, and pre-sets a document keyword number threshold. When the number of document keywords of the document to be moved is greater than the document keyword number threshold, the document to be moved is stored in the document storage subset; The document storage subset is a subset storage set of the document first storage set.

[0011] Further, when determining the management score of the document storage subset according to the storage status, it includes: The document management module obtains the management score of the document storage subset according to the following formula: ; in, Represents the management score, represents the total amount of literature, Indicates the amount of change per unit time.

[0012] Further, when determining the document management level of the document storage subset according to the management score, it includes: The document management module pre-sets a first preset management score and a second preset management score, and the first preset management score is greater than the second preset management score; When the management score is greater than or equal to the first preset management score, the document management module determines the document management level of the document storage subset as first-level management; When the management score is less than the first preset management score and greater than or equal to the second preset management score, the document management module determines the document management level of the document storage subset as secondary management; When the management score is less than the second preset management score, the document management module determines the document management level of the document storage subset as level three management.

[0013] Furthermore, when the document management module determines the document management level of the document storage subset as first-level management, it also includes: The document management module sets a hash check for the document storage subset of the primary management, checks the documents in the document storage subset within a preset time, and deletes the documents in the document storage subset based on the check result.

[0014] Compared with the prior art, the beneficial effects of the present invention are: by edge computing, the processing and storage of massive documents are distributed to multiple edge nodes, avoiding the bandwidth limitation and delay problems of the centralized cloud system, thereby improving the efficiency of document management. By clustering the documents, the system can intelligently classify the documents and decide whether to store the documents in the designated document storage subset according to the influence coefficient of each document. It helps to improve the accuracy and efficiency of document management, avoids the errors of traditional manual classification, and leads to waste of management resources. By real-time acquisition of the storage status of the document storage subset, calculating the management score of the document storage subset, and dynamically determining the document management level of the document storage subset, it improves the flexibility of management and the accuracy of resource allocation, and meets the diverse needs of document management.

[0015] On the other hand, the present application also provides a method for intelligent management of batch documents based on edge algorithm, which is used to apply the above-mentioned intelligent management system of batch documents based on edge algorithm, including: Count all documents uploaded in the edge computing data layer, extract key features of each document, perform cluster analysis on all uploaded documents based on the key features, and divide the document series according to the clustering results; Determine the impact coefficient of each document based on the document size and word proportion, and determine the average impact coefficient of each series based on the impact coefficients in each series; The average influence coefficient is compared with the average influence coefficient threshold, and a document storage set of each document in the document series is determined according to the comparison result, and the document keywords of each document in the document storage set are compared, and it is determined whether to store the document in the document storage subset according to the comparison result; The storage status of the document storage subset is collected, wherein the storage status includes the total amount of documents and the amount of change per unit time, a management score of the document storage subset is determined according to the storage status, and a document management level of the document storage subset is determined according to the management score.

[0016] It can be understood that the above-mentioned intelligent management system and method for batch documents based on edge algorithm have the same beneficial effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A functional block diagram of an intelligent batch document management system based on an edge algorithm provided in an embodiment of the present invention; Figure 2 A flowchart of an intelligent batch document management method based on an edge algorithm provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0019] In some embodiments of the present application, see Figure 1As shown, an intelligent batch document management system based on edge algorithm includes: The document collection module is configured to count all documents uploaded in the edge computing data layer, extract the key features of each document, perform cluster analysis on all uploaded documents based on the key features, and divide the document series according to the clustering results.

[0020] The document analysis module is configured to determine the influence coefficient of each document according to the document size and word proportion, and determine the average influence coefficient of each series based on the influence coefficient in each series.

[0021] The document judgment module is configured to compare the average impact coefficient with the average impact coefficient threshold, determine the document storage set of each document in the document series according to the comparison result, compare the document keywords of each document in the document storage set, and determine whether to store it in the document storage subset according to the comparison result.

[0022] The document management module is configured to collect the storage status of the document storage subset, the storage status includes the total amount of documents and the change per unit time, determine the management score of the document storage subset according to the storage status, and determine the document management level of the document storage subset according to the management score.

[0023] Specifically, in the process of using crawlers and other tools to obtain documents from academic platforms for storage and management, traditional cloud processing requires the documents to be transferred to the central server for processing, which will lead to high latency. Edge computing can directly process data on local nodes, thereby reducing response time. The edge algorithm reduces bandwidth consumption during the transmission and processing of documents, avoiding the tedious operation of local storage, thereby improving management efficiency. The edge computing data layer represents the location of uploaded documents (edge ​​nodes), and the document acquisition module counts all documents uploaded in the edge computing data layer, and extracts key features and clusters all documents, thereby intelligently grouping massive documents according to certain similarities, thereby improving the efficiency of document management. The use of edge algorithms and clustering analysis can automatically classify documents in a short time, reducing manual intervention and improving the accuracy of storage management. The document analysis module calculates the impact coefficient of each document based on the size of the document and the proportion of words, and dynamically screens the documents based on the quality and influence of the document by calculating the average impact coefficient of each document series. The impact coefficient provides a quantitative standard for the academic value of each document, thereby providing a data basis for subsequent management. The document judgment module determines the document storage set of each document in the document series by comparing the average impact coefficient and the average impact coefficient threshold, and compares the document keywords of each document in the document storage set, and determines whether to store it in the document storage subset based on the comparison results. Through quantitative standards, subjective judgments of manual screening are avoided. The document management module dynamically calculates the management score by collecting the storage status of the document storage subset. The system can evaluate the storage subset in real time and adjust the document management level of the document storage subset based on the management score, ensuring that high-value documents can obtain higher management priority and reducing the waste of management resources.

[0024] It is understandable that data processing based on edge computing can collect, analyze, judge and manage documents at the edge nodes, reducing the network delay and computing pressure brought by traditional data processing. The introduction of edge computing has improved the response speed and processing efficiency of the system. When processing massive documents, the system can accurately complete data processing and management, reducing dependence on central servers, improving system stability, ensuring that batch management of documents is always in good condition, and improving the efficiency and accuracy of document management.

[0025] In some embodiments of the present application, when extracting the key features of each document, it includes: the document collection module preprocesses all the uploaded documents. The preprocessing includes unifying the document format, adjusting the resolution of the document image, and correcting punctuation marks. Analyze the results of the preprocessing based on NLP, perform word segmentation, stop word removal, and stemming on all the preprocessed documents, and use dependency syntax analysis to parse the relationships between words to extract noun phrases. The noun phrases that conform to the syntactic structure are used as document keywords, and the ratio of the document keywords of each document to the number of words in the document is determined as the word ratio. The word ratio of each document and the document size are used as key features.

[0026] Specifically, the document collection module first preprocesses all the uploaded documents. Unifying the document format means converting the format of the uploaded documents to ensure that the format of each uploaded document is consistent, avoiding format problems from affecting subsequent management. Removing stop words means removing common words in the document that do not affect the actual semantics, such as "de" (of) and "shi" (is), to improve the effectiveness of text analysis. Stemming means reducing the words in the document to their root forms. For example, "established" and "establish" are unified as "establish" to reduce semantic redundancy in the document. Use dependency syntax analysis to parse the grammatical and semantic relationships between the words in the document. The parsing process of dependency syntax analysis is complex and relatively mature, so it will not be described in detail here. Extract noun phrases from the document. Dependency syntax analysis can accurately identify noun phrases, such as "electric power" and "energy", etc. These noun phrases reflect the theme and scope of the document. Using them as document keywords can comprehensively describe the field and main framework of the document. The ratio of the document keywords of each document to the number of words in the document is determined as the word ratio. The word ratio of each document and the document size are used as key features. The number of words in the document is the total number of remaining words in the document after stop word removal and stemming. The word ratio reflects the degree of use of the document keywords in the document. The document size reflects the overall size of the document, and its numerical value is also determined after stop word removal and stemming. Using both as the key features of the document can comprehensively represent the field and the actual content involved in the document, laying a foundation for subsequent management.

[0027] In some embodiments of the present application, when performing clustering analysis on all the uploaded documents according to the key features and dividing the document sequence according to the clustering results, it includes: the document collection module normalizes each parameter in the key features to form a document feature vector, constructs a document matrix from the document feature vectors of all the preprocessed documents, sets the number of clusters k, and calculates the clustering distance from each document feature vector to each cluster center. The clustering distance is obtained by the following formula: ; where, represents the clustering distance, Indicates the literature No. eigenvalues, Represents the cluster center No. eigenvalues, divide each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance, redetermine the position of each cluster center, and continue to redefine each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance and the redetermined cluster center, until the position of the cluster center no longer changes or reaches the preset number of iterations, then all preprocessed documents are divided into k document sequences.

[0028] Specifically, the document collection module normalizes each parameter (word proportion and document size) in the key features to ensure that all feature values ​​are in the same range, such as (0, 1], to avoid distortion of clustering results due to differences in scale between features, and to improve the accuracy and reliability of document classification. Through the k-means clustering algorithm, the reliability of document classification management is achieved. In addition, through multi-dimensional feature vectors and normalization processing, the similarities of different documents are captured in turn. The cluster centers are gradually determined through iterative optimization, and each pre-processed document is re-divided into clusters with the shortest distance. The document series corresponding to the re-determined cluster centers does not rely solely on the results determined once, ensuring the accuracy of the division results. The number of clusters k and the number of iterations can be dynamically adjusted according to the actual number of documents to avoid classification deviations caused by excessively large document counts and small number of clusters, thereby improving the adaptability of classification management and laying a foundation for subsequent management, thereby improving the efficiency of document management.

[0029] In some embodiments of the present application, when determining the influence coefficient of each document according to the document size and word proportion, the document analysis module obtains the influence coefficient of each document according to the following formula: ; in, represents the impact coefficient of the i-th document, Indicates the number of times the keyword of the i-th document appears. represents the document size of the i-th document, represents the word proportion of the i-th document, and represents the weight coefficient, and .

[0030] In some embodiments of the present application, when comparing the average influence coefficient with the average influence coefficient threshold and determining the document storage set of each document in the document series according to the comparison result, it includes: the document judgment module pre-sets the average influence coefficient threshold, the document storage set includes a first document storage set and a second document storage set, the average influence coefficient is the sum of the influence coefficients of all documents in each document series, when the average influence coefficient is greater than or equal to the average influence coefficient threshold, the document judgment module stores all documents in the document series into the first document storage set, when the average influence coefficient is less than the average influence coefficient threshold, the document judgment module stores all documents in the series into the second document storage set, and the storage management time of the first document storage set is greater than the storage management time of the second document storage set.

[0031] Specifically, by calculating the impact coefficient of a document based on multiple factors, such as document size, keyword occurrence times, and word proportion, the system can objectively evaluate the importance of each document. This avoids human subjective judgment and improves the reliability of document management. is equal to 20 times, Equal to 1MB, is equal to 0.1, and Taking 0.5, the influence coefficient of this document is 2.45. The average influence coefficient threshold is preferably 5. According to the division of the document series, the average influence coefficient of each document series is obtained, so that the documents of each document series can be stored in the corresponding document first storage set or document second storage set. The documents stored in the document first storage set are documents with generally large influence coefficients. Since it is calculated based on the document size and word proportion, the larger the value, the more document keywords involved and the higher the proportion, which represents the higher importance of the relevant field. Therefore, the storage and management time of these documents is longer, thereby ensuring the efficiency of document management and improving the flexibility of document management.

[0032] In some embodiments of the present application, when comparing the document keywords of each document in the document storage set and determining whether to store it in the document storage subset based on the comparison result, it includes: when there are documents with the same number of document keywords and the same document keywords in the first document storage set, the document judgment module determines the document as a document to be moved, the document judgment module counts the number of document keywords of each document to be moved, and pre-sets a threshold value for the number of document keywords; when the number of document keywords of the document to be moved is greater than the threshold value for the number of document keywords, the document to be moved is stored in the document storage subset, and the document storage subset is a sub-storage set of the first document storage set.

[0033] Specifically, when there are documents with the same number of document keywords and the same document keywords in the first document storage set, the document judgment module determines the document as a document to be moved. For example, when there are multiple documents with the same number of document keywords (all 5) and the same document keywords (for example, electrical, relay, relay protection, coupling and inductance) in the first document storage set, these documents are determined as documents to be moved. These documents to be moved have high similarity and often represent important research results in the field. They can accurately refer to and cite documents in related fields in subsequent research. The threshold for the number of document keywords is preferably 4. If the number of document keywords is less than or equal to the threshold for the number of document keywords, the document to be moved will not be moved and will still be stored in the first document storage set. If the number of document keywords is greater than the threshold for the number of document keywords, it indicates that the document to be moved has a detailed field research direction, and the document to be moved will be moved and stored in a sub-storage set (document storage subset) of the first document storage set, so as to facilitate its targeted management, avoid judgment errors due to human experience, and improve the reliability of intelligent management.

[0034] In some embodiments of the present application, when determining the management score of the document storage subset according to the storage status, it includes: the document management module obtains the management score of the document storage subset according to the following formula: ; in, Represents the management score, represents the total amount of literature, Indicates the amount of change per unit time.

[0035] In some embodiments of the present application, when determining the document management level of a document storage subset based on a management score, it includes: the document management module pre-sets a first preset management score and a second preset management score, the first preset management score is greater than the second preset management score, when the management score is greater than or equal to the first preset management score, the document management module determines the document management level of the document storage subset as first-level management, when the management score is less than the first preset management score and greater than or equal to the second preset management score, the document management module determines the document management level of the document storage subset as second-level management, and when the management score is less than the second preset management score, the document management module determines the document management level of the document storage subset as third-level management.

[0036] Specifically, the management levels of primary management, secondary management, and tertiary management decrease in sequence. The total number of documents indicates the number of documents that have been stored in the document storage subset. The unit time in the unit time change is set to every 30 minutes. The unit time change indicates the number of documents moved from the first document storage set to the document storage subset every 30 minutes. If 30 documents are moved within 30 minutes, the unit time change is equal to 30. The unit time in the unit time change can be adjusted according to the actual number of uploaded documents counted in the document collection module. The first preset management score is preferably 80, and the second preset management score is preferably 85. By calculating the management score, for example: when is equal to 30, is equal to 200, then is equal to 81.7, when is equal to 30, is equal to 900, then It is equal to 85.4, which enables hierarchical management of document storage subsets, ensuring the rational allocation of intelligent management resources. The higher the management score, the higher the management level and the greater the investment in resource management, thereby improving the overall management effect and resource utilization, realizing the automation of document management without manual intervention, and improving the flexibility and reliability of the management system.

[0037] In some embodiments of the present application, when the document management module determines the document management level of the document storage subset as first-level management, it also includes: the document management module sets a hash check for the first-level management document storage subset, and checks the documents in the document storage subset within a preset time, and deletes the documents in the document storage subset based on the check result.

[0038] Specifically, since the management of document storage subsets consumes a lot of resources, hash verification can verify the integrity of the documents in the document storage subset. If the hash value does not match, the problem can be discovered in time and deleted to avoid management resources being occupied by damaged documents, thereby freeing up space in the document storage subset and ensuring that the document storage subset maintains efficient management effects, realizing the automation and intelligence of document management, reducing errors caused by manual intervention, and improving management efficiency and stability.

[0039] In summary, the beneficial effects of the present invention are: by edge computing, the processing and storage of massive documents are distributed to multiple edge nodes, avoiding the bandwidth limitations and delay problems of centralized cloud systems, thereby improving the efficiency of document management. By clustering the documents, the system can intelligently classify the documents and decide whether to store the documents in the specified document storage subset based on the influence coefficient of each document. It helps to improve the accuracy and efficiency of document management, avoids the errors of traditional manual classification, and leads to waste of management resources. By real-time acquisition of the storage status of the document storage subset, calculating the management score of the document storage subset, and dynamically determining the document management level of the document storage subset. It improves the flexibility of management and the accuracy of resource allocation, and meets the diverse needs of document management.

[0040] In another preferred embodiment based on the above embodiment, refer to Figure 2 As shown, this embodiment provides a batch document intelligent management method based on edge algorithm, which is used to apply the above-mentioned batch document intelligent management system based on edge algorithm, including: S100: Count all documents uploaded in the edge computing data layer, extract key features of each document, perform cluster analysis on all uploaded documents based on the key features, and divide the document series according to the clustering results.

[0041] S200: Determine the influence coefficient of each document according to the document size and the word proportion, and determine the average influence coefficient of each series based on the influence coefficient in each series.

[0042] S300: Compare the average impact coefficient with the average impact coefficient threshold, determine the document storage set of each document in the document series according to the comparison result, compare the document keywords of each document in the document storage set, and determine whether to store it in the document storage subset according to the comparison result.

[0043] S400: Collect the storage status of the document storage subset, the storage status includes the total amount of documents and the change amount per unit time, determine the management score of the document storage subset according to the storage status, and determine the document management level of the document storage subset according to the management score.

[0044] Specifically, all uploaded documents are extracted from the edge computing data layer, and the key features of the documents are extracted through natural language processing (NLP) technology, so that all uploaded documents are clustered and analyzed using a clustering algorithm to form different document series. Each document series contains documents in similar fields. The document size can be measured by the number of words in the document. Documents with larger document sizes usually contain more descriptive information. The word ratio represents the usage of document keywords in the document. The document size and word ratio are comprehensively considered to calculate the document influence coefficient, which can accurately quantify the influence of the document and avoid human experience judgment. The average influence coefficient is compared with the average influence coefficient threshold, and the document storage set of each document in the document series is determined according to the comparison results, and the document keywords of each document in the document storage set are compared. According to the comparison results, it is determined whether to store it in the document storage subset, which improves the intelligent level of document management. The calculation of the management score can accurately determine the document management level of the document storage subset, thereby optimizing management resources and further improving the intelligent level of document management.

[0045] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0046] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0047] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An intelligent batch document management system based on edge algorithm, characterized in that: include: The document collection module is configured to count all documents uploaded in the edge computing data layer, extract key features of each document, perform cluster analysis on all uploaded documents according to the key features, and divide the document series according to the clustering results; A document analysis module is configured to determine the influence coefficient of each document according to the document size and the word proportion, and determine the average influence coefficient of each series based on the influence coefficients in each series; A document judgment module is configured to compare the average influence coefficient with the average influence coefficient threshold, determine the document storage set of each document in the document series according to the comparison result, compare the document keywords of each document in the document storage set, and determine whether to store it in the document storage subset according to the comparison result; The document management module is configured to collect the storage status of the document storage subset, the storage status including the total amount of documents and the change per unit time, determine the management score of the document storage subset according to the storage status, and determine the document management level of the document storage subset according to the management score.

2. The intelligent batch document management system based on edge algorithm according to claim 1 is characterized in that: When extracting the key features of each document, we included: The document collection module pre-processes all uploaded documents, and the pre-processing includes unifying the document format, adjusting the document image resolution, and correcting punctuation marks; Analyze the preprocessing results based on NLP, perform word segmentation, remove stop words and extract stems on all preprocessed documents, use dependency syntactic analysis to parse the relationship between words and extract noun phrases, and use the noun phrases that conform to the syntactic structure as document keywords; The ratio of the keyword of each document to the word count of the document is determined as the word ratio, and the word ratio and document size of each document are used as the key features.

3. The intelligent batch document management system based on edge algorithm according to claim 2 is characterized in that: When clustering all uploaded documents according to the key features and dividing the document series according to the clustering results, it includes: The document collection module normalizes each parameter in the key feature to form a document feature vector, constructs the document feature vectors of all pre-processed documents into a document matrix, sets the number of clusters k, and calculates the cluster distance from each document feature vector to each cluster center; The cluster distance is given by the following formula: ； in, represents the clustering distance, Indicates the literature No. eigenvalues, Represents the cluster center No. eigenvalues; Divide each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance, redetermine the position of each cluster center, and continue to redefine each preprocessed document into the document sequence corresponding to the cluster center with the closest cluster distance and the redetermined cluster center until the position of the cluster center no longer changes or reaches the preset number of iterations, then divide all preprocessed documents into k document sequences.

4. The intelligent batch document management system based on edge algorithm according to claim 3 is characterized in that: When determining the impact coefficient of each document based on the size of the document and the proportion of words, it includes: The document analysis module obtains the impact coefficient of each document according to the following formula: ； in, represents the impact coefficient of the i-th document, Indicates the number of times the keyword of the i-th document appears. represents the document size of the i-th document, represents the word proportion of the i-th document, and represents the weight coefficient, and .

5. The intelligent batch document management system based on edge algorithm according to claim 4 is characterized in that: When comparing the average impact coefficient with the average impact coefficient threshold and determining the document storage set of each document in the document series according to the comparison result, it includes: The document judgment module pre-sets an average impact coefficient threshold, and the document storage set includes a first document storage set and a second document storage set; The average impact coefficient is the sum of the impact coefficients of all documents in each document series; When the average influence coefficient is greater than or equal to the average influence coefficient threshold, the document judgment module stores all documents in the document sequence into the first document storage set; When the average impact coefficient is less than the average impact coefficient threshold, the document judgment module stores all documents in the sequence into the second document storage set; The storage management time of the first storage set of documents is greater than the storage management time of the second storage set of documents.

6. The intelligent batch document management system based on edge algorithm according to claim 5 is characterized in that: When comparing the document keywords of each document in the document storage set and determining whether to store the document in the document storage subset according to the comparison result, the method includes: When there are documents with the same number of document keywords and the same document keywords in the first document storage set, the document judgment module determines the document as a document to be moved; The document judgment module counts the number of document keywords of each document to be moved, and pre-sets a document keyword number threshold. When the number of document keywords of the document to be moved is greater than the document keyword number threshold, the document to be moved is stored in the document storage subset; The document storage subset is a subset storage set of the document first storage set.

7. The intelligent batch document management system based on edge algorithm according to claim 6 is characterized in that: When determining the management score of the document storage subset according to the storage status, it includes: The document management module obtains the management score of the document storage subset according to the following formula: ； in, Represents the management score, represents the total amount of literature, Indicates the amount of change per unit time.

8. The intelligent batch document management system based on edge algorithm according to claim 7 is characterized in that: When determining the document management level of the document storage subset according to the management score, it includes: The document management module pre-sets a first preset management score and a second preset management score, and the first preset management score is greater than the second preset management score; When the management score is greater than or equal to the first preset management score, the document management module determines the document management level of the document storage subset as first-level management; When the management score is less than the first preset management score and greater than or equal to the second preset management score, the document management module determines the document management level of the document storage subset as secondary management; When the management score is less than the second preset management score, the document management module determines the document management level of the document storage subset as level three management.

9. The intelligent batch document management system based on edge algorithm according to claim 8 is characterized in that: When the document management module determines the document management level of the document storage subset as first-level management, the method further includes: The document management module sets a hash check for the document storage subset of the primary management, checks the documents in the document storage subset within a preset time, and deletes the documents in the document storage subset based on the check result.

10. A method for intelligent management of batch documents based on edge algorithm, used for applying the intelligent management system of batch documents based on edge algorithm as claimed in any one of claims 1 to 9, characterized in that: include: Count all documents uploaded in the edge computing data layer, extract key features of each document, perform cluster analysis on all uploaded documents based on the key features, and divide the document series according to the clustering results; Determine the impact coefficient of each document based on the document size and word proportion, and determine the average impact coefficient of each series based on the impact coefficients in each series; The average influence coefficient is compared with the average influence coefficient threshold, and a document storage set of each document in the document series is determined according to the comparison result, and the document keywords of each document in the document storage set are compared, and it is determined whether to store the document in the document storage subset according to the comparison result; The storage status of the document storage subset is collected, wherein the storage status includes the total amount of documents and the amount of change per unit time, a management score of the document storage subset is determined according to the storage status, and a document management level of the document storage subset is determined according to the management score.

Images