File electronic information analysis system based on big data

Abstract

The invention discloses an archive electronic information analysis system based on big data, which relates to the technical field of information technology and data mining, and comprises a data processing module, a graph construction module, a feature learning module and an analysis module. According to the method, electronic information data of original archives are collected and preprocessed to obtain a unified archive information set, an archive association graph is constructed, and initial connection weights between nodes are dynamically adjusted through an attention mechanism to obtain an optimized archive association weight matrix; a variational coding-decoding structure is adopted to map the optimized archive association weight matrix to a potential information space, archive association features are learned and reconstructed, and reconstructed samples are obtained; according to the method, the Softmax function is utilized to calculate classification probability distribution of archive recording nodes, accurate classification and potential anomaly recognition of archives are achieved, the defect that in the prior art, nonlinear correlation in archive data cannot be processed is effectively overcome, and the automation, intelligence and accuracy level of archive information analysis is remarkably improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of information technology and data mining, and in particular to an archive electronic information analysis system based on big data. BACKGROUND

[0002] In recent years, with the deepening of social informatization, the archive electronic data generated by various institutions has shown explosive growth. These archive data are usually large in quantity, complex in structure and have a large number of nonlinear correlation relationships.

[0003] At present, a wisdom archive management method and system based on big data are disclosed in Chinese Invention Patent No. CN119396985A. The method obtains the relevant data of paper and electronic archives, uses a collection time detection model to predict the collection time of paper archives, and manages based on archive basic information and the prediction result. The user query keyword is vectorized through word embedding technology and an adjustment formula, and the query range is dynamically adjusted combined with user historical query data, so that the query result is more in line with user habits and needs, and the user experience is effectively enhanced. However, in the related technology, the archive records and data items are not abstracted as nodes to construct an archive correlation graph, and the attention mechanism is not introduced to dynamically learn and optimize the initial connection weight between nodes, so that the semantic correlation strength between archives cannot be accurately quantified. Due to the lack of a variational encoding-decoding deep feature learning model, the correlation graph cannot be mapped to a latent space to learn its robust feature representation and reconstruction, so that the system cannot calculate the classification probability distribution of the archives based on the reconstructed deep features through a Softmax function to complete accurate classification analysis and potential anomaly identification. SUMMARY

[0004] The technical problems solved by the application are that in the related technology, the archive records and data items are not abstracted as nodes to construct an archive correlation graph, and the attention mechanism is not introduced to dynamically learn and optimize the initial connection weight between nodes, so that the semantic correlation strength between archives cannot be accurately quantified. Due to the lack of a variational encoding-decoding deep feature learning model, the correlation graph cannot be mapped to a latent space to learn its robust feature representation and reconstruction, so that the system cannot calculate the classification probability distribution of the archives based on the reconstructed deep features through a Softmax function to complete accurate classification analysis and potential anomaly identification.

[0005] To solve the above technical problems, the application provides the following technical scheme: an archive electronic information analysis system based on big data, comprising a data processing module, a graph construction module, a feature learning module and an analysis module. The data processing module is used for collecting original archive electronic information data, and pre-processing the original archive electronic information data to obtain a unified archive information set. The graph construction module is configured to construct an archive association graph based on the unified archive information set, and dynamically adjust initial connection weights between nodes in the archive association graph through an attention mechanism to obtain an optimized archive association weight matrix. The feature learning module is configured to map the optimized archive association weight matrix to a latent information space through a variational encoding-decoding structure, learn and reconstruct features of the archive association to obtain reconstructed samples. The analysis module is configured to calculate a classification probability distribution of the archive record node based on the reconstructed samples through a Softmax function, analyze the classification probability distribution of the archive record node to obtain an analysis result.

[0006] As a preferred scheme of the archive electronic information analysis system based on big data, the data processing module comprises a data acquisition unit and a data preprocessing unit. The data acquisition unit is configured to acquire original archive electronic information data, and the original archive electronic information data comprises archive content data, archive structure data and archive metadata. The data preprocessing unit is configured to preprocess the original archive electronic information to obtain a unified archive information set. The preprocessing comprises data cleaning, format standardization and code unification. The unified archive information set comprises archive content elements, archive structure elements and archive metadata elements.

[0007] As a preferred scheme of the archive electronic information analysis system based on big data, the graph construction module comprises a graph structure construction unit and a dynamic weight learning unit. The graph structure construction unit is configured to construct an archive association graph based on the unified archive information set, and the processing logic comprises: The archive record node and the data item node are both taken as an archive association graph node, in an archive association strength adaptive regulation mechanism, a preset dynamically changing context window is used to count the co-occurrence relationship between data item nodes, and a cosine similarity is used to calculate the semantic association degree between data item nodes, and the calculation formula is: ; wherein, represents the semantic association degree between data item nodes, represents a data item node in the archive association graph, represents another data item node different from in the archive association graph, represents the dimension of the feature vector, represents the dimension index number of the feature vector, represents a data item node in the archive association graph in the Values ​​in each dimension Indicates the relationship between the archives in the diagram. Another different data item node in the 1st Values ​​in each dimension; The strength of the association between data item nodes is calculated to obtain the archive association index value, and the calculation formula is as follows: ; ; ; in, This indicates the value of the file-related indicator. Represents data item nodes With data item nodes The co-occurrence probability, Represents data item nodes The probability of occurrence within a preset dynamically changing context window. Represents data item nodes The probability of occurrence within a preset dynamically changing context window. This indicates the total number of preset dynamically changing context windows. Indicates a node containing data items. The number of windows, Indicates that it contains data item nodes. and data item nodes The number of windows; The frequency-archive discriminant value is obtained by calculating the discriminant weight between archival record nodes and data item nodes. The formula is as follows: ; in, Indicates the frequency-database discrimination value. Represents a data item node. Indicates the record node, Represents data item nodes At the record node The number of times it appears in Indicates the total number of records. Indicates a node containing data items. The number of archive records, Indicates the record node The total number of times all data item nodes appear in the table. Indicates the index number of the data item node in the archive record; Based on the semantic correlation between data item nodes, the archive association index value, and the frequency-archive distinguishability value, the initial connection weight between each node in the archive association graph is determined. Based on the semantic correlation between data item nodes, the archive association index value, and the frequency-archive distinguishability value, the initial connection weights between nodes in the archive association graph are determined.

[0008] As a preferred embodiment of the big data-based archival electronic information analysis system of the present invention, the dynamic weight learning unit is used to dynamically adjust the initial connection weights between nodes in the archival association graph through an attention mechanism, and its processing logic includes: Based on the initial connection weights, an attention mechanism is used to calculate the real-time association degree between nodes in the file association graph. The calculation formula is as follows: ; in, This indicates the real-time correlation between nodes in the file association graph. Represents a node Dynamic feature representation during training; Data item node Dynamic feature representation during training; This represents the learnable weight matrix. Let || denote the attention vector, and || denote the vector concatenation operation. Represents a non-linear activation function. Represents any node in the file association graph. Indicates that the file association graph is different from the node Another arbitrary node; The initial connection weights are updated based on the real-time correlation between nodes in the archive association graph to obtain the optimized archive association weight matrix.

[0009] As a preferred embodiment of the big data-based archival electronic information analysis system of the present invention, the feature learning module adopts a variational coding-decoding structure, maps the optimized archival association weight matrix to the latent information space, and performs learning and reconstruction processing on the archival association features. Its processing logic includes: The encoder is based on a graph attention network, which maps the optimized archive association weight matrix to the mean and variance vectors in the latent space. Latent variables are sampled from a multivariate Gaussian distribution defined by the mean vector and variance vector using reparameterization techniques; The decoder reconstructs and decodes the latent variables into an archive association matrix, generating reconstructed samples; The initial connection weights of the variational coding-decoding structure are optimized by minimizing the joint loss function of reconstruction error and potential spatial distribution differences.

[0010] As a preferred embodiment of the big data-based electronic information analysis system for archives described in this invention, latent variables are determined based on the encoded output of the optimized archive association weight matrix. The latent spatial parameters include a mean vector and a variance vector, calculated using the following formula: ; in, Represents a grid for encoding latent features of archives. This represents the optimization of the file association weight matrix. Represents the mean vector. This represents the variance vector.

[0011] As a preferred embodiment of the big data-based electronic information analysis system for archives described in this invention, the latent variables are generated by introducing a random sampling mechanism, and their calculation formula is as follows: ; in, Representing latent variables, This represents element-wise multiplication. This represents random noise sampled from a standard normal distribution.

[0012] As a preferred embodiment of the big data-based electronic information analysis system for archives described in this invention, the latent variables are decoded and reconstructed into an archive association matrix to generate reconstructed samples. The calculation formula is as follows: ; in, This indicates a reconstructed sample.

[0013] As a preferred embodiment of the big data-based electronic information analysis system for archives according to the present invention, the analysis module includes a classification calculation unit and a result analysis unit; The classification calculation unit is used to calculate the classification probability distribution of archive record nodes based on the reconstructed samples using the Softmax function. Its processing logic includes: The reconstructed sample is input into the Softmax function to calculate the probability value of the archive record node belonging to each of a set of predefined groups; Based on the calculated probability values ​​of each group, a classification probability distribution vector for the archive record nodes is formed.

[0014] As a preferred embodiment of the big data-based electronic information analysis system for archives according to the present invention, the result analysis unit is used to analyze the classification probability distribution of archive record nodes to obtain analysis results, and its processing logic includes: Define a preset threshold set for the classification probability distribution vector of each archive record node. Each threshold corresponds to a criterion for determining a group in the classification probability vector; For each probability value in the classification probability distribution vector With the corresponding threshold Compare the nodes and determine those that meet the conditions; like If so, the file record node is determined to belong to the corresponding group; like If so, the file record node is determined not to belong to the corresponding group; The comparison results of each archive record node in each group are summarized to generate a node state vector; The analysis results are obtained by identifying potential abnormal nodes based on the node state vectors; The analysis results include the classification status of archive record nodes, potential anomaly identifiers, and classification probability distribution vectors.

[0015] The beneficial effects of this invention are as follows: By constructing an archive association graph with archive records and data items as nodes, and introducing an attention mechanism to dynamically optimize the initial connection weights between nodes, the complex semantic associations between archives are accurately quantified. A variational coding-decoding structure based on graph attention networks is adopted to map the association graph to the latent space to learn its robust deep feature representation and reconstruct it. Based on the reconstructed samples, the Softmax function is used to realize the accurate classification probability calculation and potential anomaly identification of archive records. This effectively overcomes the shortcomings of traditional technologies in handling nonlinear associations within archive data, and significantly improves the automation, intelligence and accuracy of archive information analysis. Attached Figure Description

[0016] Figure 1 This is a basic flowchart of a big data-based electronic information analysis system for archives, provided as an embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example 1, referring to Figure 1 As an embodiment of the present invention, a big data-based electronic information analysis system for archives is provided, including a data processing module, a graph construction module, a feature learning module, and an analysis module; The data processing module is used to collect original electronic information data from archives and preprocess the original electronic information data to obtain a unified archive information set; The graph construction module is used to construct an archive association graph based on a unified archive information set, and to dynamically adjust the initial connection weights between nodes in the archive association graph through an attention mechanism to obtain an optimized archive association weight matrix. The feature learning module is used to map the optimized archive association weight matrix to the latent information space through a variational coding-decoding structure, and to learn and reconstruct the archive association features to obtain reconstructed samples. The analysis module is used to calculate the classification probability distribution of archive record nodes based on the reconstructed samples using the Softmax function, and to analyze the classification probability distribution of archive record nodes to obtain the analysis results.

[0019] This invention constructs an archive association graph using archive records and data items as nodes, and introduces an attention mechanism to dynamically optimize the initial connection weights between nodes, thereby accurately quantifying the complex semantic relationships between archives. It adopts a variational coding-decoding structure based on graph attention networks to map the association graph to a latent space to learn its robust deep feature representation and reconstruct it. Based on the reconstructed samples, the Softmax function is used to realize the accurate classification probability calculation and potential anomaly identification of archive records. This effectively overcomes the shortcomings of traditional technologies in handling nonlinear relationships within archive data, and significantly improves the automation, intelligence and accuracy of archive information analysis.

[0020] The data processing module includes a data acquisition unit and a data preprocessing unit; The data acquisition unit is used to collect original electronic information data of archives, which includes archive content data, archive structure data, and archive metadata. The data preprocessing unit is used to preprocess the original electronic information of the archives to obtain a unified archive information set; Preprocessing includes data cleaning, format standardization, and encoding standardization; The unified archival information set includes archival content elements, archival structure elements, and archival metadata elements.

[0021] In practice, structured electronic archive data from office automation systems and business management systems are automatically collected through system interfaces. Electronic documents in PDF, DOC, and XML formats are processed by a file parsing engine to extract archive content data. Metadata information of the archives, including file number, creation time, file type, and storage path, is obtained through a database connection tool. Access records and modification history of the archives are collected through a log analysis system.

[0022] The data preprocessing unit executes a three-level processing flow: the data cleaning stage employs a rule-based data verification mechanism to automatically detect and repair data format errors, remove duplicate records, and supplement missing fields, ensuring the integrity and accuracy of electronic archive data; the format standardization stage converts all types of electronic documents into PDF / A format that meets long-term preservation requirements, standardizes and encapsulates structured data using XML format, and uses UTF-8 encoding for all text data; the coding unification stage establishes an electronic archive classification and coding system, using a combined coding structure of "institution code - year - archive type - serial number" to ensure the uniqueness and standardization of electronic archive identification.

[0023] The processed unified archival information set includes three core dimensions: archival content elements (full text of electronic documents, attachments, and multimedia materials), archival structure elements (file directory hierarchy, relationships, and version control information), and archival metadata elements (core metadata, technical metadata, and management metadata), forming a complete electronic archival information resource system.

[0024] This system's electronic archive data processing solution establishes a standardized acquisition and preprocessing mechanism, enabling efficient integration and standardized management of multi-source heterogeneous electronic archive data. This significantly improves the quality consistency and usability of electronic archive data, providing a reliable data foundation for big data-based archive analysis.

[0025] The graph construction module includes a graph structure construction unit and a dynamic weight learning unit; The graph structure construction unit is used to construct an archive association graph based on a unified archive information centralization. Its processing logic includes: Both archival record nodes and data item nodes are treated as nodes in the archival association graph. In the adaptive control mechanism for archival association strength, the co-occurrence relationship between data item nodes is collected through a preset dynamically changing context window. Cosine similarity is used to calculate the semantic association degree between data item nodes, and the calculation formula is as follows: ; in, This indicates the semantic relationship between data item nodes. This represents a data item node in the file association diagram. Indicates the relationship between the archives in the diagram. Another data item node, The dimension of the feature vector. The dimension index number of the feature vector. This indicates that a data item node in the file association diagram is in the 1st position. Values ​​in each dimension Indicates the relationship between the archives in the diagram. Another different data item node in the 1st Values ​​in each dimension; The strength of the association between data item nodes is calculated to obtain the archive association index value, and the calculation formula is as follows: ; ; ; in, This indicates the value of the file-related indicator. Represents data item nodes With data item nodes The co-occurrence probability, Represents data item nodes The probability of occurrence within a preset dynamically changing context window. Represents data item nodes The probability of occurrence within a preset dynamically changing context window. This indicates the total number of preset dynamically changing context windows. Indicates a node containing data items. The number of windows, Indicates that it contains data item nodes. and data item nodes The number of windows; The frequency-archive discriminant value is obtained by calculating the discriminant weight between archival record nodes and data item nodes. The formula is as follows: ; in, Indicates the frequency-database discrimination value. Represents a data item node. Indicates the record node, Represents data item nodes At the record node The number of times it appears in Indicates the total number of records. Indicates a node containing data items. The number of archive records, Indicates the record node The total number of times all data item nodes appear in the table. Indicates the index number of the data item node in the archive record; Based on the semantic correlation between data item nodes, the archive association index value, and the frequency-archive distinguishability value, the initial connection weights between nodes in the archive association graph are determined.

[0026] In practical implementation, both archival record nodes and data item nodes are treated as nodes in the archival association graph, establishing a heterogeneous graph structure containing both types of nodes. In the adaptive control mechanism for archival association strength, the system presets a dynamically changing context window. The window size adaptively adjusts based on the co-occurrence relationship between data item nodes, initially set to 5 adjacent data item nodes. The co-occurrence relationship between statistical data item nodes is statistically analyzed in the original archival electronic information data using a sliding window. Cosine similarity is then used to calculate the semantic association degree between data item nodes, with the following formula: ; in, This indicates the semantic relationship between data item nodes. This represents a data item node in the file association diagram. Indicates the relationship between the archives in the diagram. Another data item node, The dimension of the feature vector. The dimension index number of the feature vector. This indicates that a data item node in the file association diagram is in the 1st position. Values ​​in each dimension Indicates the relationship between the archives in the diagram. Another different data item node in the 1st Values ​​in each dimension; The strength of the association between data item nodes is calculated to obtain the archive association index value, and the calculation formula is as follows: ; ; ; in, This indicates the value of the file-related indicator. Represents data item nodes With data item nodes The co-occurrence probability, Represents data item nodes The probability of occurrence within a preset dynamically changing context window. Represents data item nodes The probability of occurrence within a preset dynamically changing context window. This indicates the total number of preset dynamically changing context windows. Indicates a node containing data items. The number of windows, Indicates that it contains data item nodes. and data item nodes The number of windows; The preset dynamically changing context window is associated with the frequency-archive discrimination value of the data item node, and its change rule is defined by the following formula: ; in, This represents the size of a base context window. This indicates the size of the preset dynamically changing context window. Indicates the scaling factor; The frequency-archive discriminant value is obtained by calculating the discriminant weight between archival record nodes and data item nodes. The formula is as follows: ; in, Indicates the frequency-database discrimination value. Represents a data item node. Indicates the record node, Represents data item nodes At the record node The number of times it appears in Indicates the total number of records. Indicates a node containing data items. The number of archive records, Indicates the record node The total number of times all data item nodes appear in the table. Indicates the index number of the data item node in the archive record; Based on three indicators—semantic relevance between data item nodes, archival association index value, and frequency-archival distinguishability value—a weighted fusion strategy is used to determine the initial connection weights between nodes in the archival association graph. The formula for calculating the initial connection weights is as follows: ; in, Indicates the initial connection weights. Weighting coefficients representing semantic relevance. This represents the weighting coefficient of the file-related indicator value. The frequency-data discrimination value is represented by a weighting coefficient. ; Weighting coefficient , , Instead of fixed values, these are dynamically generated through an adaptive weight adjustment module. The graph construction module calculates based on the information entropy of the semantic association between data item nodes, the archive association index value, and the frequency-archive distinguishability value, as follows: Calculate the information entropy of the semantic association degree between data item nodes, the archive association index value, and the frequency-archive discrimination value. The higher the information entropy, the more evenly the index is distributed among nodes, and the relatively weaker the discrimination ability. The initial weight of each indicator is calculated based on the information entropy, and the indicator with the lower information entropy receives the higher initial weight. The initial weights are normalized to obtain the final dynamic weight coefficients.

[0027] Through this adaptive mechanism, the system can automatically assign higher weights to indicators with stronger discriminative power based on the characteristics of the current archival dataset, thereby making the calculation of initial connection weights more reasonable and adaptable to different scenarios. By comprehensively considering multi-dimensional association indicators, the system achieves accurate quantification of the relationships between archival data, significantly improving the quality and expressive power of the archival association graph, and laying a solid foundation for subsequent deep feature learning and analysis.

[0028] The dynamic weight learning unit is used to dynamically adjust the initial connection weights between nodes in the file association graph through an attention mechanism. Its processing logic includes: Based on the initial connection weights, an attention mechanism is used to calculate the real-time association degree between nodes in the file association graph. The calculation formula is as follows: ; in, This indicates the real-time correlation between nodes in the file association graph. Represents a node Dynamic feature representation during training; Data item node Dynamic feature representation during training; This represents the learnable weight matrix. Let || denote the attention vector, and || denote the vector concatenation operation. Represents a non-linear activation function. Represents any node in the file association graph. Indicates that the file association graph is different from the node Another arbitrary node; The initial connection weights are updated based on the real-time correlation between nodes in the archive association graph to obtain the optimized archive association weight matrix.

[0029] In practice, an attention mechanism is used to calculate the real-time correlation between nodes in the file association graph. The specific calculation process is as follows: for any two nodes in the file association graph... and First, the dynamic features of the nodes during the training process are represented. and The data is concatenated and then processed using a learnable weight matrix. Perform a linear transformation, then use the attention vector. Projection is performed, and finally, the real-time correlation degree between each node in the file association graph is calculated using the LeakyReLU nonlinear activation function. The calculation formula is as follows: ; in, This indicates the real-time correlation between nodes in the file association graph. Represents a node The dynamic feature representation during training has 128 dimensions. Data item node The dynamic feature representation during training has 128 dimensions. This represents the learnable weight matrix, with dimensions 256×256. Let || denote the attention vector, and || denote the vector concatenation operation. This represents a non-linear activation function with a negative slope parameter set to 0.2. Represents any node in the file association graph. Indicates that the file association graph is different from the node Another arbitrary node; The learnable weight matrix and attention vector in the dynamic weight learning unit are optimized through an end-to-end training process. This training process is driven by the final analysis task of the entire system and specifically includes: The data processing module, graph construction module, feature learning module, and analysis module are executed sequentially to finally obtain the classification probability distribution of the archive record nodes; The cross-entropy loss between the model's predicted results and the true labels, together with the joint loss function of the variational encoder-decoder, constitutes the total loss. The gradient of the total loss is backpropagated to each layer of the network, including the dynamic weight learning unit, using the gradient descent algorithm. Based on the calculated gradient, update the values ​​of the learnable weight matrix and attention vector.

[0030] By iteratively executing the above process, the parameters in the attention mechanism are dynamically adjusted, enabling the model to learn feature representations that best reflect the essential relationships between archive nodes.

[0031] In the attention mechanism, in order to compute nodes With nodes Real-time correlation between The system will node Dynamic feature vectors With nodes Dynamic feature vectors Vector concatenation yields an optimized archive association weight matrix, enabling adaptive learning of archive association relationships. This significantly enhances the association graph's ability to model complex archive semantics, providing a more accurate graph structure foundation for subsequent deep feature learning and analysis.

[0032] The feature learning module employs a variational coding-decoding structure to map the optimized archive association weight matrix to the latent information space and performs learning and reconstruction processing on the archive association features. Its processing logic includes: The encoder is based on a graph attention network, which maps the optimized archive association weight matrix to the mean and variance vectors in the latent space. Latent variables are sampled from a multivariate Gaussian distribution defined by the mean vector and variance vector using reparameterization techniques; The decoder reconstructs the latent variables into an archive association matrix, generating reconstructed samples; The initial connection weights of the variational coding-decoding structure are optimized by minimizing the joint loss function of reconstruction error and potential spatial distribution differences.

[0033] In practical implementation, a variational encoding-decoding structure was used to comprehensively improve feature representation capabilities, model generalization performance, feature reconstruction accuracy, computational efficiency, and adaptive learning capabilities. The optimized file association weight matrix was mapped to the latent information space, and a variational inference framework employing reparameterization techniques significantly improved the model's stability in sparse data scenarios. Through a bidirectional encoding-decoding mapping mechanism, feature compression and reconstruction were achieved while maintaining the data structure. The low-dimensional representation of the latent space reduced computational complexity while preserving information integrity. The parameters of the variational coding-decoding structure are optimized by minimizing the joint loss function, which is defined as the sum of the reconstruction error and the latent space distribution relative to the standard normal distribution. The weighted sum of divergences is expressed as: ; in, This represents the value of the joint loss function. This represents the error between the reconstructed sample and the optimized archive association weight matrix. Represents the relationship between the latent spatial distribution and the normal distribution. Divergence.

[0034] The latent variables are determined based on the encoded output of the optimized archive association weight matrix. The latent spatial parameters include the mean vector and the variance vector, and their calculation formula is as follows: ; in, Represents a grid for encoding latent features of archives. This represents the optimization of the file association weight matrix. Represents the mean vector. This represents the variance vector.

[0035] In its implementation, the encoder employs a four-layer graph attention network structure, with each layer containing eight attention heads and a hidden layer dimension of 128. After inputting the optimized archive association weight matrix, the encoder extracts the features of the archive association graph nodes through a graph attention mechanism. Finally, it outputs the mean vector and the logarithm vector of variance through two independent fully connected layers. The use of the logarithm form of the variance ensures the positive definiteness of the variance value and improves numerical stability, achieving efficient compression and abstract representation of archive association features. This significantly enhances the effect and generalization ability of feature learning, providing a high-quality latent feature foundation for subsequent archive analysis tasks.

[0036] The latent variables are generated by introducing a random sampling mechanism, and their calculation formula is as follows: ; in, Representing latent variables, This represents element-wise multiplication. This represents random noise sampled from a standard normal distribution.

[0037] In practice, latent variables are generated by introducing a random sampling mechanism, and their calculation formula is as follows: ; in, This represents a latent variable with 64 dimensions. This represents element-wise multiplication. This represents random noise sampled from a standard normal distribution. .

[0038] From the standard normal distribution Independent sampling generates random noise Its dimensions are consistent with the mean vector. Through reparameterization techniques, the variance vector is determined. Mean vector With random noise Combined, latent variables with randomness are generated. The advantage of this method lies in separating the source of randomness from the trainable parameters, allowing gradients to propagate back through a deterministic path. This achieves an effective combination of determinism and randomness, significantly improving the expressive power and generalization performance of feature learning, and providing a richer potential representation foundation for the deep analysis and generation of archival data.

[0039] The latent variables are decoded and reconstructed into an archival association matrix, generating reconstructed samples. The calculation formula is as follows: ; in, This indicates a reconstructed sample.

[0040] In its implementation, the decoder employs a three-layer graph neural network architecture, with each layer containing 128 hidden units. After inputting the latent variable z, the decoder first expands the 64-dimensional latent features to 256 dimensions through a fully connected layer, and then progressively reconstructs the relationship strength between nodes through multi-layer graph convolution operations. The final output layer uses the sigmoid function to ensure that the elements of the reconstructed association weight matrix are within the range [0,1], achieving a precise mapping from latent variables to the archival association matrix. This significantly improves the accuracy and stability of feature reconstruction, providing reliable technical support for in-depth analysis and intelligent retrieval of archival data.

[0041] The analysis module includes a classification calculation unit and a result analysis unit; The classification calculation unit is used to calculate the classification probability distribution of archive record nodes based on the reconstructed samples using the Softmax function. Its processing logic includes: The reconstructed sample is input into the Softmax function to calculate the probability value of the archive record node belonging to each of a set of predefined groups; Based on the calculated probability values ​​of each group, a classification probability distribution vector for the archive record nodes is formed.

[0042] In practice, based on the reconstructed samples, the Softmax function is used to calculate the probability distribution of each archival record node in a predefined group. First, the reconstructed samples are input into a graph attention network, and node features are mapped to the predefined group space through a fully connected layer. Then, the probability is normalized using the Softmax function to calculate the probability value of a node belonging to each group. The predefined groups are set according to archival management needs, including grouping by archival type, security classification, and retention period.

[0043] The system performs a linear transformation on the input features to obtain the original score for each group. Then, an exponential function is used to convert the score to a non-negative value. Finally, normalization ensures that the sum of the probabilities of all groups is 1. Each archival record node generates a complete probability distribution vector, where each element represents the confidence level of the node belonging to its corresponding group. The system supports dynamic adjustment of predefined groups, allowing for flexible configuration of group categories and quantities according to actual management needs. This enables accurate classification and intelligent management of archival records, significantly improving the accuracy and automation level of archival classification and providing reliable technical support for the efficient utilization of archival information resources.

[0044] The results analysis unit is used to analyze the classification probability distribution of archival record nodes and obtain analysis results. Its processing logic includes: Define a preset threshold set for the classification probability distribution vector of each archive record node. Each threshold corresponds to a criterion for determining a group in the classification probability vector; For each probability value in the classification probability distribution vector With the corresponding threshold Compare the nodes and determine those that meet the conditions; like If so, the file record node is determined to belong to the corresponding group; like If so, the file record node is determined not to belong to the corresponding group; The comparison results of each archive record node in each group are summarized to generate a node state vector; The analysis results are obtained by identifying potential abnormal nodes based on the node state vectors; The analysis results include the classification status of archive record nodes, potential anomaly indicators, and classification probability distribution vectors.

[0045] In practice, a multi-level threshold determination mechanism is established to achieve accurate classification and anomaly identification of archival record nodes. The system sets corresponding probability thresholds for each predefined group. These thresholds are dynamically adjusted based on the importance of the group and the required classification accuracy. Important groups (such as high-density classification) have higher thresholds (T). 密级 ≥0.8), the threshold setting for ordinary groups (such as file types) is moderate (T 类型 ≥0.6).

[0046] During processing, the system iterates through the classification probability distribution vector of each file record node. The probability values ​​of the classification probability distribution vector With the corresponding threshold Compare them.

[0047] For mutually exclusive groups (i.e., the same file cannot belong to two types simultaneously), the system executes a conflict resolution mechanism. If it is found that the probability values ​​of a node in multiple mutually exclusive groups all satisfy... If the conditions are met, the system selects the group with the highest probability value as the final classification of the node, and marks the node's status record with the "classification conflict resolved" flag for auditing and review.

[0048] For non-mutually exclusive groups or cases where there is no conflict, the system makes a judgment according to basic rules: when When, the record node is determined to belong to the corresponding group; when If the record node does not belong to the corresponding group, then the record node is determined to be non-member of the corresponding group. The determination results of all groups are encoded into a binary state vector, where each bit represents the node state vector of a node in a group.

[0049] Based on node state vectors, the system performs anomaly detection. Anomaly rules are explicitly predefined during system deployment: a node is marked as a "classification conflict" anomaly when it simultaneously meets threshold conditions in multiple mutually exclusive groups; it is marked as an "undefined classification" anomaly when the probability values ​​of all groups do not reach the threshold; and it is marked as a "logical inconsistency" anomaly when the probability distribution of the primary group logically contradicts that of the associated groups. These anomalous nodes are automatically pushed to a manual review queue, and the system records the anomaly type and confidence level, achieving accurate classification and intelligent review of archival records. This significantly improves the automation level and decision-making quality of archival management, providing reliable technical support for the standardized management and effective utilization of archival information resources.

[0050] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0051] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. A big data-based electronic information analysis system for archives, characterized in that: It includes a data processing module, a graph construction module, a feature learning module, and an analysis module; The data processing module is used to collect original electronic information data of archives and preprocess the original electronic information data of archives to obtain a unified archive information set. The graph construction module is used to construct an archive association graph based on a unified archive information set, and to dynamically adjust the initial connection weights between nodes in the archive association graph through an attention mechanism to obtain an optimized archive association weight matrix. The feature learning module is used to map the optimized archive association weight matrix to the latent information space through a variational coding-decoding structure, and to learn and reconstruct the archive association features to obtain reconstructed samples. The analysis module is used to calculate the classification probability distribution of archive record nodes based on the reconstructed samples using the Softmax function, and to analyze the classification probability distribution of archive record nodes to obtain analysis results.

2. The big data-based electronic information analysis system for archives as described in claim 1, characterized in that, The data processing module includes a data acquisition unit and a data preprocessing unit; The data acquisition unit is used to collect original electronic information data of archives, which includes archive content data, archive structure data and archive metadata. The data preprocessing unit is used to preprocess the original electronic information of the archives to obtain a unified archive information set; The preprocessing includes data cleaning, format standardization, and encoding standardization; The unified archival information set includes archival content elements, archival structure elements, and archival metadata elements.

3. The big data-based electronic information analysis system for archives as described in claim 1, characterized in that, The graph construction module includes a graph structure construction unit and a dynamic weight learning unit; The graph structure construction unit is used to construct an archive association graph based on a unified archive information centralization, and its processing logic includes: Both archival record nodes and data item nodes are treated as nodes in the archival association graph. In the adaptive control mechanism for archival association strength, the co-occurrence relationship between data item nodes is collected through a preset dynamically changing context window. Cosine similarity is used to calculate the semantic association degree between data item nodes, and the calculation formula is as follows: ； in, This indicates the semantic relationship between data item nodes. This represents a data item node in the file association diagram. Indicates the relationship between the archives in the diagram. Another data item node, The dimension of the feature vector. The dimension index number of the feature vector. This indicates that a data item node in the file association diagram is in the 1st position. Values ​​in each dimension Indicates the relationship between the archives in the diagram. Another different data item node in the 1st Values ​​in each dimension; The strength of the association between data item nodes is calculated to obtain the archive association index value, and the calculation formula is as follows: ； ； ； in, This indicates the value of the file-related indicator. Represents data item nodes With data item nodes The co-occurrence probability, Represents data item nodes The probability of occurrence within a preset dynamically changing context window. Represents data item nodes The probability of occurrence within a preset dynamically changing context window. This indicates the total number of preset dynamically changing context windows. Indicates a node containing data items. The number of windows, Indicates that it contains data item nodes. and data item nodes The number of windows; The frequency-archive discriminant value is obtained by calculating the discriminant weight between archival record nodes and data item nodes. The formula is as follows: ； in, Indicates the frequency-database discrimination value. Represents a data item node. Indicates the record node, Represents data item nodes At the record node The number of times it appears in Indicates the total number of records. Indicates a node containing data items. The number of archive records, Indicates the record node The total number of times all data item nodes appear in the table. Indicates the index number of the data item node in the archive record; Based on the semantic correlation between data item nodes, the archive association index value, and the frequency-archive distinguishability value, the initial connection weights between nodes in the archive association graph are determined.

4. The big data-based electronic information analysis system for archives as described in claim 3, characterized in that, The dynamic weight learning unit is used to dynamically adjust the initial connection weights between nodes in the file association graph through an attention mechanism. Its processing logic includes: Based on the initial connection weights, an attention mechanism is used to calculate the real-time association degree between nodes in the file association graph. The calculation formula is as follows: ； in, This indicates the real-time correlation between nodes in the file association graph. Represents a node Dynamic feature representation during training; Data item node Dynamic feature representation during training; This represents the learnable weight matrix. Let || denote the attention vector, and || denote the vector concatenation operation. Represents a non-linear activation function. Represents any node in the file association graph. Indicates that the file association graph is different from the node Another arbitrary node; The initial connection weights are updated based on the real-time correlation between nodes in the archive association graph to obtain the optimized archive association weight matrix.

5. The big data-based electronic information analysis system for archives as described in claim 1, characterized in that, The feature learning module adopts a variational coding-decoding structure, maps the optimized archive association weight matrix to the latent information space, and performs learning and reconstruction processing on the archive association features. Its processing logic includes: The encoder is based on a graph attention network, which maps the optimized archive association weight matrix to the mean and variance vectors in the latent space. Latent variables are sampled from a multivariate Gaussian distribution defined by the mean vector and variance vector using reparameterization techniques; The decoder reconstructs and decodes the latent variables into an archive association matrix, generating reconstructed samples; The initial connection weights of the variational coding-decoding structure are optimized by minimizing the joint loss function of reconstruction error and potential spatial distribution differences.

6. The big data-based electronic information analysis system for archives as described in claim 5, characterized in that, The latent variables are determined based on the encoded output of the optimized archive association weight matrix. The latent spatial parameters include a mean vector and a variance vector, which are calculated using the following formula: ； in, Represents a grid for encoding latent features of archives. This represents the optimization of the file association weight matrix. Represents the mean vector. This represents the variance vector.

7. The big data-based electronic information analysis system for archives as described in claim 6, characterized in that, The latent variables are generated by introducing a random sampling mechanism, and their calculation formula is as follows: ； in, Representing latent variables, This represents element-wise multiplication. This represents random noise sampled from a standard normal distribution.

8. The big data-based electronic information analysis system for archives as described in claim 7, characterized in that, The latent variables are decoded and reconstructed into an archival association matrix, generating reconstructed samples. The calculation formula is as follows: ； in, This indicates a reconstructed sample.

9. The big data-based electronic information analysis system for archives as described in claim 8, characterized in that, The analysis module includes a classification calculation unit and a result analysis unit; The classification calculation unit is used to calculate the classification probability distribution of archive record nodes based on the reconstructed samples using the Softmax function. Its processing logic includes: The reconstructed sample is input into the Softmax function to calculate the probability value of the archive record node belonging to each of a set of predefined groups; Based on the calculated probability values ​​of each group, a classification probability distribution vector for the archive record nodes is formed.

10. The big data-based electronic information analysis system for archives as described in claim 9, characterized in that, The result analysis unit is used to analyze the classification probability distribution of archival record nodes and obtain analysis results. Its processing logic includes: Define a preset threshold set for the classification probability distribution vector of each archive record node. Each threshold corresponds to a criterion for determining a group in the classification probability vector; For each probability value in the classification probability distribution vector With the corresponding threshold Compare the nodes and determine those that meet the conditions; like If so, the file record node is determined to belong to the corresponding group; like If so, the file record node is determined not to belong to the corresponding group; The comparison results of each archive record node in each group are summarized to generate a node state vector; The analysis results are obtained by identifying potential abnormal nodes based on the node state vectors; The analysis results include the classification status of archive record nodes, potential anomaly identifiers, and classification probability distribution vectors.

Images