Data mining-based storage period prediction method and system

Abstract

The application provides a data mining-based storage period prediction method and system, relates to the technical field of archive management, and comprises the following steps: analyzing a classification scheme and a storage period table to construct a labeled sample set, and training a period prediction mapping relationship by combining the text and metadata features of historical archiving data. For the data to be predicted, the modified period result is calculated by determining the rule matching degree, and the source system identifier is bound and returned. The application can improve the accuracy and automation level of the storage period prediction.

Description

Technical Field

[0001] This invention relates to the field of archives management technology, and in particular to a method and system for predicting retention periods based on data mining. Background Technology

[0002] In the field of records management, determining the retention period of electronic documents or archives is a fundamental and crucial task. Traditional methods for determining retention periods mainly rely on manual operation, which involves judging and assigning a corresponding retention period based on the specific content of the document.

[0003] With the deepening of informatization, the number of electronic documents is growing exponentially, and the types and contents of documents are becoming increasingly complex. The traditional manual judgment mode is difficult to cope with the rapid processing needs of large-scale documents when dealing with massive amounts of multi-source electronic documents, and it is easy to become a bottleneck in the process of digitizing archives. At the same time, manual judgment has the problems of subjectivity and inconsistency, resulting in the same type of documents being assigned different retention periods, which affects the standardization and seriousness of archive management.

[0004] In existing technologies, some solutions attempt to introduce automated methods based on keyword matching or simple rule engines, which reduces the burden on humans to some extent, but the level of intelligence is limited and it is difficult to cover all complex semantic scenarios and exceptions. At the same time, for text content with diverse expressions and rich semantics, using simple keyword matching is prone to misjudgment and omissions, making it difficult to meet high standards of record management requirements. Summary of the Invention

[0005] This invention provides a data mining-based method and system for predicting storage period, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of this invention provides a method for predicting retention period based on data mining, comprising:

[0007] Obtain classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set;

[0008] Historical archived data is acquired and the corresponding text fields and metadata fields are extracted. The text fields are segmented to determine the word sequence and extract the text feature vector. The metadata fields are structured and parsed to obtain the attribute feature vector. The text feature vector and the attribute feature vector are concatenated to obtain the sample feature representation and combined with the labeled sample set to solve for the term prediction mapping relationship.

[0009] The system receives a request to predict data and generates a feature representation corresponding to the data to be predicted. Based on the time prediction mapping relationship, it calculates the time candidate results and the corresponding confidence scores. It extracts the applicable rules corresponding to the data to be predicted from the rule set and calculates the rule matching score. Based on the confidence score and the rule matching score, it solves for the corrected time result.

[0010] The revised deadline result is bound to the source system identifier of the data to be predicted. The interface mapping table is queried based on the source system identifier to obtain the return interface address and the revised deadline result is returned.

[0011] In one alternative implementation,

[0012] Obtain classification scheme data and storage period table data; parse the classification scheme data to obtain a category tree structure; extract a rule set based on the storage period table data; and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set, including:

[0013] The classification scheme configuration file is read from the file management system and parsed to obtain classification scheme data; the retention period configuration file is read from the file management system and parsed to obtain retention period table data.

[0014] Extract the category identifier and parent-child relationship of each level node from the classification scheme data, construct hierarchical connections between nodes based on the parent-child relationship and determine the depth value of each node, construct a tree structure based on the category identifier, combined with the hierarchical connections and the depth value, obtain the category tree structure, and assign corresponding node identifiers to the tree nodes in the category tree structure;

[0015] The storage period table data is used to group the period rules according to the rule type and extract the condition fields. The condition fields and condition values ​​are combined into rule expressions and standardized to obtain the rule set.

[0016] Extract the node identifiers corresponding to the tree nodes in the category tree structure, extract the applicable conditions corresponding to each rule expression from the rule set, perform matching calculations on the category restriction fields in the applicable conditions and the node identifiers to obtain a matching metric value, establish an association relationship between the node identifiers and the rule expressions based on the matching metric value, and combine the node identifiers in the association relationship with the term labels corresponding to the rule expressions to obtain the labeled sample set.

[0017] In one alternative implementation,

[0018] Historical archived data is acquired and corresponding text and metadata fields are extracted. The text fields are segmented to determine word sequences and text feature vectors are extracted. The metadata fields are structured and parsed to obtain attribute feature vectors, including:

[0019] Historical archived data is read from the archives management system and the data format is identified. Based on the data format, the historical archived data is separated to extract text fields and metadata fields.

[0020] The text field is segmented into sub-words using byte-pair encoding to obtain an initial word sequence. Stop words in the initial word sequence are filtered to obtain a valid word sequence. The valid word sequence is then scanned using a context window, and a word association graph is constructed based on co-occurrence relationships. The word association graph is then subjected to graph convolution propagation to obtain a word embedding representation. The word embedding representation is then concatenated based on the valid word sequence, and positional encoding is introduced to obtain the text feature vector.

[0021] The field values ​​are extracted from the metadata field and the data type of the field values ​​is determined. The field values ​​of the data type are discretized by bucketing to construct a numerical encoding vector. The field values ​​of the data type are mapped by entity embedding to obtain a category encoding vector. The field values ​​of the data type are decomposed by Fourier transform and frequency domain features are extracted to obtain a time encoding vector. The numerical encoding vector and the category encoding vector are combined to perform interactive modeling and weighted aggregation to obtain the attribute feature vector.

[0022] In one alternative implementation,

[0023] The sample feature representation is obtained by concatenating the text feature vector with the attribute feature vector, and the maturity prediction mapping relationship is obtained by combining the concatenation of the text feature vector with the attribute feature vector and then solving the mapping relationship with the labeled sample set.

[0024] The statistical dependency between the text feature vector and the attribute feature vector is calculated using information theory methods, and a feature dependency graph is constructed. Feature clusters in the feature dependency graph are identified using a community detection algorithm. The feature dimensions within the feature clusters are scored for importance to obtain a score sequence. Based on the distribution characteristics of the score sequence, a screening threshold is determined, and feature dimensions with importance scores higher than the screening threshold are selected to obtain the sample feature representation.

[0025] Training samples, corresponding sample feature representations, and term labels are generated based on the labeled sample set. The sample feature representations are nonlinearly mapped using a kernel function method to obtain a mapping vector. Based on the mapping vector and the term labels, the mapping coefficients are obtained by kernel regression and the training samples are then used to predict the predicted output.

[0026] The residual vector is obtained by calculating the difference between the predicted output and the term label. The main error components corresponding to the residual vector are extracted by matrix factorization and a residual correction amount is constructed. The residual correction amount is fused with the mapping coefficient to obtain the corrected mapping coefficient. Based on the corrected mapping coefficient, the mapping relationship between the sample feature representation and the term prediction result is determined to obtain the term prediction mapping relationship.

[0027] In one alternative implementation,

[0028] Receiving a request for data to be predicted and generating a feature representation of the data to be predicted, and calculating candidate dates and corresponding confidence scores based on the date prediction mapping relationship, including:

[0029] The data to be predicted is separated into a text field to be predicted and a metadata field to be predicted. Lexical analysis is performed on the text field to be predicted to obtain a predicted text vector. Encoding conversion is performed on the metadata field to be predicted to obtain a predicted attribute vector.

[0030] The predicted text vector and the predicted attribute vector are concatenated by tensors to obtain the predicted input vector. The variance contribution rate of each dimension of the predicted input vector is calculated and compared with a preset variance threshold. Dimensions with a variance contribution rate greater than the preset variance threshold are retained and a feature representation to be predicted is constructed.

[0031] The feature to be predicted is input into the term prediction mapping relationship to obtain a term probability vector. The term candidate results are obtained by extracting the term categories with the highest probability values ​​from the term probability vector.

[0032] The Euclidean distance between the feature representation to be predicted and the feature representations of each sample in the labeled sample set is calculated to obtain a set of distance values. Based on the set of distance values ​​and a preset distance threshold, a set of nearest neighbor samples is determined. The frequency of the candidate deadline result in the nearest neighbor sample set is counted and the frequency ratio is calculated to obtain a neighborhood consistency score. The information entropy corresponding to the deadline probability vector is calculated and a prediction certainty score is determined based on the information entropy. The confidence score corresponding to the candidate deadline result is calculated based on the neighborhood consistency score and the prediction certainty score.

[0033] In one alternative implementation,

[0034] The applicable rules corresponding to the data to be predicted are extracted from the rule set, and the rule matching score is calculated. The revised period result is obtained based on the confidence score and the rule matching score, including:

[0035] The key field set is extracted from the data to be predicted and compared with the trigger conditions of each rule in the rule set to obtain the field matching result. Based on the field matching result, the number of matched fields is counted and the basic matching degree is calculated. The priority of each rule in the rule set is encoded to obtain the priority weight vector and combined with the basic matching degree to calculate the weighted matching degree. The rule with the largest weighted matching degree is extracted as the applicable rule and the rule matching degree score is determined.

[0036] The confidence score and the rule matching score are normalized to obtain the standard confidence score and the standard matching score, and the difference is calculated to obtain the score difference degree. The fusion weight coefficient is determined based on the score difference degree. The predicted contribution is calculated based on the standard confidence score and the fusion weight coefficient. The rule contribution is obtained by multiplying the standard matching score by the complement of the fusion weight coefficient. The comprehensive decision score is obtained by combining the predicted contribution.

[0037] Extract the term value from the term candidate results, extract the rule-specified term value from the applicable rules, calculate the difference between the term value and the rule-specified term value to obtain the term deviation, and calculate the correction deviation by combining the comprehensive decision score. Based on the term value and the correction deviation, solve for the corrected term result.

[0038] In one alternative implementation,

[0039] Binding the revised deadline result to the source system identifier of the data to be predicted, obtaining the return interface address based on the source system identifier by querying the interface mapping table, and returning the revised deadline result includes:

[0040] Extract the source system identifier from the data to be predicted, associate and bind the correction period result with the source system identifier to obtain binding result data and generate a unique identifier code, combine and encode the unique identifier code with the generation timestamp of the correction period result to obtain a traceable identifier, establish a result traceability record based on the traceable identifier and store the association relationship between the binding result data and the data to be predicted;

[0041] The initial interface address is obtained by querying the interface mapping table based on the source system identifier. The call source information of the data to be predicted is extracted from the result traceability record. The validity of the initial interface address is verified based on the call source information, and it is determined whether the initial interface address is available. If the initial interface address is unavailable, the backup interface address corresponding to the source system identifier is queried from the interface mapping table. The connectivity test is performed on the backup interface address, and the backup interface address that passes the test is used as the return interface address.

[0042] The correction period result and the traceability identifier are encapsulated into a return data packet, and the return data packet is sent to the target system corresponding to the source system identifier through the return interface address and an acknowledgment signal is received.

[0043] A second aspect of the present invention provides a data mining-based retention period prediction system, comprising:

[0044] The sample set construction module is used to acquire classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set.

[0045] The mapping relationship training module is used to acquire historical archived data and extract corresponding text fields and metadata fields. The text fields are segmented to determine word sequences and extract text feature vectors. The metadata fields are structured to obtain attribute feature vectors. The text feature vectors and attribute feature vectors are concatenated to obtain sample feature representations and combined with the labeled sample set to solve for the term prediction mapping relationship.

[0046] The term result calculation module is used to receive the call request for the data to be predicted and generate the feature representation to be predicted corresponding to the data to be predicted, calculate the term candidate result and the corresponding confidence score based on the term prediction mapping relationship, extract the applicable rule corresponding to the data to be predicted from the rule set and calculate the rule matching score, and solve for the corrected term result based on the confidence score and the rule matching score.

[0047] The result return module is used to bind the revised period result with the source system identifier of the data to be predicted, query the interface mapping table based on the source system identifier to obtain the return interface address and return the revised period result.

[0048] A third aspect of the present invention provides an electronic device, comprising:

[0049] A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0050] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0051] In this invention, a category tree is constructed by parsing the classification scheme, and a rule set is extracted by combining the retention period table. Nodes are associated with rule conditions, thereby constructing a high-quality labeled sample set. This ensures that the training data has a clear structured logic and business rule foundation, effectively overcoming prediction bias caused by rule ambiguity or inconsistent sample labeling. By integrating the text features and metadata attribute features of historical archived data, a comprehensive and in-depth sample feature representation is constructed. This word segmentation and vectorization process can capture the key semantic information of the archive content, while the structured metadata provides an objective business attribute context. The concatenated feature vector can simultaneously learn content associations and rule constraints, thereby solving for a more accurate period prediction mapping relationship. This achieves an organic combination of data-driven and rule knowledge. Through the comprehensive solution of confidence and rule matching degree, the preliminary prediction results are corrected and optimized, effectively reconciling the potential contradictions between data model prediction and rigid business rules. The corrected period results are bound to the source system identifier, and the return address is automatically located through the query interface mapping table, meeting the automated archiving needs of multiple systems and scenarios, and significantly improving the overall efficiency and intelligence level of archive management. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the data mining-based method for predicting retention periods according to an embodiment of the present invention.

[0053] Figure 2 This is a flowchart illustrating the rule matching and period correction process of the data mining-based retention period prediction method according to an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0056] Figure 1 This is a flowchart illustrating the data mining-based retention period prediction method of this invention, as shown in the embodiment. Figure 1 As shown, the method includes:

[0057] Obtain classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set;

[0058] Historical archived data is acquired and the corresponding text fields and metadata fields are extracted. The text fields are segmented to determine the word sequence and extract the text feature vector. The metadata fields are structured and parsed to obtain the attribute feature vector. The text feature vector and the attribute feature vector are concatenated to obtain the sample feature representation and combined with the labeled sample set to solve for the term prediction mapping relationship.

[0059] The system receives a request to predict data and generates a feature representation corresponding to the data to be predicted. Based on the time prediction mapping relationship, it calculates the time candidate results and the corresponding confidence scores. It extracts the applicable rules corresponding to the data to be predicted from the rule set and calculates the rule matching score. Based on the confidence score and the rule matching score, it solves for the corrected time result.

[0060] The revised deadline result is bound to the source system identifier of the data to be predicted. The interface mapping table is queried based on the source system identifier to obtain the return interface address and the revised deadline result is returned.

[0061] In one alternative implementation,

[0062] Obtain classification scheme data and storage period table data; parse the classification scheme data to obtain a category tree structure; extract a rule set based on the storage period table data; and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set, including:

[0063] The classification scheme configuration file is read from the file management system and parsed to obtain classification scheme data; the retention period configuration file is read from the file management system and parsed to obtain retention period table data.

[0064] Extract the category identifier and parent-child relationship of each level node from the classification scheme data, construct hierarchical connections between nodes based on the parent-child relationship and determine the depth value of each node, construct a tree structure based on the category identifier, combined with the hierarchical connections and the depth value, obtain the category tree structure, and assign corresponding node identifiers to the tree nodes in the category tree structure;

[0065] The storage period table data is used to group the period rules according to the rule type and extract the condition fields. The condition fields and condition values ​​are combined into rule expressions and standardized to obtain the rule set.

[0066] Extract the node identifiers corresponding to the tree nodes in the category tree structure, extract the applicable conditions corresponding to each rule expression from the rule set, perform matching calculations on the category restriction fields in the applicable conditions and the node identifiers to obtain a matching metric value, establish an association relationship between the node identifiers and the rule expressions based on the matching metric value, and combine the node identifiers in the association relationship with the term labels corresponding to the rule expressions to obtain the labeled sample set.

[0067] A connection is established with the archives management system via a data interface. The system reads the classification scheme configuration file stored in the system. This configuration file is typically stored in XML or JSO7 format and contains multi-dimensional classification information such as organizational structure, business type, and document category. When parsing this configuration file, a DOM parser or streaming parser is used to scan the file content level by level, extracting fields such as the code, name, and attribution relationship for each classification level, forming structured classification scheme data. Simultaneously, the system reads the retention period configuration file from the archives management system. This file records the retention period regulations corresponding to different archive types, including permanent, 30-year fixed-term, and 10-year fixed-term periods, as well as the applicable scope and judgment conditions for each period. The configuration file parser is used to parse this file, extracting key information such as period type, applicable conditions, and business scope, forming a retention period table.

[0068] The classification scheme data undergoes deep analysis, traversing all node records to extract the category identifier field for each node. This identifier is typically represented using an encoding system, such as dotted decimal or hierarchical encoding. Simultaneously, the parent node reference field is extracted to determine the parent-child relationship between nodes. Based on these relationships, a hierarchical connection structure is established, performing a breadth-first traversal starting from the root node. The depth of each node in the tree structure is calculated, with the root node's depth set to 0 and child node depths increasing sequentially. During the traversal, multiple child nodes under the same parent node are arranged according to their encoding order, forming an ordered sequence of sibling nodes. Using the category identifier as the unique key for each node, combined with the established hierarchical connections and calculated depth values, a tree data structure is used to organize all nodes, nesting them according to their hierarchical relationships to form a complete category tree structure. After the category tree structure is built, a globally unique node identifier is assigned to each tree node. This identifier is generated using a UUID generation algorithm or a path-based encoding method to ensure accurate node location in subsequent association mappings.

[0069] For rule extraction from the retention period table data, the period rules are grouped according to the rule type field. Rule types include rules based on business category, document nature, and time attribute. For each group of rule records, the condition fields are parsed, including multiple dimensions of constraints such as applicable business category range, document confidentiality requirements, year of formation, and responsible party type. The extracted condition fields and their corresponding condition values ​​are logically combined to construct a rule expression in the form of "business category equals accounting voucher and confidentiality level equals ordinary". The constructed rule expressions are then standardized by unifying field names to standard field names, converting condition values ​​to standardized enumeration values ​​or numerical ranges, and unifying logical operators to standard forms, resulting in a consistent set of rule expressions, thus obtaining the rule set. Each rule expression in the rule set is associated with a corresponding period tag, which identifies the applicable retention period type.

[0070] The process involves traversing the category tree structure to extract node identifiers corresponding to all tree nodes, forming a node identifier list. For each rule expression, the applicable conditions are extracted from the rule set. These conditions include a category constraint field describing the applicable classification range. The category constraint field is parsed to identify the category code or category name, and then matched against each node identifier in the node identifier list. During matching, a string similarity algorithm is used to calculate the similarity between the category constraint field and the category code corresponding to the node identifier, or a semantic matching algorithm is used to calculate the semantic distance between category names, yielding a matching metric. The matching metric ranges from 0 to 1, with higher values ​​indicating a higher degree of matching. A threshold is set based on the matching metric. When the matching metric exceeds the preset threshold, the node identifier is considered to have an association with the rule expression, and an association mapping is established between the node identifier and the rule expression. For a node identifier that may be associated with multiple rule expressions, all associations that meet the threshold condition are retained. The node identifiers in the established associations are combined with the term labels of the corresponding rule expressions to form labeled sample pairs consisting of node identifiers and term labels. Multiple labeled sample pairs are then aggregated to form a labeled sample set.

[0071] In this embodiment, by performing structured parsing and unified modeling of the classification scheme configuration file and the retention period configuration file, the standardized expression of the classification system and retention period rules is realized, avoiding the semantic inconsistency caused by relying on manual understanding of configuration rules, and improving the accuracy and consistency of data parsing. By constructing a category tree structure based on parent-child relationships and depth information, the explicit expression of the archive classification hierarchy and the management of unique node identifiers are realized, significantly enhancing the hierarchical semantic expression capability and computability of the classification system, improving the refinement and scalability of rule matching, and by matching and calculating the category node identifier with the rule application conditions and introducing a matching measurement mechanism, the automatic association between rules and classification nodes is realized, effectively reducing the cost of manual intervention and improving the accuracy and intelligence level of rule matching.

[0072] In one alternative implementation,

[0073] Historical archived data is acquired and corresponding text and metadata fields are extracted. The text fields are segmented to determine word sequences and text feature vectors are extracted. The metadata fields are structured and parsed to obtain attribute feature vectors, including:

[0074] Historical archived data is read from the archives management system and the data format is identified. Based on the data format, the historical archived data is separated to extract text fields and metadata fields.

[0075] The text field is segmented into sub-words using byte-pair encoding to obtain an initial word sequence. Stop words in the initial word sequence are filtered to obtain a valid word sequence. The valid word sequence is then scanned using a context window, and a word association graph is constructed based on co-occurrence relationships. The word association graph is then subjected to graph convolution propagation to obtain a word embedding representation. The word embedding representation is then concatenated based on the valid word sequence, and positional encoding is introduced to obtain the text feature vector.

[0076] The field values ​​are extracted from the metadata field and the data type of the field values ​​is determined. The field values ​​of the data type are discretized by bucketing to construct a numerical encoding vector. The field values ​​of the data type are mapped by entity embedding to obtain a category encoding vector. The field values ​​of the data type are decomposed by Fourier transform and frequency domain features are extracted to obtain a time encoding vector. The numerical encoding vector and the category encoding vector are combined to perform interactive modeling and weighted aggregation to obtain the attribute feature vector.

[0077] Read historical archived data from the archive management system and identify the data format. The historical archived data in the archive management system usually contains multiple formats, such as electronic document format, database record format, scanned image format, etc. For different data formats, corresponding parsers are used for identification and processing. For electronic document formats such as DOC, PDF, etc., use document parsers to extract text content; for database record formats, read structured data through database interfaces; for scanned image formats, perform optical character recognition and then extract text content. Taking financial archives as an example, it may contain financial statements in PDF format and data tables in XLS format, and identify and process these two formats separately.

[0078] After identifying the data format, perform field separation on the historical archived data based on the characteristics of different formats, and extract text fields and metadata fields. Text fields mainly include text information such as the main body content, title, and abstract of the archive; metadata fields include structured information such as the creation time, modification time, author, department, confidentiality level, file type, etc. of the archive. For example, for a personnel file, the text field may contain content such as the description of the employee's work experience and performance evaluation, while the metadata field contains information such as the employee's entry time, department number, and position level.

[0079] For the extracted text fields, use byte pair encoding technology for subword segmentation to obtain an initial token sequence. Byte pair encoding is an encoding method between the character level and the word level, which can effectively handle rare words and compound words. In specific implementation, build a vocabulary table containing common subword units, such as "finance", "statement", "confidentiality", etc. Then recursively segment the text and decompose the text into the smallest subword sequence. For example, when segmenting "annual financial statement", it may get three subwords: "annual", "finance", and "statement". After obtaining the initial token sequence, filter out the stop words in it, removing common words that contribute little to the semantics of the document, such as "of", "le", "and", etc., so as to obtain an effective token sequence.

[0080] A context window scan is performed on the effective lexical sequence to construct a lexical association graph based on co-occurrence relationships. The context window size is typically set to 5, meaning that each lexical has 5 lexical elements before and after it as its context. During the scan, when two lexical elements appear within the same context window, an edge is created in the association graph, with the edge weight being the co-occurrence frequency. For example, "finance" and "report" often appear adjacently in the text, so the edge between them has a higher weight. After constructing the lexical association graph, it is subjected to graph convolution propagation to calculate the lexical embedding representation. The graph convolution process uses the neighbor node information in the graph structure to update the representation of the central node, and after iterative propagation, a lexical embedding representation incorporating contextual semantics is obtained. The dimension of the lexical embedding representation is typically set to 128 to effectively capture the semantic information of the lexical elements. The lexical embedding representations are concatenated based on the effective lexical sequence, and positional encoding is introduced to represent the relative positions of the lexical elements in the sequence, thereby obtaining the text feature vector.

[0081] For metadata fields, field values ​​are extracted and their data types are determined. Common data types include numeric, categorical, and time types. For numeric field values, such as document page count or attachment count, binning and discretization are performed. The binning is determined based on business characteristics; for example, document page counts are divided into intervals such as 0-10 pages, 11-50 pages, 51-100 pages, and over 100 pages. Each interval is assigned an encoding to form a numeric encoding vector. For categorical field values, such as department or document type, entity embedding technology is used to map them to a low-dimensional space to obtain a category encoding vector. The entity embedding dimension is typically 64, which can effectively represent the semantic relationships between categories. For time-type field values, such as creation time or modification time, Fourier transform decomposition is performed to transform the time series to the frequency domain space, and frequency domain features are extracted to obtain a time encoding vector. Fourier transform can effectively capture the periodic characteristics of time series, such as the annual cycle of annual reports and the quarterly cycle of quarterly reports.

[0082] By combining numerical encoding vectors, categorical encoding vectors, and temporal encoding vectors, interaction modeling is performed and weighted aggregation is conducted to obtain attribute feature vectors. Interaction modeling mainly considers the mutual influence between different metadata fields, such as the association between department and document type, and the relationship between creation time and modification time. An attention mechanism is used to weight and aggregate different features, with important features receiving higher weights. For example, for financial documents, creation time and department may be more important than modification time; while for legal documents, the issuing unit and level of authority may be more critical.

[0083] In this embodiment, by performing data format recognition and field separation processing on historical archived data, the decoupled expression of text information and metadata information is achieved, avoiding information interference caused by mixed processing of different types of data. This improves the standardization of data processing and the targeting of subsequent feature extraction. By introducing byte encoding for sub-word segmentation and combining it with stop word filtering, text noise and lexical sparsity are effectively reduced, and the adaptability to out-of-vocabulary words and complex semantic structures is improved, thereby enhancing the robustness of text representation. By constructing a lexical association graph and using a graph convolution propagation mechanism to obtain lexical embedding representation, joint modeling of contextual relationships between lexical elements and global semantic structure is achieved, significantly improving the completeness and relevance of text semantic expression and enhancing the discriminative ability of feature expression. By introducing positional encoding and performing sequence concatenation of lexical embeddings, the ability of text features to characterize word order information is enhanced, and the level of refined expression of text features is improved. By interactively modeling and weighted aggregation of numerical encoding vectors and category encoding vectors, the ability to model the association between different attribute features is strengthened, and the comprehensive expression ability of overall attribute features and the adaptation effect to downstream tasks are improved.

[0084] In one alternative implementation,

[0085] The sample feature representation is obtained by concatenating the text feature vector with the attribute feature vector, and the maturity prediction mapping relationship is obtained by combining the concatenation of the text feature vector with the attribute feature vector and then solving the mapping relationship with the labeled sample set.

[0086] The statistical dependency between the text feature vector and the attribute feature vector is calculated using information theory methods, and a feature dependency graph is constructed. Feature clusters in the feature dependency graph are identified using a community detection algorithm. The feature dimensions within the feature clusters are scored for importance to obtain a score sequence. Based on the distribution characteristics of the score sequence, a screening threshold is determined, and feature dimensions with importance scores higher than the screening threshold are selected to obtain the sample feature representation.

[0087] Training samples, corresponding sample feature representations, and term labels are generated based on the labeled sample set. The sample feature representations are nonlinearly mapped using a kernel function method to obtain a mapping vector. Based on the mapping vector and the term labels, the mapping coefficients are obtained by kernel regression and the training samples are then used to predict the predicted output.

[0088] The residual vector is obtained by calculating the difference between the predicted output and the term label. The main error components corresponding to the residual vector are extracted by matrix factorization and a residual correction amount is constructed. The residual correction amount is fused with the mapping coefficient to obtain the corrected mapping coefficient. Based on the corrected mapping coefficient, the mapping relationship between the sample feature representation and the term prediction result is determined to obtain the term prediction mapping relationship.

[0089] This paper uses information theory to calculate the statistical dependencies between text feature vectors and attribute feature vectors and constructs a feature dependency graph. In practice, mutual information is used to measure the statistical correlation between different feature dimensions. The higher the mutual information value, the stronger the dependency between the two feature dimensions. The calculation method is to estimate the expected logarithm of the ratio of the joint probability distribution to the marginal probability distribution to obtain the mutual information between the two feature dimensions. For archival features, text features may include keywords in the text and titles, while attribute features may include security classification, department, creation time, etc. For example, the text feature "financial statement" and the attribute feature "finance department" may have a high mutual information value, indicating a strong statistical dependency. By calculating the mutual information between all feature dimension pairs, a feature dependency graph is constructed, where nodes represent feature dimensions, edges represent the dependency strength between features, and the edge weights are the mutual information values.

[0090] After constructing the feature dependency graph, a community detection algorithm is used to identify feature clusters in the graph. The community detection algorithm, based on the graph's topology, groups closely connected nodes into a community or cluster. In this embodiment, a density clustering method is used, the core idea of ​​which is to identify high-density regions as communities and low-density regions as community boundaries. By setting a density threshold, feature dimensions with mutual information values ​​higher than the threshold are connected to form feature clusters. For example, the three attribute features of file type, security level, and creation department may form one feature cluster, while text features such as document title, keywords, and summary may form another feature cluster.

[0091] The importance of feature dimensions within a feature cluster is scored, resulting in a score sequence. The importance score is based on the strength of the association between the feature and the term label, and is evaluated using correlation analysis and information gain methods. Correlation analysis calculates the point cross-correlation coefficient between the feature and the term label, while information gain calculates the information contribution of the feature to the term label. The final importance score is obtained by weighted averaging of the scores from the two methods. For example, for financial records, the feature dimension "document confidentiality level" might receive an importance score of 0.85, while the feature "number of revisions" might only receive a score of 0.32, indicating that the former is more important for predicting retention period.

[0092] Based on the distribution characteristics of the rating sequence, a screening threshold is determined, and feature dimensions with importance ratings higher than the threshold are selected to obtain the sample feature representation. The screening threshold is determined by analyzing the statistical distribution of the rating sequence and using the interquartile range (IMR) method to determine the outlier boundary as the threshold. Specifically, the first quartile, third quartile, and IMR of the rating sequence are calculated, and the threshold is set as the first quartile minus 1.5 times the IMR. For example, for a file sample containing 100 feature dimensions, this method may select the 30 most important feature dimensions, forming a dimensionality-reduced sample feature representation.

[0093] Training samples, along with their corresponding feature representations and expiration labels, are generated based on the labeled sample set. A training sample might be a historical archival record, such as "Financial Statement - 2023 - Finance Department - Confidential." The sample feature representation is a feature vector obtained from the previous selection step, and the expiration label might be "30 years." A kernel function is used to perform a nonlinear mapping on the sample feature representation, resulting in a mapped vector. The kernel function is chosen based on the data characteristics; for the archival retention period prediction task, a radial basis function (RBF) kernel is used, as it effectively handles nonlinear mapping relationships. The kernel function parameters are determined through cross-validation; a commonly used parameter value is a kernel width of 0.1.

[0094] The mapping coefficients are obtained using kernel regression based on the mapping vector and the time limit label, and then used to predict the training samples to obtain the predicted output. Kernel regression solves for the mapping coefficients by minimizing a regularized loss function, which includes a fitting error term and a regularization term. The regularization term adopts an elastic network form, combining the advantages of L1 and L2 regularization, achieving both feature selection and handling correlations between features. The regularization parameter is determined through grid search, with a typical value of 0.01. The obtained mapping coefficients represent the contribution weight of each mapping dimension to the final prediction result.

[0095] The residual vector is obtained by calculating the difference between the predicted output and the deadline label. The residual vector reflects the difference between the model's prediction and the actual label. The main error components corresponding to the residual vector are extracted through matrix factorization, and a residual correction is constructed. Matrix factorization uses singular value decomposition (SVD), decomposing the residual matrix into the product of left singular vectors, singular values, and right singular vectors. By selecting the k largest singular values ​​and their corresponding singular vectors, the main structure of the residual can be captured, where k is typically taken as 20% of the rank of the residual matrix. Based on the decomposition results, a residual correction is constructed, which is expressed as a linear combination of singular vectors.

[0096] The residual correction value is fused with the mapping coefficients to obtain the corrected mapping coefficients. The fusion method uses a weighted combination, with the weights determined based on the variance ratio of the correction value to the original coefficients. The corrected mapping coefficients more accurately reflect the relationship between features and the expiration date label. Based on the corrected mapping coefficients, the mapping relationship between the sample feature representation and the expiration date prediction result is determined, resulting in the expiration date prediction mapping relationship. This mapping relationship is a function; the input is the sample feature representation, and the output is the predicted storage period.

[0097] In this embodiment, by using information theory to characterize the statistical dependency relationship between text feature vectors and attribute feature vectors and constructing a feature dependency graph, a quantitative expression of the correlation between cross-modal features is achieved, which can more comprehensively reflect higher-order dependencies, thereby improving the accuracy and completeness of feature modeling. By introducing a community detection algorithm to structurally partition the feature dependency graph and form feature clusters, the self-organized grouping and structured management of features are realized, effectively enhancing the utilization of the intrinsic correlation between features and improving the rationality and stability of feature selection. By performing importance scoring within feature clusters and adaptively determining the screening threshold based on the score distribution, the automatic screening of key features is realized, significantly reducing the uncertainty caused by human intervention and improving the adaptive ability and generalization performance of feature screening.

[0098] In one alternative implementation,

[0099] Receiving a request for data to be predicted and generating a feature representation of the data to be predicted, and calculating candidate dates and corresponding confidence scores based on the date prediction mapping relationship, including:

[0100] The data to be predicted is separated into a text field to be predicted and a metadata field to be predicted. Lexical analysis is performed on the text field to be predicted to obtain a predicted text vector. Encoding conversion is performed on the metadata field to be predicted to obtain a predicted attribute vector.

[0101] The predicted text vector and the predicted attribute vector are concatenated by tensors to obtain the predicted input vector. The variance contribution rate of each dimension of the predicted input vector is calculated and compared with a preset variance threshold. Dimensions with a variance contribution rate greater than the preset variance threshold are retained and a feature representation to be predicted is constructed.

[0102] The feature to be predicted is input into the term prediction mapping relationship to obtain a term probability vector. The term candidate results are obtained by extracting the term categories with the highest probability values ​​from the term probability vector.

[0103] The Euclidean distance between the feature representation to be predicted and the feature representations of each sample in the labeled sample set is calculated to obtain a set of distance values. Based on the set of distance values ​​and a preset distance threshold, a set of nearest neighbor samples is determined. The frequency of the candidate deadline result in the nearest neighbor sample set is counted and the frequency ratio is calculated to obtain a neighborhood consistency score. The information entropy corresponding to the deadline probability vector is calculated and a prediction certainty score is determined based on the information entropy. The confidence score corresponding to the candidate deadline result is calculated based on the neighborhood consistency score and the prediction certainty score.

[0104] The data to be predicted is separated into text fields and metadata fields. Field separation employs a rule-based and pattern-matching approach, with corresponding field extraction rules designed for different formats of archival data. For structured data, field extraction is performed directly using field names or indexes; for semi-structured data, regular expressions are used to match specific patterns; and for unstructured data, natural language processing techniques are used to identify semantic boundaries. During field separation, a field type mapping table is set up to map the original fields to standardized sets of text fields or metadata fields. Text fields typically include titles, body text, and summaries, while metadata fields include creation dates, departments, and security classifications. The accuracy of field separation directly affects the subsequent feature extraction results. For different types of archives, such as official documents, financial documents, and personnel documents, a customized field extraction rule library is developed to improve separation accuracy.

[0105] Lexical analysis is performed on the text field to be predicted to obtain predicted text vectors. The lexical analysis process includes four stages: text preprocessing, word segmentation, stop word filtering, and feature quantization. Text preprocessing removes special characters, unifies capitalization and formatting; word segmentation uses the maximum matching algorithm combined with a professional dictionary, optimized for archival terminology; stop word filtering removes function words that contribute little to semantics; feature quantization uses the term frequency-inverse document frequency (IF-IF) method to calculate the weight of words in a document. IF reflects the frequency of a word's occurrence in a single document, while IF measures the prevalence of a word in the entire corpus; the two are multiplied to obtain the comprehensive weight. For long texts, a sliding window technique is used to segment the document into fixed-length segments, each segment is processed separately, and then the features are merged. In addition, considering word position information, words in important positions such as titles, first paragraphs, and last paragraphs are assigned higher weights, and the final feature values ​​are adjusted through position weighting coefficients. Word vectorization uses hash encoding technology to map high-dimensional sparse features to a fixed-dimensional dense vector space, reducing computational complexity.

[0106] The metadata fields to be predicted are encoded and transformed to obtain predicted attribute vectors. The encoding transformation employs a differentiated strategy based on the metadata type: numeric fields such as page number and attachment number are transformed to the 0-1 range using min-max normalization; categorical fields such as department and file type use one-hot encoding, with each category corresponding to an independent dimension; ordered categorical fields such as security classification use ordinal encoding to preserve the order relationship between categories; time-based fields such as creation date are decomposed into multiple time features such as year, month, day, quarter, and working day, and periodically transformed into a two-dimensional coordinate representation. For high-cardinality categorical features, such as department codes which may have hundreds of values, feature hashing is used to reduce dimensionality; for hierarchical categories, such as organizational structure, hierarchical encoding is used to preserve structural information. All encoded metadata features are uniformly normalized to ensure that different types of features are within the same numerical range, preventing any single feature from dominating the model due to its large value.

[0107] The predicted input vector is obtained by tensor concatenation of the predicted text vector and the predicted attribute vector. Tensor concatenation connects feature vectors from different sources dimensionally, forming a unified predicted input vector. The original order of the text vector and attribute vector is maintained during concatenation to ensure that the feature correspondence remains unchanged. If the text vector has a dimension of 1000 and the attribute vector has a dimension of 200, the concatenated predicted input vector will have a dimension of 1200. The variance contribution rate of each dimension of the predicted input vector is calculated using principal component analysis. First, the feature covariance matrix is ​​calculated, and then the eigenvalues ​​and eigenvectors are solved. The magnitude of the eigenvalues ​​reflects the variance contribution of the corresponding dimension. The variance contribution rate of each feature dimension is compared with a preset variance threshold, which is typically set between 0.001 and 0.01 and needs to be adjusted according to the actual data distribution. Dimensions with a variance contribution rate greater than the preset variance threshold are retained, thus constructing the feature representation to be predicted.

[0108] The term prediction probability vector is obtained by inputting the feature representation to the term prediction mapping relationship. The term prediction mapping relationship is a function that takes the feature representation as input and outputs the probability distribution of each term category. The mapping process includes two stages: feature transformation and probability calculation. Feature transformation uses a kernel function for non-linear mapping, transforming the original feature space into a high-dimensional feature space to enhance feature expressiveness. Probability calculation is based on the kernel-mapped features, calculating the probability value of each term category by correcting the mapping coefficients. Softmax normalization is used in probability calculation to ensure that the sum of all probability values ​​is 1, forming an effective probability distribution. The term categories with the highest probability values ​​are extracted from the term probability vector to obtain term candidate results. The extraction process employs a dual control strategy of probability threshold and quantity upper limit, selecting the top 3 term categories with probability values ​​exceeding 0.1 as candidate results.

[0109] The Euclidean distance between the feature representation to be predicted and the feature representations of each sample in the labeled sample set is calculated. The Euclidean distance is calculated using the L2 norm of the vector difference, reflecting the linear distance between vectors. To improve computational efficiency, a kd-tree index structure is constructed for the sample set to achieve fast nearest neighbor search. The nearest neighbor sample set is determined based on the distance value set and a preset distance threshold, which is set to twice the standard deviation of the training data. The nearest neighbor sample set contains historical samples with high feature similarity to the sample to be predicted, providing a case-based prediction basis. The frequency of candidate deadlines in the nearest neighbor sample set is counted, and the frequency ratio is calculated to obtain the neighborhood consistency score. The neighborhood consistency score reflects the degree of support for the prediction result in similar cases; it is calculated by dividing the number of times the candidate deadline appears in the nearest neighbor samples by the total number of nearest neighbor samples.

[0110] The information entropy corresponding to the probability vector of the term is calculated, and the prediction certainty score is determined based on the information entropy. The information entropy calculation uses the Shannon entropy formula, treating the probability value as the probability mass of a discrete distribution to calculate the uncertainty of the distribution. A lower entropy value indicates a more certain prediction, while a higher entropy value indicates a more uncertain prediction. The prediction certainty score is defined as the standardized inverse ratio of entropy, calculated by subtracting the actual entropy from the maximum possible entropy and then dividing by the maximum possible entropy. The maximum possible entropy corresponds to a uniform distribution, where all term categories have equal probabilities. The confidence score corresponding to the term candidate results is calculated based on the neighborhood consistency score and the prediction certainty score. The confidence score is calculated using a weighted average method, with a weight of 0.6 for the neighborhood consistency score and 0.4 for the prediction certainty score, reflecting a comprehensive consideration of both case-based and model-based prediction methods. The confidence score ranges from 0 to 1; a higher value indicates a more reliable prediction result and can serve as an important reference for archival management personnel to review prediction results.

[0111] In this embodiment, by separating the fields of the data to be predicted and constructing text vectors and attribute vectors respectively, decoupling and targeted modeling of multi-source heterogeneous information is achieved, effectively reducing feature interference and improving the accuracy and stability of feature representation. By performing tensor-level fusion on the predicted text vector and predicted attribute vector and combining it with variance contribution rate for dimensionality filtering, adaptive compression of high-dimensional features and retention of key information are achieved, effectively removing redundant features and noise information, thereby improving the compactness and discriminative ability of feature representation. By inputting the filtered feature representation into the constructed term prediction mapping relationship, probabilistic term output results are obtained, realizing the transformation from deterministic prediction to probabilistic distribution prediction, enhancing the expressive power and decision support value of the prediction results. By introducing a nearest neighbor sample filtering mechanism based on Euclidean distance, neighborhood consistency analysis is performed on the candidate term results, realizing the verification of prediction results from the perspective of data distribution, effectively improving the reliability and anti-anomaly ability of the prediction results.

[0112] Figure 2This is a flowchart illustrating the rule matching and period correction process of the data mining-based retention period prediction method according to an embodiment of the present invention.

[0113] In one alternative implementation,

[0114] The applicable rules corresponding to the data to be predicted are extracted from the rule set, and the rule matching score is calculated. The revised period result is obtained based on the confidence score and the rule matching score, including:

[0115] The key field set is extracted from the data to be predicted and compared with the trigger conditions of each rule in the rule set to obtain the field matching result. Based on the field matching result, the number of matched fields is counted and the basic matching degree is calculated. The priority of each rule in the rule set is encoded to obtain the priority weight vector and combined with the basic matching degree to calculate the weighted matching degree. The rule with the largest weighted matching degree is extracted as the applicable rule and the rule matching degree score is determined.

[0116] The confidence score and the rule matching score are normalized to obtain the standard confidence score and the standard matching score, and the difference is calculated to obtain the score difference degree. The fusion weight coefficient is determined based on the score difference degree. The predicted contribution is calculated based on the standard confidence score and the fusion weight coefficient. The rule contribution is obtained by multiplying the standard matching score by the complement of the fusion weight coefficient. The comprehensive decision score is obtained by combining the predicted contribution.

[0117] Extract the term value from the term candidate results, extract the rule-specified term value from the applicable rules, calculate the difference between the term value and the rule-specified term value to obtain the term deviation, and calculate the correction deviation by combining the comprehensive decision score. Based on the term value and the correction deviation, solve for the corrected term result.

[0118] The process involves extracting a set of key fields from the data to be predicted and comparing them item by item with the triggering conditions of each rule in the rule set to obtain field matching results. Key field extraction employs pattern recognition technology, with field extraction rules designed specifically for the archival metadata structure. For official documents, key fields include document type, security classification, and issuing department; for financial documents, key fields include accounting subject, business type, and fiscal year; and for personnel documents, key fields include personnel category, job level, and personnel change type. Different strategies are employed for different data formats during extraction: structured data is extracted directly through field mapping; semi-structured data is extracted through regular expression matching; and unstructured data is extracted through semantic analysis. After extraction, the set of key fields is compared with the triggering conditions of each rule in the rule set. The comparison uses a combination of exact matching and fuzzy matching. Exact matching requires completely identical field values, while fuzzy matching supports wildcards, range matching, and semantic similarity calculation. For example, for the document type field, "meeting minutes" and "meeting records" are considered a successful match in fuzzy matching mode.

[0119] The basic matching degree is calculated by counting the number of matched fields based on the field matching results. The basic matching degree reflects the degree of conformity between the rule triggering conditions and the key fields of the archive. It is calculated by dividing the number of matched fields by the total number of fields in the rule triggering conditions. Importance weights are assigned to different fields, with key fields such as security classification and document type having higher weights than secondary fields such as page count and attachment count. The priority of each rule in the rule set is encoded to obtain a priority weight vector. Priority encoding is based on the rule's applicability and business importance, using an exponential decay function to convert the priority number into a weight value; the higher the priority, the greater the weight. For example, a basic rule applicable to all archives has a priority of 1 and a weight of 0.5; a rule applicable to archives of a specific department has a priority of 2 and a weight of 0.25; a rule applicable to a specific document type has a priority of 3 and a weight of 0.125, and so on. A weighted matching degree is calculated by combining the basic matching degree with the priority weight. The rule with the highest weighted matching degree is extracted as the applicable rule, and its matching degree score is determined. The rule matching degree score equals the weighted matching degree of that rule, reflecting the degree of fit between the rule and the archive.

[0120] The confidence score and rule matching score are normalized separately to obtain the standard confidence score and standard matching score. Normalization uses a min-max scaling method to linearly transform the original scores to the interval between 0 and 1. If the original confidence score is 0.78, the minimum confidence score is 0.5, and the maximum confidence score is 0.95, then the standard confidence score is 0.622. The difference between the standard confidence score and the standard matching score is calculated to obtain the score difference. The score difference reflects the consistency between the machine learning prediction result and the rule matching result; the smaller the difference, the more consistent the two methods are. The fusion weight coefficient is determined based on the score difference using a non-linear transformation function of the difference. When the difference is small, the fusion weight coefficient is close to 0.5, indicating equal fusion; when the difference is large, the fusion weight coefficient is biased towards the higher score. The transformation function uses a sigmoid form, with the score difference as input and the weight coefficient between 0 and 1 as output. For example, when the score difference is 0.2, the fusion weight coefficient is 0.55; when the score difference is 0.8, the fusion weight coefficient is 0.85.

[0121] The predicted contribution is calculated based on the standard confidence score and the fusion weight coefficient. The calculation method is to multiply the standard confidence score by the fusion weight coefficient. The predicted contribution reflects the degree of influence of the machine learning prediction result in the final decision. The rule contribution is obtained by multiplying the standard matching score by the complement of the fusion weight coefficient; the complement equals 1 minus the fusion weight coefficient. The rule contribution reflects the degree of influence of the rule matching result in the final decision. The comprehensive decision score is obtained by combining the predicted contribution and the rule contribution. The comprehensive decision score integrates data-driven prediction results with expert knowledge rule judgments, providing a more reliable basis for decision-making.

[0122] Extract the term values ​​from the candidate term results, and extract the rule-specified term values ​​from the applicable rules. Term values ​​are usually expressed in years, such as 10 years, 30 years, etc., and can be permanently stored as a special value of 99. Calculate the difference between the term value and the rule-specified term value to obtain the term deviation. The term deviation reflects the gap between the prediction result and the rule judgment; a positive deviation indicates the predicted term is longer than the rule term, and a negative deviation indicates the predicted term is shorter than the rule term. Combine the comprehensive decision score to calculate the corrected deviation, and multiply the term deviation by the proportion of the rule's contribution to the comprehensive decision score. When the rule's contribution is high, the corrected deviation is close to the original deviation; when the rule's contribution is low, the corrected deviation is close to 0. Based on the term value and the corrected deviation, the corrected term result is obtained by subtracting the corrected deviation from the term value.

[0123] In this embodiment, by comparing key fields in the data to be predicted with triggering conditions in the rule set item by item, and constructing a weighted matching mechanism based on rule priority, the applicability of rules is quantitatively evaluated and the optimal rule is automatically selected, significantly improving the accuracy of rule matching and decision-making efficiency. By normalizing the confidence score and rule matching score obtained from the prediction and constructing a difference-driven fusion weight mechanism, adaptive fusion between data-driven prediction results and rule-driven results is achieved, effectively enhancing the adaptability and decision-making flexibility under different data scenarios. By introducing a collaborative calculation method for prediction contribution and rule contribution, a unified comprehensive decision score is constructed, significantly improving the comprehensiveness and stability of the decision results. By quantitatively analyzing the deviation between the prediction deadline and the rule-specified deadline, and dynamically correcting it based on the comprehensive decision score, refined correction of the prediction results is achieved, effectively reducing systematic bias and improving the accuracy of the results.

[0124] In one alternative implementation,

[0125] Binding the revised deadline result to the source system identifier of the data to be predicted, obtaining the return interface address based on the source system identifier by querying the interface mapping table, and returning the revised deadline result includes:

[0126] Extract the source system identifier from the data to be predicted, associate and bind the correction period result with the source system identifier to obtain binding result data and generate a unique identifier code, combine and encode the unique identifier code with the generation timestamp of the correction period result to obtain a traceable identifier, establish a result traceability record based on the traceable identifier and store the association relationship between the binding result data and the data to be predicted;

[0127] The initial interface address is obtained by querying the interface mapping table based on the source system identifier. The call source information of the data to be predicted is extracted from the result traceability record. The validity of the initial interface address is verified based on the call source information, and it is determined whether the initial interface address is available. If the initial interface address is unavailable, the backup interface address corresponding to the source system identifier is queried from the interface mapping table. The connectivity test is performed on the backup interface address, and the backup interface address that passes the test is used as the return interface address.

[0128] The correction period result and the traceability identifier are encapsulated into a return data packet, and the return data packet is sent to the target system corresponding to the source system identifier through the return interface address and an acknowledgment signal is received.

[0129] Extracting the source system identifier from the data to be predicted. The data to be predicted typically contains metadata information, where the source system identifier is a key field indicating the original source of the archival data. The extraction process employs field mapping technology, locating and extracting the source system identifier from the metadata according to predefined field mapping rules. Corresponding extraction strategies are designed for different formats of data to be predicted: structured data is extracted directly by field name; semi-structured data is extracted by locating and extracting using path expressions; and unstructured data is extracted by text pattern matching. The extracted source system identifier is usually a unique identifier, such as a system code or system name. For example, extracting the source system identifier "OA2023" from the archival metadata indicates that the archive comes from an office automation system.

[0130] The process involves associating the revised retention period with the source system identifier to obtain binding result data and generate a unique identifier. The association and binding operation establishes a mapping relationship between the predicted retention period and the archive source information, forming the binding result data. During the binding process, a key-value pair storage structure is used, with the source system identifier as the key and the revised retention period result as the value. The unique identifier generation employs a multi-segment concatenation technique, combining information such as the timestamp, random number, and source system identifier. A 4-digit random number is generated using the millisecond precision of the current timestamp, and the last 4 digits of the hash value of the source system identifier are extracted and concatenated according to the format "timestamp-random number-hash value" to form the unique identifier. For example, if the timestamp is 1647852369123, the random number is 5824, and the last 4 digits of the hash value of the source system identifier "OA2023" are 7B2C, the generated unique identifier would be "1647852369123-5824-7B2C".

[0131] A traceable identifier is obtained by combining the unique identifier and the timestamp generated by the revised deadline. The combined encoding employs a hierarchical encoding strategy, using the unique identifier as the main body and the timestamp as supplementary information. During the encoding process, the timestamp is converted into a human-readable date and time format, such as "20230315142609" representing March 15, 2023, at 2:26:09 PM. The timestamp and unique identifier are then connected using a separator to form the traceable identifier. For example, based on the aforementioned unique identifier and timestamp, the traceable identifier is "1647852369123-5824-7B2C@20230315142609". The design of the traceable identifier follows the principles of readability and uniqueness, ensuring that the identifier is easy for manual identification while guaranteeing system-level uniqueness.

[0132] A result traceability record is established based on traceability identifiers, storing the association between the bound result data and the data to be predicted. The result traceability record contains multiple fields: traceability identifier, correction deadline result, source system identifier, prediction time, and original request data. The association storage adopts a graph data model, connecting the data nodes to be predicted and the bound result data nodes through a "prediction result" edge. Edge attributes include traceability identifiers and generation time. The relationship storage supports bidirectional queries: querying the corresponding prediction result through the data to be predicted, or tracing back to the original data through the prediction result. Distributed storage technology ensures query efficiency under large-scale data, and indexes are created for key fields to accelerate retrieval. Data version control is implemented during storage, recording data change history and supporting result auditing and backtracking.

[0133] The initial interface address is obtained by querying the interface mapping table based on the source system identifier. The interface mapping table is a configuration table that stores the mapping relationship between the source system identifier and its corresponding interface information. The mapping table structure contains multiple fields: source system identifier, primary interface address, backup interface address, interface type, authentication method, etc. The query process uses key-value lookup, using the source system identifier as the key to retrieve the corresponding interface configuration information.

[0134] The call source information for the data to be predicted is extracted from the result traceability records. This information includes the identity of the request initiator, IP address, and request time. During extraction, the call source information is obtained by parsing the original request data fields from the result traceability records. The validity of the initial interface address is verified based on this information, and its availability is determined. The verification process includes three stages: permission verification, connectivity verification, and load verification. Permission verification confirms whether the call source has permission to access the target interface; connectivity verification checks interface reachability by sending heartbeat packets; and load verification checks whether the target system's current load status allows for new requests. If the initial interface address is unavailable, the backup interface address corresponding to the source system identifier is retrieved from the interface mapping table. Backup interface addresses are typically redundancies or load balancers of the primary interface.

[0135] Connectivity tests are performed on the backup interface addresses, and the backup interface address that passes the test is used as the return interface address. The connectivity test employs a multi-level probing strategy: first, a TCP port probe is performed to confirm the port's open status; then, an HTTPOPTIONS request is sent to verify the interface service's availability; finally, a lightweight test request is sent to verify the interface's functional integrity. Timeout control is set during the test, with a typical timeout threshold of 2000 milliseconds. If multiple backup interfaces pass the test, the optimal interface is selected as the return interface address based on response time and load conditions. If all interfaces are unavailable, an alarm mechanism is triggered and an error log is recorded.

[0136] The revised deadline result and traceability identifier are encapsulated into a return data packet. Data encapsulation uses a standardized format, such as JSON or XML, to ensure compatibility across systems. The return data packet structure consists of a header and a body: the header contains metadata such as timestamp, version number, and signature; the body contains business data such as the revised deadline result and traceability identifier. The data packet design follows a minimalist principle, containing only necessary information to reduce transmission load. The return data packet is sent to the target system corresponding to the source system identifier via the return interface address, and an acknowledgment signal is received. The sending process uses HTTP POST or WebService calls, selecting the appropriate communication protocol based on the interface type. The request includes necessary authentication information, such as tokens and signatures, to ensure communication security. A request retry mechanism is set up; when there is a network error or the target system is temporarily unavailable, retrying according to an exponential backoff strategy, with a maximum of 3 retries. After successfully receiving the acknowledgment signal from the target system, the result traceability record is updated, the transmission status is marked as "completed," and the acknowledgment time and acknowledgment identifier are recorded.

[0137] In this embodiment, by associating and binding the correction deadline result with the source system identifier and generating a unique identifier, and introducing timestamp combinations to form a traceable identifier, a full-link association between the prediction result and the original data source is achieved, significantly improving the traceability and auditability of the result data. By establishing result traceability records based on the traceable identifier and storing the mapping relationship between the data to be predicted and the result data, complete traceability and association management of the data processing process is achieved, effectively enhancing the system's capabilities in problem localization, responsibility tracing, and data reproduction. By introducing an interface mapping table and combining it with the call source information to perform validity verification and status judgment on the interface address, dynamic verification and adaptive selection of the interface call path are achieved, significantly improving the system's stability and adaptability in complex network environments. By automatically switching to the backup interface and performing connectivity testing when the main interface is unavailable, a fault tolerance and redundancy guarantee mechanism at the interface level is achieved, effectively reducing the risk of interface call failure and improving the continuity and reliability of system services.

[0138] A second aspect of the present invention provides a data mining-based retention period prediction system, comprising:

[0139] The sample set construction module is used to acquire classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set.

[0140] The mapping relationship training module is used to acquire historical archived data and extract corresponding text fields and metadata fields. The text fields are segmented to determine word sequences and extract text feature vectors. The metadata fields are structured to obtain attribute feature vectors. The text feature vectors and attribute feature vectors are concatenated to obtain sample feature representations and combined with the labeled sample set to solve for the term prediction mapping relationship.

[0141] The term result calculation module is used to receive the call request for the data to be predicted and generate the feature representation to be predicted corresponding to the data to be predicted, calculate the term candidate result and the corresponding confidence score based on the term prediction mapping relationship, extract the applicable rule corresponding to the data to be predicted from the rule set and calculate the rule matching score, and solve for the corrected term result based on the confidence score and the rule matching score.

[0142] The result return module is used to bind the revised period result with the source system identifier of the data to be predicted, query the interface mapping table based on the source system identifier to obtain the return interface address and return the revised period result.

[0143] A third aspect of the present invention provides an electronic device, comprising:

[0144] A processor and a memory for storing processor-executable instructions, wherein the processor is configured to invoke instructions stored in the memory to perform the aforementioned method.

[0145] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0146] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data mining-based method for predicting retention period, characterized in that, include: Obtain classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set; Historical archived data is acquired and the corresponding text fields and metadata fields are extracted. The text fields are segmented to determine the word sequence and extract the text feature vector. The metadata fields are structured and parsed to obtain the attribute feature vector. The text feature vector and the attribute feature vector are concatenated to obtain the sample feature representation and combined with the labeled sample set to solve for the term prediction mapping relationship. The system receives a request to predict data and generates a feature representation corresponding to the data to be predicted. Based on the time prediction mapping relationship, it calculates the time candidate results and the corresponding confidence scores. It extracts the applicable rules corresponding to the data to be predicted from the rule set and calculates the rule matching score. Based on the confidence score and the rule matching score, it solves for the corrected time result. The revised deadline result is bound to the source system identifier of the data to be predicted. The interface mapping table is queried based on the source system identifier to obtain the return interface address and the revised deadline result is returned.

2. The method according to claim 1, characterized in that, Obtain classification scheme data and storage period table data; parse the classification scheme data to obtain a category tree structure; extract a rule set based on the storage period table data; and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set, including: The classification scheme configuration file is read from the file management system and parsed to obtain classification scheme data; the retention period configuration file is read from the file management system and parsed to obtain retention period table data. Extract the category identifier and parent-child relationship of each level node from the classification scheme data, construct hierarchical connections between nodes based on the parent-child relationship and determine the depth value of each node, construct a tree structure based on the category identifier, combined with the hierarchical connections and the depth value, obtain the category tree structure, and assign corresponding node identifiers to the tree nodes in the category tree structure; The storage period table data is used to group the period rules according to the rule type and extract the condition fields. The condition fields and condition values ​​are combined into rule expressions and standardized to obtain the rule set. Extract the node identifiers corresponding to the tree nodes in the category tree structure, extract the applicable conditions corresponding to each rule expression from the rule set, perform matching calculations on the category restriction fields in the applicable conditions and the node identifiers to obtain a matching metric value, establish an association relationship between the node identifiers and the rule expressions based on the matching metric value, and combine the node identifiers in the association relationship with the term labels corresponding to the rule expressions to obtain the labeled sample set.

3. The method according to claim 1, characterized in that, The process involves acquiring historical archived data and extracting corresponding text and metadata fields. The text fields are then segmented to determine word sequences and text feature vectors are extracted. The metadata fields are then parsed in a structured manner to obtain attribute feature vectors, including: Historical archived data is read from the archives management system and the data format is identified. Based on the data format, the historical archived data is separated to extract text fields and metadata fields. The text field is segmented into sub-words using byte-pair encoding to obtain an initial word sequence. Stop words in the initial word sequence are filtered to obtain a valid word sequence. The valid word sequence is then scanned using a context window, and a word association graph is constructed based on co-occurrence relationships. The word association graph is then subjected to graph convolution propagation to obtain a word embedding representation. The word embedding representation is then concatenated based on the valid word sequence, and positional encoding is introduced to obtain the text feature vector. The field values ​​are extracted from the metadata field and the data type of the field values ​​is determined. The field values ​​of the data type are discretized by bucketing to construct a numerical encoding vector. The field values ​​of the data type are mapped by entity embedding to obtain a category encoding vector. The field values ​​of the data type are decomposed by Fourier transform and frequency domain features are extracted to obtain a time encoding vector. The numerical encoding vector and the category encoding vector are combined to perform interactive modeling and weighted aggregation to obtain the attribute feature vector.

4. The method according to claim 1, characterized in that, The sample feature representation is obtained by concatenating the text feature vector with the attribute feature vector, and the maturity prediction mapping relationship is obtained by combining the concatenation of the text feature vector with the attribute feature vector and then solving the mapping relationship with the labeled sample set. The statistical dependency between the text feature vector and the attribute feature vector is calculated using information theory methods, and a feature dependency graph is constructed. Feature clusters in the feature dependency graph are identified using a community detection algorithm. The feature dimensions within the feature clusters are scored for importance to obtain a score sequence. Based on the distribution characteristics of the score sequence, a screening threshold is determined, and feature dimensions with importance scores higher than the screening threshold are selected to obtain the sample feature representation. Training samples, corresponding sample feature representations, and term labels are generated based on the labeled sample set. The sample feature representations are nonlinearly mapped using a kernel function method to obtain a mapping vector. Based on the mapping vector and the term labels, the mapping coefficients are obtained by kernel regression and the training samples are then used to predict the predicted output. The residual vector is obtained by calculating the difference between the predicted output and the term label. The main error components corresponding to the residual vector are extracted by matrix factorization and a residual correction amount is constructed. The residual correction amount is fused with the mapping coefficient to obtain the corrected mapping coefficient. Based on the corrected mapping coefficient, the mapping relationship between the sample feature representation and the term prediction result is determined to obtain the term prediction mapping relationship.

5. The method according to claim 1, characterized in that, Receiving a request for data to be predicted and generating a feature representation of the data to be predicted, and calculating candidate dates and corresponding confidence scores based on the date prediction mapping relationship, including: The data to be predicted is separated into a text field to be predicted and a metadata field to be predicted. Lexical analysis is performed on the text field to be predicted to obtain a predicted text vector. Encoding conversion is performed on the metadata field to be predicted to obtain a predicted attribute vector. The predicted text vector and the predicted attribute vector are concatenated by tensors to obtain the predicted input vector. The variance contribution rate of each dimension of the predicted input vector is calculated and compared with a preset variance threshold. Dimensions with a variance contribution rate greater than the preset variance threshold are retained and a feature representation to be predicted is constructed. The feature to be predicted is input into the term prediction mapping relationship to obtain a term probability vector. The term candidate results are obtained by extracting the term categories with the highest probability values ​​from the term probability vector. The Euclidean distance between the feature representation to be predicted and the feature representations of each sample in the labeled sample set is calculated to obtain a set of distance values. Based on the set of distance values ​​and a preset distance threshold, a set of nearest neighbor samples is determined. The frequency of the candidate deadline result in the nearest neighbor sample set is counted and the frequency ratio is calculated to obtain a neighborhood consistency score. The information entropy corresponding to the deadline probability vector is calculated and a prediction certainty score is determined based on the information entropy. The confidence score corresponding to the candidate deadline result is calculated based on the neighborhood consistency score and the prediction certainty score.

6. The method according to claim 1, characterized in that, The applicable rules corresponding to the data to be predicted are extracted from the rule set, and the rule matching score is calculated. The revised period result is obtained based on the confidence score and the rule matching score, including: The key field set is extracted from the data to be predicted and compared with the trigger conditions of each rule in the rule set to obtain the field matching result. Based on the field matching result, the number of matched fields is counted and the basic matching degree is calculated. The priority of each rule in the rule set is encoded to obtain the priority weight vector and combined with the basic matching degree to calculate the weighted matching degree. The rule with the largest weighted matching degree is extracted as the applicable rule and the rule matching degree score is determined. The confidence score and the rule matching score are normalized to obtain the standard confidence score and the standard matching score, and the difference is calculated to obtain the score difference degree. The fusion weight coefficient is determined based on the score difference degree. The predicted contribution is calculated based on the standard confidence score and the fusion weight coefficient. The rule contribution is obtained by multiplying the standard matching score by the complement of the fusion weight coefficient. The comprehensive decision score is obtained by combining the predicted contribution. Extract the term value from the term candidate results, extract the rule-specified term value from the applicable rules, calculate the difference between the term value and the rule-specified term value to obtain the term deviation, and calculate the correction deviation by combining the comprehensive decision score. Based on the term value and the correction deviation, solve for the corrected term result.

7. The method according to claim 1, characterized in that, Binding the revised deadline result to the source system identifier of the data to be predicted, obtaining the return interface address based on the source system identifier by querying the interface mapping table, and returning the revised deadline result includes: Extract the source system identifier from the data to be predicted, associate and bind the correction period result with the source system identifier to obtain binding result data and generate a unique identifier code, combine and encode the unique identifier code with the generation timestamp of the correction period result to obtain a traceable identifier, establish a result traceability record based on the traceable identifier and store the association relationship between the binding result data and the data to be predicted; The initial interface address is obtained by querying the interface mapping table based on the source system identifier. The call source information of the data to be predicted is extracted from the result traceability record. The validity of the initial interface address is verified based on the call source information, and it is determined whether the initial interface address is available. If the initial interface address is unavailable, the backup interface address corresponding to the source system identifier is queried from the interface mapping table. The connectivity test is performed on the backup interface address, and the backup interface address that passes the test is used as the return interface address. The correction period result and the traceability identifier are encapsulated into a return data packet, and the return data packet is sent to the target system corresponding to the source system identifier through the return interface address and an acknowledgment signal is received.

8. A data mining-based retention period prediction system, used to implement the method of any one of claims 1-7, characterized in that, include: The sample set construction module is used to acquire classification scheme data and storage period table data, parse the classification scheme data to obtain a category tree structure, extract a rule set based on the storage period table data, and associate and map the node identifiers in the category tree structure with the applicable conditions in the rule set to obtain a labeled sample set. The mapping relationship training module is used to acquire historical archived data and extract corresponding text fields and metadata fields. The text fields are segmented to determine word sequences and extract text feature vectors. The metadata fields are structured to obtain attribute feature vectors. The text feature vectors and attribute feature vectors are concatenated to obtain sample feature representations and combined with the labeled sample set to solve for the term prediction mapping relationship. The term result calculation module is used to receive the call request for the data to be predicted and generate the feature representation to be predicted corresponding to the data to be predicted, calculate the term candidate result and the corresponding confidence score based on the term prediction mapping relationship, extract the applicable rule corresponding to the data to be predicted from the rule set and calculate the rule matching score, and solve for the corrected term result based on the confidence score and the rule matching score. The result return module is used to bind the revised period result with the source system identifier of the data to be predicted, query the interface mapping table based on the source system identifier to obtain the return interface address and return the revised period result.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images