A text classification method and system of clustering classification fusion
By using a clustering and classification fusion method to dynamically adjust feature weights and classification thresholds, the problems of low accuracy and insufficient adaptability in document classification are solved, achieving efficient processing of multi-format documents and improving system stability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2026-01-21
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technologies for document classification suffer from low classification accuracy, insufficient adaptive optimization capabilities, poor performance when handling multi-format documents, and difficulties in deployment and maintenance, especially lacking a deep collaborative mechanism for combining rule-based classification and cluster analysis.
By employing a clustering and classification fusion approach, and through dynamic weight calculation, a multimodal clustering framework, and a multi-evidence fusion decision framework, combined with filename, title, content, and regional location features, the feature weights and classification thresholds are dynamically adjusted to achieve multi-dimensional feature representation and adaptive optimization of documents.
It significantly improves the accuracy of document classification and the system's adaptability, reduces maintenance costs and difficulty, can handle multi-format documents, and provides intuitive explanations of decision-making processes.
Smart Images
Figure CN122153064A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of text classification technology, and in particular to a text classification method and system that integrates clustering and classification. Background Technology
[0002] In today's digital age, businesses and organizations face the challenge of managing massive amounts of electronic documents. Every day, tens of thousands of PDF documents, scanned images, and electronic files need to be accurately categorized and effectively managed. These documents may originate from different business departments, have varying formatting standards and content characteristics, and traditional document management methods are no longer sufficient to meet the demands of modern management.
[0003] In recent years, deep learning technology has demonstrated tremendous potential in document classification. Models such as convolutional neural networks and recurrent neural networks can automatically learn feature representations of documents, reducing reliance on manual feature engineering. In particular, the emergence of pre-trained language models, such as BERT and the GPT series, has greatly improved the accuracy of text classification. These models, through pre-training on large-scale corpora, acquire rich linguistic knowledge and are able to better understand the semantic content of documents.
[0004] Current technologies have significant shortcomings in feature utilization. Most systems over-rely on text content features while neglecting other important document attributes. Secondly, the combination of rule-based classification and cluster analysis is rigid and inefficient, simply chaining or paralleling the two methods without a deep collaborative mechanism. Adaptive optimization capabilities are severely lacking; most existing technologies maintain fixed parameter configurations after deployment, unable to self-adjust and optimize based on actual operating conditions. This is particularly true for key parameters such as feature weight settings and classification threshold selection, which often rely on expert experience and lack a scientific data-driven optimization mechanism. Performance is poor when handling multi-format documents. Different document formats (such as native PDFs, scanned images, and photos) require different preprocessing methods, but existing technologies often use a uniform processing flow, failing to optimize for the characteristics of different formats. Furthermore, practical deployment and maintenance present significant difficulties, making it hard to meet the real-time requirements of enterprise applications. Summary of the Invention
[0005] This application provides a text classification method and system that integrates clustering and classification, which can solve the problem of low classification accuracy caused by the lack of clustering analysis in classification systems.
[0006] To achieve the above objectives, according to a first aspect of this application, a text classification method based on clustering and classification fusion is provided, the method comprising the following steps:
[0007] S1. Preprocess and extract features from the input file to obtain multi-dimensional features; S2. Use a dynamic weighting mechanism to classify documents based on multi-dimensional features to obtain the original classification results; S3. Multi-dimensional features are processed through a multimodal clustering framework to obtain clustering results; S4. Based on the multi-evidence fusion and confidence propagation decision framework, the original classification results and clustering results are processed to obtain the final classification result.
[0008] Optionally, S1 further includes the following steps: After the input file is processed by OCR, filename pattern features, title keyword features, content keyword features, and regional location features are extracted. For a given document, filename features, title features, content features, and regional features are extracted. The above features are combined to construct multi-dimensional features.
[0009] Furthermore, the dynamic weight calculation mechanism includes the following steps: Based on the classification rules, the available feature types in the multi-dimensional features are detected, and the sum of the basic weights is calculated and then normalized to obtain the multi-dimensional feature combination. The following processing is performed on the combination of multi-dimensional features: A filename matching strategy is adopted, which uses multi-level similarity calculation. If the filename pattern features completely contain the filename features, the filename matching score is set to full score. If the filename pattern features partially contain the filename features, the filename matching score is calculated based on the length of the longest common substring and the character matching ratio. The title matching score is determined based on the length of the common prefix between the title keyword features and the title features; A keyword semantic matching mechanism is adopted. The semantic embedding and cosine similarity calculation are performed through the Sentence-BERT model to obtain the keyword semantic embedding. The content keyword features and content features are compared, and the semantic matching degree is calculated through cosine similarity to obtain the keyword matching score based on semantic similarity. A precise regional location matching mechanism is adopted, and the intersection-union ratio (IUGR) of regional location features is calculated based on OCR coordinate information. A high score threshold for regional features is set. When the IUGR exceeds the high score threshold, the IUGR score is set to full score; otherwise, the IUGR of the regional location features is used as the IUGR score. The combined score, matching tag, and confidence level are obtained by combining the file name matching score, title matching score, keyword matching score, and intersection-union ratio (IU) score, and then using dynamic weighting for weighted fusion.
[0010] Optionally, the dynamic weight calculation mechanism further includes the following steps: The matching tags are sorted from highest to lowest confidence level; Keyword features are extracted from the title, and document titles are identified based on OCR structured data. An abnormal filename detection mechanism is employed, which identifies abnormal filenames based on character distribution features, calculates the ratio of numbers to letters in abnormal filenames, and compares it with an anomaly threshold. In comparison, when the ratio of numbers to letters exceeds the anomaly threshold... When the comparison is complete, reset the file name title weight to zero; after the comparison is complete, output the original classification results. .
[0011] Furthermore, the multimodal clustering framework includes the following steps: The TF-IDF vectorization technique is used to extract title features from the multi-dimensional features using the following formula. Text content features and regional location characteristics :
[0012]
[0013]
[0014] in, For normalization function, For filename, Keywords for the entire document content To return a text block Content, For text blocks, The set of all text blocks; Set cluster weights to include title features Text content features and regional location characteristics Concatenating the clustering weights yields an enhanced document representation:
[0015]
[0016] in, For title feature weights, Weights for text content features. For regional location feature weights, For text length features, Weights for text length features; During training, the optimal weight combination is found by optimizing the objective function:
[0017] in, For weighted combinations, For balance coefficient, To adjust the RAND Corporation index, The contour coefficient is used; the enhanced multidimensional features are finally obtained based on the optimal weight combination.
[0018] Optionally, the multimodal clustering framework further includes the following steps: A multi-dimensional representation space is constructed based on the enhanced multi-dimensional features, wherein the first dimension of the multi-dimensional representation space is... One document for:
[0019] Will After combining the documents, the output is the enhanced multi-dimensional feature combination. ; Calculate the intra-cluster compactness measure:
[0020] in, For document The average distance to other documents in the same cluster. For document Average distance to the nearest other document cluster; The model parameter set obtained through training is adopted using a model persistence mechanism. It is serialized and saved, and then directly loaded and used during the inference phase; among which, For TF-IDF vectorization, For clustering models, The optimized weights; model parameter set The clustering results are output using the k-means clustering algorithm. .
[0021] Furthermore, the multi-evidence fusion and confidence propagation decision framework includes the following steps: Original classification results and clustering results A high-confidence direct assignment strategy is adopted for classifications with confidence levels exceeding a confidence threshold. The document's category is directly determined:
[0022] in, This is a high-confidence document. For the first One document Confidence level; Each high-confidence document is assigned a corresponding category label. ; To handle documents with similar overall classification scores, define a conflict detection function:
[0023]
[0024] in For collision detection functions, Threshold for conflict judgment; For conflicting documents Confidence calibration is performed, among which A collection of conflicting documents; Calculate the cluster Category distribution:
[0025] in, It is a counting function; The conflict resolution decision function is defined as follows:
[0026] For unassigned documents, identify the set of unassigned documents:
[0027] Identify the set of unassigned categories:
[0028] Set a greedy allocation strategy to select the most suitable document for each unassigned category based on the matching scoring function:
[0029] in, For document Cluster category The level of support.
[0030] Optionally, the multi-evidence fusion and confidence propagation decision framework further includes the following steps: Propagate category labels to the entire cluster for assigned documents. and its categories Maintaining semantic consistency within a cluster through a cluster propagation function:
[0031] Set the allocation strategy for the cluster-dominant category to allocate the remaining unallocated documents, selecting the category that appears most frequently among the already allocated documents in the same cluster. As the category of this document:
[0032] For completely isolated documents, assign the "uncategorized" label.
[0033] Optionally, the multi-evidence fusion and confidence propagation decision framework further includes the following steps: Based on the original classification results and clustering results, a confidence assessment is performed, and the overall confidence score is calculated.
[0034]
[0035] in, Documents classified by dynamic weighting mechanism Assign support to the current category to documents in the same cluster. For classification confidence weighting coefficients; The output classification results include the following fine-grained evaluation metrics:
[0036]
[0037]
[0038]
[0039] in, For overall accuracy, This represents the number of samples that are actually positive and are predicted as positive by the model. This represents the number of samples that were actually classified as negative but were predicted as positive by the model. For accuracy, For recall rate, This represents the number of samples that were actually positive but were predicted as negative by the model. For F1 score, For true category labels.
[0040] To achieve the above objectives, according to a second aspect of this application, a text classification system based on clustering, classification, and fusion is also provided. The system executes the text classification method based on clustering, classification, and fusion described in the first aspect. The system includes a file input extraction module, a classification module, a clustering module, and a fusion module. The file input extraction module receives an input file and outputs multi-dimensional features. The classification module receives the multi-dimensional features and outputs the original classification result. The clustering module receives the multi-dimensional features and outputs the clustering result. The fusion module receives the original classification result and the clustering result and outputs the classification result.
[0041] In this application, a deep collaborative mechanism is constructed between a dynamic weight calculation mechanism and a multimodal clustering framework. The clustering results can serve as auxiliary constraints or prior information to correct the discrimination boundary of the dynamic weight calculation mechanism. The high-confidence results output by the dynamic weight calculation mechanism can in turn guide the updating and adjustment of the multimodal clustering framework. Through the dynamic feedback mechanism, the dynamic weight calculation mechanism and the multimodal clustering framework mutually verify and optimize each other, effectively reducing the probability of misclassification and misclustering, and significantly improving the overall classification accuracy and the system's adaptability in complex scenarios. Multi-dimensional features are adopted, and the weights of each feature are dynamically adjusted through an intelligent weight optimization algorithm, avoiding the limitations of expression caused by a single feature dimension, making the document representation more comprehensive and accurate. This application automatically adjusts feature weights and classification thresholds, and performs adaptive parameter updates in real time or periodically according to the system's operating status and data changes, so that the system always maintains a near-optimal performance state. This significantly reduces the reliance on manual experience-based parameter tuning and frequent maintenance, which not only improves the stability and robustness of the system operation, but also greatly reduces the maintenance costs and management difficulty of the system in long-term deployment and practical application. Attached Figure Description
[0042] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0043] Figure 1 This is a flowchart of a text classification method based on clustering and classification fusion according to an embodiment of this application; Figure 2 This is a structural diagram of a text classification system based on clustering and classification fusion provided in an embodiment of this application. Detailed Implementation
[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0045] Example 1 like Figure 1 As shown, this embodiment provides a text classification method that combines clustering and classification, the method comprising the following steps: S1. Preprocess and extract features from the input file to obtain multi-dimensional features; After the input file is processed by OCR, filename pattern features, title keyword features, content keyword features, and regional location features are extracted. For a given document, filename features, title features, content features, and regional features are extracted. The above features are combined to construct multi-dimensional features.
[0046] S2. Use a dynamic weighting mechanism to classify documents based on multi-dimensional features to obtain the original classification results.
[0047] S3. Multi-dimensional features are processed through a multimodal clustering framework to obtain clustering results.
[0048] S4. Based on the multi-evidence fusion and confidence propagation decision framework, the original classification results and clustering results are processed to obtain the final classification result.
[0049] Example 2 This embodiment is based on Embodiment 1.
[0050] The dynamic weight calculation mechanism includes the following steps: Based on the classification rules, the available feature types in the multi-dimensional features are detected, and the sum of the basic weights is calculated and then normalized to obtain the multi-dimensional feature combination. The following processing is performed on the combination of multi-dimensional features: A filename matching strategy is adopted, which uses multi-level similarity calculation. If the filename pattern features completely contain the filename features, the filename matching score is set to full score. If the filename pattern features partially contain the filename features, the filename matching score is calculated based on the length of the longest common substring and the character matching ratio. The title matching score is determined based on the length of the common prefix between the title keyword features and the title features; A keyword semantic matching mechanism is adopted. The semantic embedding and cosine similarity calculation are performed through the Sentence-BERT model to obtain the keyword semantic embedding. The content keyword features and content features are compared, and the semantic matching degree is calculated through cosine similarity to obtain the keyword matching score based on semantic similarity. A precise regional location matching mechanism is adopted, and the intersection-union ratio (IUGR) of regional location features is calculated based on OCR coordinate information. A high score threshold for regional features is set. When the IUGR exceeds the high score threshold, the IUGR score is set to full score; otherwise, the IUGR of the regional location features is used as the IUGR score. The combined score, matching tag, and confidence level are obtained by combining the file name matching score, title matching score, keyword matching score, and intersection-union ratio (IU) score, and then using dynamic weighting for weighted fusion.
[0051] The matching tags are sorted from highest to lowest confidence level; Keyword features are extracted from the title, and document titles are identified based on OCR structured data. An abnormal filename detection mechanism is employed, which identifies abnormal filenames based on character distribution features, calculates the ratio of numbers to letters in abnormal filenames, and compares it with an anomaly threshold. In comparison, when the ratio of numbers to letters exceeds the anomaly threshold... When the comparison is complete, reset the file name title weight to zero; after the comparison is complete, output the original classification results. .
[0052] Example 3 The multimodal clustering framework includes the following steps: The TF-IDF vectorization technique is used to extract title features from the multi-dimensional features using the following formula. Text content features and regional location characteristics :
[0053]
[0054]
[0055] in, For normalization function, For filename, Keywords for the entire document content To return a text block Content, For text blocks, The set of all text blocks; Set cluster weights to include title features Text content features and regional location characteristics Concatenating the clustering weights yields an enhanced document representation:
[0056]
[0057] in, For title feature weights, Weights for text content features. For regional location feature weights, For text length features, Weights for text length features; During training, the optimal weight combination is found by optimizing the objective function:
[0058] in, For weighted combinations, For balance coefficient, To adjust the RAND Corporation index, The contour coefficient is used; the enhanced multidimensional features are finally obtained based on the optimal weight combination.
[0059] A multi-dimensional representation space is constructed based on the enhanced multi-dimensional features, wherein the first dimension of the multi-dimensional representation space is... One document for:
[0060] Will After combining the documents, the output is the enhanced multi-dimensional feature combination. ; Calculate the intra-cluster compactness measure:
[0061] in, For document The average distance to other documents in the same cluster. For document Average distance to the nearest other document cluster; The model parameter set obtained through training is adopted using a model persistence mechanism. It is serialized and saved, and then directly loaded and used during the inference phase; among which, For TF-IDF vectorization, For clustering models, The optimized weights; model parameter set The clustering results are output using the k-means clustering algorithm. .
[0062] Example 4 The multi-evidence fusion and confidence propagation decision framework includes the following steps: Original classification results and clustering results A high-confidence direct assignment strategy is adopted for classifications with confidence levels exceeding a confidence threshold. The document's category is directly determined:
[0063] in, This is a high-confidence document. For the first One document Confidence level; Each high-confidence document is assigned a corresponding category label. ; To handle documents with similar overall classification scores, define a conflict detection function:
[0064]
[0065] in For collision detection functions, Threshold for conflict judgment; For conflicting documents Confidence calibration is performed, among which A collection of conflicting documents; Calculate the cluster Category distribution:
[0066] in, It is a counting function; The conflict resolution decision function is defined as follows:
[0067] For unassigned documents, identify the set of unassigned documents:
[0068] Identify the set of unassigned categories:
[0069] Set a greedy allocation strategy to select the most suitable document for each unassigned category based on the matching scoring function:
[0070] in, For document Cluster category The level of support.
[0071] Propagate category labels to the entire cluster for assigned documents. and its categories Maintaining semantic consistency within a cluster through a cluster propagation function:
[0072] Set the allocation strategy for the cluster-dominant category to allocate the remaining unallocated documents, selecting the category that appears most frequently among the already allocated documents in the same cluster. As the category of this document:
[0073] For completely isolated documents, assign the "uncategorized" label.
[0074] Based on the original classification results and clustering results, a confidence assessment is performed, and the overall confidence score is calculated.
[0075]
[0076] in, Documents classified by dynamic weighting mechanism Assign support to the current category to documents in the same cluster. For classification confidence weighting coefficients; The output classification results include the following fine-grained evaluation metrics:
[0077]
[0078]
[0079]
[0080] in, For overall accuracy, This represents the number of samples that are actually positive and are predicted as positive by the model. This represents the number of samples that were actually classified as negative but were predicted as positive by the model. For accuracy, For recall rate, This represents the number of samples that were actually positive but were predicted as negative by the model. For F1 score, For true category labels.
[0081] Example 5 like Figure 2 As shown, this embodiment provides a text classification system that integrates clustering, classification, and fusion. The system executes the text classification method integrating clustering, classification, and fusion described in embodiments 1-4. The system includes a file input extraction module, a classification module, a clustering module, and a fusion module. The file input extraction module receives input files and outputs multi-dimensional features. The classification module receives multi-dimensional features and outputs the original classification result. The clustering module receives multi-dimensional features and outputs the clustering result. The fusion module receives the original classification result and the clustering result and outputs the classification result.
[0082] Example 6 This embodiment provides five categories: "Registration Form," "Investigation Report," "Service Commitment," "Publicly Circulated Official Letter," and "Summary of Suggestions, Complaints, and Objections," totaling 98 original documents as overall data. For each category, two documents are randomly selected as training data, with 3-5 categories enabled each time. For the enabled categories, up to 10 original documents are randomly selected as test data. The enabled categories must include "Publicly Circulated Official Letter." The data selection in this embodiment uses data lacking obvious clustering, which puts greater pressure on the clustering part.
[0083] For different numbers of categories, this embodiment will conduct 5 experiments to test the accuracy results. The test results of this embodiment are as follows: The first experiment used 5 categories, with a total of 46 original files, and the accuracy rates were 91.30%, 91.30%, 95.65%, 91.30%, and 95.65%, respectively; The second experiment used 4 categories, with "registration form" hidden, with a total of 40 original files, and the accuracy rates were 95.00%, 97.50%, 92.50%, 100.00%, and 97.50%, respectively; The third experiment used 3 categories, with "registration form" and "service commitment" hidden, with a total of 30 original files, and the accuracy rates were 93.33%, 86.67%, 93.33%, 86.67%, and 93.33%, respectively.
[0084] This application has the following advantages or beneficial effects: At the technical architecture level, this application breaks through the limitations of traditional single technical routes and innovatively realizes deep collaboration between rule classification and cluster analysis; through a dynamic feedback mechanism, the two are mutually verified and optimized. This collaborative mechanism not only improves the classification accuracy, but also enhances the system's adaptability. In terms of feature engineering, this application achieves true multi-dimensional feature fusion: compared with the existing technology that mainly relies on the single feature pattern of text content, this application simultaneously mines multi-source features such as file name semantics, layout structure, coordinate position, and document length, and dynamically adjusts the importance of each feature through intelligent weight optimization algorithm, so as to make the document representation more comprehensive and accurate. In terms of adaptive optimization capability, this application has a significant advantage: the parameter configuration of traditional systems often relies on human experience, while this application, through data-driven optimization algorithms, can automatically adjust feature weights and classification thresholds according to actual operating results, so that the system always maintains the best performance state. This self-evolution capability greatly reduces the maintenance cost of the system. When dealing with complex document scenarios, this application demonstrates stronger robustness: whether it is a native PDF, scanned image or photo document, the system can achieve ideal processing results through an adaptive preprocessing process. In particular, in the OCR text extraction and layout analysis stages, the accuracy of information extraction is significantly improved through multi-strategy fusion technology. This application also represents a significant breakthrough in engineering practicality: the system adopts a modular design, which has good scalability and maintainability; through algorithm optimization, the processing efficiency is greatly improved while ensuring accuracy; and it provides intuitive and interpretable output to help users understand the classification decision-making process. These improvements make this application not only technologically advanced, but also easier to deploy and apply in real-world environments.
[0085] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various steps of the aforementioned text classification method and system embodiments of clustering and classification fusion, and can achieve the same beneficial effects as the aforementioned text classification method and system embodiments of clustering and classification fusion. To avoid repetition, they will not be described again here.
[0086] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.
[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.
[0088] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute some steps of the transmission and reception methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0089] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principles described in this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A text classification method that combines clustering and classification, characterized in that, The method includes the following steps: S1. Preprocess and extract features from the input file to obtain multi-dimensional features; S2. Use a dynamic weighting mechanism to classify documents based on multi-dimensional features to obtain the original classification results; S3. Multi-dimensional features are processed through a multimodal clustering framework to obtain clustering results; S4. Based on the multi-evidence fusion and confidence propagation decision framework, the original classification results and clustering results are processed to obtain the final classification result.
2. The text classification method based on clustering and classification fusion according to claim 1, characterized in that, S1 further includes the following steps: After the input file is processed by OCR, filename pattern features, title keyword features, content keyword features, and regional location features are extracted. For a given document, filename features, title features, content features, and regional features are extracted. The above features are combined to construct multi-dimensional features.
3. The text classification method based on clustering and classification fusion according to claim 1, characterized in that, The dynamic weight calculation mechanism includes the following steps: Based on the classification rules, the available feature types in the multi-dimensional features are detected, and the sum of the basic weights is calculated and then normalized to obtain the multi-dimensional feature combination. The following processing is performed on the combination of multi-dimensional features: A filename matching strategy is adopted, which uses multi-level similarity calculation. If the filename pattern features completely contain the filename features, the filename matching score is set to full score. If the filename pattern features partially contain the filename features, the filename matching score is calculated based on the length of the longest common substring and the character matching ratio. The title matching score is determined based on the length of the common prefix between the title keyword features and the title features; A keyword semantic matching mechanism is adopted. The semantic embedding and cosine similarity calculation are performed through the Sentence-BERT model to obtain the keyword semantic embedding. The content keyword features and content features are compared, and the semantic matching degree is calculated through cosine similarity to obtain the keyword matching score based on semantic similarity. A precise regional location matching mechanism is adopted to calculate the intersection-union ratio of regional location features based on OCR coordinate information; Set a high score threshold for regional features. When the cross-union ratio (CUN) exceeds the high score threshold for regional features, set the CUN calculation score to full score. Otherwise, use the CUN of the regional location features as the CUN calculation score. The combined score, matching tag, and confidence level are obtained by combining the file name matching score, title matching score, keyword matching score, and intersection-union ratio (IU) score, and then using dynamic weighting for weighted fusion.
4. The text classification method based on clustering and classification fusion according to claim 3, characterized in that, The dynamic weight calculation mechanism also includes the following steps: The matching tags are sorted from highest to lowest confidence level; Extract keyword features from the title and identify the document title based on OCR structured data; An abnormal filename detection mechanism is employed, which identifies abnormal filenames based on character distribution features, calculates the ratio of numbers to letters in abnormal filenames, and compares it with an abnormality threshold. In comparison, when the ratio of numbers to letters exceeds the anomaly threshold... When the comparison is complete, reset the file name title weight to zero; after the comparison is complete, output the original classification results. .
5. The text classification method based on clustering and classification fusion according to claim 1, characterized in that, The multimodal clustering framework includes the following steps: The TF-IDF vectorization technique is used to extract title features from the multi-dimensional features using the following formula. Text content features and regional location characteristics : in, For normalization function, For filename, Keywords for the entire document content To return a text block Content, For text blocks, The set of all text blocks; Set cluster weights to include title features Text content features and regional location characteristics Concatenating the clustering weights yields an enhanced document representation: in, For title feature weights, Weights for text content features. Weights for regional location features. For text length features, Weights for text length features; During training, the optimal weight combination is found by optimizing the objective function: in, For weighted combinations, For balance coefficient, To adjust the RAND Corporation index, The contour coefficient is used; the enhanced multidimensional features are finally obtained based on the optimal weight combination.
6. The text classification method based on clustering and classification fusion according to claim 5, characterized in that, The multimodal clustering framework also includes the following steps: A multi-dimensional representation space is constructed based on the enhanced multi-dimensional features, wherein the first dimension of the multi-dimensional representation space is... One document for: Will After combining the documents, the output is the enhanced multi-dimensional feature combination. ; Calculate the intra-cluster compactness measure: in, For document The average distance to other documents in the same cluster. For document The average distance to the nearest other document cluster; The model parameter set obtained through training is adopted using a model persistence mechanism. It is serialized and saved, and then directly loaded and used during the inference phase; among which, For TF-IDF vectorization, For clustering models, The optimized weights; model parameter set The clustering results are output using the k-means clustering algorithm. .
7. The text classification method based on clustering and classification fusion according to claim 1, characterized in that, The multi-evidence fusion and confidence propagation decision framework includes the following steps: Original classification results and clustering results A high-confidence direct assignment strategy is adopted, for classifications with confidence levels exceeding a confidence threshold. The document's category is directly determined: in, This is a high-confidence document. For the first One document Confidence level; Each high-confidence document is assigned a corresponding category label. ; To handle documents with similar overall classification scores, define a conflict detection function: in For collision detection functions, Threshold for conflict judgment; For conflicting documents Confidence calibration is performed, among which A collection of conflicting documents; Calculate the cluster Category distribution: in, It is a counting function; The conflict resolution decision function is defined as follows: For unassigned documents, identify the set of unassigned documents: Identify the set of unassigned categories: Set a greedy allocation strategy to select the most suitable document for each unassigned category based on the matching scoring function: in, For document Cluster category The level of support.
8. The text classification method based on clustering and classification fusion according to claim 7, characterized in that, The multi-evidence fusion and confidence propagation decision framework also includes the following steps: Propagate category labels to the entire cluster for assigned documents. and its categories Maintaining semantic consistency within a cluster through a cluster propagation function: Set the allocation strategy for the cluster-dominant category to allocate the remaining unallocated documents, selecting the category that appears most frequently among the already allocated documents in the same cluster. As the category of this document: For completely isolated documents, assign the "uncategorized" label.
9. The text classification method based on clustering and classification fusion according to claim 8, characterized in that, The multi-evidence fusion and confidence propagation decision framework also includes the following steps: Based on the original classification results and clustering results, a confidence assessment is performed, and the overall confidence score is calculated. in, Documents classified by the dynamic weighting mechanism. Assign support to the current category to documents in the same cluster. For classification confidence weighting coefficients; The output classification results include the following fine-grained evaluation metrics: in, For overall accuracy, This represents the number of samples that are actually positive and are predicted as positive by the model. This represents the number of samples that were actually classified as negative but were predicted as positive by the model. For accuracy, For recall rate, This represents the number of samples that were actually positive but were predicted as negative by the model. For F1 score, For true category labels.
10. A text classification system that integrates clustering and classification, characterized in that, The system executes the text classification method of clustering, classification and fusion as described in any one of claims 1-9; the system includes a file input extraction module, a classification module, a clustering module and a fusion module; the file input extraction module receives input files and outputs multi-dimensional features; The classification module receives multi-dimensional features and outputs the raw classification result. The clustering module receives multi-dimensional features and outputs clustering results. The fusion module receives the original classification results and clustering results, and outputs the classification results.