Text classification method and system based on multi-modal feature fusion
By constructing a multimodal feature library and a domain classification model, and combining multi-dimensional detection and dynamic optimization strategies, the problems of single feature dimensions and poor domain adaptability in traditional text classification methods are solved, and automated detection and accurate classification of text are achieved.
Patent Information
- Application Number
- CN202511813498.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-04
- Publication Date
- 2026-03-27
AI Technical Summary
Traditional text classification methods rely on single semantic features, making it difficult to capture text structure patterns and scene-related information. Furthermore, they have poor adaptability to domain differences, resulting in low classification accuracy and weak generalization ability.
We construct a multimodal feature library for text domains, combine it with a domain classification model, filter text features through multidimensional detection, adjust and fuse feature weights, dynamically optimize classification strategies, and integrate multimodal feature extraction and human feedback mechanisms to achieve automated detection and accurate classification of text.
It improves the accuracy and adaptability of text classification, reduces the cost of manual annotation, supports automated classification of text across domains, and is suitable for massive amounts of text data in multiple scenarios.
Smart Images
Figure CN121743498A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of text classification technology, specifically a text classification method and system based on multimodal feature fusion. Background Technology
[0002] With the explosive growth of internet text data (such as news, e-commerce reviews, and medical reports), efficient and accurate text classification has become a core requirement for information filtering and value mining. Traditional text classification methods rely on single semantic features (such as bag-of-words models and TF-IDF), making it difficult to capture text structural patterns and scene-related information. Furthermore, they suffer from poor adaptability to domain differences, resulting in low classification accuracy and weak generalization ability. While multimodal feature technology can integrate multi-dimensional text information, how to effectively fuse semantic, structural, and scene features, and how to build adaptive classification models that fit different domains, remain pressing technical challenges. Summary of the Invention
[0003] This invention addresses the problems of single feature dimension, poor domain adaptability, and low classification accuracy in traditional methods by providing a text classification method and system based on multimodal feature fusion. By constructing a multimodal feature library for the text domain and combining it with a domain classification model, it achieves automated text detection, accurate classification, and dynamic optimization of classification strategies.
[0004] Firstly, the present invention provides a text classification method based on multimodal feature fusion, and the technical solution adopted to solve the above-mentioned technical problems is as follows:
[0005] A text classification method based on multimodal feature fusion includes the following steps:
[0006] S1. Complete the collection and preprocessing of multi-source text data, define and associate core features, store them, and build a multimodal feature library to support subsequent classification;
[0007] S2. Obtain the text to be classified. First, screen qualified texts through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, extract the features of qualified texts based on the multimodal feature library constructed in step S1 and generate the corresponding multimodal feature data.
[0008] S3. Based on the multimodal feature data corresponding to qualified texts, the preliminary classification of texts is achieved through a domain classification model that adjusts feature weights, fuses multimodal features, and adapts them. The final classification result of the texts is determined by combining contradiction detection, weight calculation, and weighted voting of multi-result verification fusion.
[0009] S4. Based on the final text classification results, through the construction of a classification strategy library, dynamic optimization of strategies, automated execution and effect monitoring, the classification strategy is iteratively optimized and efficiently implemented, continuously improving the adaptability, stability and accuracy of text classification.
[0010] S5. Through human feedback optimization mechanism and text domain change awareness, continuously optimize multimodal feature library and domain classification model to provide adaptive and high-quality feature support for classification process.
[0011] Optionally, step S1 specifically includes:
[0012] S1.1 Collection of multi-source text data: For structured text data, extract text metadata, tag data, and statistical feature data; for unstructured text data, collect original text content, semantic association data, and scene description data.
[0013] S1.2 Preprocessing of multi-source text data: Perform data cleaning operations on the text metadata, tag data and statistical feature data extracted from structured text data: remove duplicates in tag data, standardize the format of the extracted data, and standardize the statistical feature data from different sources to ensure data consistency and usability;
[0014] Using natural language processing techniques, semantic vectors, keywords, and syntactic structure information are extracted from the raw text content of unstructured text data;
[0015] By using feature alignment technology, multi-dimensional features corresponding to the same text entities in multi-source text data are associated to ensure feature consistency;
[0016] S1.3 Based on the cleaned, extracted and aligned multi-dimensional text features mentioned above, further define the core features of the text domain, including semantic features, structural features and scene features combined with domain labels, establish the relationship between features, and use a distributed feature database for storage to build a multimodal feature library. This multimodal feature library supports efficient feature query and matching operations.
[0017] Further optionally, step S2 specifically includes:
[0018] S2.1. Obtain the text to be classified. First, use the rule engine to filter it to retain valid text and exclude garbled and blank text. Then, by matching the preset domain keywords, preliminarily determine the domain to which the valid text belongs. Finally, based on the completeness and clarity of the valid text, complete the classification of text quality levels, thereby filtering out qualified text.
[0019] S2.2. Based on the qualified texts obtained through screening, multimodal feature extraction is performed: ① Semantic feature extraction: semantic vectors of qualified texts are generated using a pre-trained language model to capture deep semantic information of the text; ② Structural feature extraction: structural information such as the number of paragraphs, sentence length, keyword density, and punctuation distribution of qualified texts are extracted, and structural feature vectors are constructed; ③ Scene feature extraction: combined with the domain results of the aforementioned judgment, scene features in the multimodal feature library are matched to generate scene feature vectors containing domain type and application scenario ID; finally, the above three types of features are integrated to form multimodal feature data.
[0020] Further optionally, step S3 specifically includes:
[0021] S3.1 Based on the multimodal feature data corresponding to qualified texts, firstly, through the attention mechanism, the weights of multimodal features are dynamically adjusted according to the domain and quality level of the text. Then, a feature fusion strategy that combines concatenation and attention is adopted to integrate the multimodal features into a feature vector of a unified dimension. Subsequently, for the specific domain of the text, the corresponding domain classification model is called to output the preliminary classification results of qualified texts.
[0022] S3.2 For the preliminary classification results of qualified text, first compare them with the stored data in the multimodal feature library to detect contradictions between feature pointers and classification results. Then, assign weights to the corresponding classification results based on the historical reliability data of each feature. Finally, use a weighted voting algorithm to comprehensively calculate the classification results corresponding to different features to determine the final classification result of the text and ensure classification accuracy.
[0023] Preferably, the domain classification model is based on a general pre-trained model and fine-tuned using multimodal feature data of the corresponding domain. This domain classification model will further learn the multimodal association rules of the specific domain on the basis of the basic knowledge learned by the general pre-training, thereby ensuring that the classification results are accurately adapted to the target domain scenario.
[0024] Further optionally, step S4 specifically includes:
[0025] S4.1 Based on the multimodal feature data corresponding to qualified texts, and combined with the experience of feature fusion strategies accumulated in the final classification process of qualified texts and the experience of domain classification model adaptation, a classification strategy library is constructed; and the classification strategies in the classification strategy library are prioritized based on classification efficiency, classification accuracy and domain adaptability indicators, so as to provide a basis for subsequent classification execution and strategy optimization.
[0026] S4.2. Verify the final text classification results using a manual verification mechanism to obtain accuracy feedback data; dynamically adjust the feature weights and fusion methods in the feature fusion strategy based on this feedback data.
[0027] S4.3 Automatically invoke the optimized classification strategy from step S4.2 and apply it to the current text classification task and the iterative correction of historical classification results to automate the classification process; at the same time, monitor the accuracy and execution efficiency of the text classification process in real time, continuously collect feedback on the execution effect of the classification strategy, verify the effectiveness and stability of the classification strategy, and provide data support for the iterative optimization of the classification strategy.
[0028] Preferably, when performing step S4, for newly added domain text, its feature types and distribution characteristics are analyzed, personalized classification strategy recommendations are generated, updated to the classification strategy library, and the priorities are reordered to improve the adaptability of the classification strategy to multi-domain scenarios.
[0029] Further optionally, step S5 specifically includes:
[0030] S5.1. Verify the final text classification results using a manual verification mechanism to obtain accuracy feedback data; optimize the correlation between features in the multimodal feature library based on this feedback data to enhance the influence of high-accuracy features; and simultaneously, incorporate new text cases, classification strategies, and classification results into the multimodal feature library.
[0031] S5.2 Real-time monitoring of new domain text data and text format changes, automatic updating of domain labels and structural feature definitions in the multimodal feature library; and incremental learning to update the multimodal feature library and domain classification model to adapt to the dynamic changes of text data, continuously providing adaptive and high-quality feature support for the classification process.
[0032] Secondly, the present invention provides a text classification system based on multimodal feature fusion, and the technical solution adopted to solve the above-mentioned technical problems is as follows:
[0033] A text classification system based on multimodal feature fusion, used to implement the method described in the first aspect, specifically includes:
[0034] The text processing and feature library construction module is used to collect and preprocess historical multi-source text data, define and associate core features and store them, and build a multimodal feature library to support subsequent classification.
[0035] The text filtering and feature generation module is used to obtain the text to be classified. First, qualified texts are filtered through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, features of qualified texts are extracted based on the constructed multimodal feature library to generate corresponding multimodal feature data.
[0036] The text classification result determination module is used to perform preliminary text classification based on the multimodal feature data corresponding to qualified texts, through feature weight adjustment, multimodal feature fusion and adaptation of the domain classification model, and to determine the final text classification result by combining contradiction detection, weight calculation and weighted voting of multi-result verification fusion.
[0037] The strategy library construction and optimization module is used to build a classification strategy library based on the final text classification results, dynamically optimize the strategy, automate the execution and monitor the effect, and achieve iterative optimization and efficient implementation of the classification strategy, so as to continuously improve the adaptability, stability and accuracy of text classification.
[0038] The continuous optimization module is used to continuously optimize the multimodal feature library and domain classification model through human feedback optimization mechanism and text domain change perception, so as to continuously provide adaptive and high-quality feature support for the classification process.
[0039] The text classification method and system based on multimodal feature fusion of the present invention have the following advantages compared with the prior art:
[0040] 1. This invention constructs a multimodal feature library for the text domain and combines it with a classification model adapted to the domain to achieve automated detection, accurate classification, and dynamic optimization of classification strategies for text. This reduces the cost of manual annotation, improves classification efficiency and generalization ability, and provides a scalable and intelligent classification solution for text data in multiple domains and scenarios. It overcomes the problems of single feature dimension, poor domain adaptability, and low classification accuracy in traditional methods.
[0041] 2. This invention integrates multimodal feature extraction, feature weight optimization, and classification result feedback functions, supporting automated classification of cross-domain (such as news, e-commerce, and medical) texts, significantly reducing reliance on manual annotation, improving classification efficiency and accuracy, and is suitable for the intelligent classification needs of massive and multi-scenario text data. Attached Figure Description
[0042] Appendix Figure 1 This is a flowchart of the method according to Embodiment 1 of the present invention;
[0043] Appendix Figure 2 This is a system module connection block diagram of Embodiment 2 of the present invention. Detailed Implementation
[0044] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.
[0045] Example 1:
[0046] Combined with appendix Figure 1This embodiment proposes a text classification method based on multimodal feature fusion, which includes the following steps:
[0047] S1. Complete the collection and preprocessing of multi-source text data, define and associate core features, store them, and build a multimodal feature library to support subsequent classification; this step specifically includes the following operations:
[0048] S1.1 Collection of multi-source text data: For structured text data, extract text metadata (such as title, author, publication time), tag data (such as manually labeled classification results, domain tags) and statistical feature data (such as keyword frequency, sentence length distribution); for unstructured text data, collect original text content, semantic association data and scene description data.
[0049] S1.2 Preprocessing of Multi-Source Text Data: Perform data cleaning operations on the text metadata (such as title, author, publication time), tag data (such as manually labeled classification results, domain tags), and statistical feature data (such as keyword frequency, sentence length distribution) extracted from structured text data: remove duplicates in the tag data, unify the format specifications of the extracted data (such as data type, naming rules, storage format), and standardize the statistical feature data from different sources (such as Z-score standardization, Min-Max standardization) to ensure data consistency and usability;
[0050] Using natural language processing techniques (such as the BERT model and word segmentation tools), semantic vectors, keywords, and syntactic structure information are extracted from the raw text content of unstructured text data;
[0051] By using feature alignment technology, multi-dimensional features corresponding to the same text entities in multi-source text data are associated to ensure feature consistency;
[0052] S1.3 Based on the cleaned, extracted, and aligned multi-dimensional text features mentioned above, further define the core features of the text domain, including semantic features (such as word vectors and sentence vectors), structural features (such as paragraph hierarchy and keyword positions), and scene features combined with domain tags (such as domain tags and user group attributes). Establish the relationships between features, such as "semantic vectors correspond to text content", "scene features are associated with domain tags", and "structural features affect classification weights". Use a distributed feature database (such as Elasticsearch) to store the features and build a multimodal feature library. This multimodal feature library supports efficient feature query and matching operations.
[0053] S2. Obtain the text to be classified. First, filter qualified texts through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, extract features of qualified texts based on the multimodal feature library constructed in step S1 to generate corresponding multimodal feature data. This step specifically includes the following operations:
[0054] S2.1. Obtain the text to be classified. First, use a rule engine (such as character length validation and special character filtering) to filter the text to retain valid text and exclude meaningless content such as garbled text and blank text. Then, by matching preset domain keywords (such as "symptoms" and "drugs" in the medical field, and "products" and "prices" in the e-commerce field), the domain to which the valid text belongs is initially determined. Finally, based on the completeness (such as whether it contains a title and body text) and clarity (such as whether there are ambiguous expressions) of the valid text, the text quality level is divided, thereby filtering out qualified text.
[0055] S2.2. Based on the qualified texts obtained through screening, multimodal feature extraction is performed: ① Semantic feature extraction: semantic vectors of qualified texts are generated using pre-trained language models such as RoBERTa and GPT to capture deep semantic information of the text; ② Structural feature extraction: structural information such as the number of paragraphs, sentence length, keyword density, and punctuation distribution of qualified texts are extracted, and structural feature vectors are constructed; ③ Scene feature extraction: combined with the domain results of the aforementioned judgment, scene features in the multimodal feature library are matched to generate scene feature vectors containing information such as domain type and application scenario ID; finally, the above three types of features are integrated to form multimodal feature data.
[0056] S3. Based on the multimodal feature data corresponding to qualified texts, a preliminary text classification is achieved through a domain classification model that adjusts feature weights, fuses multimodal features, and adapts the data. Finally, the text classification result is determined by combining contradiction detection, weight calculation, and weighted voting through multi-result verification fusion. This step specifically includes the following operations:
[0057] S3.1 Based on the multimodal feature data corresponding to qualified texts, firstly, through an attention mechanism, the weights of multimodal features are dynamically adjusted according to the domain and quality level of the text (e.g., medical texts emphasize semantic features, while news texts emphasize structural features). Then, a feature fusion strategy that combines concatenation and attention is adopted to integrate the multimodal features into a feature vector of a unified dimension. Subsequently, for the specific domain of the text, the corresponding domain classification model (e.g., medical text classification model, e-commerce text classification model) is called to output the preliminary classification results of qualified texts.
[0058] The domain classification model used in this step is based on a general pre-trained model (such as BERT for text modality or CLIP for multimodality) and is built by fine-tuning the multimodal feature data of the corresponding domain. The domain classification model will further learn the multimodal association rules of the specific domain on the basis of the basic knowledge learned by the general pre-training, thereby ensuring that the classification results are accurately adapted to the target domain scenario.
[0059] S3.2 For the preliminary classification results of qualified text, first compare them with the stored data in the multimodal feature library to detect contradictions between feature pointing and classification results (such as semantic features pointing to one domain but scene features pointing to another domain). Then, assign weights to the corresponding classification results according to the historical reliability data of each feature (such as semantic feature classification accuracy and scene feature classification accuracy). Finally, use a weighted voting algorithm to comprehensively calculate the classification results corresponding to different features to determine the final classification result of the text and ensure classification accuracy.
[0060] S4. Based on the final text classification results, through the construction of a classification strategy library, dynamic strategy optimization, automated execution, and effect monitoring, the classification strategy is iteratively optimized and efficiently implemented, continuously improving the adaptability, stability, and accuracy of text classification. This step specifically includes the following operations:
[0061] S4.1 Based on the multimodal feature data corresponding to qualified texts, combined with the experience of feature fusion strategies accumulated in the final classification process of qualified texts (such as the semantic + scene feature-dominated fusion mode for medical texts) and the experience of domain classification model adaptation, a classification strategy library is constructed; and the classification strategies in the classification strategy library are prioritized based on classification efficiency, classification accuracy and domain adaptability indicators, so as to provide a basis for subsequent classification execution and strategy optimization.
[0062] S4.2. Verify the final text classification results using a manual verification mechanism and obtain accuracy feedback data. Based on this feedback data, dynamically adjust the feature weights and fusion methods in the feature fusion strategy (e.g., if the classification accuracy of structural features is low in a certain scenario, reduce their weight in the fusion strategy).
[0063] S4.3 Automatically invoke the optimized classification strategy from step S4.2 and apply it to the current text classification task and the iterative correction of historical classification results to automate the classification process; at the same time, monitor the accuracy and execution efficiency of the text classification process in real time, continuously collect feedback on the execution effect of the classification strategy, verify the effectiveness and stability of the classification strategy, and provide data support for the iterative optimization of the classification strategy.
[0064] When performing this step, for newly added domain texts, analyze their feature types and distribution characteristics, generate personalized classification strategy recommendations (e.g., for newly added legal text domains, recommend a fusion strategy of "semantic + keyword structural features"), update to the classification strategy library and reorder priorities to improve the adaptability of classification strategies to multi-domain scenarios.
[0065] S5. Through a human feedback optimization mechanism and text domain change awareness, continuously optimize the multimodal feature library and domain classification model to provide adaptive and high-quality feature support for the classification process. This step specifically includes the following operations:
[0066] S5.1. Verify the final text classification results using a manual verification mechanism to obtain accuracy feedback data; optimize the correlation between features in the multimodal feature library based on this feedback data to enhance the influence of high-accuracy features; and simultaneously, incorporate new text cases, classification strategies, and classification results into the multimodal feature library.
[0067] S5.2 Real-time monitoring of new domain text data (such as emerging industry reports) and text format changes (such as short texts from short video platforms) automatically updates the domain labels and structural feature definitions in the multimodal feature library; and uses incremental learning to update the multimodal feature library and domain classification model to adapt to the dynamic changes in text data, continuously providing adaptive and high-quality feature support for the classification process.
[0068] Example 2:
[0069] Combined with appendix Figure 2 This embodiment proposes a text classification system based on multimodal feature fusion, which is used to implement the method described in Embodiment 1, specifically including:
[0070] The text processing and feature library construction module is used to collect and preprocess historical multi-source text data, define and associate core features and store them, and build a multimodal feature library to support subsequent classification.
[0071] The text filtering and feature generation module is used to obtain the text to be classified. First, qualified texts are filtered through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, features of qualified texts are extracted based on the constructed multimodal feature library to generate corresponding multimodal feature data.
[0072] The text classification result determination module is used to perform preliminary text classification based on the multimodal feature data corresponding to qualified texts, through feature weight adjustment, multimodal feature fusion and adaptation of the domain classification model, and to determine the final text classification result by combining contradiction detection, weight calculation and weighted voting of multi-result verification fusion.
[0073] The strategy library construction and optimization module is used to build a classification strategy library based on the final text classification results, dynamically optimize the strategy, automate the execution and monitor the effect, and achieve iterative optimization and efficient implementation of the classification strategy, so as to continuously improve the adaptability, stability and accuracy of text classification.
[0074] The continuous optimization module is used to continuously optimize the multimodal feature library and domain classification model through human feedback optimization mechanism and text domain change perception, so as to continuously provide adaptive and high-quality feature support for the classification process.
[0075] In summary, the text classification method and system based on multimodal feature fusion of the present invention can realize automated text detection, accurate classification, and dynamic optimization of classification strategies, reduce manual annotation costs, improve classification efficiency and generalization ability, and provide scalable and intelligent classification solutions for text data in multiple fields and scenarios. It overcomes the problems of single feature dimension, poor domain adaptability, and low classification accuracy in traditional methods.
[0076] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.
Claims
1. A text classification method based on multimodal feature fusion, characterized in that, Includes the following steps: S1. Complete the collection and preprocessing of historical multi-source text data, define and associate core features and store them, and build a multimodal feature library to support subsequent classification; S2. Obtain the text to be classified. First, screen qualified texts through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, extract the features of qualified texts based on the multimodal feature library constructed in step S1 and generate the corresponding multimodal feature data. S3. Based on the multimodal feature data corresponding to qualified texts, the preliminary classification of texts is achieved through a domain classification model that adjusts feature weights, fuses multimodal features, and adapts them. The final classification result of the texts is determined by combining contradiction detection, weight calculation, and weighted voting of multi-result verification fusion. S4. Based on the final text classification results, through the construction of a classification strategy library, dynamic optimization of strategies, automated execution and effect monitoring, the classification strategy is iteratively optimized and efficiently implemented, continuously improving the adaptability, stability and accuracy of text classification. S5. Through human feedback optimization mechanism and text domain change awareness, continuously optimize multimodal feature library and domain classification model to provide adaptive and high-quality feature support for classification process.
2. The text classification method based on multimodal feature fusion according to claim 1, characterized in that, Step S1 specifically includes: S1.1 Collection of multi-source text data: For structured text data, extract text metadata, tag data, and statistical feature data; for unstructured text data, collect original text content, semantic association data, and scene description data. S1.2 Preprocessing of multi-source text data: Perform data cleaning operations on the text metadata, tag data and statistical feature data extracted from structured text data: remove duplicates in tag data, standardize the format of the extracted data, and standardize the statistical feature data from different sources to ensure data consistency and usability; Using natural language processing techniques, semantic vectors, keywords, and syntactic structure information are extracted from the raw text content of unstructured text data; By using feature alignment technology, multi-dimensional features corresponding to the same text entities in multi-source text data are associated to ensure feature consistency; S1.3 Based on the cleaned, extracted and aligned multi-dimensional text features mentioned above, further define the core features of the text domain, including semantic features, structural features and scene features combined with domain labels, establish the relationship between features, and use a distributed feature database for storage to build a multimodal feature library. This multimodal feature library supports efficient feature query and matching operations.
3. The text classification method based on multimodal feature fusion according to claim 2, characterized in that, Step S2 specifically includes: S2.
1. Obtain the text to be classified. First, use the rule engine to filter it to retain valid text and exclude garbled and blank text. Then, by matching the preset domain keywords, preliminarily determine the domain to which the valid text belongs. Finally, based on the completeness and clarity of the valid text, complete the classification of text quality levels, thereby filtering out qualified text. S2.
2. Based on the qualified texts obtained through screening, multimodal feature extraction is performed: ① Semantic feature extraction: semantic vectors of qualified texts are generated using a pre-trained language model to capture deep semantic information of the text; ② Structural feature extraction: structural information such as the number of paragraphs, sentence length, keyword density, and punctuation distribution of qualified texts are extracted, and structural feature vectors are constructed; ③ Scene feature extraction: combined with the domain results of the aforementioned judgment, scene features in the multimodal feature library are matched to generate scene feature vectors containing domain type and application scenario ID; finally, the above three types of features are integrated to form multimodal feature data.
4. The text classification method based on multimodal feature fusion according to claim 3, characterized in that, Step S3 specifically includes: S3.1 Based on the multimodal feature data corresponding to qualified texts, firstly, through the attention mechanism, the weights of multimodal features are dynamically adjusted according to the domain and quality level of the text. Then, a feature fusion strategy that combines concatenation and attention is adopted to integrate the multimodal features into a feature vector of a unified dimension. Subsequently, for the specific domain of the text, the corresponding domain classification model is called to output the preliminary classification results of qualified texts. S3.2 For the preliminary classification results of qualified text, first compare them with the stored data in the multimodal feature library to detect contradictions between feature pointers and classification results. Then, assign weights to the corresponding classification results based on the historical reliability data of each feature. Finally, use a weighted voting algorithm to comprehensively calculate the classification results corresponding to different features to determine the final classification result of the text and ensure classification accuracy.
5. The text classification method based on multimodal feature fusion according to claim 4, characterized in that, The domain classification model is built on a general pre-trained model and fine-tuned using multimodal feature data of the corresponding domain. The domain classification model will further learn the multimodal association rules of the specific domain on the basis of the basic knowledge learned by the general pre-training, thereby ensuring that the classification results are accurately adapted to the target domain scenario.
6. The text classification method based on multimodal feature fusion according to claim 4, characterized in that, Step S4 specifically includes: S4.1 Based on the multimodal feature data corresponding to qualified texts, and combined with the experience of feature fusion strategies accumulated in the final classification process of qualified texts and the experience of domain classification model adaptation, a classification strategy library is constructed; and the classification strategies in the classification strategy library are prioritized based on classification efficiency, classification accuracy and domain adaptability indicators, so as to provide a basis for subsequent classification execution and strategy optimization. S4.
2. Verify the final text classification results using a manual verification mechanism to obtain accuracy feedback data; dynamically adjust the feature weights and fusion methods in the feature fusion strategy based on this feedback data. S4.3 Automatically invoke the optimized classification strategy from step S4.2 and apply it to the current text classification task and the iterative correction of historical classification results to automate the classification process; at the same time, monitor the accuracy and execution efficiency of the text classification process in real time, continuously collect feedback on the execution effect of the classification strategy, verify the effectiveness and stability of the classification strategy, and provide data support for the iterative optimization of the classification strategy.
7. A text classification method based on multimodal feature fusion according to claim 6, characterized in that, When performing step S4, for newly added domain text, analyze its feature types and distribution characteristics, generate personalized classification strategy recommendations, update the classification strategy library and reorder the priorities to improve the adaptability of classification strategies to multi-domain scenarios.
8. A text classification method based on multimodal feature fusion according to claim 6, characterized in that, Step S5 specifically includes: S5.
1. Verify the final text classification results using a manual verification mechanism to obtain accuracy feedback data; optimize the correlation between features in the multimodal feature library based on this feedback data to enhance the influence of high-accuracy features; and simultaneously, incorporate new text cases, classification strategies, and classification results into the multimodal feature library. S5.2 Real-time monitoring of new domain text data and text format changes, automatic updating of domain labels and structural feature definitions in the multimodal feature library; and incremental learning to update the multimodal feature library and domain classification model to adapt to the dynamic changes of text data, continuously providing adaptive and high-quality feature support for the classification process.
9. A text classification system based on multimodal feature fusion, characterized in that, It is used to implement the method according to any one of claims 1-8, specifically comprising: The text processing and feature library construction module is used to collect and preprocess historical multi-source text data, define and associate core features and store them, and build a multimodal feature library to support subsequent classification. The text filtering and feature generation module is used to obtain the text to be classified. First, qualified texts are filtered through multi-dimensional detection of text validity, domain affiliation, and quality level. Then, features of qualified texts are extracted based on the constructed multimodal feature library to generate corresponding multimodal feature data. The text classification result determination module is used to perform preliminary text classification based on the multimodal feature data corresponding to qualified texts, through feature weight adjustment, multimodal feature fusion and adaptation of the domain classification model, and to determine the final text classification result by combining contradiction detection, weight calculation and weighted voting of multi-result verification fusion. The strategy library construction and optimization module is used to build a classification strategy library based on the final text classification results, dynamically optimize the strategy, automate the execution and monitor the effect, and achieve iterative optimization and efficient implementation of the classification strategy, so as to continuously improve the adaptability, stability and accuracy of text classification. The continuous optimization module is used to continuously optimize the multimodal feature library and domain classification model through human feedback optimization mechanism and text domain change perception, so as to continuously provide adaptive and high-quality feature support for the classification process.