Information text extraction and classification system and method based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an artificial intelligence-based information text information extraction and classification method for customs information texts by utilizing a domestic artificial intelligence framework. According to the method, the advantages of a domestic artificial intelligence framework can be fully utilized, and a professional natural language processing technology is combined, so that efficient and accurate analysis of complex information texts is realized; meanwhile, the method has good universality and generalization ability, can adapt to the requirements of different fields and scenes, and provides powerful support for intelligent analysis of information texts; according to the method, a main body framework based on a domestic artificial intelligence framework Ernie knowledge enhancement pre-training model is constructed, so that efficient processing and accurate analysis of multi-field information texts are realized.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based information extraction and classification method for customs information text using a domestic artificial intelligence framework. Background Art

[0002] With the rapid development of information technology, the sources of information are becoming increasingly diverse, including but not limited to public news reports, professional reports, internal case materials and intelligence information. These information texts often contain rich information, especially in specific areas such as risk prevention and control, where their information value is particularly important. However, compared with traditional texts, information texts have the characteristics of complex logic, nested entities and multi-level classification, which makes traditional algorithms face huge challenges in processing these texts. Most traditional algorithms use general structured recognition and extraction methods, which makes it difficult to achieve intelligent and precise analysis goals, and cannot fully explore and utilize the deep information in information texts.

[0003] In recent years, the rise of artificial intelligence deep learning technology has provided new ideas and methods for the intelligent analysis of information texts. Domestic and foreign scholars and enterprises have conducted a lot of research in this field, using natural language processing (NLP) models to achieve structured processing of text information, significantly improving the efficiency and accuracy of information extraction. For example, with the support of natural language processing technology, large-scale text entity or relationship extraction in many fields such as biology, chemistry, and medicine has achieved remarkable results. At the same time, methods based on generative models have also demonstrated powerful capabilities in writing optimization and document analysis tasks. China Customs and other institutions have also actively introduced internationally advanced artificial intelligence frameworks to achieve effective extraction of information such as countries, items, quantities, and dates.

[0004] Despite this, current AI text analysis technology still has many shortcomings. First, when dealing with specific languages ​​such as Chinese, foreign AI frameworks and models often have language and context differences, resulting in low adaptability. Second, many existing technologies have limited intelligence, and it is difficult to further improve the recognition accuracy, especially when dealing with complex information texts. In addition, the high reliance on foreign technologies and frameworks also limits the autonomy and controllability of the technology, posing potential security risks. Summary of the invention

[0005] In view of the above problems, the present invention aims to provide a more intelligent, accurate and autonomous information extraction and classification system method for information texts. The method should be able to fully utilize the advantages of the domestic artificial intelligence framework and combine professional natural language processing technology to achieve efficient and accurate analysis of complex information texts. At the same time, the method should also have good versatility and generalization capabilities, be able to adapt to the needs of different fields and scenarios, and provide strong support for the intelligent analysis of information texts.

[0006] The technical solution adopted by the present invention is: an information text extraction and classification system based on artificial intelligence, characterized in that it includes:

[0007] a. The main model module realizes multi-task collaborative processing through its multi-layer architecture and supports a wide range of text information extraction and classification functions;

[0008] b. Pre-processing module, used to pre-process the customs information to be processed;

[0009] c. a text extraction module connected to the subject model module for performing information extraction;

[0010] d. a text classification module, connected to the subject model module, for performing text classification;

[0011] e. Integration module, used to integrate the extracted information and classification results.

[0012] An information text extraction and classification method based on artificial intelligence comprises the following steps:

[0013] S1. Collect open source information data from multiple fields as training data;

[0014] S2. Preprocessing the training data to meet the input and output format requirements of the language model;

[0015] S3. Build a knowledge-enhanced pre-training model based on the domestic artificial intelligence framework as the main model;

[0016] S4, training the subject model using the preprocessed training data to construct a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities;

[0017] S5. Apply the general text extraction model and the general text classification model to the information text to be processed to perform information extraction and text classification and post-integration.

[0018] The multi-layer architecture of the subject model includes a general representation layer and a specific representation layer. The general representation layer is used to obtain the same underlying abstract features in different task paradigms, and the specific representation layer contains specific representation modules for natural language understanding and natural language generation.

[0019] The preprocessing step includes removing redundant spaces, punctuation marks, special characters, unifying date formats and currency units, and performing format conversion.

[0020] The main model adopts the Ernie knowledge-enhanced pre-training model, and its general representation layer and specific representation layer both use the Transformer-XL structure as the backbone, where the general representation layer contains 48 Transformer layers, 4096 hidden units and 64 attention heads, and the specific representation layer contains 12 Transformer layers, 768 hidden units and 12 attention heads.

[0021] The general text extraction model is tuned through a multi-task unified modeling approach, the input layer uses a structural model based on a prompt mechanism, and the output layer uses a structured extraction language to represent the extraction results in a unified form.

[0022] The general text classification model is tuned through a multi-task unified semantic matching method, and the classification task is modeled as a matching task between labels and texts. The decoding layer uses directed label links to calculate scores, and the output layer outputs associated pairs of labels and text content.

[0023] The method also includes a step of evaluating the general text extraction model and the general text classification model, wherein the text extraction uses accuracy, recall rate and F1 score as evaluation indicators, and the text classification uses MacroF1 and Micro F1 as evaluation indicators.

[0024] It also includes steps of injecting domain-specific expertise into general text extraction models and general text classification models to obtain text extraction and classification models with domain characteristics.

[0025] The domain knowledge injection step includes: collecting domain-specific expertise, text extraction annotation data, and text classification annotation data, and using these data to tune the general model to generate a domain-specific text extraction and classification model.

[0026] The specific field is the customs field, the professional knowledge includes tariff policies, trade regulations, inspection and quarantine systems, customs laws and professional terms, and the annotated data includes customs information text extraction annotated data and text classification annotated data.

[0027] The beneficial effects of the present invention are as follows: the information text extraction and classification system and method based on artificial intelligence provided by the present invention realizes efficient processing and accurate analysis of information texts in multiple fields by constructing a main framework based on the Ernie knowledge enhancement pre-training model of the domestic artificial intelligence framework; specifically, the technical solution has the following significant beneficial effects:

[0028] (I) Efficient information extraction capability: Through the optimized design of the general representation layer and the specific representation layer, the model can quickly and accurately extract structured information from complex information texts, including entities, relationships, and events, greatly improving the efficiency and accuracy of information extraction.

[0029] (ii) Powerful text classification capabilities: The model has extensive text classification capabilities and can automatically identify and classify information texts in different fields. It supports multi-label classification and effectively solves the problem of inaccurate category recognition in traditional classification methods.

[0030] (III) Good generalization and adaptability: The model is tuned through self-supervised learning tasks and back-propagation algorithms, so that the model can quickly adapt to new tasks and has the potential for wide application in different fields and scenarios, thus improving the generalization and practicality of the model.

[0031] (IV) Autonomous and controllable security assurance: The domestic artificial intelligence framework is adopted to reduce dependence on external technology, enhance the data isolation and protection mechanism of the model, and improve the security and autonomous controllability of the system.

[0032] (V) Significantly improve the efficiency of customs information analysis: The optimized model is applied to the field of customs information analysis, realizing the automatic parsing and recognition of a large amount of unlabeled customs information texts, improving the intelligence level and efficiency of customs information processing, and providing strong support for rapid retrieval analysis, risk warning, etc.

[0033] (VI) Improvement of industry adaptability and recognition accuracy: Enhanced training of the model through data sets in specific fields has further improved the adaptability and recognition accuracy of the model in industry applications, providing a reliable basis for accurate decision-making in specific fields such as customs. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the information text extraction model architecture in the present invention;

[0035] Figure 2 It is a schematic diagram of the information text classification model architecture in the present invention;

[0036] Figure 3 It is a schematic diagram of an example of customs information text marking in the present invention. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments:

[0038] Example 1: Information text extraction and classification system based on artificial intelligence framework

[0039] like Figure 1-3As shown, this embodiment adopts a domestic artificial intelligence framework to solve core technical problems, especially focusing on enhancing security and controllability. This framework is integrated with domestic models, which not only provides a more excellent data isolation and protection mechanism, but also ensures higher controllability of the algorithm and model training process. In addition, it has shown significant advantages in integration with domestic data platforms and security technologies. In order to further improve processing capabilities, this embodiment also incorporates text extraction technology based on natural language processing, which can automatically extract structured information from massive unstructured texts, covering multiple elements such as entities, relationships and events, among which entity extraction specifically refers to identifying information with specific meanings from text. At the same time, we have also introduced text classification technology, which is a natural language processing technology that can automatically classify text into preset categories, and is widely used in fields such as sentiment analysis and topic identification.

[0040] This embodiment is an information text extraction and classification system based on artificial intelligence, including: a subject model module, which realizes multi-task collaborative processing through its multi-layer architecture and supports a wide range of text information extraction and classification functions; a preprocessing module, which is used to preprocess the customs information to be processed; a text extraction module, which is connected to the subject model module and is used to perform information extraction; a text classification module, which is connected to the subject model module and is used to perform text classification; an integration module, which is used to integrate the extracted information and classification results. Experiments have shown that the model has good information extraction and classification effects. The experimental process specifically includes the following steps:

[0041] S1. Subject model design: It includes the subject model module, which realizes multi-task collaborative processing through its multi-layer architecture and supports a wide range of text information extraction and classification functions.

[0042] Collect open source information data from the financial, legal, and medical fields as initial training data. Preprocess the initial training data, including removing extra spaces, punctuation marks, special characters, unifying date formats and currency units, and converting the training data to meet the input and output format requirements of the language model.

[0043] This embodiment also includes a preprocessing module for preprocessing the customs information to be processed; the model of this embodiment selects the Ernie knowledge enhancement pre-training model based on the domestic artificial intelligence framework as the main model, and the model has the ability to coordinate processing between multiple task paradigms. The model architecture consists of a general representation layer and a specific representation layer. The general representation layer can obtain the same underlying abstract features in different task paradigms, such as vocabulary information and syntactic information, etc. The specific representation layer includes a natural language understanding (NLU) specific representation module and a natural language generation (NLG) specific representation module. Based on this upper and lower two-layer architecture, with less training corpus and time cost, tuning the specific representation layer can quickly and effectively improve the recognition efficiency and adaptability of specific tasks.

[0044] Both the general representation layer and task-specific representation layer of ERNIE 3.0 use the Transformer-XL structure as the backbone. In the pre-training task of the new algorithm module, this embodiment uses a general representation layer with 48 transformer layers, 4096 hidden units and 64 attention heads; a specific representation layer with 12 transformer layers, 768 hidden units and 12 attention heads, using the GeLU activation function and Adam optimization algorithm, with the parameters setting the maximum sequence length of the context to 512, the memory length of language generation to 128, the total batch size to 6144, and the learning rate to 1e-4. Through word perception, structure perception and knowledge perception pre-training tasks, the model has the ability to understand, generate and reason.

[0045] S2. Generalizing model-specific capabilities: preprocessing the training data to meet the input and output format requirements of the language model.

[0046] To conduct information text analysis, we first need to initialize the model with a pre-trained parameter combination, and then tune the specific representation layer of the pre-trained main model so that the adjusted model can obtain broad information text extraction and classification capabilities.

[0047] (1) Construct a general text extraction model, including a text extraction module, connected to the main model module, for performing information extraction; and have extensive text processing and information extraction capabilities. The generalization stage includes:

[0048] The encoding layer of the knowledge-enhanced model is used as the main model, and it is tuned in a multi-task unified modeling manner. Drawing on the general information extraction framework paradigm, a structured model based on a prompt mechanism is used in the input layer to guide the model to adaptively discriminate the extraction target during the training process. The decoded main model results use a structured extraction language in the output layer to represent the extraction results of different tasks in a unified form.

[0049] The input expression is

[0050]

[0051] Where: s represents the defined structure extraction mode; × represents the input text; y represents the extracted and generated structured result; Represents a connection symbol. The overall input form of text extraction is

[0052]

[0053] Where: s i represents the component of the i-th structural extraction mode; x i Represents the content of the ith original text sequence. Expanding the above formula, the final model input result is

[0054]

[0055] Where: [spot] means the entity is followed by s i represents the i-th entity of different categories, [asso] represents a relationship or event, and a i It represents the i-th relationship or event of different categories, and [te×t] means it is followed by text content.

[0056] The preprocessed initialized training data is used to optimize the parameters of the specific representation layer of the knowledge enhancement pre-training model, and the language model parameters are updated using self-supervised learning tasks and back-propagation algorithms, so that the model has a wide range of text extraction capabilities and can extract key information that is not limited to industry fields and extraction targets.

[0057] (2) Construct a general text classification model, including a text classification module, which is connected to the main model module and is used to perform text classification; it has a wide range of text classification capabilities.

[0058] The generalization stage includes: using the knowledge-enhanced model encoding layer as the main model, tuning in a multi-task unified semantic matching manner, and uniformly modeling the classification task as a matching task between labels and texts. In the decoding layer, directional tag links are used to decode the input content into word pairs between labels and texts and calculate the scores, and the output layer outputs the associated pairs of label nouns and text content.

[0059] The input expression is

[0060] H = Encoder[l1,l2,...,l i , t1, t2, ..., t i , M]

[0061] Where: l i represents the i-th label sequence, t irepresents the i-th text sequence, and M is a mask matrix used to determine which sequence pairs can pay attention to each other. The expression of the pair connection score is

[0062]

[0063] Where: (l i , t j ) represents the connection between the label and the text tag pair; FFNN label / FFNN text Both represent feedforward neural networks, where data will propagate in one direction and there is no feedback connection in the process; R j-i Indicates rotation position embedding, which can inject relative position information. Denote the embedding representations of labels and text tokens, respectively.

[0064] The preprocessed initialized training data is used to optimize the parameters of the specific representation layer of the knowledge-enhanced pre-training model, and the language model parameters are updated using self-supervised learning tasks and back-propagation algorithms, so that the model has extensive text classification capabilities and supports the transfer of label knowledge between different fields and numerous "general classification" tasks.

[0065] In the parameter optimization process, the hyperparameters that need to be adjusted for the generalization training of the above two models mainly include training cycle, maximum learning rate, batch processing size, and maximum text segmentation length, so that the corresponding parameters reach the optimal range. After tuning, the general text extraction and general text classification models are obtained, which not only have extensive text extraction and classification capabilities, but also can adapt to new tasks through a small amount of gradient updates.

[0066] S3. Model Evaluation

[0067] (1) Information Text Extraction Model

[0068] In this embodiment, the extraction generally uses precision p, recall R and F1 score as evaluation indicators. In the process of model optimization, improving the precision can reduce the probability of misidentifying entities and misclassifying categories, and improving the recall can capture more correct entities and reduce omissions. There is often a need to balance these two indicators. Under the premise of balance, both should be maintained at a relatively high level to achieve optimal performance. The expressions of the three indicators are as follows:

[0069]

[0070] Where: Tp is the number of correctly predicted entities; Fp is the number of entities that are not of this type but are mistakenly predicted to be of this type; Fn represents the number of entities of this type of text that are mistakenly predicted to be of other categories.

[0071] (2) Information Text Classification Model

[0072] Text classification evaluation generally uses Macro F1 and Micro F1 as evaluation indicators. Macro F1 is the macro average of the F1 score, which takes the average of the F1 scores of each category to avoid the model performance being dominated by the large number of categories. MicroF1 comprehensively evaluates the overall performance of the model on all categories. Through continuous parameter tuning, various evaluation indicators are optimized. The expressions related to the indicators are as follows:

[0073]

[0074] Where: n represents the total number of all categories; represents the F1 score of the ith category; represents the total precision of all categories; represents the total recall of all categories.

[0075] This embodiment also includes an integration module for integrating the extracted information and classification results, integrating and sorting the information identified by the extraction model and text classification model in this embodiment, so as to construct a complete information text analysis result. For the same information text, the text extraction model and the text classification model are called in turn for identification. First, the information text and the extraction prompt label are used as the input of the text extraction model to obtain the extraction result, and then the same information text and classification prompt label are used as the input of the text classification model to obtain the classification result, and the disassembled hierarchical relationship is recombined and finally stored in a record. As the model continues to undergo adaptive iterative training and upgrading, its intelligence and accuracy are also gradually improving.

[0076] Example 2: Customs information text extraction and classification method based on artificial intelligence

[0077] The current application of information extraction technology in the field of customs information analysis mainly has the following problems: First, foreign models and frameworks have differences in different languages ​​and contexts, and have low adaptability to domestic customs information text features; second, there is a lack of accurate identification of information subject categories; third, the models and frameworks are highly dependent on foreign countries and are difficult to achieve independent control. To solve the above problems, based on the new model of the fusion of extraction and classification technology proposed in this embodiment, professional training corpus in the customs field is used to manually mark a small amount of information data, and the professional knowledge in the customs field and information text extraction and annotation data are injected into the general text extraction model to obtain a customs information text extraction model. The professional knowledge in the customs field and information text classification and annotation data are injected into the general text classification model to obtain a customs information text classification model. Automatic parsing and identification of a large number of unlabeled customs information texts are realized, rapid retrieval analysis and risk warning are completed, and the precision and intelligence of customs information analysis are realized.

[0078] In view of the special industry background and professional terminology of customs information text, the information text extraction and text classification model are optimized and adapted to improve the accuracy and applicability of the model. Model training includes data acquisition and analysis, data preprocessing, feature selection, data annotation, model training, and score generation.

[0079] Step 1: Data acquisition and analysis: Collect open source information data from multiple fields as training data.

[0080] Domain knowledge injection includes: collecting and organizing professional knowledge in the customs field, extracting and annotating data from customs information text, and classifying and annotating data from customs information text. The professional knowledge includes: tariff policies, trade regulations, inspection and quarantine systems, customs laws, professional terms, etc.; the customs information text extraction and annotating data and text classification and annotating data include: annotated customs information collected through various channels such as news reports, customs announcements, social media, etc., including comprehensive information such as seized information and endangered information.

[0081] Step 2: Data preprocessing: preprocess the training data to meet the input and output format requirements of the language model;

[0082] Data preprocessing includes: after collecting multi-source data, processing the mapping association of business multi-source data sets, that is, normalizing the field values ​​with the same definition in the multi-source data so that their specific values ​​are mapped to the values ​​of the standard code or parameter. In practice, a unified code or parameter library is established and improved. For example, in the field of customs, standardized mapping of enterprise codes, country (region) codes, and customs district codes is required, among which enterprise codes need to be mapped to the latest standard code, the unified social credit code.

[0083] Step 3: Feature Selection

[0084] For different semantic categories such as seized and endangered, corresponding structured information extraction labels and text classification labels are constructed. For example, the seized category includes seizure information, violation process information, customs declaration information, risk type, law enforcement type, etc. Seizure information mainly includes time, location, goods, value, quantity, etc.; violation process information includes source, destination, violation method, etc.; customs declaration information includes customs declaration number, company information, etc.; risk type includes 39 single-level and multi-level risk classifications; law enforcement type includes law enforcement code, etc.

[0085] Step 4: Data labeling

[0086] Customs information is annotated according to the label. Text extraction and annotation refers to finding the keywords corresponding to the prompt label from the information text and associating them. Text classification and annotation refers to dividing a single information text into corresponding prompt labels, thereby constructing a customs information text sample set including a training set, an evaluation set, and a test set. In the process of selecting the original information text, ensure that the sample set has sufficient diversity to cover the main knowledge points and label range in the customs field, and ensure that the selected samples are of practical significance.

[0087] Step 5: Model training

[0088] Construct a knowledge-enhanced pre-trained model based on a domestic artificial intelligence framework as the main model; use the preprocessed training data to train the main model to construct a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities; apply the general text extraction model and the general text classification model to the information text to be processed to perform information extraction and post-text classification integration.

[0089] By adopting the few-sample learning method, we can quickly achieve good performance through a small amount of training data or a small number of training iterations, quickly adapt to and learn new customs information analysis tasks, embed professional information in the customs field into the general text extraction and classification model, and enhance the information analysis ability of the language model in the customs field.

[0090] The model adopts a structure with 12 transformer layers, 768 hidden units and 12 attention heads. It takes the customs information labeled data set as input and calculates the output. The parameters of the encoding layer and decoding layer in the model are optimized through back propagation. In the process of continuous full parameter adjustment, the encoding representation of professional customs terms in customs information is completed; the decoding layer predicts the starting and ending positions of customs information entities for entity extraction tasks, and predicts the connection relationship between customs information labels and text for classification tasks; and finally outputs the customs information entities and classifications in structured representation at the output layer.

[0091] Step 6: Optimize and evaluate

[0092] The model is tuned in the following two ways: First, through hyperparameter adjustment, the parameter optimization of customs information analysis is realized. After iterative hyperparameter tuning, it is evaluated that when the training cycle is 10, the maximum learning rate is 1e-5, and the batch processing size is 16, the model effect is better. Second, the feature optimization of customs information analysis is realized by improving feature engineering. First, all the involved labels are set and the corresponding learning samples are added to ensure the original probability of effective recognition and improve the recognition rate of new labels. Then, for situations involving hierarchical classification (such as infringement risk under risk type), only the sub-classification (such as infringement risk) is taken as the actual classification label. According to the business need that a multi-level classification can only produce one classification result, the prediction results are processed by taking the highest probability category at the same level.

[0093] For entity evaluation, this embodiment uses accuracy p, recall R and F1 score as evaluation indicators; for classification evaluation, Macro F1 and Micro F1 are used as evaluation indicators. The model effect was verified by 220 test samples. The results of entity recognition were 92.75% accuracy, 95.79% recall and 94.25 F1 score. The results of classification recognition were MacroF1 of 75.32 and Micro F1 of 90.68. The prediction accuracy of the two models after tuning increased by 8.34% and 19% respectively, and the recognition effect was significantly improved. At the same time, a total of 200 new texts in the non-sample set were used for prediction. The entity recognition accuracy was 90.14%, and the classification recognition accuracy Macro F1 was 74.8. Compared with the recognition effect of the sample test set, the accuracy rate decreased less. It can be seen that in the field of customs information text, the model has good entity extraction and classification generalization capabilities.

[0094] In this embodiment, the customs information data to be extracted is obtained, including paper documents and electronic documents, and the data sources include: news reports, customs announcements, and social media. The paper documents are scanned, image processed, and optical character recognition technology is used to convert the text information into an electronic text format. The electronic document data is preprocessed, including removing unnecessary spaces, punctuation marks, special characters, and encoding conversion. At the same time, the document is divided into chapters according to the title of the information, and each piece of information is analyzed and processed. A text extraction model with customs domain knowledge is used to perform customized structured information extraction on the customs information data. For different semantic categories such as seized information and endangered information, the corresponding structured information extraction tags that have been constructed are used. For different semantic categories of information text, a text extraction model with customs domain expertise is used, and corresponding prompt tags are used to achieve refined information extraction. In this embodiment, a text classification model with customs domain knowledge is used to perform customized text classification on the customs information data. For different semantic categories such as seized information and endangered information, the corresponding text classification labels that have been constructed are used to implement multi-label text classification for information texts of different semantic categories using a text classification model with professional knowledge in the customs field and corresponding prompt labels. In this embodiment, the information identified by the text extraction model and the text classification model are integrated and sorted to construct a complete customs information text analysis result.

[0095] For the same information text, the text extraction model and the text classification model are called successively for recognition. First, the information text and the extraction prompt label are used as the input of the text extraction model to obtain the extraction result, and then the same information text and the classification prompt label are used as the input of the text classification model to obtain the classification result, and the disassembled hierarchical relationship is recombined, and finally stored in a record. In this embodiment, the information text extraction and classification model has been used to automatically identify key information of 3,374 various unstructured information texts, which has consolidated the customs information analysis work. For example, the customs information text marking example Figure 3 As the model continues to undergo adaptive iterative training and upgrades, its intelligence and accuracy are gradually improving.

[0096] The model of the integrated text extraction and classification algorithm proposed in this embodiment can quickly and automatically process and analyze information texts, and has universality and generalization. The model has achieved good results in the application of information text analysis in the customs field. Practice shows that after enhanced training with a specific field data set, the model will have better industry adaptability and higher recognition accuracy.

[0097] Example 3: Financial information text extraction and classification based on artificial intelligence

[0098] This embodiment aims to apply the information text extraction and classification system and method based on artificial intelligence to the financial field, so as to achieve efficient and accurate processing of financial information.

[0099] Step 1: Data collection and preprocessing

[0100] Collect open source information data in the financial field, including news reports, professional reports, market analysis, etc. Preprocess the data, including removing extra spaces, punctuation marks, special characters, unifying date formats and currency units, and converting formats to meet the input and output format requirements of the language model.

[0101] Step 2: Main model construction and training

[0102] The Ernie knowledge-enhanced pre-trained model based on the domestic artificial intelligence framework is used as the main model. The multi-layer architecture of the model includes a general representation layer and a specific representation layer, where the general representation layer is used to obtain the same underlying abstract features in different task paradigms, and the specific representation layer contains specific representation modules for natural language understanding and natural language generation.

[0103] The main model is trained using preprocessed financial information data to build a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities. During the training process, the model is tuned using a multi-task unified modeling method and a multi-task unified semantic matching method.

[0104] Step 3: Text extraction and classification

[0105] The trained general text extraction model and general text classification model are applied to the financial information text to be processed. The text extraction module can automatically extract key information from financial information, such as company name, stock price, financial data, etc. The text classification module can automatically classify financial information into different categories, such as stock market dynamics, economic policies, financial market analysis, etc.

[0106] Step 4: Integration and Output

[0107] The integration module integrates the results of text extraction and text classification to generate a structured financial information analysis report. The report includes the extracted key information and classification results, as well as analysis and suggestions based on this information.

[0108] Step 5: Model evaluation and optimization

[0109] The text extraction and text classification models were evaluated, with accuracy, recall and F1 scores used as evaluation indicators for text extraction and Macro F1 and Micro F1 used as evaluation indicators for text classification. Based on the evaluation results, the models were further optimized and adjusted to improve their performance and accuracy in financial information processing.

[0110] Step 6: Domain Knowledge Injection

[0111] In order to further improve the adaptability and recognition accuracy of the model in the financial field, we collected professional knowledge, text extraction annotation data, and text classification annotation data in the financial field. We used these data to tune the general model to generate a text extraction and classification model with financial field characteristics.

[0112] The information text extraction and classification system and method based on artificial intelligence have achieved remarkable results in the financial field through the application of Example 3. The model can quickly and accurately process a large amount of financial information, extract key information and classify it, providing strong decision support for financial institutions and investors.

[0113] Example 4: Medical information text extraction and classification based on artificial intelligence

[0114] This embodiment aims to apply the information text extraction and classification system and method based on artificial intelligence to the medical field to achieve efficient and accurate processing of medical information.

[0115] Step 1: Data collection and preprocessing

[0116] Collect open source information data in the medical field, including medical research reports, clinical trial results, medical policies, etc. Preprocess the data, including removing extra spaces, punctuation, special characters, unifying medical terms and formats, and converting the formats to meet the input and output format requirements of the language model.

[0117] Step 2: Main model construction and training

[0118] The Ernie knowledge-enhanced pre-trained model based on the domestic artificial intelligence framework is used as the main model. The multi-layer architecture of the model includes a general representation layer and a specific representation layer, which can obtain the same underlying abstract features in different task paradigms and contains specific representation modules for natural language understanding and natural language generation.

[0119] The main model is trained using preprocessed medical information data to build a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities. During the training process, the model is tuned using a multi-task unified modeling method and a multi-task unified semantic matching method based on the characteristics of medical information.

[0120] Step 3: Text extraction and classification

[0121] The trained general text extraction model and general text classification model are applied to the medical information text to be processed. The text extraction module can automatically extract key information from medical information, such as disease names, drug names, treatment methods, etc. The text classification module can automatically classify medical information into different categories, such as clinical research, disease prevention, medical policy, etc.

[0122] Step 4: Integration and Output

[0123] The integration module integrates the results of text extraction and text classification to generate a structured medical information analysis report. The report includes the extracted key information and classification results, as well as analysis and suggestions based on this information. This helps medical institutions and researchers quickly understand the core content of medical information and provide support for medical decision-making and research.

[0124] Step 5: Model evaluation and optimization

[0125] The text extraction and text classification models were evaluated, with accuracy, recall and F1 scores used as evaluation indicators for text extraction and Macro F1 and Micro F1 used as evaluation indicators for text classification. Based on the evaluation results, the models were further optimized and adjusted to improve their performance and accuracy in medical information processing.

[0126] Step 6: Domain Knowledge Injection

[0127] In order to further improve the adaptability and recognition accuracy of the model in the medical field, we collected professional knowledge, text extraction annotation data, and text classification annotation data in the medical field. We used this data to tune the general model to generate a text extraction and classification model with medical field characteristics. This helps the model to more accurately understand and process professional terms and complex content in medical information.

[0128] The information text extraction and classification system and method based on artificial intelligence through the application of Example 4 has achieved remarkable results in the medical field. The model can quickly and accurately process a large amount of medical information, extract key information and classify it, providing strong decision support for medical institutions and researchers. At the same time, the versatility and generalization ability of the system also enable it to adapt to the needs of different fields and scenarios, providing strong support for the intelligent analysis of information texts.

[0129] It should be further explained that in Examples 3 and 4, during the steps of text extraction and classification, integration and output, and model evaluation and optimization, the model first uses a directional tag linking technique in the decoding layer to decode the input content into word pair links between the label and the text, and calculates the corresponding scores; then, in the output layer, the results of the main model are uniformly represented by a structured extraction language, and the extraction results of different tasks are displayed in a consistent form, and finally the association pairs of the label noun and the text content are output, and the input expression is used for the calculation and evaluation of the entire model, and the expressions involved are the same as those in Examples 1 and 2.

[0130] Embodiment 5: Cross-language information text extraction and classification system and method based on artificial intelligence

[0131] This embodiment is mainly based on the foundation of Embodiment 1, and utilizes the domestic artificial intelligence framework to perform an artificial intelligence-based information extraction and classification method for cross-language information text.

[0132] With the deepening development of globalization, the demand for processing cross-language information texts is increasing; these texts cover multiple languages, such as Chinese, English, French, etc., and often contain rich cross-domain information; however, traditional artificial intelligence text analysis technology faces huge challenges in processing cross-language information texts, which are mainly manifested in the following aspects: First, the differences in grammar, vocabulary and context between different languages ​​lead to low model adaptability; second, it is difficult to extract entities, relationships and events in cross-language texts, and it is difficult to improve the accuracy; third, existing technologies often rely on models and frameworks of specific languages, making it difficult to achieve cross-language universality and generalization capabilities.

[0133] Based on this, the present invention aims to provide a more intelligent, accurate and cross-language capable information text extraction and classification system method. This method can make full use of the advantages of the domestic artificial intelligence framework and combine cross-language natural language processing technology to achieve efficient and accurate analysis of cross-language information texts. At the same time, the method should also have good versatility and generalization capabilities, be able to adapt to the needs of different languages ​​and fields, and provide strong support for the intelligent analysis of cross-language information texts.

[0134] This embodiment mainly provides a cross-language information text extraction and classification system based on artificial intelligence, including:

[0135] a. The main model module realizes multi-task collaborative processing through its multi-layer architecture and supports a wide range of text information extraction and classification functions;

[0136] b. Preprocessing module, used to preprocess the cross-language information to be processed, including language detection and text translation;

[0137] c. a text extraction module connected to the subject model module for performing information extraction;

[0138] d. a text classification module, connected to the subject model module, for performing text classification;

[0139] e. Integration module, used to integrate the extracted information and classification results;

[0140] fCross-language adaptation module, used to enhance the adaptability and generalization ability of the model between different languages.

[0141] The method for extracting and classifying cross-language information text using the cross-language information text extraction and classification system based on artificial intelligence includes the following steps:

[0142] S1. Collect multi-language and multi-domain open source information data as training data;

[0143] S2. Preprocessing the training data, including language detection, text translation and format conversion, to meet the input and output format requirements of the language model;

[0144] S3. Build a cross-language knowledge enhancement pre-training model based on the domestic artificial intelligence framework as the main model;

[0145] S4, training the subject model using the preprocessed training data to construct a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities;

[0146] S5. Use the cross-language adaptation module to enhance the adaptability and generalization ability of the model between different languages;

[0147] S6. Apply the universal text extraction model and the universal text classification model to the cross-language information text to be processed to perform information extraction and post-text classification integration.

[0148] In this embodiment, the multi-layer architecture of the subject model includes a general representation layer and a specific representation layer. The general representation layer is used to obtain the same underlying abstract features in different task paradigms, and the specific representation layer includes specific representation modules for natural language understanding and natural language generation.

[0149] The language detection in the preprocessing step is used to identify the language type of the text, and the text translation translates the text in the non-target language into the target language to meet the needs of model processing.

[0150] The main model adopts a cross-language knowledge enhanced pre-training model based on a domestic artificial intelligence framework. Both its general representation layer and specific representation layer use the Transformer-XL structure as the backbone, and the model's generalization ability between different languages ​​is enhanced through cross-language pre-training tasks.

[0151] The general text extraction model is tuned through a multi-task unified modeling approach, the input layer uses a structural model based on a prompt mechanism, and the output layer uses a structured extraction language to represent the extraction results in a unified form.

[0152] The universal text classification model is tuned by a multi-task unified semantic matching method, and the classification task is modeled as a matching task between labels and texts. The decoding layer uses directional tag links to calculate scores, and the output layer outputs association pairs of labels and text content. The method also includes a step of evaluating the universal text extraction model and the universal text classification model, wherein text extraction uses accuracy, recall rate and F1 score as evaluation indicators, and text classification uses Macro F1 and Micro F1 as evaluation indicators. The cross-language adaptation module includes cross-language pre-training tasks, cross-language parameter sharing and cross-language fine-tuning strategies to enhance the adaptability and generalization ability of the model between different languages.

[0153] The specific execution operation flow of the system in this embodiment is as follows:

[0154] Step 1: Data collection and preprocessing

[0155] Data collection: Collect multi-language, multi-field open-source information data from multiple public data sources, including but not limited to news reports, professional reports, market analysis, etc.; ensure that the data covers multiple languages ​​such as Chinese, English, French, and multiple fields such as finance, law, and medicine;

[0156] Language detection: Use a deep learning-based language detection model to identify the language of the collected information text and mark the language type of each text.

[0157] Text translation: For texts in non-target languages ​​(such as English and French), we use advanced machine translation models to translate them into the target language (such as Chinese); ensuring that the translated texts retain the semantics and information integrity of the original text.

[0158] Format conversion: remove unnecessary spaces, punctuation marks, special characters, etc. in the text, unify the date format, currency unit, etc. to meet the input and output format requirements of the language model.

[0159] Step 2: Main model construction and training

[0160] Model selection: A cross-language knowledge-enhanced pre-training model based on a domestic artificial intelligence framework is used as the main model; this model has the ability to collaborate between multi-task paradigms and can process text data in multiple languages.

[0161] Model architecture: The multi-layer architecture of the model includes a general representation layer and a specific representation layer; the general representation layer is used to obtain the same underlying abstract features in different task paradigms, such as vocabulary information and syntactic information; the specific representation layer contains specific representation modules for natural language understanding and natural language generation, which are used to process specific language tasks.

[0162] Model training: Use preprocessed cross-language information data to train the main model. During the training process, use cross-language pre-training tasks to enhance the model's generalization ability between different languages. Through word perception, structure perception and knowledge perception pre-training tasks, the model is equipped with the ability to understand, generate and reason.

[0163] Step 3: Cross-language adaptation

[0164] Cross-language pre-training tasks: During the training process, cross-language pre-training tasks are introduced to enable the model to be exposed to text data in multiple languages ​​during training; through strategies such as cross-language shared vocabulary and cross-language masked language models, the model's adaptability to different languages ​​is improved.

[0165] Cross-language parameter sharing: Share some parameters between different language branches of the model, such as the parameters of the common representation layer; through parameter sharing, the parameter differences between different languages ​​are reduced and the generalization ability of the model is improved.

[0166] Cross-language fine-tuning strategy: fine-tune the model on the target language data; during the fine-tuning process, keep the parameters of the general representation layer unchanged and only adjust the parameters of the specific representation layer; through fine-tuning, make the model better adapt to the characteristics of the target language.

[0167] Step 4: Text extraction and classification

[0168] Text extraction: The cross-language information text to be processed is input into the trained general text extraction model; the model automatically extracts key information from the text, such as company name, stock price, disease name, etc., and represents it in a structured form.

[0169] Text classification: The cross-language information text to be processed is input into the trained general text classification model; the model automatically classifies the text into different categories, such as stock market trends, economic policies, clinical research, etc., and outputs the associated pairs of labels and text content.

[0170] Step 5: Integration and Output

[0171] Information integration: Integrate the results of text extraction and text classification; sort and combine the extracted key information and classification results in a certain format to generate a structured cross-language information text analysis report.

[0172] Report output: The integrated analysis report is output to the user; the report includes the extracted key information, classification results, and analysis and suggestions based on this information, which helps users quickly understand the core content of cross-language information texts.

[0173] Step 6: Model evaluation and optimization

[0174] Model evaluation: Perform model evaluation on text extraction and text classification models; text extraction uses accuracy, recall and F1 score as evaluation indicators; text classification uses Macro F1 and Micro F1 as evaluation indicators; the performance of the model is quantitatively evaluated through evaluation indicators.

[0175] Model optimization: Optimize and adjust the model based on the evaluation results; improve the performance and accuracy of the model in cross-language information text processing by adjusting model parameters and optimizing training strategies; at the same time, continuously collect new cross-language information data to update and iteratively train the model to adapt to the ever-changing language environment and text features.

[0176] When this embodiment is used for processing Chinese-English cross-language information text, first, according to the needs of Chinese-English cross-language information text processing, data such as news reports and professional reports in Chinese and English are collected as training data; in the preprocessing stage, a language detection model is used to identify the language type of the text, and a machine translation model is used to translate the English text into Chinese; then, the processed data is used to train the cross-language knowledge enhancement pre-training model; in the text extraction and classification stage, the Chinese-English cross-language information text to be processed is input into the model, key information is extracted and classified; finally, the extraction and classification results are integrated to generate a structured analysis report.

[0177] When this embodiment is aimed at processing multilingual financial information text, first, according to the multilingual financial information text processing needs, financial information data including Chinese, English, French and other languages ​​are collected as training data; in the preprocessing stage, the text is subjected to language detection, translation and format conversion; then, the processed data is used to train the cross-language knowledge enhancement pre-training model; in the text extraction and classification stage, the multilingual financial information text to be processed is input into the model, key information such as company name and stock price is extracted, and classification is performed on stock market dynamics, economic policies, etc.; finally, the extraction and classification results are integrated to generate a structured financial information analysis report.

[0178] It can be seen that when this embodiment realizes cross-language text information extraction and classification in Chinese, combining natural language processing (NLP) technology and cross-language processing strategy is the key and end point of software execution. Based on this, this embodiment will propose a detailed step to enable the operator to achieve this goal in a Chinese environment:

[0179] 1. Data collection and preprocessing

[0180] Data collection: Collect cross-language text data from multiple channels, including Chinese and other target languages ​​(such as English, French, etc.); ensure that the data covers multiple fields to improve the generalization ability of the model.

[0181] Preprocessing:

[0182] Language Detection: Use the language detection tool to identify the language type of the text.

[0183] Text Translation: For non-Chinese texts, use high-quality machine translation models to translate them into Chinese.

[0184] Unify the format: remove extra spaces, punctuation marks, and special characters from the text, unify the date format, currency unit, etc., to ensure consistency in the text format.

[0185] 2. Model selection and construction

[0186] Choose a model: Choose a pre-trained model based on the Transformer architecture, such as BERT, ERNIE, etc. These models perform well when processing Chinese text; ensure that the model supports cross-lingual processing or is enhanced by cross-lingual pre-training tasks.

[0187] Build the model:

[0188] Multi-layer architecture: The multi-layer architecture of the design model includes a general representation layer and a specific representation layer; the general representation layer is used to obtain the underlying abstract features of texts in different languages, and the specific representation layer is used to handle language-specific tasks.

[0189] Cross-language adaptation: Add cross-language adaptation modules to the model, such as cross-language parameter sharing and cross-language fine-tuning strategies, to improve the model's generalization ability across different languages.

[0190] 3. Model training and optimization

[0191] Training data preparation: Use the preprocessed cross-language text data to train the model; ensure that the training data contains sufficient Chinese and other target language texts.

[0192] Model training: A multi-task unified modeling approach is used for model training, while optimizing text extraction and text classification tasks. During the training process, cross-language pre-training tasks are introduced, such as cross-language masked language models and cross-language shared vocabularies, to enhance the model's cross-language processing capabilities.

[0193] Model optimization: Use indicators such as accuracy, recall, and F1 score to evaluate the text extraction model; use indicators such as Macro F1 and Micro F1 to evaluate the text classification model; and tune the model based on the evaluation results, such as adjusting model parameters and optimizing training strategies.

[0194] 4. Information extraction and classification

[0195] Text extraction: Translate the cross-language text to be processed into Chinese (if it has not been translated yet); input the Chinese text into the trained text extraction model to extract key information such as entities, relations, events, etc.

[0196] Text classification: Input Chinese text into the trained text classification model and classify it into predefined categories; output the classification results, including the associated pairs of labels and text content.

[0197] 5. Results integration and output

[0198] Result integration integrates the results of text extraction and text classification to generate a structured analysis report; the extracted key information and classification results are sorted and combined according to a certain format.

[0199] Report output: Output the integrated analysis report in a user-friendly format, such as PDF, Excel, etc.; provide further analysis and suggestions based on the extracted information and classification results.

[0200] 6. Cross-language processing strategy

[0201] Cross-language vocabulary mapping: Establish a cross-language vocabulary mapping table to map vocabulary from different languages ​​into a unified semantic space; during model training and use, use the vocabulary mapping table to enhance cross-language processing capabilities.

[0202] Cross-language semantic understanding: Use cross-language semantic understanding models, such as multilingual BERT and XLM-R, to improve the model's ability to understand the semantics of texts in different languages; introduce cross-language semantic understanding mechanisms in text extraction and classification tasks to improve the accuracy of cross-language processing.

[0203] 7. Continuous iteration and optimization

[0204] Data update: Regularly collect new cross-language text data to update and iteratively train the model; ensure that the model can handle the ever-changing language environment and text features.

[0205] Model improvement: Improve and optimize the model according to actual application needs; introduce new cross-language processing technologies and strategies to improve the performance and generalization ability of the model; through the above steps, operators can realize cross-language text information extraction and classification in a Chinese environment; this not only helps to improve the accuracy and efficiency of cross-language text processing, but also provides strong support for information exchange and understanding between different languages.

[0206] Through the application of this embodiment 5 and its specific implementation methods, the artificial intelligence-based cross-language information text extraction and classification system and method provided by the present invention can efficiently and accurately process multilingual information texts, extract key information and classify them, and provide strong support for the intelligent analysis of cross-language information texts; at the same time, the versatility and generalization ability of the system also enable it to adapt to the needs of different languages ​​and fields, and has broad application prospects.

Claims

1. An information text extraction and classification system based on artificial intelligence, characterized in that: include: a. The main model module realizes multi-task collaborative processing through its multi-layer architecture and supports a wide range of text information extraction and classification functions; b. Pre-processing module, used to pre-process the customs information to be processed; c. a text extraction module connected to the subject model module for performing information extraction; d. a text classification module, connected to the subject model module, for performing text classification; e. Integration module, used to integrate the extracted information and classification results.

2. An information text extraction and classification method based on artificial intelligence, characterized by: The following steps are involved: S1. Collect open source information data from multiple fields as training data; S2. Preprocessing the training data to meet the input and output format requirements of the language model; S3. Build a knowledge-enhanced pre-training model based on the domestic artificial intelligence framework as the main model; S4, training the subject model using the preprocessed training data to construct a general text extraction model with extensive text processing and information extraction capabilities and a general text classification model with extensive text classification capabilities; S5. Apply the general text extraction model and the general text classification model to the information text to be processed to perform information extraction and text classification and post-integration.

3. The information text extraction and classification system method based on artificial intelligence according to claim 1 or 2 is characterized in that: The multi-layer architecture of the subject model includes a general representation layer and a specific representation layer. The general representation layer is used to obtain the same underlying abstract features in different task paradigms, and the specific representation layer contains specific representation modules for natural language understanding and natural language generation.

4. The information text extraction and classification system method based on artificial intelligence according to claim 1 or 2, characterized in that: The preprocessing step includes removing redundant spaces, punctuation marks, special characters, unifying date formats and currency units, and performing format conversion.

5. The information text extraction and classification system method based on artificial intelligence according to claim 1 or 2, characterized in that: The main model adopts the Ernie knowledge-enhanced pre-training model, and its general representation layer and specific representation layer both use the Transformer-XL structure as the backbone, wherein the general representation layer contains 48 Transformer layers, 4096 hidden units and 64 attention heads, and the specific representation layer contains 12 Transformer layers, 768 hidden units and 12 attention heads.

6. The information text extraction and classification method based on artificial intelligence according to claim 2 is characterized by: The general text extraction model is tuned through a multi-task unified modeling approach, the input layer uses a structural model based on a prompt mechanism, and the output layer uses a structured extraction language to represent the extraction results in a unified form.

7. The information text extraction and classification method based on artificial intelligence according to claim 2 is characterized by: The general text classification model is tuned through a multi-task unified semantic matching method, and the classification task is modeled as a matching task between labels and texts. The decoding layer uses directed label links to calculate scores, and the output layer outputs associated pairs of labels and text content.

8. The information text extraction and classification method based on artificial intelligence according to claim 2 is characterized by: The method also includes a step of evaluating the general text extraction model and the general text classification model, wherein the text extraction uses accuracy, recall rate and F1 score as evaluation indicators, and the text classification uses Macro F1 and Micro F1 as evaluation indicators.

9. The information text extraction and classification method based on artificial intelligence according to claim 2 is characterized by: It also includes steps of injecting domain-specific expertise into general text extraction models and general text classification models to obtain text extraction and classification models with domain characteristics.

10. The method for intelligent extraction and classification of information text based on artificial intelligence according to claim 9, characterized in that: The specific field is the customs field, the professional knowledge includes tariff policies, trade regulations, inspection and quarantine systems, customs laws and professional terms, and the annotated data includes customs information text extraction annotated data and text classification annotated data.

Images