A method and device for predicting scholar's title and age based on MT5 pre-training model

Through the method based on the MT5 pre-trained model, scholar information is crawled and cleaned, and the professional title and age output are generated using the fine-tuning model, which solves the problem of low efficiency and accuracy of scholar information prediction in the existing technology, and achieves higher prediction results.

CN114297366BActive Publication Date: 2024-12-03BEIJING KNOWLEDGE ATLAS TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111371097.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-18
Publication Date
2024-12-03
Estimated Expiration
2041-11-18

AI Technical Summary

Technical Problem

It is difficult for the prior art to accurately predict the title and age of scholars, especially in Internet data where scholars have unstructured information and complex semantic relationships. Traditional methods cannot be effectively utilized, resulting in low prediction efficiency and accuracy.

Method used

The method based on the MT5 pre-trained model is adopted to crawl the web page information related to the scholar's name and organization, clean text and feature extraction, and use the fine-tuned MT5 model to generate the output results of the scholar's title and age to improve the prediction accuracy.

Benefits of technology

By denoising and cleaning text content, the MT5 model has been used to significantly improve the prediction effect of scholars' professional title and age, achieving higher accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114297366B_ABST
    Figure CN114297366B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for predicting the academic title and age of scholars based on an MT5 pre-training model, wherein the method includes: crawling web search results according to the annotated scholar's name and institution to obtain training text; cleaning the text data of the training text to extract text information, wherein, for abnormal text exceeding the preset length of the model, RNN feature extraction is performed to obtain a first text vector with dimension matching; the content of the normal text is serialized to be converted into a second text vector, and the first text vector is simultaneously input into the fine-tuned MT5 pre-training model to train and generate corresponding text answers; the text answer is structured, and the trained model is evaluated to generate the output results of the academic title and age; the output results are presented to the user using a web page framework to complete the model engineering deployment. The present invention greatly improves the prediction effect of the academic title and age by denoising and cleaning the text-irrelevant content and using the training model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of information extraction, and in particular to a method and device for predicting scholar's professional title and age based on an MT5 pre-training model. Background Art

[0002] At present, the information resources on the Internet are rich, often unstructured and diverse in types, making it difficult to accurately retrieve relevant record information. For example, a certain type of event can be classified according to the location where it occurred, which cannot be accomplished by search engines. Therefore, there is an urgent need for technical means to quickly and accurately obtain information, and thus information extraction technology has emerged. Taking the academic circle as an example, there are hundreds of millions of experts and scholars in the world, and most of the data of these experts and scholars are presented on the Internet in heterogeneous and unstructured forms, which contain a large amount of valuable data. Researching the basic information of scholars provides an objective basis and auxiliary decision-making support for the accurate positioning of scholars and the introduction of scholar talents.

[0003] The basic information of scholars, such as age, education, professional title, work unit, etc., is of great significance to individuals or institutions. Therefore, how to extract valuable field information from massive and messy data has become a hot issue of concern in academia and industry.

[0004] In the related art, the conventional method of predicting professional titles and ages mainly relies on information such as scholars' personal relationships and academic achievements. However, the applicant found that in actual scenarios, most scholars do not have clear basic information, and the semantic relationships of web page information are relatively complex. For example: it is difficult for scholars to retrieve information on personal relationships and the time of publication of achievements, and the scholar's professional title or age is not mentioned in the web search text, only relevant information such as his work experience and study experience is provided. In this way, the complex semantic relationships cannot be learned according to traditional feature engineering and classification algorithms, resulting in the inability to accurately judge their professional titles and ages. In addition, conventional methods usually use their fixed scientific research results or relationship information, which is not real-time. Therefore, there is an urgent need for a solution that can improve the accuracy and efficiency of scholars' professional title and age prediction. Summary of the invention

[0005] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the purpose of the present invention is to use the scholar information crawled according to the scholar name and institution search results as training text, and train the information extraction through the MT5 multilingual generative model to predict the scholar's title and age. A scholar title and age prediction method based on the MT5 pre-training model is proposed.

[0007] Another object of the present invention is to provide a device for predicting scholar's professional title and age based on the MT5 pre-training model.

[0008] To achieve the above-mentioned purpose, the present invention proposes a method for predicting academic titles and ages based on the MT5 pre-training model, comprising the following steps:

[0009] S1, crawls web search results according to the annotated scholar names and institutions to obtain training texts, and stores the training texts in a local database;

[0010] S2, based on the training text, the text data of the training text is cleaned to extract the text information; wherein, for the abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same dimension as the model; the text information includes: the education experience, work experience, and research experience information in the scholar's profile;

[0011] S3, serializing the content of the normal text to convert it into a second text vector, and inputting the first text vector and the second text vector into the fine-tuned MT5 pre-trained model for model training to train and generate corresponding text answers;

[0012] S4, converts the text answers into structured data and evaluates the trained model to generate output results of scholars’ titles and ages;

[0013] S5, present the output results to the user using the web page framework to complete the model engineering deployment.

[0014] The scholar's title and age prediction method based on the MT5 pre-training model of the embodiment of the present invention, crawls web search results according to the annotated scholar's name and institution to obtain training text; the text data of the training text is cleaned to extract text information, wherein, for abnormal text exceeding the preset length of the model text, RNN feature extraction is performed to obtain the first text vector of dimension matching; the content of the normal text is serialized to be converted into a second text vector, and the fine-tuned MT5 pre-training model is input together with the first text vector at the same time, and the training generates the corresponding text answer; the text answer is structured, and the trained model is evaluated to generate the output result of the scholar's title and age; the output result is presented to the user using a web page framework to complete the model engineering deployment. The present invention greatly improves the prediction effect of the scholar's title and age by denoising and cleaning the text-irrelevant content and using the training model.

[0015] In addition, the scholar title and age prediction method based on the MT5 pre-training model according to the above embodiment of the present invention may also have the following additional technical features:

[0016] Furthermore, in step S2, the text data of the training text is cleaned to extract text information, including:

[0017] The text data is cleaned through regular expressions and noise reduction to extract text information. Among them, the detailed content of the web page is processed by noise reduction, which uses Trafilatura to capture the preset information content of the web page while retaining the structure, and based on the combination of DOM inspection, xpath expression and rules, the preset information content text is extracted and noise is deleted.

[0018] Furthermore, in step S2, RNN feature extraction is performed on abnormal text in the text information that exceeds the preset length of the MT5 pre-trained model, and a first text vector with the same dimension as the model is output, including:

[0019] The default sequence of text content vectors is {z1,z2,z3,...,z 3565}, extract the semantic features of the text content vector sequence through the recurrent neural network RNN, connect the semantic features to the 3565*1024 fully connected layer, and get the vector dimension of {z1,z2,z3,...,z 1024}, the formula is as follows:

[0020] a <t>< / t> =tanh(W ax x 〈t> +W aa a 〈t-1〉 +b a )

[0021]

[0022] Calculate the hidden layer a and predicted value y at the t-th time step, and combine the two sets of parameters with the activation a of the previous layer and the current data x to obtain the first text vector, as shown in the following formula:

[0023] a 〈t〉 =tanh(H a [a 〈t-1 〉,x 〈t〉 ]+b a )

[0024] The output dimension of the first text vector is adjusted to the dimension of the model embedding through the fully connected layer.

[0025] Further, step S3 includes:

[0026] S3.1, serialize the text content, use T5Tokenizer to perform word segmentation and pad_token connection between the name, organization and text content to distinguish the semantic key information, and save the text sequence after word segmentation to the local database;

[0027] S3.2, based on the text sequence after word segmentation, load the training text data saved in S1 to fine-tune the MT5 pre-trained model.

[0028] Further, step S3.2 includes:

[0029] S3.21, fine-tune the calculation loss and attention mask parts of the MT5 pre-trained model to calculate the loss of generated content;

[0030] S3.22, load the fine-tuned MT5 pre-trained model, divide the text into training set, test set and validation set, and pad the batch data;

[0031] S3.23, read the vocabulary of the MT5 pre-trained model, encode the concatenated question templates through the vocabulary, and input the generated encoding list into the fine-tuned MT5 pre-trained model;

[0032] S3.24, generating a score for each word in the vocabulary through the fine-tuned MT5 pre-trained model, and selecting multiple words in the vocabulary whose probability exceeds the threshold, and obtaining the generated results to predict the next word;

[0033] S3.25, based on the generated results, calculate the cross entropy loss between the vocabulary with the maximum probability value and the actual result;

[0034] S3.26, adjust the model parameters and repeat S3.23-S3.25 until the model converges.

[0035] Further, step S4 includes:

[0036] S4.1, load the trained MT5 pre-trained model, input the test text data, perform word segmentation on the test text data, and input it into the trained MT5 pre-trained model to obtain the output result;

[0037] S4.2, based on the output results, the BLEU and ROUGE of the answer generated by the fine-tuned MT5 pre-trained model are calculated through the formula.

[0038] Furthermore, the formulas for calculating the BLEU and ROUGE of the answers generated by the fine-tuned MT5 pre-trained model include:

[0039]

[0040] in,

[0041]

[0042]

[0043]

[0044]

[0045] Where LCS(C,S) is the length of the longest common subsequence of C and S, m and n represent the length of the manual standard summary and the machine automatic summary respectively, R LCS ,P LCS Represent the recall rate and precision rate, F LCS It is Rouge-L, as a comprehensive evaluation index.

[0046] Optionally, step S5 includes:

[0047] S5.1, load the trained MT5 pre-trained model to encapsulate it into a model interface;

[0048] S5.2, obtain the scholar’s ​​name and institution information input by the front-end user through the flask framework;

[0049] S5.3, automatically pass the input content through the scrapy framework, obtain the search web page and parse it;

[0050] S5.4, repeat the above text preprocessing and word segmentation process to process the data into the model input format;

[0051] S5.5, based on the packaged model interface, the server visualizes the model to the user, completing the model engineering deployment.

[0052] Furthermore, step S1 includes:

[0053] S1.1, conduct a joint search of the scholar’s ​​name and institution through a search engine to obtain multiple web pages;

[0054] S1.2, parse the contents of multiple web pages through xpath to obtain summary information and detailed text content.

[0055] To achieve the above object, the present invention proposes, on the other hand, a device for predicting academic titles and ages based on the MT5 pre-training model, comprising:

[0056] A web data crawling module is used to crawl web search results according to the annotated scholar's name and institution to obtain training texts, and store the training texts in a local database;

[0057] The data preprocessing module is used to clean the text data of the training text based on the training text to extract the text information; wherein, for the abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output the first text vector with the same dimension as the model; the text information includes: the education experience, work experience, and research experience information in the scholar's profile;

[0058] The model training module is used to serialize the content of the normal text to convert it into a second text vector, and input the first text vector and the second text vector into the fine-tuned MT5 pre-trained model for model training to train and generate corresponding text answers;

[0059] The result generation module is used to convert text answers into structured data and evaluate the trained model to generate output results of scholars' titles and ages;

[0060] The engineering deployment module is used to present the output results to users using a web page framework to complete the engineering deployment of the model.

[0061] The embodiment of the present invention provides a scholar's professional title and age prediction device based on an MT5 pre-training model, a webpage data crawling module, used to crawl webpage search results according to the annotated scholar's name and organization to obtain training text, and store the training text in a local database; a data preprocessing module, used to clean the text data of the training text based on the training text to extract text information; wherein, for abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same model dimension; the text information includes: education experience, work experience, and research experience information in the scholar's profile; a model training module, used to serialize the content of the normal text to convert it into a second text vector, and input the first text vector and the second text vector into a fine-tuned MT5 pre-training model for model training to train and generate corresponding text answers; a result generation module, used to form structured data from the text answer, and evaluate the trained model to generate output results of the scholar's professional title and age; an engineering deployment module, used to present the output results to users using a webpage framework to complete the model engineering deployment. The present invention can establish an MT5 pre-training model for predicting academic titles and ages, improve the prediction effect, and display the output results on a web page.

[0062] Beneficial effects of the present invention:

[0063] The present invention utilizes the search capability of Internet engines to reduce noise and clean irrelevant content in texts, and uses the names and institutions of scholars to predict professional titles and ages, thereby greatly improving the prediction effect of scholars' professional titles and ages.

[0064] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0066] Figure 1 A flowchart of a method for predicting scholar's professional title and age based on an MT5 pre-training model according to an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of a framework of a scholar's professional title and age prediction algorithm based on an MT5 pre-training model according to an embodiment of the present invention;

[0068] Figure 3 The present invention is a schematic diagram of the structure of a device for predicting scholar's professional title and age based on the MT5 pre-training model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0070] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0071] The following describes a method and device for predicting scholar's title and age based on the MT5 pre-training model according to an embodiment of the present invention with reference to the accompanying drawings. First, the method for predicting scholar's title and age based on the MT5 pre-training model according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0072] Figure 1 It is a flowchart of a method for predicting scholar's professional title and age based on MT5 pre-training model according to an embodiment of the present invention.

[0073] like Figure 1 As shown, the scholar title and age prediction method based on the MT5 pre-training model includes the following steps:

[0074] S1, crawls web search results according to the annotated scholar names and institutions to obtain training texts, and stores the training texts in the local database.

[0075] Specifically, we use the scrapy framework to search for the names and institutions of a group of scholars through the Google search engine. The search method mainly uses the address + name + institution concatenation method. Then we use xpath and regular expressions to parse the web page text content, obtain its summary information and detailed text content information, and save them in csv.

[0076] S2, based on the training text, clean the text data of the training text to extract the text information; wherein, for the abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same dimension as the model; the text information includes: education experience, work experience, and research experience information in the scholar's profile.

[0077] Specifically, the parsed web page details are processed for noise reduction. The Trafilatura module is used to extract the main text of the web page while retaining some structure. Based on the combination of DOM inspection, XPath expression and rules, the main text content is robustly extracted and noise removed. The text containing the scholar's research experience, education experience and resume is saved locally for the prediction of the scholar's title and age.

[0078] Specifically, for abnormal text in the text information that exceeds the preset length of the MT5 pre-trained model, RNN feature extraction is performed to output a first text vector with the same dimension as the model, including:

[0079] In the cleaned text, some texts are found to be too long beyond the model input length. If they are directly truncated here, it will affect the model's reasoning and semantic understanding. Therefore, the processing method is as follows: Assume that the obtained text content vector sequence is {z1,z2,z3,...,z 3565} After extracting the semantic features through the recurrent neural network RNN, and then connecting it to the 3565*1024 fully connected layer, we get a two-dimensional {z1,z2,z3,...,z 1024}. The formula is as follows:

[0080] a 〈t〉 =tanh(W ax x 〈t〉 +W aa a 〈t-1〉 +b a )

[0081]

[0082] Calculate the hidden layer a and predicted value y at the t-th time step, and combine the two sets of parameters with the activation a of the previous layer and the current data x. The following formula:

[0083] a 〈t〉 =tanh(H a [a 〈t-1〉 ,x 〈t〉 ]+b a )

[0084] Then, the output dimension of the final vector is adjusted to the dimension of the model embedding through the fully connected layer. After completing the above long text data preprocessing, the long text data is saved in a pkl file.

[0085] S3, serialize the content of the normal text to convert it into a second text vector, and input the first text vector and the second text vector into the fine-tuned MT5 pre-trained model for model training to train and generate corresponding text answers.

[0086] Specifically, in S3, including S3.1, the saved text content is serialized, multilingual word segmentation is performed using T5Tokenizer, and the name, organization, and text content are connected by pad_token, and processed into the following format: [CLS]+name+[PAD]+org+[PAD]+text+[SEP]; this enables the model to distinguish semantically important information during learning. Subsequently, the text sequence with word segmentation and the long text information are saved together, which is now the input of the model.

[0087] S3.2, then load the saved training data and fine-tune the pre-trained model. This mainly includes:

[0088] S3.2.1, fine-tune the calculation loss and attention mask part of the pre-trained model so that it only calculates the loss of the generated content and ignores characters such as MASK, CLS and PAD.

[0089] S3.2.2, load the fine-tuned pre-trained model, divide the text into training set, test set, and validation set, and pad the batch data to align all text data.

[0090] S3.2.3, reading the vocabulary of the MT5 pre-trained model, encoding the concatenated question templates using the vocabulary, and inputting the generated encoding list into the training model;

[0091] S3.2.4, generating a score for each word in the vocabulary through the MT5 pre-trained model, and selecting the top 10 words with the highest probability in the vocabulary to predict the next word.

[0092] S3.2.5, after obtaining the generated results, calculate the vocabulary score and the actual result to calculate the cross entropy loss.

[0093] S3.2.6, adjust the model parameters and repeat steps S3.2.3-S3.2.5 until the model converges and the loss value no longer decreases.

[0094] As an example, the present invention serializes the stored text content, uses T5Tokenizer to perform multilingual word segmentation, and connects the name, organization and text content with pad_token to process it into the following format: [CLS]+name+[PAD]+org+[PAD]+text+[SEP]; this enables the model to distinguish semantically important information during learning.

[0095] As an example, the present invention saves the text sequence and long text information of the word segmentation together as the input of the model. The MT5-base version pre-trained model is selected, and it is first fine-tuned, including fine-tuning the calculation loss and attention mask parts, so that it only calculates the loss of the generated content, and ignores characters such as MASK, CLS and PAD. Load the fine-tuned pre-trained model, and divide the text into training set, test set, and verification set according to 8:1:1, and pad the batch data to align all text data. And process the data in np-array format into tensor. Take the vocabulary of the MT5 pre-trained model, encode the spliced ​​question template through the vocabulary, and input the generated encoding list into the training model; generate the score of each word in the vocabulary through the MT5 pre-trained model, and select the top 10 words with the highest probability in the vocabulary to predict the next word. Calculate the cross entropy loss by calculating the vocabulary score and the actual result, and continue training until the model loss no longer decreases.

[0096] S4, converts the text answers into structured data and evaluates the trained model to generate output results of scholars’ titles and ages.

[0097] Specifically, load the above trained generative model and input the test text data. Perform word segmentation on the text data as in the above training data and input it into the model. After obtaining the output results, calculate the BLEU and ROUGE of the answer generated by the fine-tuned pre-trained model using the following formula.

[0098]

[0099] in,

[0100]

[0101]

[0102]

[0103]

[0104] Where LCS(C,S) is the length of the longest common subsequence of C and S, m and n represent the length of the manual standard summary and the machine automatic summary respectively (generally the number of words contained), R LCS , P LCS Represent the recall rate and precision rate respectively. The final F LCs This is what we call Rouge-L. As a comprehensive evaluation indicator, we can get the training effect of the model and evaluate the model.

[0105] S5, present the output results to the user using the web page framework to complete the model engineering deployment.

[0106] Specifically, take the trained model to a web server with access rights, load the pre-trained model and vocabulary information, and encapsulate it into an interface. Obtain the scholar's name and institution information entered by the front-end user through the request module in the flask framework. Automatically pass the input content through the scrapy framework to the search web page and parse it. Repeat the above text preprocessing and word segmentation process to process the data into the model input format. Through the encapsulated model interface, it is visually displayed to the user through the server.

[0107] Figure 2 It is a framework diagram of a scholar's professional title and age prediction algorithm based on an MT5 pre-training model according to an embodiment of the present invention, and is a process-based display of the above steps.

[0108] Through the above steps, the acquisition of web search text data based on the name of the scholar and the information of the institution is realized, the text data is cleaned by regular expression and noise reduction method, and the text information that may contain research experience, work experience, character profile, etc. is extracted, and the overlong text exceeding the MT5 pre-training model is subjected to RNN feature extraction, and converted into the embedding dimension of the model input; the text content is serialized, and the text is input into the fine-tuned MT5 multilingual generative training model, and the answers to each round of questions are formed into structured data, and the trained model is evaluated; the model output results are presented to the user using the web page framework to realize engineering deployment. The present invention utilizes the retrieval ability of the Internet engine, de-noises and cleans the text irrelevant content, predicts the correct answer, and greatly improves the prediction effect of the scholar's title and age.

[0109] It should be noted that there are many ways to implement the scholar title and age prediction method, but no matter what the specific implementation method is, as long as the method solves the problem of scholar title and age prediction and improves the prediction effect, it is a solution to the existing technical problems and has corresponding effects.

[0110] In order to implement the above embodiment, Figure 3 As shown, this embodiment also provides a device 10 for predicting academic titles and ages based on the MT5 pre-trained model. The device 10 includes: a web page data crawling module 100, a data preprocessing module 200, a model training module 300, a result generation module 400 and an engineering deployment module 500.

[0111] The webpage data crawling module 100 is used to crawl the webpage search results according to the annotated scholar's name and institution to obtain the training text, and store the training text in the local database;

[0112] The data preprocessing module 200 is used to clean the text data of the training text based on the training text to extract the text information; wherein, for the abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same dimension as the model; the text information includes: the education experience, work experience, and research experience information in the scholar's profile;

[0113] The model training module 300 is used to serialize the content of the normal text to convert it into a second text vector, and input the first text vector and the second text vector into the fine-tuned MT5 pre-training model for model training to train and generate corresponding text answers;

[0114] A result generation module 400 is used to convert the text answers into structured data and evaluate the trained model to generate output results of the scholar's title and age;

[0115] The engineering deployment module 500 is used to present the output results to the user using a web page framework to complete the engineering deployment of the model.

[0116] According to the scholar's professional title and age prediction device based on the MT5 pre-training model of the embodiment of the present invention, a web data crawling module is used to crawl web search results according to the annotated scholar's name and institution to obtain training text, and store the training text in a local database; a data preprocessing module is used to clean the text data of the training text based on the training text to extract text information; wherein, for abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same model dimension; the text information includes: education experience, work experience, and research experience information in the scholar's profile; a model training module is used to serialize the content of the normal text to convert it into a second text vector, and input the first text vector and the second text vector into the fine-tuned MT5 pre-training model for model training to train and generate corresponding text answers; a result generation module is used to form structured data from the text answer, and evaluate the trained model to generate output results of the scholar's professional title and age; an engineering deployment module is used to present the output results to the user using a web page framework to complete the model engineering deployment. The present invention can train the MT5 multilingual generative model for predicting academic titles and ages, improves the accuracy and efficiency of predictions, and can present the output results to users using a web page framework.

[0117] It should be noted that the aforementioned explanation of the embodiment of the method for predicting scholar's title and age based on the MT5 pre-training model is also applicable to the device for predicting scholar's title and age based on the MT5 pre-training model in this embodiment, and will not be repeated here.

[0118] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0120] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for predicting academic titles and ages based on the MT5 pre-training model, characterized in that: The following steps are involved: S1, crawling web page search results according to the annotated scholar names and institutions to obtain training texts, and storing the training texts in a local database; S2, based on the training text, cleaning the text data of the training text to extract text information; wherein, for the abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same dimension as the model; the text information includes: education experience, work experience, and research experience information in the scholar's profile; S3, serializing the content of the normal text to convert it into a second text vector, and inputting the first text vector and the second text vector into the fine-tuned MT5 pre-trained model for model training to train and generate corresponding text answers; S4, forming the text answer into structured data, and evaluating the MT5 pre-trained model to generate output results of the scholar's title and age; S5, presenting the output result to the user using a web page framework to complete the engineering deployment of the model; The S3 includes: S3.1, serialize the text content, use T5Tokenizer to perform word segmentation and pad_token connection of the name, organization and text content to distinguish the semantic key information, and save the text sequence after word segmentation to the local database; S3.2, based on the text sequence after word segmentation, loading the training text to fine-tune the MT5 pre-training model; S3.2, including: S3.21, fine-tune the calculation loss and attention mask parts of the MT5 pre-trained model to calculate the loss of generated content; S3.22, load the fine-tuned MT5 pre-trained model, divide the text into training set, test set and validation set, and pad the batch data; S3.23, reading the vocabulary of the MT5 pre-trained model, encoding the spliced ​​question templates using the vocabulary, and inputting the generated encoding list into the fine-tuned MT5 pre-trained model; S3.24, generating a score for each word in the vocabulary through the fine-tuned MT5 pre-trained model, and selecting multiple words in the vocabulary whose probability exceeds the threshold, and obtaining the generated results to predict the next word; S3.25, based on the generated results, calculate the cross entropy loss between the vocabulary with the maximum probability value and the actual result; S3.26, adjust the model parameters and repeat S3.23-S3.25 until the model converges.

2. The method for predicting scholar's professional title and age based on MT5 pre-training model according to claim 1, characterized in that: The text data of the training text is cleaned to extract text information, including: The text data is cleaned by regular expression and noise reduction to extract text information; wherein, the detailed content of the web page is processed by noise reduction, which is to use Trafilatura to capture the preset information volume text of the web page while retaining the structure, and based on the combination of DOM inspection, xpath expression and rules, the preset information volume text is extracted and noise is deleted.

3. The scholar title and age prediction method based on the MT5 pre-training model according to claim 1 is characterized in that: The method of performing RNN feature extraction on the abnormal text in the text information that exceeds the preset length of the MT5 pre-trained model and outputting a first text vector with the same dimension as the model includes: The default sequence of text content vectors is {z1,z2,z3,...,z 3565 }, extract the semantic features of the text content vector sequence through the recurrent neural network RNN, connect the semantic features to the 3565*1024 fully connected layer, and obtain a vector with dimensions {z1,z2,z3,...,z 1024 }, the formula is as follows: a <t>< / t> =tanh(W ax x <t>< / t> +W aa a <t-1>< / t-1> +b a ) Calculate the hidden layer a and predicted value y at the t-th time step, and combine the two sets of parameters with the activation a of the previous layer and the current data x to obtain the first text vector, as shown in the following formula: a <t>< / t> =tanh(H a [a <t-1>< / t-1> ,x <t>< / t> ]+b a ) The output dimension of the first text vector is adjusted to the dimension of the model embedding through the fully connected layer.

4. The method for predicting scholar's professional title and age based on MT5 pre-training model according to claim 1, characterized in that: The S4 comprises: S4.1, load the trained MT5 pre-trained model, input the test text data, perform word segmentation on the test text data, and input it into the trained MT5 pre-trained model to obtain an output result; S4.2, based on the output result, the BLEU and ROUGE of the answer generated by the fine-tuned MT5 pre-trained model are calculated by formula.

5. The method for predicting scholar's professional title and age based on MT5 pre-training model according to claim 4 is characterized in that: The formula is: in, Where LCS(C,S) is the length of the longest common subsequence of C and S, m and n represent the length of the manual standard summary and the machine automatic summary respectively, R LCS ,P LCS Represent the recall rate and precision rate, F LCS It is Rouge-L, as a comprehensive evaluation index.

6. The method for predicting scholar's professional title and age based on MT5 pre-training model according to claim 5, characterized in that: The method further includes: S5, S5.1, loading the trained MT5 pre-trained model to encapsulate it into a model interface; S5.2, obtain the scholar’s ​​name and institution information input by the front-end user through the flask framework; S5.3, automatically pass the input content through the scrapy framework, obtain the search web page and parse it; S5.4, repeat the above text preprocessing and word segmentation process to process the data into the model input format; S5.5, based on the packaged model interface, the server visualizes the model to the user, completing the model engineering deployment.

7. The method for predicting scholar's professional title and age based on MT5 pre-training model according to claim 1, characterized in that: Said S1 comprises: S1.1, conduct a joint search of the scholar’s ​​name and institution through a search engine to obtain multiple web pages; S1.2, parsing the contents of the plurality of web pages through xpath to obtain summary information and detailed text contents.

8. A device for predicting academic titles and ages based on the MT5 pre-training model, characterized in that: include: A web data crawling module is used to crawl web search results according to the annotated scholar's name and institution to obtain training text, and store the training text in a local database; A data preprocessing module is used to clean the text data of the training text based on the training text to extract text information; wherein, for abnormal text in the text information that exceeds the preset length of the MT5 pre-training model, RNN feature extraction is performed to output a first text vector with the same dimension as the model; the text information includes: education experience, work experience, and research experience information in the scholar's profile; A model training module, used for serializing the content of the normal text to convert it into a second text vector, and inputting the first text vector and the second text vector into the fine-tuned MT5 pre-trained model for model training to train and generate corresponding text answers; A result generation module, used to form the text answer into structured data, and evaluate the MT5 pre-trained model to generate output results of the scholar's title and age; An engineering deployment module is used to present the output results to the user using a web page framework to complete the engineering deployment of the model; The model training module is used to serialize the text content, perform word segmentation using T5Tokenizer and pad_token connect the name, organization and text content to distinguish the semantic key information, and save the text sequence after word segmentation to the local database, and load the training text based on the text sequence after word segmentation to fine-tune the MT5 pre-training model; The step of loading the training text based on the text sequence after word segmentation to fine-tune the MT5 pre-trained model includes: fine-tuning the calculation loss and attention mask part of the MT5 pre-trained model to calculate the loss of generated content; loading the fine-tuned MT5 pre-trained model, dividing the text into a training set, a test set and a validation set, and padding the batch data; reading the vocabulary of the MT5 pre-trained model, encoding the spliced ​​question template through the vocabulary, and inputting the generated encoding list into the fine-tuned MT5 pre-trained model; generating a score for each word in the vocabulary through the fine-tuned MT5 pre-trained model, and selecting multiple words in the vocabulary whose probability exceeds a threshold, to obtain a generation result to predict the next word; based on the generation result, calculating the cross entropy loss between the word with the maximum probability value and the actual result; adjusting the model parameters, and repeatedly performing the above-mentioned fine-tuning steps until the model converges.