A multi-text feature adapter enhanced professional literacy named entity recognition method

Abstract

The application discloses a kind of multi-text feature adapter enhanced professional accomplishment named entity recognition method.The application is according to the meaning of content to professional accomplishment named entity and is labeled, and based on "BIO" method, professional accomplishment named entity is labeled with character tag;And with the characteristics and advantages of BERT model in the field of natural language processing, multi-text feature adapter is integrated into BERT to fine-tune the model, and MFEBERT model is proposed;Professional accomplishment named entity recognition model of MFEBERT+BiLSTM+CRF is constructed, MFEBERT utilizes the multi-text feature adapter to fuse the character-level features, lexical-level and part-of-speech-level fusion features of professional accomplishment named entity, learns the constraint conditions of professional accomplishment named entity by BiLSTM+CRF, and finally realizes the intelligent identification of professional accomplishment named entity, provides important technical support for scientific, accurate and effective construction of professional accomplishment evaluation index system, and helps to promote the innovative construction and application of education evaluation system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of named entity recognition, and particularly relates to a professional accomplishment named entity recognition method enhanced by a multi-text feature adapter. BACKGROUND

[0002] The professional accomplishment evaluation index system is crucial for assessing the ability level of professional talents, as it provides a set of standards for educational institutions, employers, and individuals to judge the depth of knowledge and proficiency of skills in a person's field. However, the existing methods for constructing these evaluation indexes often rely on the traditional mode of expert consultation, which, although has its historical value and certain accuracy, also exposes some obvious shortcomings.

[0003] Specifically, on the one hand, the traditional method has a significant bottleneck in time efficiency, requiring a large amount of time to collect and organize the opinions of different experts; on the other hand, due to the high dependence on individual professional insights, the evaluation system formed ultimately is easily influenced by personal subjective factors, and may not fully reflect the consensus or latest development trends of the entire industry. In addition, the communication and exchange among experts also have certain limitations, which further affects the integrity and consistency of the evaluation index system. SUMMARY

[0004] The present application aims to overcome the limitations of the professional accomplishment evaluation index construction method based on expert consultation, and provides a professional accomplishment named entity recognition method enhanced by a multi-text feature adapter.

[0005] The first aspect of the present application provides a professional accomplishment named entity recognition method enhanced by a multi-text feature adapter, comprising the following steps:

[0006] Obtaining professional accomplishment named entities; annotating the professional accomplishment named entities according to content meaning to form a professional accomplishment named entity dataset; dividing the professional accomplishment named entity dataset into a training set and a test set;

[0007] Performing character tagging and word segmentation processing on the professional accomplishment named entities to form a plurality of professional accomplishment named vocabularies;

[0008] Constructing a multi-text feature adapter, and constructing a professional accomplishment-oriented named entity recognition integrated model based on the multi-text feature adapter;

[0009] Using the professional accomplishment-oriented named entity recognition integrated model to predict the professional accomplishment named vocabularies to obtain professional accomplishment named entity results;

[0010] Merging the professional accomplishment named entity results with a similarity higher than a preset similarity threshold.

[0011] Further, the dimensions of the content meaning specifically include moral character, professional knowledge and skills, high-level ability and lifelong development ability; the professional accomplishment named entity of each dimension of the content meaning is included in the divided training set and test set.

[0012] Further, the professional accomplishment named entity is subjected to word labeling and word segmentation processing, specifically including the following steps:

[0013] Designing word labels based on the content meaning of the professional accomplishment named entity;

[0014] Using a word segmentation tool to divide each professional accomplishment named entity into unequal and disjointed professional accomplishment named vocabularies;

[0015] Each word in each vocabulary is assigned a word label.

[0016] Further, the word labels include three first-level categories of starting characters, intermediate characters and non-entity characters; the dimensions of the content meaning are classified twice under each category except the non-entity characters to obtain multiple second-level categories; in the step of assigning a word label to each word in each vocabulary, the assigned word label is a word label under the second-level category.

[0017] Further, the professional accomplishment-oriented named entity recognition integrated model specifically includes an MFEBERT layer, a BiLSTM layer and a CRF layer;

[0018] The MFEBERT layer includes a BERT embedding layer, a first Transformer layer, a multi-text feature adapter and a second Transformer layer; the MFEBERT layer is used to fuse the character-level features, vocabulary-level features and part-of-speech-level features of the professional accomplishment named vocabulary to obtain the vector representation of the professional accomplishment named vocabulary;

[0019] The BiLSTM layer is used to mine the bidirectional semantic dependency of the vocabulary components of the professional accomplishment named entity in the sentence based on the vector representation to obtain the probability of different professional accomplishment named entity label categories to which the word vector belongs;

[0020] The CRF layer is used to learn the constraint relationship within the professional accomplishment named entity label to obtain the predicted professional accomplishment named entity result as the model output.

[0021] Further, the multi-text feature adapter includes a part-of-speech embedding layer, a vocabulary embedding layer, a bilinear attention network and a normalization layer.

[0022] The part-of-speech embedding layer is used for part-of-speech tagging and part-of-speech embedding of the professional literacy named entity vocabulary, to obtain a part-of-speech embedding vector;

[0023] The vocabulary embedding layer is used for vocabulary embedding of the professional literacy named entity vocabulary, to obtain a vocabulary embedding vector;

[0024] The bilinear attention network is used for feature fusion of the part-of-speech embedding vector and the vocabulary embedding vector, to obtain a multi-dimensional word feature vector;

[0025] The normalization layer is used for vector addition and normalization of the multi-dimensional word feature vector and the character feature vector, to obtain a fusion feature vector.

[0026] Further, the use of the multi-text feature adapter for feature fusion of the character-level feature, the vocabulary-level feature and the part-of-speech-level feature of the professional literacy named entity vocabulary to obtain a fusion feature vector specifically includes the following steps:

[0027] In the BERT embedding layer, each character in the professional literacy named entity vocabulary is embedded to form a sub-embedding vector;

[0028] In the first Transformer layer, the semantic relationship between each sub-embedding vector and other sub-embedding vectors in the sentence is extracted through an attention mechanism to obtain a character feature vector;

[0029] In the part-of-speech embedding layer, the professional literacy named entity vocabulary is part-of-speech tagged to obtain a part-of-speech tagging result; the part-of-speech tagging result is one-hot encoded to form a part-of-speech embedding vector;

[0030] In the vocabulary embedding layer, the professional literacy named entity vocabulary is vocabulary embedded using a word2vec model to obtain a vocabulary embedding vector;

[0031] In the bilinear attention network, the part-of-speech embedding vector and the vocabulary embedding vector are feature fused to obtain a multi-dimensional word feature vector;

[0032] In the normalization layer, the multi-dimensional word feature vector and the character feature vector are vector added and normalized to obtain a fusion feature vector;

[0033] In the second Transformer layer, the vector representation of the professional literacy named entity vocabulary is obtained through an attention mechanism.

[0034] Further, the feature fusion of the part-of-speech embedding vector and the vocabulary embedding vector in the bilinear attention network to obtain a multi-dimensional word feature vector specifically includes the following steps:

[0035] The part-of-speech embedding vector and the vocabulary embedding vector are feature fused to obtain a word feature vector;

[0036] The dimensions of the word feature vector are kept consistent with the dimensions of the character feature vector using a nonlinear transformation, and a word feature matrix is formed;

[0037] The attention matrix of the professional literacy named entity is calculated using a bilinear transformation matrix, and the attention matrix is dot product operated with the word feature matrix to obtain a multi-dimensional word feature vector.

[0038] Further, the professional literacy named entity results with a similarity higher than a preset similarity threshold are merged, and the similarity between different professional literacy named entity results is specifically judged by cosine similarity.

[0039] Further, the similarity between different professional literacy named entity results is judged by cosine similarity, and is calculated by the following formula:

[0040]

[0041] Wherein, E1 and E2 each represent a professional literacy named entity result, and i represents each character in the professional literacy named entity result.

[0042] The second aspect of the application provides an electronic device, comprising a memory and a processor;

[0043] The memory is used for storing a program;

[0044] The processor executes the program to realize the professional literacy named entity recognition method of the first aspect.

[0045] The third aspect of the application provides a computer readable storage medium, characterized in that the storage medium stores a program, and the program is executed by a processor to realize the professional literacy named entity recognition method of the first aspect.

[0046] The embodiment of the application also discloses a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. The processor of the computer device can read the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the foregoing method.

[0047] The embodiment of the application has the following beneficial effects: the multi-text feature adapter enhanced professional accomplishment named entity recognition method provided by the embodiment of the application integrates the multi-text feature adapter into BERT for model fine-tuning by means of the characteristics and advantages of the BERT model in the field of natural language processing, and proposes an MFEBERT model; a MFEBERT+BiLSTM+CRF professional accomplishment named entity recognition model is constructed, the MFEBERT utilizes the multi-text feature adapter to fuse the character-level features, lexical-level features and part-of-speech-level fusion features of the professional accomplishment named entity, learns the constraint conditions of the professional accomplishment named entity through BiLSTM+CRF, and finally realizes intelligent identification of the professional accomplishment named entity, thereby providing important technical support for scientifically, accurately and effectively constructing a professional accomplishment evaluation index system, and helping to promote the innovative construction and application of the education evaluation system.

[0048] Additional aspects and advantages of the application will be described in the following description part, some of which will become apparent from the following description, or will be understood through practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0050] Figure 1 It is a whole flowchart of the multi-text feature adapter enhanced professional accomplishment named entity recognition method of the present application.

[0051] Figure 2 It is a process example diagram of labeling professional accomplishment named entities of the present application.

[0052] Figure 3 It is a result example diagram of the sentence containing professional accomplishment named entities after the word segmentation processing of the present application.

[0053] Figure 4 It is an example diagram of the professional accomplishment named entity labeling of the present application.

[0054] Figure 5 It is an architecture diagram of the professional accomplishment named entity recognition integrated model of the present application.

[0055] Figure 6 It is a model architecture diagram of the multi-text feature adapter of the present application.

[0056] Figure 7 It is an operation flowchart of the professional accomplishment named entity recognition integrated model in an embodiment of the present application.

[0057] Figure 8 is an example diagram of the similarity of the professional literacy named entity feature vector determined by the present application.

[0058] Figure 9 is a schematic diagram of the structure of an electronic device according to the present application.

[0059] Figure 10 is a schematic diagram of the structure of a computer-readable storage medium according to the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0061] BERT pre-training large language model is a natural language processing technology based on deep learning method, which can adapt to downstream tasks such as Chinese named entity recognition through fine-tuning technology, and provides a solution instead of complex and large feature engineering. At present, it has achieved remarkable results in various natural language processing tasks, such as accurate named entity recognition in information extraction tasks, identification of key information in reading comprehension tasks, understanding of questions and construction of intelligent and fast question and answer systems, high-accuracy classification in text classification tasks such as sentiment analysis, mail detection, topic classification, and construction of knowledge graph.

[0062] With the characteristics and advantages of BERT model in the field of natural language processing, the present application proposes a multi-text feature adapter enhanced professional literacy named entity recognition method based on BERT model, proposes a multi-text feature adapter (Text-based Feature Adapter, TFA), and integrates TFA into BERT for model fine-tuning, and proposes a MFEBERT model; a MFEBERT+BiLSTM+CRF professional literacy named entity recognition model is constructed, MFEBERT utilizes the multi-text feature adapter TFA to fuse the character-level features, word-level and part-of-speech-level fusion features of professional literacy named entity, and learns the constraint conditions of professional literacy named entity through BiLSTM+CRF, and finally realizes the intelligent identification of professional literacy named entity, provides important technical support for the scientific, accurate and effective construction of professional literacy evaluation index system, and helps to promote the innovative construction and application of education evaluation system.

[0063] As shown in Figure 1 The first embodiment of the present application provides a multi-text feature adapter enhanced professional literacy named entity recognition method, which comprises the following steps:

[0064] S1. Obtain professional accomplishment named entities; label the professional accomplishment named entities according to content meanings to form a professional accomplishment named entity dataset; divide the professional accomplishment named entity dataset into a training set and a test set;

[0065] S2. Perform character tagging and word segmentation processing on the professional accomplishment named entities to form a plurality of professional accomplishment named vocabularies.

[0066] S3. Construct a multi-text feature adapter, and construct a professional accomplishment-oriented named entity recognition integrated model based on the multi-text feature adapter.

[0067] S4. Use the professional accomplishment-oriented named entity recognition integrated model to predict the professional accomplishment named vocabularies to obtain professional accomplishment named entity results.

[0068] S5. Merge the professional accomplishment named entity results with a similarity higher than a preset similarity threshold.

[0069] The present application aims to overcome the limitations of the professional accomplishment evaluation index construction method based on expert consultation, and provides a multi-text feature adapter enhanced professional accomplishment named entity recognition method. In the embodiment of the present application, the professional accomplishment named entities are divided into four dimensions of moral character, professional knowledge and skills, high-level ability, and lifelong development ability and are labeled by collecting higher education professional talent training objectives to form a professional accomplishment named entity dataset. A series of new labels pointing to professional accomplishment are constructed using the "BIO" method, and the professional accomplishment named entities are processed by character labeling. Word segmentation processing is performed using jieba. A multi-text feature adapter TFA is proposed, and MFEBERT model is constructed by integrating BERT. The MFEBERT+BiLSTM+CRF integrated model for professional accomplishment named entity recognition is constructed, the character-level features, vocabulary-level and part-of-speech-level fusion features are deeply mined, and the relationship between characters, vocabularies, parts of speech and named entities is extracted to realize accurate recognition of professional accomplishment named entities. Finally, the cosine similarity is used to realize the deduplication and merging of semantically similar professional accomplishment named entities, and to provide professional accomplishment named entity data support for constructing professional accomplishment-oriented evaluation indexes.

[0070] The implementation process of each step of the embodiment of the present application will be described in detail below.

[0071] S1. Obtain professional accomplishment named entities; label the professional accomplishment named entities according to content meanings to form a professional accomplishment named entity dataset; divide the professional accomplishment named entity dataset into a training set and a test set.

[0072] In step S1, the embodiment of the present application collects large-scale texts from different universities, such as specific professional talent training programs and professional post requirements, and divides the professional competence required by students into four dimensions: moral character, professional knowledge and skills, higher-order ability, and lifelong development ability, according to the professional talent training target, to form a professional competence named entity data set.

[0073] As shown in the example, Figure 2 The embodiment of the present application obtains professional competence named entities from talent training program texts, which include four types of labels: moral character, professional knowledge and skills, higher-order ability, and lifelong development ability.

[0074] After forming the professional competence named entity data set, the embodiment of the present application divides it into a training set and a test set according to the hierarchical division principle, so that the training set and the test set each include professional competence named entities of each dimension content meaning.

[0075] S2. Perform character labeling and word segmentation processing on the professional competence named entity to form a plurality of professional competence named entities.

[0076] In step S2, the professional competence named entity is subjected to character labeling and word segmentation processing, which includes the following steps:

[0077] S2-1. Design character labels based on the content meaning of the professional competence named entity.

[0078] In step S2-1, the embodiment of the present application performs character labeling on the professional competence named entity based on the content meaning. The BIO labeling method can label a word as one of the three primary categories: start character (B, Begin), intermediate character (I, Inside), and non-entity character (O, Outside). Based on the BIO labeling, the embodiment of the present application classifies the dimensions other than the non-entity character into two categories in combination with the content meaning, to obtain a plurality of secondary categories. Specifically, the embodiment of the present application involves four dimensions: "moral character (Morality)", "professional knowledge and skills (Knowledge and Skills)", "higher-order ability (Higher-Order Skills)", and "lifelong development ability (Lifelong Development Skills)", so that the following nine secondary categories can be obtained as character labels of the professional competence named entity: "B-MOR", "I-MOR", "B-KNS", "I-KNS", "B-HOS", "I-HOS", "B-LDS", "I-LDS", and "O".

[0079] S2-2. Each professional literacy named entity is divided into unequal and disjoint professional literacy named vocabularies using a word segmentation tool;

[0080] In the embodiment of the present application, the jieba library under the Python environment is used as a word segmentation tool to perform word segmentation processing on the professional literacy named entity, so that each professional literacy named entity is divided into unequal and disjoint vocabularies, and the subsequent model is prepared to mine the word-level features of the professional literacy named entity. Specifically, the professional literacy named entity is first decomposed into each single character, and then the single characters are processed to form a plurality of vocabularies. As a specific embodiment, the word segmentation effect of the professional literacy named entity in the sentence "having the system development ability in the professional field, being able to comprehensively use the theory and technical means to realize the related hardware and software system" is as shown in the following table. Figure 3

[0081] In other embodiments, the spaCy, HanLP, THULAC, etc. word segmentation library can also be used as a word segmentation tool.

[0082] S2-3. Each character in each vocabulary obtained by division is respectively assigned a character label.

[0083] In the embodiment of the present application, the nine secondary categories obtained in step S2-1 are used to label each vocabulary. As a specific embodiment, "system development ability" is a professional literacy named entity in the "professional knowledge and skills" dimension, and then "system", "opening", "development", "ability", "power" are marked as [‘B-KNS’,‘I-KNS’,‘I-KNS’,‘I-KNS’,‘I-KNS’,‘I-KNS’], and the characters of other non-professional literacy named entities are marked as ‘O’. The labeling effect is as shown in the following table. Figure 4

[0084] S3. Constructing a multi-text feature adapter, and constructing a professional literacy oriented named entity recognition integrated model based on the multi-text feature adapter.

[0085] In the embodiment of the present application, the professional literacy oriented named entity recognition integrated model specifically includes an MFEBERT layer, a BiLSTM layer and a CRF layer.

[0086] As shown in the following table, the MFEBERT layer includes a BERT embedding layer, a Transformer layer and a multi-text feature adapter. Figure 5

[0087] The MFEBERT layer is used to perform feature fusion on the character-level features, vocabulary-level features and part-of-speech-level features of the professional literacy named vocabulary, to obtain the vector representation of the professional literacy named vocabulary;

[0088] ​​​The BiLSTM layer is used to mine the bidirectional semantic dependency of the lexical components of the professional literacy named entity in the sentence based on the vector representation, and obtain the probability of the word vector belonging to different professional literacy named entity label categories.

[0089] The CRF layer is used to learn the constraint relationship inside the professional literacy named entity label, and obtain the predicted professional literacy named entity result as the model output.

[0090] As a preferred embodiment, the multi-text feature adapter TFA architecture, as shown in Figure 6 includes a part-of-speech embedding layer, a lexical embedding layer, a bilinear attention network, and a normalization layer.

[0091] The part-of-speech embedding layer is used to perform part-of-speech tagging and part-of-speech embedding on the professional literacy named entity, and obtain a part-of-speech embedding vector

[0092] The lexical embedding layer is used to perform lexical embedding on the professional literacy named entity, and obtain a lexical embedding vector

[0093] The bilinear attention network is used to perform feature fusion on the part-of-speech embedding vector and the lexical embedding vector to obtain a multi-dimensional word feature vector

[0094] The normalization layer is used to perform vector addition and normalization on the multi-dimensional word feature vector and the character feature vector to obtain a fused feature vector

[0095] S4. Using the professional literacy-oriented named entity recognition integration model to predict the professional literacy named entity, obtaining the professional literacy named entity result.

[0096] In the embodiment of the application, the professional literacy named entity is predicted using the professional literacy-oriented named entity recognition integration model, and the professional literacy named entity result is obtained, which specifically includes the following steps:

[0097] In the BERT embedding layer, the professional literacy named entity is embedded in each character to form a sub-embedding vector.

[0098] In the Transformer layer, the embodiment of the application extracts the semantic relationship between each sub-embedding vector and other sub-embedding vectors in the sentence through the attention mechanism to obtain a word feature vector. Specifically, the Transformer layer of the embodiment of the application includes L layers of Transformer, which are divided into k layers of Transformer as the first Transformer layer and (L-k) layers of Transformer as the second Transformer layer. The word embedding vector is first input into the k layers of Transformer, and each layer of Transformer extracts the semantic relationship between each word and other words in the sentence through the attention mechanism to output a word feature vector

[0099] In the part-of-speech embedding layer, the embodiment of the application uses the posseg library to perform part-of-speech tagging (Part of Speech Tagging) on the professional literacy named vocabulary to obtain a part-of-speech tagging result; and performs one-hot encoding on the part-of-speech tagging result to form a part-of-speech embedding vector

[0100] In the vocabulary embedding layer, the embodiment of the application uses a word2vec model to perform vocabulary embedding on the professional literacy named vocabulary to obtain a vocabulary embedding vector

[0101] In the bilinear attention network, the embodiment of the application performs feature fusion on the part-of-speech embedding vector and the vocabulary embedding vector to obtain a multi-dimensional word feature vector

[0102] Preferably, the part-of-speech embedding vector and the vocabulary embedding vector are fused to obtain a multi-dimensional word feature vector, specifically including the following steps:

[0103] The part-of-speech embedding vector and the vocabulary embedding vector are fused to obtain a word feature vector The feature fusion is represented by the following formula:

[0104]

[0105] The dimension of the word feature vector is made consistent with the dimension of the word feature vector by using a nonlinear transformation, to form a word feature matrix

[0106] The attention matrix of the professional literacy named entity is calculated using a bilinear transformation matrix:

[0107]

[0108] W aLearn the representation relationship between the two features, and get the attention matrix of the word feature vector after the softmax function The attention matrix is obtained with the word feature matrix XP i w The dot product operation is performed to obtain a multi-dimensional word feature vector

[0109] In the normalization layer, the embodiment of the application performs vector addition and normalization on the multi-dimensional word feature vector and the word feature vector to obtain a fusion feature vector

[0110] In the second Transformer layer, the embodiment of the application obtains the vector representation of the professional literacy named entity through the attention mechanism.

[0111] In the BiLSTM layer, the embodiment of the application mines the bidirectional semantic dependency of the lexical component of the professional literacy named entity in the sentence based on the vector representation, to obtain the probability of the word vector belonging to different professional literacy named entity label categories.

[0112] In the CRF layer, the embodiment of the application learns the constraint relationship within the professional literacy named entity label to obtain the predicted professional literacy named entity result as the model output.

[0113] The implementation effect of the professional literacy-oriented named entity recognition integrated model is as shown in Figure 7 The MFEBERT layer mines the character-level features, lexical-level and part-of-speech-level fusion features of the professional literacy named entity, and outputs the word vector The BiLSTM layer mines the bidirectional semantic dependency of the lexical component of the professional literacy named entity in the sentence through the BiLSTM network, and outputs the probability of the corresponding word vector belonging to different professional literacy named entity label categories. Then, the CRF layer learns the constraint relationship within the professional literacy named entity label, and finally outputs the predicted professional literacy named entity result.

[0114] S5. The professional literacy named entity results with a similarity higher than a preset similarity threshold are merged.

[0115] In the embodiment of the application, the similarity between different professional literacy named entity results is determined by cosine similarity, and the formula is as follows:

[0116]

[0117] Wherein, E1, E2 respectively represent one professional accomplishment named entity result, i represents each word in the professional accomplishment named entity result, the cosine similarity value range is [-1, 1], the cosine similarity of two professional accomplishment named entity vectors is closer to 1, it is explained that the two have similar semantics, closer to -1, it is explained that the semantic information possessed by the two is not similar, through the cosine similarity, the professional accomplishment named entity with similar word composition and text semantics can be classified into a class, finally, through merging the same word group, extending and supplementing the word group, the professional accomplishment named entity deduplication based on text semantics is realized, and the three-level dimension professional accomplishment named entity recognition and extraction are completed.

[0118] As a specific embodiment, as shown in Figure 8 The system development ability and the system analysis and development ability are professional knowledge and skill dimension professional accomplishment named entities with similar semantics, so the cosine value of the feature vectors of the two is 0.72188510, close to 1, so it is judged that the two are similar, and they can be merged into "system analysis and development ability".

[0119] The embodiment of the application has at least the following advantages:

[0120] 1. The application proposes a multi-text feature adapter enhanced professional accomplishment named entity recognition method for professional accomplishment named entity recognition task, and constructs an MFEBERT+BiLSTM+CRF model, wherein the MFEBERT layer deeply integrates the character-level features, the lexical-level features and the part-of-speech-level features in the professional accomplishment named entity through the multi-text feature adapter TFA, can recognize the features of the professional accomplishment named entity based on the lexical structure and the part-of-speech connection relationship, effectively adapt to the diversity and complexity of the professional accomplishment descriptive text, improve the generalization ability of the model to different professional accomplishment named entities, and can dynamically identify the professional accomplishment named entity based on the updated text, and update the professional accomplishment evaluation index system to the forefront in time.

[0121] 2. In the MFEBERT+BiLSTM+CRF model constructed by the application, the BERT model is fine-tuned through the multi-text feature adapter TFA in the MFEBERT layer, only the parameters in the TFA need to be trained, and the character-level features, the lexical-level features and the part-of-speech-level features can be deeply integrated, which improves the training efficiency of the BERT model while improving the representation ability of the model to the professional accomplishment named entity.

[0122] 3. The multi-text feature adapter enhanced professional literacy named entity recognition method provided by the present application aims to improve the efficiency and accuracy of the construction of a wide range of professional literacy evaluation index system, and to improve the usability of the professional literacy named entity after accurate recognition, the cosine similarity of the word vector of the recognized professional literacy named entity is compared, the similar professional literacy named entities are merged, and the professional literacy named entities with different meanings and not containing each other are retained, providing accurate data reference for the construction of the professional literacy evaluation index system.

[0123] 4. The present application innovatively uses cosine similarity to judge the similarity of professional literacy named entity word vectors, and merges the professional literacy named entities with similar semantics, which helps the evaluator to avoid inefficient and tedious manual deduplication work, and provides accurate data reference.

[0124] 5. The present application effectively overcomes the disadvantages of the experience-centered expert consultation method, and breaks through the limitations of the traditional evaluation index system construction method. Through the automatic processing of professional talent training plan and professional post description text, not only can the professional literacy named entity be efficiently and accurately identified in a larger scale text, but also the objectivity and professionalism of the construction of the professional literacy evaluation system of different majors in colleges and universities can be effectively improved.

[0125] Figure 9 is the structure schematic diagram of the electronic device provided by the second embodiment of the present application. The memory of the present embodiment stores program instructions for implementing the multi-text feature adapter enhanced professional literacy named entity recognition method of any of the above embodiments. The processor is used to execute the program instructions stored in the memory to perform multi-text feature adapter enhanced professional literacy named entity recognition. The processor can also be referred to as CPU (Central Processing Unit). The processor can be an integrated circuit chip with signal processing capability. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0126] The contents of the method in the first embodiment of the present application are applicable to the present electronic device embodiment. The present electronic device embodiment specifically implements the same functions as the above method embodiments, and achieves the same beneficial effects as the above method embodiments.

[0127] Figure 10The computer readable storage medium of the third embodiment of the present application is a structural schematic diagram. The computer readable storage medium of the fourth embodiment of the present application stores program instructions capable of realizing the professional literacy named entity recognition method enhanced by the multi-text feature adapter, wherein the program instructions can be stored in the storage medium in the form of a software product, and include instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned computer readable storage medium includes various media capable of storing program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet computer.

[0128] The contents of the method in the first embodiment of the present application are applicable to the computer readable storage medium embodiment, the computer readable storage medium embodiment specifically realizes the same functions as the method embodiment, and achieves the same beneficial effects as the method.

[0129] The embodiment also provides a computer program product, which, when running on a computer, causes the computer to execute the related steps to realize the multi-text feature adapter enhanced professional literacy named entity recognition method provided by the above embodiment.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of the related data need to comply with the relevant laws, regulations and standards of the country and region, and provide corresponding operation entrances for the user to choose authorization or refusal.

[0131] Those skilled in the art can understand that the modules in the device in the embodiments of the present application can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments of the present application can be combined into a module or a unit or a component, and furthermore can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, all combinations of all features disclosed in the present specification (including the corresponding claims, the abstract and the drawings) and all processes or units of any methods or apparatuses disclosed thus can be adopted. Unless explicitly stated otherwise, each feature disclosed in the present specification (including the corresponding claims, the abstract and the drawings) can be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0132] Through the description of the above embodiments, those skilled in the art can understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0133] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered listing of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with such an instruction execution system, apparatus, or device. For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical apparatus), a portable computer diskette (magnetic apparatus), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical apparatus), and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or another suitable medium upon which the program is printed, because the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic

[0134] In addition, various embodiments described in this specification are described with reference to particular examples. The description as can be made in this specification is not necessarily mutually exclusive. For example, specific details of one or more embodiments can be combined with specific details of one or more other embodiments. In addition, descriptions of various embodiments can apply to other embodiments. For example, features described in one embodiment can be applied to another embodiment, unless described otherwise. In addition, it should be understood that descriptions of various embodiments are intended to be illustrative, and not necessarily mutual exclusive. Various features are described which can be used in various combinations. Therefore, various features are not necessarily mutually exclusive in different embodiments, but can be combined in any manner. In addition, it should be understood that the use of "or" in describing various embodiments is intended to encompass both exclusive and inclusive cases. For example, "A or B" can mean "A but not B" or "B but not A" or "A and B" unless otherwise indicated.

[0135] It should be understood that various aspects of the application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, or a combination thereof, can be used: a discrete logic circuit having logic gates for implementing logic functions upon data signals, an application specific integrated circuit having appropriate combinational logic gates, a programmable gate array (PGA), a field programmable gate array (FPGA), or the like.

[0136] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the description of the specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0137] In addition, the terms "first", "second", and the like used in the embodiments of the present application are only for the purpose of description, and can not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the embodiments. Therefore, the features defined with "first", "second" and the like in the embodiments of the present application can be explicitly or implicitly indicated to include at least one of the features. In the description of the present application, the meaning of the word "plurality" is at least two or more, for example, two, three, four, etc., unless otherwise specifically limited in the embodiments.

[0138] In the embodiments of the present application, the terms "comprising" and "including" or any other variation thereof are intended to cover a non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element, and furthermore, components, features, elements with the same name in different embodiments of the present application can have the same meaning or different meanings, and the specific meaning thereof should be determined in the light of its explanation in the specific embodiment or further in combination with the context in the specific embodiment.

[0139] Although the embodiments of the present application have been shown and described above, it is to be understood that the above-described embodiments are exemplary only, and are not to be taken in a limiting sense, but in their conception and true scope thereof, it is understood that changes in the above-described embodiments come within the scope of the present application. It will be readily apparent to those skilled in the art that certain modifications can be made to the above-described embodiments without departing from the scope of the present application. Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the present application. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

Claims

1. A method for multi-textual feature adapter augmented professional literacy named entity recognition, characterized in that, The method comprises the following steps: obtaining professional literacy named entities; annotating the professional literacy named entities according to content meanings to form a professional literacy named entity dataset; dividing the professional literacy named entity dataset into a training set and a test set; performing character tagging and word segmentation processing on the professional literacy named entities to form a plurality of professional literacy named vocabularies; constructing a multi-text feature adapter, and constructing a professional literacy-oriented named entity recognition integrated model based on the multi-text feature adapter; using the professional literacy-oriented named entity recognition integrated model to predict the professional literacy named vocabularies to obtain professional literacy named entity results; merging professional literacy named entity results with a similarity higher than a preset similarity threshold; The professional literacy-oriented named entity recognition integrated model specifically comprises an MFEBERT layer, a BiLSTM layer and a CRF layer; The MFEBERT layer comprises a BERT embedding layer, a first Transformer layer, the multi-text feature adapter and a second Transformer layer; the MFEBERT layer is used for performing feature fusion on character-level features, word-level features and part-of-speech-level features of the professional literacy named vocabularies to obtain vector representations of the professional literacy named vocabularies; The BiLSTM layer is used for mining bidirectional semantic dependencies of word components of professional literacy named entities in a sentence based on the vector representations to obtain probabilities of different professional literacy named entity label categories to which word vectors belong; The CRF layer is used for learning constraint relationships among professional literacy named entity labels to obtain predicted professional literacy named entity results as model outputs; The multi-text feature adapter comprises a part-of-speech embedding layer, a word embedding layer, a bilinear attention network and a normalization layer; The part-of-speech embedding layer is used for performing part-of-speech tagging and part-of-speech embedding on the professional literacy named vocabularies to obtain part-of-speech embedding vectors; The word embedding layer is used for performing word embedding on the professional literacy named vocabularies to obtain word embedding vectors; The bilinear attention network is used for performing feature fusion on the part-of-speech embedding vectors and the word embedding vectors to obtain multi-dimensional word feature vectors; The normalization layer is used for performing vector addition and normalization on the multi-dimensional word feature vectors and word feature vectors to obtain fusion feature vectors; The use of the multi-text feature adapter to perform feature fusion on character-level features, word-level features and part-of-speech-level features of the professional literacy named vocabularies to obtain fusion feature vectors specifically comprises the following steps: In the BERT embedding layer, embedding each character in the professional literacy named vocabulary to form a sub-embedding vector; In the first Transformer layer, extracting semantic relationships between each sub-embedding vector and other sub-embedding vectors in a sentence through an attention mechanism to obtain a word feature vector; In the part-of-speech embedding layer, performing part-of-speech tagging on the professional literacy named vocabularies to obtain part-of-speech tagging results; performing one-hot encoding on the part-of-speech tagging results to form part-of-speech embedding vectors; In the vocabulary embedding layer, the professional literacy named vocabulary is subjected to vocabulary embedding using a word2vec model to obtain a vocabulary embedding vector; In the bilinear attention network, the part-of-speech embedding vector and the vocabulary embedding vector are subjected to feature fusion to obtain a multi-dimensional word feature vector; In the normalization layer, the multi-dimensional word feature vector and the character feature vector are subjected to vector addition and normalization to obtain a fusion feature vector; In the second Transformer layer, the vector representation of the professional literacy named vocabulary is obtained through an attention mechanism.

2. The method of claim 1, wherein, The dimensions of the content meaning specifically include moral character, professional knowledge and skills, high-order ability, and lifelong development ability; and the training set and the test set after the division each include the professional literacy named entity of each dimension of content meaning.

3. The method of claim 1, wherein, The professional literacy named entity is subjected to literal labeling and word segmentation processing, specifically including the following steps: Designing literal labels based on the content meaning of the professional literacy named entity to be labeled; Dividing each professional literacy named entity into unequal and disjoint professional literacy named vocabularies using a word segmentation tool; Assigning each literal in each vocabulary a literal label.

4. The method of claim 3, wherein, The literal labels include three primary categories of start character, intermediate character, and non-entity character; each category except the non-entity character is subjected to secondary classification in the dimension of the content meaning to obtain multiple secondary categories; in the step of assigning each literal in each vocabulary a literal label, the literal label assigned is a literal label in the secondary category.

5. The method of claim 1, wherein, In the bilinear attention network, the part-of-speech embedding vector and the vocabulary embedding vector are subjected to feature fusion to obtain a multi-dimensional word feature vector, specifically including the following steps: The part-of-speech embedding vector and the vocabulary embedding vector are subjected to feature fusion to obtain a word feature vector; Using a nonlinear transformation to make the dimension of the word feature vector consistent with the dimension of the character feature vector to form a word feature matrix; Using a bilinear transformation matrix to calculate an attention matrix of the professional literacy named entity, and performing a dot product operation between the attention matrix and the word feature matrix to obtain a multi-dimensional word feature vector.

6. The method of claim 1, wherein, The professional literacy named entity results with a similarity higher than a preset similarity threshold are subjected to merging processing, specifically by judging the similarity between different professional literacy named entity results through cosine similarity.

7. The method of claim 6, wherein the multi-text feature adapter enhanced professional literacy named entity recognition method is characterized by, The similarity between different professional literacy named entity results is judged through cosine similarity by the following formula: ； wherein E 1、 E 2each denotes one professional literacy named entity result, i denotes each word in the professional literacy named entity result.

Images