Professional accomplished named entity recognition method enhanced by multiple text feature adapters

Abstract

The invention discloses a professional accomplished named entity recognition method enhanced by a multi-text feature adapter. According to the method, professional accomplishment named entities are labeled according to content meanings, and text labeling is performed on the professional accomplishment named entities based on a BIO method; by means of the characteristics and advantages of a BERT model in the field of natural language processing, a multi-text feature adapter is fused into BERT for model fine adjustment, and an MFEBERT model is proposed; an MFEBERT + BiLSTM + CRF professional accomplished named entity recognition model is constructed, the MFEBERT utilizes a multi-text feature adapter to fuse character-level features, vocabulary-level and part-of-speech-level fusion features of professional accomplished named entities, constraint conditions of the professional accomplished named entities are learned through BiLSTM + CRF, and finally intelligent recognition of the professional accomplished named entities is achieved. Important technical support is provided for scientifically, accurately and effectively constructing a professional accomplishment evaluation index system, and innovative construction and application of an education evaluation system are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of named entity recognition, and in particular to a professional quality named entity recognition method enhanced by a multi-text feature adapter. Background Art

[0002] The professional quality evaluation index system is crucial for evaluating the ability level of professionals. It provides educational institutions, employers and individuals with a set of measurement standards for judging the depth of knowledge and proficiency of a person in their field. However, the existing methods for constructing these evaluation indicators often rely on the traditional model of expert consultation. Although this method has its historical value and certain accuracy, it also exposes some obvious shortcomings.

[0003] Specifically, on the one hand, the traditional method has an obvious bottleneck in time efficiency, and it takes a lot of time to collect and organize the opinions of different experts; on the other hand, due to the high reliance on personal professional insights, the final evaluation system is easily affected by personal subjective factors and may not fully reflect the consensus of the entire industry or the latest development trends. In addition, there are certain limitations in the communication between experts, which further affects the integrity and consistency of the evaluation index system. Summary of the invention

[0004] The present invention aims to overcome the limitations of a method for constructing a professional quality evaluation index based on expert consultation, and to provide a professional quality named entity recognition method enhanced by a multi-text feature adapter.

[0005] The first aspect of the present invention provides a method for professional quality named entity recognition enhanced by a multi-text feature adapter, comprising the following steps:

[0006] Acquire professional literacy named entities; annotate the professional literacy named entities according to content meanings to form a professional literacy named entity dataset; divide the professional literacy named entity dataset into a training set and a test set;

[0007] Perform text labeling and word segmentation on the professional quality naming entities to form multiple professional quality naming vocabularies;

[0008] Constructing a multi-text feature adapter, and constructing a named entity recognition integration model oriented to professional literacy based on the multi-text feature adapter;

[0009] Using the professional literacy-oriented named entity recognition integrated model to predict the professional literacy naming vocabulary, and obtain professional literacy named entity results;

[0010] The professional quality named entity results with similarity higher than the preset similarity threshold are merged.

[0011] Furthermore, the dimensions of the content meaning specifically include moral character, professional knowledge and skills, high-level abilities and lifelong development abilities; the divided training set and test set both include professional quality named entities of the content meaning of each dimension.

[0012] Furthermore, the text labeling and word segmentation processing of the professional quality named entity specifically includes the following steps:

[0013] Design text labels based on the content and meaning of the professional literacy named entities;

[0014] Use word segmentation tools to divide each professional literacy named entity into different non-overlapping professional literacy named words;

[0015] Assign a text label to each word in each word obtained by the division.

[0016] Furthermore, the text labels include three primary categories: starting characters, middle characters and non-entity characters; each category except the non-entity characters is secondary classified based on the dimension of content meaning to obtain multiple secondary categories; in the step of assigning a text label to each character in each vocabulary obtained by the division, the assigned text labels are text labels under the secondary categories.

[0017] Furthermore, the named entity recognition integration model for professional literacy specifically includes the MFEBERT layer, the BiLSTM layer, and the CRF layer;

[0018] The MFEBERT layer includes a BERT embedding layer, a first Transformer layer, the multi-text feature adapter and a second Transformer layer; the MFEBERT layer is used to perform feature fusion on the text-level features, word-level features and part-of-speech-level features of the professional literacy naming vocabulary to obtain a vector representation of the professional literacy naming vocabulary;

[0019] The BiLSTM layer is used to mine the bidirectional semantic dependencies of the lexical components of the professional literacy named entity in the sentence based on the vector representation, and obtain the probability that the word vector belongs to different professional literacy named entity label categories;

[0020] The CRF layer is used to learn the constraint relationship within the professional literacy named entity label and obtain the predicted professional literacy named entity result as the model output.

[0021] Further, the multi-text feature adapter includes a part-of-speech embedding layer, a vocabulary embedding layer, a bilinear attention network and a normalization layer;

[0022] The part-of-speech embedding layer is used to perform part-of-speech tagging and part-of-speech embedding on the professional literacy naming vocabulary to obtain a part-of-speech embedding vector;

[0023] The vocabulary embedding layer is used to embed the professional literacy naming vocabulary to obtain a vocabulary embedding vector;

[0024] The bilinear attention network is used to perform feature fusion on the part-of-speech embedding vector and the vocabulary embedding vector to obtain a multi-dimensional word feature vector;

[0025] The normalization layer is used to perform vector addition and normalization on the multi-dimensional word feature vector and the character feature vector to obtain a fused feature vector.

[0026] Furthermore, the multi-text feature adapter is used to perform feature fusion on the text-level features, vocabulary-level features and part-of-speech-level features of the professional literacy naming vocabulary to obtain a fused feature vector, which specifically includes the following steps:

[0027] In the BERT embedding layer, each word in the professional literacy naming vocabulary is embedded to form a sub-embedding vector;

[0028] In the first Transformer layer, the semantic relationship between each sub-embedding vector and other sub-embedding vectors in the sentence is extracted through the attention mechanism to obtain the word feature vector;

[0029] In the part-of-speech embedding layer, the professional literacy naming vocabulary is tagged with parts of speech to obtain a part-of-speech tagging result; the part-of-speech tagging result is one-hot encoded to form a part-of-speech embedding vector;

[0030] In the vocabulary embedding layer, the word2vec model is used to embed the professional literacy naming vocabulary to obtain a vocabulary embedding vector;

[0031] In a bilinear attention network, the part-of-speech embedding vector is feature-fused with the vocabulary embedding vector to obtain a multi-dimensional word feature vector;

[0032] In the normalization layer, the multi-dimensional word feature vector and the character feature vector are vector-added and normalized to obtain a fused feature vector;

[0033] In the second Transformer layer, the vector representation of professional literacy naming vocabulary is obtained through the attention mechanism.

[0034] Furthermore, in the bilinear attention network, the part-of-speech embedding vector is feature-fused with the vocabulary embedding vector to obtain a multi-dimensional word feature vector, which specifically includes the following steps:

[0035] Performing feature fusion on the part-of-speech embedding vector and the vocabulary embedding vector to obtain a word feature vector;

[0036] Using nonlinear transformation to make the dimension of the word feature vector consistent with the dimension of the character feature vector to form a word feature matrix;

[0037] The attention matrix of the professional quality named entity is calculated using a bilinear transformation matrix, and a dot product operation is performed on the attention matrix and the word feature matrix to obtain a multi-dimensional word feature vector.

[0038] Furthermore, the professional quality named entity results having a similarity higher than a preset similarity threshold are merged, and the similarity between different professional quality named entity results is specifically determined by cosine similarity.

[0039] Furthermore, the similarity between named entity results of different professional qualities is determined by cosine similarity, which is calculated by the following formula:

[0040]

[0041] Among them, E 1 、E 2 Each represents a professional quality named entity result, and i represents each word in the professional quality named entity result.

[0042] A second aspect of the present invention provides an electronic device, comprising a memory and a processor;

[0043] The memory is used to store programs;

[0044] The processor executes the program to implement the professional quality named entity recognition method enhanced by a multi-text feature adapter as described in the first aspect.

[0045] The third aspect of the present invention provides a computer-readable storage medium, characterized in that the storage medium stores a program, and the program is executed by a processor to implement the professional quality named entity recognition method enhanced by a multi-text feature adapter as described in the first aspect.

[0046] The embodiment of the present invention also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the above method.

[0047] The embodiments of the present invention have the following beneficial effects: the embodiment of the present invention provides a multi-text feature adapter enhanced professional quality named entity recognition method, which uses the characteristics and advantages of the BERT model in the field of natural language processing, integrates the multi-text feature adapter into BERT for model fine-tuning, and proposes the MFEBERT model; constructs the MFEBERT+BiLSTM+CRF professional quality named entity recognition model, and MFEBERT uses the multi-text feature adapter to integrate the text-level features, vocabulary-level and part-of-speech-level fusion features of professional quality named entities, and learns the constraints of professional quality named entities through BiLSTM+CRF, and finally realizes the intelligent recognition of professional quality named entities, which provides important technical support for the scientific, accurate and effective construction of a professional quality evaluation index system, and helps promote the innovative construction and application of the education evaluation system.

[0048] Additional aspects and advantages of the present invention will be given in part in the following description, and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 It is an overall flow chart of a professional quality named entity recognition method enhanced by a multi-text feature adapter of the present invention.

[0051] Figure 2 It is an example diagram of the process of labeling professional literacy named entities in the present invention.

[0052] Figure 3 This is an example diagram of the result after the present invention performs word segmentation processing on a sentence containing a professional quality named entity.

[0053] Figure 4 This is an example diagram of text labeling of professional quality named entities in the present invention.

[0054] Figure 5 It is an architectural diagram of the professional literacy-oriented named entity recognition integration model of the present invention.

[0055] Figure 6 It is a model architecture diagram of the multi-text feature adapter of the present invention.

[0056] Figure 7 It is a schematic diagram of the operation flow of a named entity recognition integration model for professional literacy in one embodiment of the present invention.

[0057] Figure 8 This is an example diagram of the result of judging the similarity of feature vectors of professional quality named entities according to the present invention.

[0058] Fig. 9 It is a structural schematic diagram of an electronic device of the present invention.

[0059] Fig.10 It is a schematic diagram of the structure of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The BERT pre-trained large language model is a natural language processing technology based on deep learning methods. It can adapt to downstream tasks such as Chinese named entity recognition through fine-tuning technology, and provides a solution to replace complex and large feature engineering. It has achieved remarkable results in a variety of natural language processing tasks. For example, it can complete accurate named entity recognition in information extraction tasks, identify key information in text in reading comprehension tasks, understand problems and build intelligent and fast question-answering systems, help sentiment analysis, email detection, topic classification, etc. to achieve high-accuracy classification in text classification tasks, and perform well in building knowledge graphs.

[0062] With the help of the characteristics and advantages of the BERT model in the field of natural language processing, the present invention proposes a professional literacy named entity recognition method enhanced by a multi-text feature adapter based on the BERT model, proposes a multi-text feature adapter (Text-based Feature Adapter, TFA), integrates TFA into BERT for model fine-tuning, and proposes the MFEBERT model; constructs the MFEBERT+BiLSTM+CRF professional literacy named entity recognition model, MFEBERT uses the multi-text feature adapter TFA to integrate the text-level features, vocabulary-level and part-of-speech-level fusion features of professional literacy named entities, learns the constraints of professional literacy named entities through BiLSTM+CRF, and finally realizes the intelligent recognition of professional literacy named entities, which provides important technical support for the scientific, accurate and effective construction of a professional literacy evaluation index system, and helps to promote the innovative construction and application of the education evaluation system.

[0063] like Figure 1 As shown, the first embodiment of the present invention provides a professional quality named entity recognition method enhanced by a multi-text feature adapter, comprising the following steps:

[0064] S1. Obtain professional literacy named entities; annotate professional literacy named entities according to content meaning to form a professional literacy named entity dataset; divide the professional literacy named entity dataset into a training set and a test set;

[0065] S2. Label and segment the professional literacy naming entities to form multiple professional literacy naming vocabularies;

[0066] S3. Build a multi-text feature adapter, and build a named entity recognition integration model for professional literacy based on the multi-text feature adapter;

[0067] S4. Use the professional literacy-oriented named entity recognition integrated model to predict professional literacy naming vocabulary and obtain professional literacy named entity results;

[0068] S5. Merge the professional competence named entity results whose similarity is higher than the preset similarity threshold.

[0069] The present invention aims to overcome the limitations of the method for constructing professional quality evaluation indicators based on expert consultation, and provides a method for identifying professional quality named entities enhanced by a multi-text feature adapter. The embodiment of the present invention collects the professional talent training goals of higher education, divides the professional quality named entities into four dimensions of moral character, professional knowledge and skills, high-order ability, and lifelong development ability, and marks them to form a professional quality named entity data set, uses the "BIO" method to construct a series of new labels pointing to professional quality and performs text labeling processing on the professional quality named entities, and uses jieba for word segmentation processing; proposes to construct a multi-text feature adapter TFA, and integrates BERT to construct an MFEBERT model; constructs an MFEBERT+BiLSTM+CRF integrated model for professional quality named entity recognition, deeply mines text-level features, vocabulary-level and part-of-speech-level fusion features, and extracts the relationship between text, vocabulary, part-of-speech and named entities, so as to realize accurate recognition of professional quality named entities; finally, cosine similarity is used to realize the deduplication and merging of semantically similar professional quality named entities, so as to provide professional quality named entity data support for constructing evaluation indicators for professional quality.

[0070] The implementation process of each step of the embodiment of the present invention is described in detail below.

[0071] S1. Obtain professional literacy named entities; label professional literacy named entities according to their content meanings to form a professional literacy named entity dataset; divide the professional literacy named entity dataset into a training set and a test set.

[0072] In step S1, the embodiment of the present invention collects a large amount of texts such as talent training programs for specific majors and descriptions of professional job requirements for the majors from different universities, and divides the professional qualities required by students into four dimensions according to the professional talent training goals: moral character, professional knowledge and skills, high-level abilities, and lifelong development abilities, so as to form a professional quality named entity dataset.

[0073] For example, Figure 2 As shown, an embodiment of the present invention obtains professional quality named entities from the talent training program text, which includes four types of labels: moral character, professional knowledge and skills, high-level capabilities, and lifelong development capabilities.

[0074] After forming the professional quality named entity dataset, the embodiment of the present invention divides it into a training set and a test set according to the hierarchical division principle, so that the divided training set and test set both include professional quality named entities with content meanings of each dimension.

[0075] S2. Perform text labeling and word segmentation on the named entities of professional competence to form multiple named vocabularies of professional competence.

[0076] In step S2, the professional literacy named entity is subjected to text labeling and word segmentation processing, which specifically includes the following steps:

[0077] S2-1. Design text labels based on the meaning of the content of the named entities being annotated.

[0078] In step S2-1, in the embodiment of the present invention, the professional quality named entity is labeled based on "BIO". The BIO labeling method can mark a word as one of the three primary categories of the starting character (B, Begin), the middle character (I, Inside) and the non-entity character (O, Outside). On the basis of the BIO labeling, the embodiment of the present invention combines the content meaning to perform secondary classification on the dimensions other than the non-entity characters in the primary category to obtain multiple secondary categories. Specifically, the embodiment of the present invention involves four dimensions of "morality", "knowledge and skills", "higher-order skills" and "lifelong development skills", so the following nine secondary categories can be obtained as the text labels of the professional quality named entity: "B-MOR", "I-MOR", "B-KNS", "I-KNS", "B-HOS", "I-HOS", "B-LDS", "I-LDS", "O".

[0079] S2-2. Use a word segmentation tool to divide each professional literacy named entity into unequal numbers of non-overlapping professional literacy named words;

[0080] In the embodiment of the present invention, the jieba library in the Python environment is used as a word segmentation tool to perform word segmentation on professional quality named entities, so that each professional quality named entity is divided into different non-overlapping words, in preparation for the subsequent model to mine the vocabulary-level features of professional quality named entities. Specifically, the embodiment of the present invention first decomposes the professional quality named entity into each single word, and then groups the single words to form multiple words. As a specific embodiment, "it has the ability to develop systems in professional fields and can comprehensively use theoretical and technical means to realize related software and hardware systems." The word segmentation effect of this section of professional quality named entities is as follows: Figure 3 shown.

[0081] In some other embodiments, word segmentation libraries such as spaCy, HanLP, and THULAC may also be used as word segmentation tools.

[0082] S2-3. Assign a text label to each character in each word obtained by the division.

[0083] In the embodiment of the present invention, each word is labeled using the nine secondary categories obtained in step S2-1. As a specific embodiment, "system development capability" is a professional quality named entity in the "professional knowledge and skills" dimension, and "system", "system", "development", "development", "ability", and "power" are marked as ['B-KNS','I-KNS','I-KNS','I-KNS','I-KNS','I-KNS'] respectively, and other non-professional quality named entity characters are marked as 'O'. The labeling effect is as follows: Figure 4 shown.

[0084] S3. Build a multi-text feature adapter, and build a named entity recognition integration model for professional literacy based on the multi-text feature adapter.

[0085] In the embodiment of the present invention, the professional quality-oriented named entity recognition integrated model specifically includes an MFEBERT layer, a BiLSTM layer and a CRF layer.

[0086] like Figure 5 As shown in Figure 1, the MFEBERT layer includes a BERT embedding layer, a Transformer layer, and a multi-text feature adapter.

[0087] The MFEBERT layer is used to fuse the text-level features, word-level features, and part-of-speech-level features of professional literacy naming vocabulary to obtain the vector representation of professional literacy naming vocabulary;

[0088] The BiLSTM layer is used to mine the bidirectional semantic dependencies of the lexical components of professional literacy named entities in sentences based on vector representation, and obtain the probability that the word vector belongs to different professional literacy named entity label categories;

[0089] The CRF layer is used to learn the constraints within the professional literacy named entity labels and obtain the predicted professional literacy named entity results as the model output.

[0090] As a preferred embodiment, the multi-text feature adapter TFA architecture is as follows Figure 6 As shown, it includes a part-of-speech embedding layer, a vocabulary embedding layer, a bilinear attention network, and a normalization layer.

[0091] Among them, the part-of-speech embedding layer is used to perform part-of-speech tagging and part-of-speech embedding on professional literacy naming vocabulary to obtain the part-of-speech embedding vector

[0092] The vocabulary embedding layer is used to embed the professional literacy naming vocabulary and obtain the vocabulary embedding vector

[0093] Bilinear attention network is used to embed the part-of-speech vector and vocabulary embedding vectors Perform feature fusion to obtain a multi-dimensional word feature vector

[0094] The normalization layer is used to transform the multidimensional word feature vector Character vector with word Perform vector addition and normalization to obtain the fused feature vector

[0095] S4. Use the professional literacy-oriented named entity recognition integrated model to predict professional literacy naming vocabulary and obtain professional literacy named entity results.

[0096] In an embodiment of the present invention, a professional literacy named entity recognition integrated model is used to predict professional literacy named words to obtain professional literacy named entity results, which specifically includes the following steps:

[0097] In the BERT embedding layer, the embodiment of the present invention embeds each word in the professional literacy naming vocabulary to form a sub-embedding vector.

[0098] In the Transformer layer, the embodiment of the present invention extracts the semantic relationship between each sub-embedding vector and other sub-embedding vectors in the sentence through the attention mechanism to obtain a character feature vector. Specifically, the Transformer layer of the embodiment of the present invention includes L layers of Transformers, which are divided into k layers of Transformers as the first Transforme layer and (Lk) layers of Transformers as the second Transformer layer. First, the character embedding vector is input into the k layers of Transformers. Each layer of Transformers extracts the semantic relationship between each character and other characters in the sentence through the attention mechanism, and outputs a character feature vector.

[0099] In the part-of-speech embedding layer, the embodiment of the present invention uses the posseg library to perform part-of-speech tagging on the professional literacy naming vocabulary to obtain the part-of-speech tagging result; the part-of-speech tagging result is one-hot encoded to form a part-of-speech embedding vector

[0100] In the vocabulary embedding layer, the embodiment of the present invention uses the word2vec model to embed the professional literacy naming vocabulary to obtain the vocabulary embedding vector

[0101] In the bilinear attention network, the embodiment of the present invention performs feature fusion of the part-of-speech embedding vector and the vocabulary embedding vector to obtain a multi-dimensional word feature vector

[0102] Preferably, the part-of-speech embedding vector and the vocabulary embedding vector are feature-fused to obtain a multi-dimensional word feature vector, which specifically includes the following steps:

[0103] The word feature vector is obtained by fusing the part-of-speech embedding vector with the vocabulary embedding vector. Feature fusion is expressed by the following formula:

[0104]

[0105] Use nonlinear transformation to make word feature vector The dimension of the word feature vector The dimensions are consistent, forming the word feature matrix

[0106] Use the bilinear transformation matrix to calculate the attention matrix of the professional literacy named entity:

[0107]

[0108] By training W aLearn the representation relationship between the two features and obtain the attention matrix of the word feature vector after the softmax function. The attention matrix and word feature matrix XP i w Perform dot product operation to obtain multi-dimensional word feature vector

[0109] In the normalization layer, the embodiment of the present invention performs vector addition and normalization on the multi-dimensional word feature vector and the character feature vector to obtain a fused feature vector

[0110] In the second Transformer layer, the embodiment of the present invention obtains the vector representation of the professional literacy naming vocabulary through the attention mechanism.

[0111] In the BiLSTM layer, the embodiment of the present invention mines the bidirectional semantic dependency of the lexical components of the professional literacy named entity in the sentence based on the vector representation, and obtains the probability that the word vector belongs to different professional literacy named entity label categories;

[0112] In the CRF layer, the embodiment of the present invention learns the constraint relationship within the professional literacy named entity tag and obtains the predicted professional literacy named entity result as the model output.

[0113] The implementation effect of the named entity recognition integrated model for professional literacy is as follows Figure 7 As shown in the figure, the MFEBERT layer mines the text-level features, vocabulary-level and part-of-speech-level fusion features of professional literacy named entities and outputs word vectors The BiLSTM layer is used to pass the input word vector through the BiLSTM network to mine the bidirectional semantic dependency of the lexical components of the professional literacy named entity in the sentence, and the probability of the corresponding word vector belonging to different professional literacy named entity label categories is output. The CRF layer is then used to learn the constraint relationship within the professional literacy named entity label, and finally the predicted professional literacy named entity result is output.

[0114] S5. Merge the professional competence named entity results whose similarity is higher than the preset similarity threshold.

[0115] In the embodiment of the present invention, the similarity between named entity results of different professional qualities is specifically determined by cosine similarity, and the formula is as follows:

[0116]

[0117] Among them, E 1 、E 2Each represents a professional quality named entity result, and i represents each word in the professional quality named entity result. The value range of cosine similarity is [-1,1]. The closer the cosine similarity of two professional quality named entity vectors is to 1, the more similar the two have, and the closer it is to -1, the more different the semantic information they have. Through cosine similarity, professional quality named entities with similar word composition and text semantics can be clustered into one category. Finally, by merging the same phrases and expanding the supplementary phrases, the professional quality named entity deduplication based on text semantics is realized, and the professional quality named entity recognition and extraction of three-level dimensions is completed.

[0118] As a specific example, Figure 8 As shown in the figure, "system development capability" and "system analysis and development capability" are semantically similar professional quality named entities belonging to the professional knowledge and skills dimension. Therefore, the cosine value of their feature vectors is 0.72188510, which is close to 1. Therefore, the two are judged to be similar and can be merged into "system analysis and development capability".

[0119] The embodiments of the present invention have at least the following advantages:

[0120] 1. Aiming at the task of professional quality named entity recognition, the present invention proposes a professional quality named entity recognition method enhanced by a multi-text feature adapter, and constructs a MFEBERT+BiLSTM+CRF model, wherein the MFEBERT layer deeply integrates the text-level features, vocabulary-level features and part-of-speech-level features in the professional quality named entity through the multi-text feature adapter TFA, and can identify the characteristics of the professional quality named entity based on the vocabulary structure and the part-of-speech connection relationship, effectively adapt to the diversity and complexity of professional quality descriptive texts, improve the generalization ability of the model for professional quality named entity recognition of different professions, and can dynamically identify professional quality named entities based on updated texts, and timely update the professional quality evaluation index system to the cutting-edge level.

[0121] 2. In the MFEBERT+BiLSTM+CRF model constructed by the present invention, the BERT model is fine-tuned through the multi-text feature adapter TFA at the MFEBERT layer. Only the parameters in TFA need to be trained to deeply integrate the text-level features, vocabulary-level features and part-of-speech-level features, thereby effectively improving the training efficiency of the BERT model while improving the model's ability to represent professional quality named entities.

[0122] 3. The multi-text feature adapter enhanced professional quality named entity recognition method proposed in the present invention aims to improve the efficiency and accuracy of constructing a professional quality evaluation index system for a wide range of professions. In order to improve the usability of professional quality named entities after accurate recognition, the word vectors of the recognized professional quality named entities are compared by cosine similarity, and relatively similar professional quality named entities are merged, while professional quality named entities with different meanings and that do not contain each other are retained, thereby providing accurate data reference for the construction of a professional quality evaluation index system.

[0123] 4. The present invention innovatively uses cosine similarity to judge the similarity of professional quality named entity word vectors, merges semantically similar professional quality named entities, and completes data processing such as deduplication and merging for the subsequent construction of a professional quality evaluation index system. It helps evaluation system builders avoid inefficient and tedious manual deduplication work and provides them with accurate data references.

[0124] 5. The present invention effectively overcomes the drawbacks of the experience-centered expert consultation method and breaks through the limitations of the traditional evaluation index system construction method. Through the automated processing of professional talent training programs and professional job description texts, it can not only achieve efficient and accurate identification of professional quality named entities in larger-scale texts, but also effectively improve the objectivity and professionalism of the professional quality evaluation system construction of different majors in colleges and universities.

[0125] Fig. 9 : is a schematic diagram of the structure of the electronic device proposed in the second embodiment of the present invention. In the memory of this embodiment, program instructions for implementing the professional quality named entity recognition method enhanced by the multi-text feature adapter of any of the above embodiments are stored. The processor is used to execute the program instructions stored in the memory to perform professional quality named entity recognition enhanced by the multi-text feature adapter. Among them, the processor can also be called CPU (Central Processing Unit). The processor may be an integrated circuit chip with signal processing capabilities. The processor can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0126] The contents of the method in the first embodiment of the present invention are all applicable to the electronic device embodiment. The functions specifically implemented by the electronic device embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

[0127] Fig.10It is a schematic diagram of the structure of the computer-readable storage medium of the third embodiment of the present invention. The computer-readable storage medium of the fourth embodiment of the present invention stores program instructions that can implement the professional literacy named entity recognition method enhanced by the above-mentioned multi-text feature adapter, wherein the program instructions can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0128] The contents of the method in the first embodiment of the present invention are all applicable to the computer-readable storage medium embodiment. The functions specifically implemented by the computer-readable storage medium embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method.

[0129] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement the professional quality named entity recognition method enhanced by the multi-text feature adapter provided in the above embodiment.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0131] Those skilled in the art will appreciate that the modules in the devices in the embodiments of the present invention may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments of the present invention may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including corresponding claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including corresponding claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0132] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0133] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0134] In addition, each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. In particular, for embodiments such as devices and equipment, since they are basically similar to method embodiments, the relevant parts can refer to the partial description of the method embodiments. The embodiments of the devices and equipment described above are merely schematic, wherein the modules, units, etc. described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed in multiple places, such as nodes of a system network. Specifically, some or all of the modules and units may be selected according to actual needs to achieve the purpose of the above-mentioned embodiment scheme. Those skilled in the art can understand and implement it without paying creative labor.

[0135] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0136] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0137] In addition, the terms "first", "second", etc. used in the embodiments of the present invention are only used for descriptive purposes and should not be understood as indicating or implying relative importance, or implicitly indicating the number of technical features indicated in the present embodiment. Therefore, the features defined by the terms "first", "second", etc. in the embodiments of the present invention can explicitly or implicitly indicate that the embodiment includes at least one of the features. In the description of the present invention, the word "multiple" means at least two or two or more, such as two, three, four, etc., unless otherwise clearly and specifically defined in the embodiments.

[0138] In the embodiments of the present invention, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present invention may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context in the specific embodiment.

[0139] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and cannot be construed as limitations of the present invention, and those of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention. Those skilled in the art will readily come to think of other embodiments of the present invention after considering the specification and practicing the present invention. This application is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art that are not disclosed in the present invention. The specification and embodiments are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

Claims

1. A professional quality named entity recognition method enhanced by a multi-text feature adapter, characterized in that: The steps include: Acquire professional literacy named entities; annotate the professional literacy named entities according to content meanings to form a professional literacy named entity dataset; divide the professional literacy named entity dataset into a training set and a test set; Perform text labeling and word segmentation on the professional quality naming entities to form multiple professional quality naming vocabularies; Constructing a multi-text feature adapter, and constructing a named entity recognition integration model oriented to professional literacy based on the multi-text feature adapter; Using the professional literacy-oriented named entity recognition integrated model to predict the professional literacy naming vocabulary, and obtain professional literacy named entity results; The professional quality named entity results with similarity higher than the preset similarity threshold are merged.

2. According to claim 1, a multi-text feature adapter enhanced professional literacy named entity recognition method is characterized in that: The dimensions of the content meaning specifically include moral character, professional knowledge and skills, high-level abilities and lifelong development abilities; the divided training set and test set both include professional quality named entities of the content meaning of each dimension.

3. According to claim 1, a multi-text feature adapter enhanced professional literacy named entity recognition method is characterized in that: The text labeling and word segmentation processing of the professional literacy named entity specifically includes the following steps: Design text labels based on the content and meaning of the professional literacy named entities; Use word segmentation tools to divide each professional literacy named entity into different non-overlapping professional literacy named words; Assign a text label to each word in each word obtained by the division.

4. According to claim 3, a multi-text feature adapter-enhanced professional literacy named entity recognition method is characterized in that: The text labels include three primary categories: starting characters, middle characters and non-entity characters; each category except non-entity characters is secondary classified based on the dimension of content meaning to obtain multiple secondary categories; in the step of assigning a text label to each character in each vocabulary obtained by the division, the assigned text labels are text labels under the secondary categories.

5. According to claim 1, a multi-text feature adapter enhanced professional literacy named entity recognition method is characterized in that: An integrated model for named entity recognition for professionalism, including the MFEBERT layer, the BiLSTM layer, and the CRF layer; The MFEBERT layer includes a BERT embedding layer, a first Transformer layer, the multi-text feature adapter and a second Transformer layer; the MFEBERT layer is used to perform feature fusion on the text-level features, word-level features and part-of-speech-level features of the professional literacy naming vocabulary to obtain a vector representation of the professional literacy naming vocabulary; The BiLSTM layer is used to mine the bidirectional semantic dependencies of the lexical components of the professional literacy named entity in the sentence based on the vector representation, and obtain the probability that the word vector belongs to different professional literacy named entity label categories; The CRF layer is used to learn the constraint relationship within the professional literacy named entity label and obtain the predicted professional literacy named entity result as the model output.

6. According to claim 5, a multi-text feature adapter enhanced professional literacy named entity recognition method is characterized in that: The multi-text feature adapter includes a part-of-speech embedding layer, a vocabulary embedding layer, a bilinear attention network and a normalization layer; The part-of-speech embedding layer is used to perform part-of-speech tagging and part-of-speech embedding on the professional literacy naming vocabulary to obtain a part-of-speech embedding vector; The vocabulary embedding layer is used to embed the professional literacy naming vocabulary to obtain a vocabulary embedding vector; The bilinear attention network is used to perform feature fusion on the part-of-speech embedding vector and the vocabulary embedding vector to obtain a multi-dimensional word feature vector; The normalization layer is used to perform vector addition and normalization on the multi-dimensional word feature vector and the character feature vector to obtain a fused feature vector.

7. According to claim 6, a multi-text feature adapter enhanced professional literacy named entity recognition method is characterized in that: The use of the multi-text feature adapter to fuse the text-level features, vocabulary-level features, and part-of-speech-level features of the professional literacy naming vocabulary to obtain a fused feature vector specifically includes the following steps: In the BERT embedding layer, each word in the professional literacy naming vocabulary is embedded to form a sub-embedding vector; In the first Transformer layer, the semantic relationship between each sub-embedding vector and other sub-embedding vectors in the sentence is extracted through the attention mechanism to obtain the word feature vector; In the part-of-speech embedding layer, the professional literacy naming vocabulary is tagged with parts of speech to obtain a part-of-speech tagging result; the part-of-speech tagging result is one-hot encoded to form a part-of-speech embedding vector; In the vocabulary embedding layer, the word2vec model is used to embed the professional literacy naming vocabulary to obtain a vocabulary embedding vector; In a bilinear attention network, the part-of-speech embedding vector is feature-fused with the vocabulary embedding vector to obtain a multi-dimensional word feature vector; In the normalization layer, the multi-dimensional word feature vector and the character feature vector are vector-added and normalized to obtain a fused feature vector; In the second Transformer layer, the vector representation of professional literacy naming vocabulary is obtained through the attention mechanism.

8. According to claim 7, a multi-text feature adapter-enhanced professional literacy named entity recognition method is characterized in that: In the bilinear attention network, the part-of-speech embedding vector is feature-fused with the vocabulary embedding vector to obtain a multi-dimensional word feature vector, which specifically includes the following steps: Performing feature fusion on the part-of-speech embedding vector and the vocabulary embedding vector to obtain a word feature vector; Using nonlinear transformation to make the dimension of the word feature vector consistent with the dimension of the character feature vector to form a word feature matrix; The attention matrix of the professional literacy named entity is calculated using a bilinear transformation matrix, and a dot product operation is performed on the attention matrix and the word feature matrix to obtain a multi-dimensional word feature vector.

9. The method for professional quality named entity recognition enhanced by a multi-text feature adapter according to claim 1, characterized in that: The professional quality named entity results whose similarity is higher than a preset similarity threshold are merged, and the similarity between different professional quality named entity results is specifically determined by cosine similarity.

10. The method for professional quality named entity recognition enhanced by a multi-text feature adapter according to claim 9, characterized in that: The similarity between named entity results of different professional qualities is determined by cosine similarity, which is calculated by the following formula: Among them, E1 and E2 each represent a professional quality named entity result, and i represents each word in the professional quality named entity result.

Images