A method for automatically generating expert ability labels based on semantic analysis

Abstract

This invention discloses an automatic generation method for expert ability labels based on semantic analysis, comprising the following steps: segmenting unstructured text related to expert abilities into sentences and words to form an original text embedding representation sequence; performing semantic normalization through thesaurus query and contextual dynamic semantic clustering to obtain an enhanced text embedding representation sequence; extracting content semantic and positional semantic features using an improved DeBERTa model, optimizing through a semantic residual routing mechanism to obtain expert ability semantic features; and generating an expert ability label sequence through conditional random field structure modeling and Viterbi algorithm decoding. This invention effectively improves the accuracy and semantic understanding depth of expert ability label generation.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method for automatically generating expert competency tags based on semantic analysis. Background Technology

[0002] With the continuous development of information technology and intelligent applications, the automatic extraction and analysis of expert competency information has gradually become an important research direction in the fields of knowledge management and intelligent talent recommendation. Currently, the identification and classification of expert competency information mainly relies on semantic analysis methods for unstructured text. Existing technologies typically employ traditional word vector mapping models and conditional random field models to achieve automated annotation and extraction of expert competency information through word segmentation, feature extraction, and label sequence prediction of expert text.

[0003] Generally, standardized word embedding models are used to obtain text semantic vectors. Basic semantic feature extraction methods are then used to generate expert ability feature representations. Finally, traditional conditional random field models are used to model and predict the label sequence, completing the generation of expert ability labels. However, due to the complex expressions, ambiguous word meanings, and significant differences in synonym expressions in expert ability-related texts, traditional semantic modeling methods are insufficient in both semantic understanding depth and accuracy when dealing with such complex texts. This is especially true in real-world scenarios with numerous technical terms and diverse expressions, making it difficult to achieve accurate semantic uniformity and effective feature extraction.

[0004] Meanwhile, existing label prediction methods typically neglect the semantic diversity and dynamic contextual features of expert competency texts, relying solely on simple word embeddings for feature extraction. They lack deep semantic normalization processing for synonyms and near-synonyms, resulting in significant deficiencies in the accuracy, stability, and consistency of automatically generated expert competency labels. Particularly in professional applications, this deficiency easily leads to labeling bias and misidentification of expert competencies, thereby reducing the reliability of the generated labels and failing to meet the practical application needs of intelligent management and precise recommendation of professional talent.

[0005] Therefore, how to provide a method for automatically generating expert competency tags based on semantic analysis is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an automatic generation method for expert ability labels based on semantic analysis. Addressing the shortcomings of existing technologies in terms of semantic understanding depth and modeling accuracy when processing unstructured text related to expert abilities, this invention proposes a label generation technique that combines an improved DeBERTa model with thesaurus query, context-based dynamic semantic clustering, decoupled self-attention mechanism, semantic residual routing mechanism, and conditional random field probabilistic graphical modeling and Viterbi decoding. This invention offers the advantages of high label generation accuracy, deep semantic understanding, and good robustness.

[0007] An automatic generation method for expert capability tags based on semantic analysis according to an embodiment of the present invention includes:

[0008] The unstructured text related to expert capabilities is segmented into sentences and words, and then converted into word embedding vectors to form the original text embedding representation sequence.

[0009] Semantic analysis is performed on the original text embedding representation sequence. Based on the thesaurus query and context dynamic semantic clustering, a set of synonym embedding vectors is determined. Semantic adjustment and normalization are performed by calculating the offset between the embedding vectors in the set and the center vector, resulting in an enhanced text embedding representation sequence.

[0010] By utilizing the decoupled self-attention mechanism of the improved DeBERTa model, the content semantic features and positional semantic features of the enhanced text embedding representation sequence are extracted respectively, resulting in a preliminary semantic feature representation sequence.

[0011] By utilizing the semantic residual routing mechanism of the improved DeBERTa model, the initial semantic feature representation sequence is adaptively filtered and combined layer by layer to obtain an optimized expert capability semantic feature representation sequence.

[0012] Based on the pre-established label transition probabilities and boundary constraints, a global structural model of the expert capability semantic feature representation sequence is performed using conditional random fields to obtain a probabilistic graphical model of the expert capability label sequence.

[0013] The Viterbi algorithm is used to decode the probabilistic graphical model to obtain a preliminary expert capability label sequence;

[0014] The prediction error signal is calculated based on the preliminary expert capability label sequence and the real label sequence in the training set. The gradient information is then backpropagated to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the label generation model.

[0015] A label generation model is used to process new unstructured text data related to expert capabilities to obtain the final expert capability label sequence.

[0016] Optionally, the step of segmenting and word-segmenting the unstructured text related to expert capabilities and converting it into word embedding vectors to form the original text embedding representation sequence specifically involves:

[0017] Based on the preset text segmentation rules, the sentence boundaries in unstructured text related to expert capabilities are determined by periods, question marks, exclamation marks, and semicolons to obtain a set of text sentences;

[0018] Based on a pre-trained vocabulary, a forward maximum matching word segmentation operation is performed on each sentence in the text sentence set to obtain a word sequence;

[0019] The word sequence is examined to identify uncommon words outside the pre-trained vocabulary. Each word is mapped to a character embedding vector, which is then concatenated into a combined word embedding vector to replace the uncommon words and obtain an updated word sequence.

[0020] The pre-trained word embedding model is used to map the updated word sequence one by one into word embedding vectors, forming an initial word embedding vector sequence;

[0021] Based on the original order of the sentences in the text sentence set, the initial word embedding vector sequence is concatenated sentence by sentence, and inter-sentence separator embedding vectors are inserted between sentences to obtain the original text embedding representation sequence.

[0022] Optionally, the step of performing semantic analysis on the original text embedding representation sequence to obtain the enhanced text embedding representation sequence specifically involves:

[0023] By using a thesaurus query method, each word embedding vector in the original text embedding representation sequence is queried and matched to determine the set of word embedding vectors that are synonyms or near-synonyms.

[0024] Based on the context-dynamic semantic clustering mechanism, the vector cosine similarity method is used to cluster the word embedding vector set to generate a context-dynamic semantic cluster set;

[0025] Calculate the Euclidean distance between each word embedding vector in the context dynamic semantic cluster set and the corresponding cluster set center vector to determine the semantic offset of the word embedding vector;

[0026] Calculate the difference vector between each word embedding vector and the cluster set center vector, and perform semantic adjustment on the word embedding vector based on the difference vector;

[0027] Normalize the vector magnitude of the semantically adjusted word embedding vector to obtain the semantically normalized word embedding vector;

[0028] Following the word embedding vector order of the original text embedding representation sequence, the semantically normalized word embedding vectors are concatenated into an enhanced text embedding representation sequence.

[0029] Optionally, the step of extracting the content semantic features and positional semantic features of the enhanced text embedding representation sequence to obtain a preliminary semantic feature representation sequence is as follows:

[0030] The enhanced text embedding representation sequence is input into the improved DeBERTa model. Through a decoupled self-attention mechanism, each word embedding vector is mapped to the content semantic subspace and the position semantic subspace respectively, generating two independent subspace embedding representations.

[0031] For the content semantic subspace embedding representation, a decoupled self-attention mechanism is used to calculate the dot product between each word embedding vector and other word embedding vectors in the sequence to obtain the content semantic self-attention score matrix;

[0032] The content semantic self-attention score matrix is ​​subjected to softmax normalization to obtain the content semantic self-attention weight matrix.

[0033] The content semantic self-attention weight matrix and the content semantic subspace embedding representation are multiplied together to obtain the content semantic feature representation sequence.

[0034] For the positional semantic subspace embedding representation, a decoupled self-attention mechanism is used to calculate the relative positional distance between each word embedding vector and other word embedding vectors in the sequence, and the positional semantic self-attention score matrix is ​​obtained.

[0035] The location semantic self-attention score matrix is ​​subjected to softmax normalization to obtain the location semantic self-attention weight matrix;

[0036] The positional semantic self-attention weight matrix and the positional semantic subspace embedding representation are multiplied by matrix to obtain the positional semantic feature representation sequence. The content semantic feature representation sequence and the positional semantic feature representation sequence are then concatenated and fused to obtain the preliminary semantic feature representation sequence.

[0037] Optionally, the improved DeBERTa model includes a word embedding module, a semantic offset alignment module, a decoupled self-attention module, a semantic residual routing module, and a deep semantic extraction module:

[0038] The word embedding module is used to perform dimensional mapping on the enhanced text embedding representation sequence to generate an initial word embedding representation sequence;

[0039] The semantic offset alignment module is used to perform semantic adjustment and normalization on the word embedding vectors in the initial word embedding representation sequence using a thesaurus query method and a context-based dynamic semantic clustering mechanism, and outputs a semantically enhanced word embedding representation sequence.

[0040] The decoupled self-attention module is used to map the semantically enhanced word embedding representation sequence to the content semantic subspace and the position semantic subspace respectively through the decoupled self-attention mechanism. By calculating the content semantic self-attention score matrix and the position semantic self-attention score matrix respectively, the content semantic feature representation sequence and the position semantic feature representation sequence are obtained. The content semantic feature representation sequence and the position semantic feature representation sequence are then concatenated and fused to output the preliminary semantic feature representation sequence.

[0041] The semantic residual routing module is used to calculate the saliency score of the preliminary semantic feature representation sequence through the semantic residual routing mechanism, and to filter and fuse the preliminary semantic feature representation sequence step by step based on the saliency score, and output the optimized expert ability semantic feature representation sequence.

[0042] The deep semantic extraction module is used to extract high-order semantic features from the route-optimized semantic feature representation sequence step by step, and output the final expert capability semantic feature representation sequence.

[0043] Optionally, the step of performing layer-by-layer adaptive filtering and combination of the preliminary semantic feature representation sequence to obtain an optimized expert capability semantic feature representation sequence specifically involves:

[0044] The initial semantic feature representation sequence is input into the improved DeBERTa model, and the embedding vector of each word is dimensionally split according to different semantic granularities to obtain multiple semantic sub-vectors;

[0045] The semantic residual routing mechanism in the improved DeBERTa model is used to calculate the saliency score of semantic sub-vectors at different granularities. The saliency score is obtained by calculating the cosine similarity between each semantic sub-vector and other semantic sub-vectors at the same level.

[0046] The saliency score is normalized to obtain the adaptive routing weight value;

[0047] Based on the adaptive routing weight values, semantic sub-vectors with weight values ​​higher than the preset weight threshold are retained, and the retained semantic sub-vectors are weighted and summed dimension by dimension to obtain the routing filtering results at each granularity.

[0048] The routing filtering results of two adjacent different granularities are concatenated into vectors, and a dimensionality reduction operation is performed through a linear transformation function to output the fused high-order semantic feature representation vector.

[0049] The higher-order semantic feature representation vectors are reordered according to the word embedding vector position order of the initial semantic feature representation sequence to obtain the optimized expert ability semantic feature representation sequence.

[0050] Optionally, the probabilistic graphical model for constructing the expert capability label sequence specifically includes:

[0051] Extract the label feature representation corresponding to each word embedding vector from the optimized expert ability semantic feature representation sequence to form the input feature sequence;

[0052] Based on a corpus of expert capability tags annotated manually, the tag transition probability is determined by statistically analyzing the transition frequency between tags, and a tag transition probability matrix is ​​constructed.

[0053] Based on a corpus of expert capability tags with manual annotation, the frequency of each tag appearing at the beginning and end of the sequence is counted to determine the tag boundary probability and form a tag boundary constraint vector.

[0054] The input feature sequence is fed into the conditional random field structure decoding layer. The mapping between each word embedding vector and the candidate label in the input feature sequence is represented as a state feature function, the label transition probability is represented as a transition feature function, and the label values ​​at the start and end positions of the sequence are constrained by the label boundary constraint vector.

[0055] Within the decoding layer of the conditional random field structure, the sum of the state feature function and the transition feature function is calculated for each input feature sequence to obtain the local potential value of the label corresponding to each position, and then accumulated to obtain the global potential function.

[0056] Within the decoding layer of the conditional random field structure, the global situational energy function is transformed by an exponential function and normalized to obtain the probability values ​​of the label sequence, thus forming a probabilistic graphical model of the expert capability label sequence.

[0057] Optionally, the step of using the Viterbi algorithm to decode the probabilistic graphical model to obtain a preliminary expert capability label sequence specifically involves:

[0058] The probability values ​​of candidate label nodes at each position in the probabilistic graphical model of the expert capability label sequence are transformed into log probability values ​​to form a log probability matrix of label nodes.

[0059] Using the Viterbi algorithm, the initial path score of the label node at the beginning of the sequence is determined based on the log probability value of each candidate label node at the first position in the label node log probability matrix.

[0060] Based on the label transition probability matrix, calculate the cumulative path score for each candidate label node at the second position of the sequence. The cumulative path score is the sum of the path score at the previous position and the label transition probability value.

[0061] Starting from the third position in the sequence, calculate the cumulative path score from each candidate label node in the previous position to the current candidate label node, retain the path with the highest score as the optimal path score for the current candidate label node, and record the corresponding label node in the previous position.

[0062] Proceed position by position to the end position of the sequence, obtain the optimal path score of the candidate label node at the end position of the sequence and the recorded label node at the previous position;

[0063] The optimal path scores of candidate label nodes at the end position of the sequence are compared, and the label node with the highest path score is determined as the end position label node. The label nodes are then backtracked from the label nodes to the beginning position of the sequence to determine the label nodes at each position.

[0064] Based on the backtracked sequence of label nodes, arrange them in the order of their positions to obtain the globally optimal preliminary expert capability label sequence.

[0065] Optionally, the step of calculating the prediction error signal based on the preliminary expert capability label sequence and the real label sequence in the training set, and backpropagating the gradient information to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the label generation model, specifically involves:

[0066] Based on the preliminary expert capability label sequence and the real labeled expert capability label sequence in the training set, calculate the conditional probability of each real label node in the label probabilistic graphical model.

[0067] The negative logarithm of the conditional probability of the true label node at each location is taken and accumulated to obtain the total loss value of the negative log-likelihood loss function as the prediction error signal;

[0068] Based on the prediction error signal, the gradient information of the negative log-likelihood loss function with respect to each trainable parameter of the improved DeBERTa model and the semantic adjustment and normalization process is calculated by backpropagation.

[0069] The gradient information is backpropagated step by step to the word embedding module, semantic offset alignment module, decoupled self-attention module, semantic residual routing module and deep semantic extraction module of the improved DeBERTa model to update the corresponding trainable parameters.

[0070] The gradient information is backpropagated to the semantic adjustment and normalization process to update the set center vector parameters and related parameters of semantic normalization.

[0071] The training data is processed in multiple rounds using the updated and improved DeBERTa model and semantic adjustment and normalization parameters until the loss value meets the convergence threshold.

[0072] Save the parameters of each module after the convergence threshold is met to obtain the trained label generation model.

[0073] Optionally, the step of using a label generation model to process new unstructured text data related to expert capabilities to obtain the final expert capability label sequence specifically involves:

[0074] The unstructured text data related to new expert capabilities is scanned character by character, and the data is segmented into sentences according to preset identifiers to obtain a set of sentences.

[0075] For each sentence in the sentence set, perform forward maximum matching word segmentation, and embed and concatenate uncommon words at the character level to obtain an initial word embedding vector sequence;

[0076] By using a thesaurus query method and a context-based dynamic semantic clustering mechanism, the initial word embedding vector sequence is semantically adjusted and normalized to obtain an enhanced text embedding representation sequence.

[0077] The enhanced text embedding representation sequence is input into the label generation model, and the content semantic features and location semantic features are extracted and fused to obtain a preliminary semantic feature representation sequence.

[0078] By utilizing the semantic residual routing mechanism in the label generation model, the preliminary semantic feature representation sequence is adaptively filtered and combined to obtain the optimized expert capability semantic feature representation sequence.

[0079] The optimized expert ability semantic feature representation sequence is input into the conditional random field structure decoding layer in the label generation model, and the Viterbi algorithm is used for decoding.

[0080] Based on the decoding results of the Viterbi algorithm, the globally optimal expert capability label sequence of the text to be processed is obtained.

[0081] The beneficial effects of this invention are:

[0082] (1) This invention achieves in-depth semantic analysis and high-precision automatic generation of unstructured text related to expert ability by integrating the improved DeBERTa model with the conditional random field structure decoding mechanism. This effectively improves the depth of semantic understanding and the accuracy of label prediction, and enhances the consistency and robustness of label generation in complex expert ability description scenarios.

[0083] (2) By introducing a semantic offset alignment mechanism and a semantic residual routing mechanism, this invention achieves accurate semantic normalization of expert ability descriptions with different expression methods and adaptive optimization combination of multi-granular semantic features, which significantly improves the accuracy and stability of semantic feature extraction and shows better adaptability in expert ability analysis scenarios with complex professional terms and diverse expression methods.

[0084] (3) In terms of semantic understanding of complex expert ability text, this invention effectively solves the problems of insufficient semantic modeling and low label recognition accuracy in the prior art by using a decoupled self-attention mechanism and a global optimization decoding mechanism for label sequences. It breaks through the bottleneck of the prior art in handling diverse expression methods, achieves significant improvement in the accuracy and reliability of automatic generation of expert ability labels, and effectively enhances the application effect in the field of intelligent management and precise recommendation of expert talents. Attached Figure Description

[0085] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0086] Figure 1 This is a schematic diagram of the overall process of an automatic generation method for expert capability tags based on semantic analysis proposed in this invention.

[0087] Figure 2 This is a schematic diagram of the improved DeBERTa model structure for an automatic generation method of expert capability labels based on semantic analysis proposed in this invention.

[0088] Figure 3 This is a schematic diagram of the decoding process of the conditional random field structure decoding layer in the automatic generation method of expert ability labels based on semantic analysis proposed in this invention. Detailed Implementation

[0089] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0090] refer to Figures 1-3 A method for automatically generating expert ability tags based on semantic analysis, comprising:

[0091] Unstructured text related to expert capabilities is processed by sentence segmentation and word segmentation, and then converted into corresponding word embedding vectors to form the original text embedding representation sequence;

[0092] Semantic analysis is performed on the original text embedding representation sequence. The word embedding vectors belonging to the same or similar expressions are identified by the thesaurus query. The set of embedding vectors with the same semantic meaning is determined based on the context dynamic semantic clustering mechanism. By calculating the offset between each embedding vector in the set and the center vector corresponding to the set of embedding vectors, the word embedding vectors are semantically adjusted and normalized to obtain the enhanced text embedding representation sequence.

[0093] By utilizing the decoupled self-attention mechanism in the improved DeBERTa model, content semantic features and positional semantic features are extracted from the enhanced text embedding representation sequence to obtain a preliminary semantic feature representation sequence.

[0094] By utilizing the semantic residual routing mechanism in the improved DeBERTa model, the initial semantic feature representation sequence is adaptively filtered layer by layer, and semantic features of different granularities are combined to obtain the optimized expert capability semantic feature representation sequence.

[0095] The semantic feature representation sequence of expert ability is used as the input feature sequence of the conditional random field structure decoding layer. Based on the pre-established label transition probability and boundary constraints, the global structure model of the input feature sequence is performed to obtain the probabilistic graphical model of the expert ability label sequence.

[0096] The Viterbi algorithm is used to decode the probabilistic graphical model of the expert capability label sequence to obtain the globally optimal preliminary expert capability label sequence.

[0097] Based on the preliminary expert capability label sequence and the real labeled expert capability label sequence in the training set, the prediction error signal is obtained by calculating the negative log-likelihood loss function, and the gradient information corresponding to the prediction error signal is backpropagated to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the trained label generation model.

[0098] A label generation model is used to process the new unstructured text data related to expert capabilities to obtain the final expert capability label sequence.

[0099] In this embodiment, the process of performing text segmentation and word segmentation on unstructured text related to expert capabilities, and converting it into corresponding word embedding vectors to form the original text embedding representation sequence, specifically involves:

[0100] According to the pre-defined text segmentation rules, the unstructured text related to expert ability is scanned character by character. The boundary markers of sentence ends are identified by periods, question marks, exclamation marks and semicolons. The boundary positions of all sentences in the text are determined. The unstructured text is then segmented according to the determined sentence boundary positions to obtain a text sentence set consisting of multiple independent sentences.

[0101] Based on a pre-built pre-trained vocabulary, a forward maximum matching word segmentation operation is performed on each sentence in the text sentence set from the beginning to the end. The longest matching word in the vocabulary is given priority, and the word segmentation result of each sentence is determined word by word in turn to obtain a word sequence composed of words in the vocabulary.

[0102] The word sequence is checked word by word to identify rare words not included in the pre-trained vocabulary. For each rare word, character-level embedding is performed. Specifically, each character that makes up the rare word is mapped to a corresponding pre-trained character embedding vector. Then, the character embedding vectors are concatenated one by one according to the order in which each character appears in the original rare word to form a combined word embedding vector. The combined word embedding vector replaces the position of the corresponding rare word in the original word sequence to obtain the updated word sequence after rare word processing.

[0103] Using a pre-trained word embedding model, each word in the updated word sequence is mapped one by one. Each word is represented as a numerical vector with a fixed number of dimensions, forming an initial word embedding vector sequence.

[0104] Based on the original order of the sentences in the text sentence set, the word embedding vectors belonging to the same sentence in the initial word embedding vector sequence are sequentially concatenated. Predefined inter-sentence separator embedding vectors are inserted between the word embedding vectors of each sentence to obtain the original text embedding representation sequence.

[0105] In this embodiment, the semantic analysis of the original text embedding representation sequence involves identifying word embedding vectors belonging to synonymous or near-synonymous expressions using a thesaurus query method, determining a set of embedding vectors with the same semantic meaning based on a context-based dynamic semantic clustering mechanism, and semantically adjusting and normalizing the word embedding vectors by calculating the offset between each embedding vector in the set and the corresponding center vector of the embedding vector set, thereby obtaining an enhanced text embedding representation sequence. Specifically:

[0106] The thesaurus query method is used to query and match each word embedding vector in the original text embedding representation sequence one by one. Based on the word identifiers corresponding to the word embedding vectors, the set of word embedding vectors that are synonyms or near-synonyms in the preset thesaurus is determined. The thesaurus query method is as follows: based on the pre-constructed thesaurus, the word identifiers corresponding to each word embedding vector in the original text embedding representation sequence are input into the thesaurus for matching and retrieval. The word identifiers are used as search keys to query the pre-stored set of synonyms or near-synonyms in the thesaurus. If synonyms or near-synonyms are found, the set of word embedding vectors that are synonyms or near-synonyms corresponding to the word identifiers is determined based on the search results.

[0107] Based on the context-dynamic semantic clustering mechanism, all word embedding vectors in the word embedding vector set are clustered using the vector cosine similarity method. The semantic similarity between word embedding vectors is determined by calculating the cosine similarity between every two word embedding vectors. Word embedding vectors that meet the clustering conditions are selected according to a preset similarity threshold to generate a context-dynamic semantic clustering set. The context-dynamic semantic clustering mechanism is as follows: based on the context window, the semantic similarity of each word embedding vector in the word embedding vector set is calculated. Taking each word embedding vector to be clustered as the center, the context window is expanded forward and backward by a preset length. The semantic similarity between the word embedding vector to be clustered and other word embedding vectors in the window is calculated using cosine similarity. Word embedding vectors that meet the conditions are selected according to the preset similarity threshold, thereby dynamically determining the set of word embedding vectors with the same semantic meaning.

[0108] For all word embedding vectors in the context dynamic semantic clustering set, calculate the Euclidean distance between each word embedding vector and the corresponding cluster set center vector, and define the Euclidean distance as the semantic offset of the word embedding vector;

[0109] The semantic offset is processed by a vector offset correction method. Specifically, the difference vector between each word embedding vector and the cluster set center vector is calculated, and the position of the original word embedding vector is adjusted according to the direction and magnitude of the difference vector to achieve semantic adjustment of the word embedding vector.

[0110] The semantically adjusted word embedding vectors are normalized to the unit vector space, and the vector magnitude normalization method is used to uniformly adjust the magnitude of the word embedding vectors to 1, thus obtaining the semantically normalized word embedding vectors.

[0111] Based on the original positional order of the word embedding vectors in the original text embedding representation sequence, all word embedding vectors that have undergone semantic adjustment and normalization are sequentially reassembled to obtain the enhanced text embedding representation sequence.

[0112] In this embodiment, the decoupled self-attention mechanism in the improved DeBERTa model is used to extract content semantic features and positional semantic features from the enhanced text embedding representation sequence to obtain a preliminary semantic feature representation sequence, specifically as follows:

[0113] The enhanced text embedding representation sequence is input into the improved DeBERTa model. Through a decoupled self-attention mechanism, each word embedding vector is mapped to the content semantic subspace and the position semantic subspace respectively, generating two independent subspace embedding representations.

[0114] The decoupled self-attention mechanism is as follows:

[0115] Each word embedding vector in the input is mapped to the content semantic subspace and the position semantic subspace respectively. The content semantic self-attention score matrix and the position semantic self-attention score matrix are calculated based on the content semantic subspace embedding representation and the position semantic subspace embedding representation respectively.

[0116] The content semantic self-attention score matrix is ​​obtained by calculating the dot product between each word embedding vector and all other word embedding vectors in the sequence, and the position semantic self-attention score matrix is ​​obtained by calculating the relative positional distance between each word embedding vector and all other word embedding vectors in the sequence.

[0117] The content semantic self-attention score matrix and the position semantic self-attention score matrix are normalized using the softmax function to obtain the corresponding content semantic self-attention weight matrix and position semantic self-attention weight matrix.

[0118] Then, perform matrix multiplication operations between the content semantic self-attention weight matrix and the position semantic self-attention weight matrix and their respective subspace embedding representations to obtain the content semantic feature representation sequence and the position semantic feature representation sequence, respectively.

[0119] The decoupled self-attention mechanism in the improved DeBERTa model is used to perform self-attention computation on the embedding representation of the content semantic subspace. The content semantic self-attention score matrix is ​​obtained by calculating the dot product between each word embedding vector and all other word embedding vectors in the sequence.

[0120] The softmax function is used to normalize the content semantic self-attention score matrix to obtain the content semantic self-attention weight matrix corresponding to each word embedding vector;

[0121] The content semantic self-attention weight matrix and the content semantic subspace embedding representation are multiplied together to obtain the content semantic feature representation sequence.

[0122] We utilize the decoupled self-attention mechanism in the improved DeBERTa model to perform self-attention computation on the positional semantic subspace embedding representation, construct a relative position matrix, and calculate the positional semantic self-attention score matrix based on the relative positional distance between each word embedding vector and all other word embedding vectors in the sequence.

[0123] The positional semantic self-attention score matrix is ​​normalized using the softmax function to obtain the positional semantic self-attention weight matrix corresponding to each word embedding vector;

[0124] The positional semantic self-attention weight matrix and the positional semantic subspace embedding representation are multiplied by matrix to obtain the positional semantic feature representation sequence. The content semantic feature representation sequence and the positional semantic feature representation sequence are then concatenated and fused to obtain the preliminary semantic feature representation sequence.

[0125] In this embodiment, the improved DeBERTa model includes a word embedding module, a semantic offset alignment module, a decoupled self-attention module, a semantic residual routing module, and a deep semantic extraction module:

[0126] The word embedding module is used to receive an enhanced text embedding representation sequence, perform dimension mapping on each word embedding vector in the enhanced text embedding representation sequence, and generate an initial word embedding representation sequence.

[0127] The semantic offset alignment module is used to perform semantic unification processing on the initial word embedding representation sequence. Specifically, it identifies the set of word embedding vectors in the initial word embedding representation sequence that are synonymous or near-synonymous based on the thesaurus query method, calculates the semantic offset between each word embedding vector in the set and the corresponding set center vector through the context dynamic semantic clustering mechanism, performs semantic adjustment and normalization on the word embedding vectors in the set based on the semantic offset, and outputs the semantically enhanced word embedding representation sequence.

[0128] The decoupled self-attention module is used to extract content semantic features and positional semantic features from the semantically enhanced word embedding representation sequence through a decoupled self-attention mechanism. Specifically, it maps each word embedding vector in the semantically enhanced word embedding representation sequence to the content semantic subspace and the positional semantic subspace, calculates the dot product between each word embedding vector in the content semantic subspace and other word embedding vectors in the sequence to generate a content semantic self-attention score matrix, calculates the relative positional distance between each word embedding vector in the positional semantic subspace and other word embedding vectors in the sequence to generate a positional semantic self-attention score matrix, performs softmax function normalization on the content semantic self-attention score matrix and the positional semantic self-attention score matrix respectively, and then performs matrix multiplication with the corresponding subspace embedding representation to obtain the content semantic feature representation sequence and the positional semantic feature representation sequence. Finally, the content semantic feature representation sequence and the positional semantic feature representation sequence are concatenated and fused to output the preliminary semantic feature representation sequence.

[0129] The semantic residual routing module is used to adaptively filter and combine semantic features in the preliminary semantic feature representation sequence step by step through the semantic residual routing mechanism. Specifically, it calculates the saliency scores of semantic features of different granularities in the preliminary semantic feature representation sequence, sets adaptive routing weights for each granularity semantic feature according to the saliency scores, filters and merges the semantic features in the preliminary semantic feature representation sequence step by step based on the routing weights, and outputs the optimized expert capability semantic feature representation sequence.

[0130] The deep semantic extraction module receives a sequence of semantic feature representations optimized by routing, extracts high-order semantic features step by step through multiple serially stacked Transformer structures, and after aggregation of the semantic feature representation sequences output step by step, outputs the final expert capability semantic feature representation sequence.

[0131] In this embodiment, the semantic residual routing mechanism in the improved DeBERTa model is used to adaptively filter the initial semantic feature representation sequence layer by layer, and semantic features of different granularities are combined to obtain an optimized expert capability semantic feature representation sequence. Specifically:

[0132] The initial semantic feature representation sequence is input into the improved DeBERTa model. Based on the different semantic granularities of the feature embedding vector, the dimensions of each word embedding vector are split to obtain multiple semantic sub-vectors of different granularities.

[0133] The semantic residual routing mechanism in the improved DeBERTa model is used to calculate the saliency score of semantic sub-vectors of different granularities one by one. The saliency score is obtained by calculating the cosine similarity between the current granularity semantic sub-vector and other granularity semantic sub-vectors at the same level, and the importance of each granularity semantic sub-vector is represented by the value of the cosine similarity. The semantic residual routing mechanism is as follows:

[0134] Based on the different semantic granularities of the feature embedding vectors, each word embedding vector is dimensionally segmented to obtain multiple semantic sub-vectors of different granularities;

[0135] The saliency score is calculated for each semantic subvector at different granularities. Specifically, the cosine similarity between each semantic subvector at different granularities and other semantic subvectors at the same level is calculated to obtain a saliency score that represents the importance of semantic features.

[0136] The saliency scores are normalized to obtain the corresponding adaptive routing weight values. Specifically, the saliency score of each granular semantic subvector is divided by the sum of the saliency scores of all granular semantic subvectors.

[0137] Based on the adaptive routing weight values, the semantic sub-vectors at each granularity are filtered, retaining the semantic sub-vectors with weight values ​​higher than the preset weight threshold and discarding the semantic sub-vectors with weight values ​​lower than the threshold. The retained semantic sub-vectors are then summed dimension by dimension to output the routing filtering results at each granularity.

[0138] A step-by-step fusion strategy is adopted for routing filtering results of different granularities. Specifically, the routing filtering results of two adjacent granularities are concatenated into vectors, and the concatenated vectors are reduced in dimensionality by a linear transformation function to obtain the fused high-order semantic feature representation vector.

[0139] The saliency scores of semantic subvectors of different granularities are normalized to obtain the corresponding adaptive routing weight values. The adaptive routing weight values ​​are obtained by dividing the saliency score of each semantic subvector of granularity by the sum of the saliency scores of all semantic subvectors of granularity at the same level.

[0140] The semantic subvectors at each granularity are filtered and combined based on the adaptive routing weight values. Semantic subvectors with adaptive routing weight values ​​higher than the preset weight threshold within the same granularity are retained, while other semantic subvectors are discarded. The retained semantic subvectors are then summed dimension by dimension to obtain the routing filtering results for each granularity.

[0141] The routing filtering results at each granularity are fused step by step. Specifically, the routing filtering results of two adjacent different granularities are concatenated into vectors in turn, and the concatenated vectors are reduced in dimensionality by a linear transformation function to output the fused high-order semantic feature representation vector.

[0142] The fused high-order semantic feature representation vector is reordered according to the positional order of the original word embedding vectors in the preliminary semantic feature representation sequence to obtain the optimized expert ability semantic feature representation sequence.

[0143] In this embodiment, the step of using the expert capability semantic feature representation sequence as the input feature sequence of the conditional random field structure decoding layer, and performing global structural modeling on the input feature sequence based on pre-established label transition probabilities and boundary constraints to obtain a probabilistic graphical model of the expert capability label sequence is as follows:

[0144] Extract the label feature representation corresponding to each word embedding vector from the optimized expert ability semantic feature representation sequence, and arrange them in sequence to form the input feature sequence;

[0145] Based on the corpus of expert capability tags annotated by humans, the transition frequency between each tag and all other tags is counted by the maximum likelihood estimation method. The proportion of the transition frequency to the occurrence frequency of the corresponding tag is used as the transition probability between tags, and a tag transition probability matrix is ​​constructed.

[0146] Based on the expert capability tag corpus with manual annotation, the frequency of each tag appearing at the beginning and end of the sequence is counted, and the proportion of the frequency to the total frequency of the corresponding tag is determined as the tag boundary probability, forming a tag boundary constraint vector;

[0147] The input feature sequence is fed into the decoding layer of the conditional random field structure. According to the definition of the conditional random field structure, the mapping relationship between the word embedding vector corresponding to each position in the input feature sequence and each candidate label is represented as the state feature function, and each label transition relationship in the label transition probability matrix is ​​represented as the transition feature function. At the same time, the probability values ​​in the label boundary constraint vector are used as the transition probability constraints of the initial state and the final state, respectively.

[0148] The conditional random field structure decoding layer is:

[0149] The sequence labeling decoding structure based on linear chain conditional random fields consists of an input node layer and a label node layer. Each node in the input node layer corresponds to a word embedding vector at a position in the input feature sequence, and each node in the label node layer corresponds to a candidate label value.

[0150] Each node in the input node layer is mapped to all nodes in the label node layer through a state feature function, and adjacent nodes in the label node layer are mapped to each other through a transition feature function.

[0151] The initial state node is connected to the label node at the first position of the input sequence, and the label node at the last position of the input sequence is connected to the termination state node through the label boundary probability provided by the label boundary constraint vector, so as to constrain the range of label values ​​at the beginning and end positions of the sequence.

[0152] The state characteristic function is calculated as follows:

[0153] Using the word embedding vector corresponding to each position in the input feature sequence and each candidate label as input, the pre-trained linear mapping weight vector is used to perform a dimension-wise multiplication operation with the current word embedding vector, and the results are summed to obtain the local feature score of the word embedding vector at the current position under each candidate label value. The local feature score is the state feature function value between the corresponding position and the candidate label.

[0154] The calculation method for the transfer characteristic function is as follows:

[0155] Based on the label transition probabilities provided by the label transition probability matrix, for each combination of labels between the current position and the next position in the input feature sequence, the corresponding label transition probability in the label transition probability matrix is ​​found, and the probability value is directly used as the feature function value for the transition of the label at the current position to the label at the next position.

[0156] Within the decoding layer of the Conditional Random Field structure, the sum of the state feature function and transition feature function values ​​at each position of the input feature sequence is calculated one by one, which serves as the local potential value of the label corresponding to each position. All local potential values ​​are then accumulated to obtain the global potential function corresponding to the input feature sequence.

[0157] Within the conditional random field structure decoding layer, an exponential function transformation is performed on the global state energy function corresponding to the input feature sequence and then normalized to obtain the probability value of each possible label sequence. The probability values ​​of all label sequences are then used to construct a probabilistic graphical model of the expert capability label sequence.

[0158] In this embodiment, the step of using the Viterbi algorithm to decode the probabilistic graphical model of the expert capability label sequence to obtain the globally optimal preliminary expert capability label sequence specifically involves:

[0159] In the probabilistic graphical model of the expert capability label sequence, the probability values ​​corresponding to all candidate label nodes at each position are transformed into log probability values ​​to form the initial log probability matrix of label nodes.

[0160] Using the Viterbi algorithm, the initial path score of each label node at the beginning of the sequence is determined based on the log probability value of each candidate label node at the first position in the label node log probability matrix.

[0161] Based on the label transition probability value provided by the label transition probability matrix, the cumulative path score of each candidate label node at the second position of the sequence is calculated successively. The cumulative path score is the sum of the path score of the previous sequence position and the transition probability value from the label node at the previous position to the current label node.

[0162] Starting from the third position in the sequence, take all candidate label nodes at each sequence position as the current calculation object, calculate the cumulative path score from each candidate label node at the previous position to the current candidate label node, compare the cumulative path scores of all paths, retain the path with the largest cumulative path score, and take the cumulative path score as the optimal path score of the current candidate label node, while recording the corresponding label node at the previous position.

[0163] Continue moving towards the end of the sequence position by position until the end position is reached, and obtain the optimal path score of all candidate label nodes at the end position and the previous position label node.

[0164] The optimal path scores of all candidate label nodes at the end position of the sequence are compared, and the label node with the largest path score is determined as the global path end label node. Then, starting from the end label node, the label node is traced back to the beginning position of the sequence one by one according to the previous position label node, and the label node at each position is determined in turn.

[0165] Based on the tag node sequence obtained through backtracking, the tag nodes are arranged in the original position order in the sequence to obtain the globally optimal preliminary expert capability tag sequence.

[0166] In this embodiment, the step of obtaining a prediction error signal by calculating the negative log-likelihood loss function based on the preliminary expert capability label sequence and the real-labeled expert capability label sequence in the training set, and then backpropagating the gradient information corresponding to the prediction error signal to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the trained label generation model, specifically:

[0167] Based on the preliminary expert capability label sequence and the real labeled expert capability label sequence in the training set, the corresponding label nodes are compared at each sequence position, and the conditional probability value of the real label node at each sequence position in the label probabilistic graphical model is calculated.

[0168] Based on the conditional probability values ​​of the real label nodes corresponding to each sequence position, the negative logarithm is taken and then accumulated position by position to obtain the total loss value of the negative log-likelihood loss function, which is used as the prediction error signal.

[0169] Based on the prediction error signal, the gradient information of the negative log-likelihood loss function with respect to each trainable parameter in the improved DeBERTa model and semantic adjustment and normalization process is calculated by backpropagation.

[0170] The obtained gradient information is backpropagated step by step to the parameters in the word embedding module, semantic offset alignment module, decoupled self-attention module, semantic residual routing module and deep semantic extraction module of the improved DeBERTa model, and the trainable parameters in each module are updated respectively.

[0171] The obtained gradient information is simultaneously backpropagated to the set center vector parameters and semantic normalization parameters in the semantic adjustment and normalization process, and the parameters are updated accordingly.

[0172] Using the updated and improved DeBERTa model and semantic adjustment and normalization parameters, the training data is reprocessed and repeated for multiple rounds of training until the total loss value of the negative log-likelihood loss function meets the preset convergence threshold, and then the parameter update is stopped.

[0173] Save the parameters of each module after the convergence threshold is met to obtain the trained label generation model.

[0174] In this embodiment, the label generation model is used to process the new unstructured text data related to expert capabilities to obtain the final expert capability label sequence, specifically as follows:

[0175] The unstructured text data related to the new expert capabilities to be processed is scanned character by character, and the sentence is segmented with the preset sentence end marker as the boundary to obtain the sentence set of the text to be processed;

[0176] For each sentence in the sentence set, perform forward maximum matching word segmentation, and complete the character-level embedding and concatenation operation of uncommon words based on the pre-trained vocabulary to obtain an initial word embedding vector sequence;

[0177] The word embedding vector set with synonym or near-synonymous relationship in the initial word embedding vector sequence is identified by using a thesaurus query method. Then, the semantic offset between each word embedding vector in the set and the set center vector is calculated by the context dynamic semantic clustering mechanism. Finally, the word embedding vectors are semantically adjusted and normalized one by one to obtain an enhanced text embedding representation sequence.

[0178] The enhanced text embedding representation sequence is input into the trained label generation model, and the content semantic feature representation sequence and the position semantic feature representation sequence are extracted and then concatenated and fused to obtain the preliminary semantic feature representation sequence.

[0179] By utilizing the semantic residual routing mechanism in the trained label generation model, the preliminary semantic feature representation sequence is adaptively filtered and combined layer by layer to obtain the optimized expert ability semantic feature representation sequence.

[0180] The optimized expert ability semantic feature representation sequence is input into the conditional random field structure decoding layer in the label generation model to obtain the label sequence probabilistic graphical model corresponding to the text to be processed, and the Viterbi algorithm is used to decode the label sequence probabilistic graphical model.

[0181] Based on the decoding results of the Viterbi algorithm, the globally optimal expert capability label sequence of the text to be processed is obtained.

[0182] Example 1:

[0183] To verify the feasibility of this invention in practice, it was applied to the automated annotation and analysis of talent competency information in a research institution, aiming to solve the problems of low efficiency and inaccuracy in manual annotation of expert competency labels in actual work. The expert competency text data collected by this institution consisted of a large amount of unstructured text, covering multiple disciplines, containing numerous professional terms, and employing diverse descriptive methods. Traditional technologies struggled to accurately analyze the deep semantics of such text, resulting in poor consistency and a high error rate in label classification results, severely impacting the application effectiveness of the talent intelligent recommendation and knowledge management system.

[0184] In practice, the method of this invention is first used to process the collected unstructured text related to expert abilities. Specifically, the system scans each text character by character, determines sentence boundaries using preset sentence end markers, and then performs word segmentation on each sentence using the forward maximum matching method. For rare words in the segmentation results that do not match the pre-trained vocabulary, word embedding at the character level is used to process them, ultimately forming an initial word embedding vector sequence. After completing the above steps, the system automatically uses a pre-built thesaurus to query the initial word embedding vector sequence one by one, identifies semantically similar word embedding vector sets, and determines the center vector of the same semantic set through a context-based dynamic semantic clustering mechanism. The semantic offset between each word embedding vector and the center vector is calculated, and semantic adjustment and normalization are performed to obtain an enhanced text embedding representation sequence.

[0185] Subsequently, the system inputs the enhanced text embedding representation sequence into the improved DeBERTa model. Internally, this model first utilizes a decoupled self-attention mechanism to extract features from content semantics and location semantics separately, obtaining content semantic feature representation sequences and location semantic feature representation sequences. These two sequences are then concatenated and fused to form a preliminary semantic feature representation sequence. Next, using the model's internal semantic residual routing mechanism, the preliminary semantic feature representation sequence undergoes multi-granularity adaptive feature filtering and combination layer by layer to obtain a more optimized expert capability semantic feature representation sequence.

[0186] Next, the system inputs the obtained expert ability semantic feature representation sequence into the conditional random field (CRF) structure decoding layer. First, the system statistically obtains the label transition probability matrix and boundary constraint vector based on a manually annotated corpus of expert ability labels. Then, within the CRF structure decoding layer, it constructs state feature functions and transition feature functions, and calculates the local and global potential energy values ​​of the labels corresponding to each sequence position, thereby obtaining a probabilistic graphical model of the expert ability label sequence. Subsequently, the Viterbi algorithm is used to decode the probabilistic graphical model of the label sequence to obtain the globally optimal preliminary expert ability label sequence.

[0187] During the model optimization and training phase, the system calculates the negative log-likelihood loss function based on the expert capability label sequence and the preliminary label sequence manually annotated in the training set to obtain the prediction error signal. Then, through gradient backpropagation, it jointly optimizes the relevant parameters in the improved DeBERTa model's word embedding module, semantic offset alignment module, decoupled self-attention module, semantic residual routing module, deep semantic extraction module, and semantic adjustment and normalization processes. After actual iterative training, the improved DeBERTa model in this invention is finally determined to have a word embedding dimension of 256, a self-attention mechanism using 4 attention heads, a significance score threshold of 0.65 for the semantic residual routing module, and a learning rate of 0.0005. After 27 complete iterations of training, the negative log-likelihood loss function converges to 0.058 on the validation set.

[0188] After the model was optimized, the researchers randomly selected five expert competency texts for automatic label generation testing and conducted a detailed comparative analysis with actual manually labeled texts. The results are shown in the table below.

[0189] Table 1 Comparison of Expert Competency Label Prediction Results and Manually Labeled Labels

[0190] Sample number Actual number of tags Predict the number of tags Correctly predict the number of labels Label accuracy (%) Tag recall (%) A01 12 11 11 100.0 91.7 A02 15 15 14 93.3 93.3 A03 9 10 9 90.0 100.0 A04 14 13 13 100.0 92.9 A05 11 11 10 90.9 90.9

[0191] Analysis of the data in Table 1 clearly shows that the expert capability labels generated by the method of this invention exhibit significant advantages in both accuracy and recall. In sample A01, with 12 actual labels, 11 of the automatically generated labels accurately matched the manually labeled ones, achieving a label accuracy of 100.0% and a recall of 91.7%. Sample A02 showed label accuracy and recall both exceeding 93%, significantly higher than the generally lower than 80% of existing traditional methods. In sample A03, the number of predicted labels was slightly higher than the actual number, but this effectively improved the recall to 100%, while the accuracy also reached 90%. Overall, the method of this invention demonstrates high accuracy in label classification and significant advantages in semantic recognition, especially for complex text data with diverse expressions and numerous technical terms, showing a practical effect far superior to traditional techniques.

[0192] In summary, this embodiment fully demonstrates that the expert capability label automatic generation method proposed in this invention can significantly improve label generation efficiency, label prediction accuracy and consistency in practical applications, effectively solve the problems of insufficient semantic understanding accuracy, high manual cost and poor classification effect faced by traditional methods, and has good application value and promotion prospects.

[0193] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for automatically generating an expert ability label based on semantic analysis, characterized in that, include: The unstructured text related to expert capabilities is segmented into sentences and words, and then converted into word embedding vectors to form the original text embedding representation sequence. Semantic analysis is performed on the original text embedding representation sequence. Based on the thesaurus query and context dynamic semantic clustering, a set of synonym embedding vectors is determined. Semantic adjustment and normalization are performed by calculating the offset between the embedding vectors in the set and the center vector, resulting in an enhanced text embedding representation sequence. By utilizing the decoupled self-attention mechanism of the improved DeBERTa model, the content semantic features and positional semantic features of the enhanced text embedding representation sequence are extracted respectively, resulting in a preliminary semantic feature representation sequence. By utilizing the semantic residual routing mechanism of the improved DeBERTa model, the initial semantic feature representation sequence is adaptively filtered and combined layer by layer to obtain an optimized expert capability semantic feature representation sequence. Based on the pre-established label transition probabilities and boundary constraints, a global structural model of the expert capability semantic feature representation sequence is performed using conditional random fields to obtain a probabilistic graphical model of the expert capability label sequence. The Viterbi algorithm is used to decode the probabilistic graphical model to obtain a preliminary expert capability label sequence; The prediction error signal is calculated based on the preliminary expert capability label sequence and the real label sequence in the training set. The gradient information is then backpropagated to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the label generation model. A label generation model is used to process new unstructured text data related to expert capabilities to obtain the final expert capability label sequence; The improved DeBERTa model includes a word embedding module, a semantic offset alignment module, a decoupled self-attention module, a semantic residual routing module, and a deep semantic extraction module. The word embedding module is used to perform dimensional mapping on the enhanced text embedding representation sequence to generate an initial word embedding representation sequence; The semantic offset alignment module is used to perform semantic adjustment and normalization on the word embedding vectors in the initial word embedding representation sequence using a thesaurus query method and a context-based dynamic semantic clustering mechanism, and outputs a semantically enhanced word embedding representation sequence. The decoupled self-attention module is used to map the semantically enhanced word embedding representation sequence to the content semantic subspace and the position semantic subspace respectively through the decoupled self-attention mechanism. By calculating the content semantic self-attention score matrix and the position semantic self-attention score matrix respectively, the content semantic feature representation sequence and the position semantic feature representation sequence are obtained. The content semantic feature representation sequence and the position semantic feature representation sequence are then concatenated and fused to output the preliminary semantic feature representation sequence. The semantic residual routing module is used to calculate the saliency score of the preliminary semantic feature representation sequence through the semantic residual routing mechanism, and to filter and fuse the preliminary semantic feature representation sequence step by step based on the saliency score, and output the optimized expert ability semantic feature representation sequence. The deep semantic extraction module is used to extract high-order semantic features from the route-optimized semantic feature representation sequence step by step, and output the final expert capability semantic feature representation sequence. The process of progressively adaptively filtering and combining the preliminary semantic feature representation sequence to obtain an optimized expert capability semantic feature representation sequence is as follows: The initial semantic feature representation sequence is input into the improved DeBERTa model, and the embedding vector of each word is dimensionally split according to different semantic granularities to obtain multiple semantic sub-vectors; The semantic residual routing mechanism in the improved DeBERTa model is used to calculate the saliency score of semantic sub-vectors at different granularities. The saliency score is obtained by calculating the cosine similarity between each semantic sub-vector and other semantic sub-vectors at the same level. The saliency score is normalized to obtain the adaptive routing weight value; Based on the adaptive routing weight values, semantic sub-vectors with weight values ​​higher than the preset weight threshold are retained, and the retained semantic sub-vectors are weighted and summed dimension by dimension to obtain the routing filtering results at each granularity. The routing filtering results of two adjacent different granularities are concatenated into vectors, and a dimensionality reduction operation is performed through a linear transformation function to output the fused high-order semantic feature representation vector. The higher-order semantic feature representation vectors are reordered according to the word embedding vector position order of the initial semantic feature representation sequence to obtain the optimized expert ability semantic feature representation sequence. 2.The method of claim 1, wherein, The process of segmenting and word-segmenting unstructured text related to expert capabilities, and converting it into word embedding vectors to form the original text embedding representation sequence, specifically involves: Based on the preset text segmentation rules, the sentence boundaries in unstructured text related to expert capabilities are determined by periods, question marks, exclamation marks, and semicolons to obtain a set of text sentences; Based on a pre-trained vocabulary, a forward maximum matching word segmentation operation is performed on each sentence in the text sentence set to obtain a word sequence; The word sequence is examined to identify uncommon words outside the pre-trained vocabulary. Each word is mapped to a character embedding vector, which is then concatenated into a combined word embedding vector to replace the uncommon words and obtain an updated word sequence. The pre-trained word embedding model is used to map the updated word sequence one by one into word embedding vectors, forming an initial word embedding vector sequence; Based on the original order of the sentences in the text sentence set, the initial word embedding vector sequence is concatenated sentence by sentence, and inter-sentence separator embedding vectors are inserted between sentences to obtain the original text embedding representation sequence. 3.The method of claim 1, wherein, The semantic analysis of the original text embedding representation sequence to obtain the enhanced text embedding representation sequence is specifically as follows: By using a thesaurus query method, each word embedding vector in the original text embedding representation sequence is queried and matched to determine the set of word embedding vectors that are synonyms or near-synonyms. Based on the context-dynamic semantic clustering mechanism, the vector cosine similarity method is used to cluster the word embedding vector set to generate a context-dynamic semantic cluster set; Calculate the Euclidean distance between each word embedding vector in the context dynamic semantic cluster set and the corresponding cluster set center vector to determine the semantic offset of the word embedding vector; Calculate the difference vector between each word embedding vector and the cluster set center vector, and perform semantic adjustment on the word embedding vector based on the difference vector; Normalize the vector magnitude of the semantically adjusted word embedding vector to obtain the semantically normalized word embedding vector; Following the word embedding vector order of the original text embedding representation sequence, the semantically normalized word embedding vectors are concatenated into an enhanced text embedding representation sequence.

4. The method for automatically generating expert competency tags based on semantic analysis according to claim 1, characterized in that, The process involves extracting the content semantic features and positional semantic features of the enhanced text embedding representation sequence to obtain a preliminary semantic feature representation sequence, specifically as follows: The enhanced text embedding representation sequence is input into the improved DeBERTa model. Through a decoupled self-attention mechanism, each word embedding vector is mapped to the content semantic subspace and the position semantic subspace respectively, generating two independent subspace embedding representations. For the content semantic subspace embedding representation, a decoupled self-attention mechanism is used to calculate the dot product between each word embedding vector and other word embedding vectors in the sequence to obtain the content semantic self-attention score matrix; The content semantic self-attention score matrix is ​​subjected to softmax normalization to obtain the content semantic self-attention weight matrix. The content semantic self-attention weight matrix and the content semantic subspace embedding representation are multiplied together to obtain the content semantic feature representation sequence. For the positional semantic subspace embedding representation, a decoupled self-attention mechanism is used to calculate the relative positional distance between each word embedding vector and other word embedding vectors in the sequence, and the positional semantic self-attention score matrix is ​​obtained. The location semantic self-attention score matrix is ​​subjected to softmax normalization to obtain the location semantic self-attention weight matrix; The positional semantic self-attention weight matrix and the positional semantic subspace embedding representation are multiplied by matrix to obtain the positional semantic feature representation sequence. The content semantic feature representation sequence and the positional semantic feature representation sequence are then concatenated and fused to obtain the preliminary semantic feature representation sequence.

5. The method for automatically generating expert competency tags based on semantic analysis according to claim 1, characterized in that, The probabilistic graphical model for the expert capability label sequence is constructed as follows: Extract the label feature representation corresponding to each word embedding vector from the optimized expert ability semantic feature representation sequence to form the input feature sequence; Based on a corpus of expert capability tags annotated manually, the tag transition probability is determined by statistically analyzing the transition frequency between tags, and a tag transition probability matrix is ​​constructed. Based on a corpus of expert capability tags with manual annotation, the frequency of each tag appearing at the beginning and end of the sequence is counted to determine the tag boundary probability and form a tag boundary constraint vector. The input feature sequence is fed into the conditional random field structure decoding layer. The mapping between each word embedding vector and the candidate label in the input feature sequence is represented as a state feature function, the label transition probability is represented as a transition feature function, and the label values ​​at the start and end positions of the sequence are constrained by the label boundary constraint vector. Within the decoding layer of the conditional random field structure, the sum of the state feature function and the transition feature function is calculated for each input feature sequence to obtain the local potential value of the label corresponding to each position, and then accumulated to obtain the global potential function. Within the decoding layer of the conditional random field structure, the global situational energy function is transformed by an exponential function and normalized to obtain the probability values ​​of the label sequence, thus forming a probabilistic graphical model of the expert capability label sequence.

6. The method of claim 1, wherein the method further comprises: The process of using the Viterbi algorithm to decode the probabilistic graphical model and obtain a preliminary expert capability label sequence is as follows: The probability values ​​of candidate label nodes at each position in the probabilistic graphical model of the expert capability label sequence are transformed into log probability values ​​to form a log probability matrix of label nodes. Using the Viterbi algorithm, the initial path score of the label node at the beginning of the sequence is determined based on the log probability value of each candidate label node at the first position in the label node log probability matrix. Based on the label transition probability matrix, calculate the cumulative path score for each candidate label node at the second position of the sequence. The cumulative path score is the sum of the path score at the previous position and the label transition probability value. Starting from the third position in the sequence, calculate the cumulative path score from each candidate label node in the previous position to the current candidate label node, retain the path with the highest score as the optimal path score for the current candidate label node, and record the corresponding label node in the previous position. Proceed position by position to the end position of the sequence, obtain the optimal path score of the candidate label node at the end position of the sequence and the recorded label node at the previous position; The optimal path scores of candidate label nodes at the end position of the sequence are compared, and the label node with the highest path score is determined as the end position label node. The label nodes are then backtracked from the label nodes to the beginning position of the sequence to determine the label nodes at each position. Based on the backtracked sequence of label nodes, arrange them in the order of their positions to obtain the globally optimal preliminary expert capability label sequence.

7. The method of claim 1, wherein the method further comprises: The process involves calculating the prediction error signal based on the preliminary expert capability label sequence and the real label sequence in the training set, and backpropagating the gradient information to the improved DeBERTa model and the semantic adjustment and normalization process to obtain the label generation model. Specifically: Based on the preliminary expert capability label sequence and the real labeled expert capability label sequence in the training set, calculate the conditional probability of each real label node in the label probabilistic graphical model. The negative logarithm of the conditional probability of the true label node at each location is taken and accumulated to obtain the total loss value of the negative log-likelihood loss function as the prediction error signal; Based on the prediction error signal, the gradient information of the negative log-likelihood loss function with respect to each trainable parameter of the improved DeBERTa model and the semantic adjustment and normalization process is calculated by backpropagation. The gradient information is backpropagated step by step to the word embedding module, semantic offset alignment module, decoupled self-attention module, semantic residual routing module and deep semantic extraction module of the improved DeBERTa model to update the corresponding trainable parameters. The gradient information is backpropagated to the semantic adjustment and normalization process to update the set center vector parameters and related parameters of semantic normalization. The training data is processed in multiple rounds using the updated and improved DeBERTa model and semantic adjustment and normalization parameters until the loss value meets the convergence threshold. Save the parameters of each module after the convergence threshold is met to obtain the trained label generation model.

8. The method of claim 1, wherein the method further comprises: The process of using a label generation model to process new unstructured text data related to expert capabilities to obtain the final expert capability label sequence is as follows: The unstructured text data related to new expert capabilities is scanned character by character, and the data is segmented into sentences according to preset identifiers to obtain a set of sentences. For each sentence in the sentence set, perform forward maximum matching word segmentation, and embed and concatenate uncommon words at the character level to obtain an initial word embedding vector sequence; By using a thesaurus query method and a context-based dynamic semantic clustering mechanism, the initial word embedding vector sequence is semantically adjusted and normalized to obtain an enhanced text embedding representation sequence. The enhanced text embedding representation sequence is input into the label generation model, and the content semantic features and location semantic features are extracted and fused to obtain a preliminary semantic feature representation sequence. By utilizing the semantic residual routing mechanism in the label generation model, the preliminary semantic feature representation sequence is adaptively filtered and combined to obtain the optimized expert capability semantic feature representation sequence. The optimized expert ability semantic feature representation sequence is input into the conditional random field structure decoding layer in the label generation model, and the Viterbi algorithm is used for decoding. Based on the decoding results of the Viterbi algorithm, the globally optimal expert capability label sequence of the text to be processed is obtained.

Images