Chinese medical named entity recognition method and device based on multi-level adaptive semantic enhancement
By employing a multi-level adaptive semantic enhancement mechanism, combined with the ERNIE-Health pre-trained model and techniques such as convolution and gating, the semantic weights are dynamically adjusted and reconstructed, solving the problem of insufficient semantic difference capture capability in Chinese medical named entity recognition and improving the model's recognition performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF TECH
- Filing Date
- 2024-07-22
- Publication Date
- 2026-06-05
Smart Images
Figure CN118898252B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of named entity recognition technology, specifically to a method and apparatus for Chinese medical named entity recognition based on multi-level adaptive semantic enhancement. Background Technology
[0002] Named entity recognition (NER), an important task in natural language processing, aims to identify entities with specific meanings from Chinese text, such as place names, organization names, personal names, and dates. This task has wide applications in information extraction, question answering systems, and machine translation. Traditional NER methods include rule-based templates and statistical methods. Currently, deep learning-based methods are becoming a research hotspot, often combining neural networks and attention mechanisms to extract features and enhance the model's recognition capabilities.
[0003] The Chinese Named Entity Recognition (CNER) task in the Chinese medical field faces numerous challenges. For example, Chinese text lacks the use of spaces to separate words like English does, resulting in the absence of explicit word boundaries. Furthermore, the same character or word may have different meanings in different contexts, leading to polysemy and ambiguity. Although a large number of feature enhancement methods have been researched in CNER, existing literature lacks methods for dynamically adjusting semantic weights across different dimensions. Feature weights often tend to assign the same value across different contexts, thus limiting their ability to adjust feature weights based on different semantic contributions, resulting in insufficient ability of models to capture important semantic differences. Summary of the Invention
[0004] To overcome the above-mentioned shortcomings, this invention proposes a Chinese medical named entity recognition method and apparatus based on multi-level adaptive semantic enhancement in the medical field.
[0005] This invention dynamically adjusts and reconstructs semantic weights at both the character and sentence levels through an innovative compression and decompression mechanism, integrating the weight adjustments from the character and sentence domains into a unified and effective multi-level adaptive semantic enhancement module. Using the ERNIE-Health pre-trained model specifically designed for the medical field and employing techniques such as convolution, gating mechanisms, and multilayer perceptrons, the model's ability to recognize Chinese named entities is significantly improved. This method better captures semantic differences within the context, overcomes the limitations of existing methods in feature weight allocation, and enhances the overall performance of the CNER task.
[0006] The present invention achieves the above objectives through the following technical solutions:
[0007] A Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement includes the following steps:
[0008] (1) A Chinese text of length n is represented as T = {C1, C2, ..., C...} N}, construct character C i Features include character features, boundary features, radical features, and pinyin features;
[0009] (2) Character-level C is generated using the ERNIE-Health pre-trained model. i The features are transformed into vector representations, including character feature vectors e c Boundary eigenvector e b , radical feature vector e r Pinyin feature vector e p ;
[0010] (3) Input the four character-level features into the character layer adaptive semantic enhancement module. Use convolutional layers to compress the character features, perform nonlinear transformation through gating mechanism and ReLU activation function, dynamically adjust semantic weights, and use multilayer perceptron for decompression to obtain the enhanced character-level features;
[0011] (4) Input the enhanced character-level features into the sentence-level adaptive semantic enhancement module, and adaptively learn the contribution of different characters in the sentence through compression and decompression mechanisms;
[0012] (5) Input the enhanced features obtained by the multi-level adaptive semantic enhancement module into the BiLSTM-CRF module for label prediction.
[0013] Preferably, the boundary feature extraction in step (1) is as follows:
[0014] (1.1) Boundary feature extraction rules: Construct a medical dictionary D, match it with characters, and use the "BMESN" relation pattern to represent the boundary features of each character. Where B indicates that the character starts, M indicates that the character is inside a word, E indicates that the character ends, S indicates that the character is a word on its own, and N indicates that the character is not related to a word.
[0015] Preferably, step (2) converts the character features into a vector representation as follows:
[0016] (2.1) For character features, boundary features, radical features, and pinyin features, the ERNIE-Health pre-trained model is used to transform them into 50-dimensional feature vectors, with each character's vector representation being C. j It consists of four 50-dimensional character-level features:
[0017] C j =[(e c ) 50 , (eb ) 50 , (e r ) 50 , (e p ) 50 (1)
[0018] Among them, e c Represents the character feature vector, e b Represents the boundary eigenvector, e r Represents the feature vector of a radical, e p This represents the feature vector of the pinyin.
[0019] Preferably, step (3) involves representing C as a vector. j The adaptive semantic enhancement module at the input character layer dynamically adjusts semantic weights, including:
[0020] (3.1) Compression is performed using convolutional layers, and the feature vector is represented as a feature matrix as follows:
[0021]
[0022] The formula for convolution operation is as follows:
[0023]
[0024] Where X is the input feature matrix, k is the convolution kernel size, and W... conv It is the convolution kernel weight, b conv It is a bias;
[0025] (3.2) Nonlinear transformation is performed using a gating mechanism and the ReLU activation function to dynamically adjust semantic weights. The gating mechanism is used for feature selection:
[0026] G=σ(W g X conv +b g (4)
[0027] Among them, W g and b g These are the weights and biases of the gating mechanism, and σ is the sigmoid function;
[0028] The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional reconstructed semantic weight matrix, as shown in the following formula:
[0029] X relu =ReLU(G·X) conv (5)
[0030] (3.3) Use a multilayer perceptron (MLP) to decompress the low-dimensional semantic weight matrix back into a high-dimensional space to generate a high-dimensional weight matrix:
[0031] X expanded =MLP(X relu (6)
[0032] (3.4) Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced character-level semantic features:
[0033]
[0034] Preferably, step (4) inputs the enhanced character-level features into the sentence-level adaptive semantic enhancement module, and adaptively learns the contribution of different characters in the sentence through compression and decompression mechanisms, specifically including the following steps:
[0035] (4.1) Compression is performed using convolutional layers. First, T = {C1, C2, ..., C...} N The feature vector of each character is represented as a feature matrix:
[0036]
[0037] After convolutional compression, the 200-dimensional enhanced feature vector is transformed into a sentence-level one-dimensional feature vector:
[0038]
[0039] (4.2) Nonlinear transformation is performed using a gating mechanism and the ReLU activation function to dynamically adjust semantic weights. The gating mechanism is used to select features:
[0040]
[0041] Among them, W g and b g σ represents the weights and biases of the gating mechanism, and σ is the activation function.
[0042] The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional reconstructed semantic weight matrix, as shown in the following formula:
[0043]
[0044] (4.3) Use a multilayer perceptron (MLP) to decompress the low-dimensional semantic weight matrix back into the high-dimensional space to generate a high-dimensional weight matrix:
[0045] T expanded =MLP(T relu (12)
[0046] (4.4) Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced sentence-level semantic features:
[0047]
[0048] A second aspect of the present invention relates to a Chinese medical named entity recognition device based on multi-level adaptive semantic enhancement, comprising a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement a Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement of the present invention.
[0049] A third aspect of the present invention relates to a computer-readable storage medium, characterized in that it stores a program thereon, which, when executed by a processor, implements a Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement according to the present invention.
[0050] This invention dynamically adjusts and reconstructs semantic weights at both the character and sentence levels through an innovative compression and decompression mechanism, integrating the weight adjustments from the character and sentence domains into a unified and effective multi-level adaptive semantic enhancement module. Using the ERNIE-Health pre-trained model specifically designed for the medical field and employing techniques such as convolution, gating mechanisms, and multilayer perceptrons, the model's ability to recognize Chinese named entities is significantly improved. This method better captures semantic differences within the context, overcomes the limitations of existing methods in feature weight allocation, and enhances the overall performance of the CNER task.
[0051] The beneficial effects of this invention are as follows: The multi-level adaptive semantic enhancement mechanism (MASE) proposed in this invention significantly improves the performance of Chinese named entity recognition by dynamically adjusting and reconstructing semantic weights at both the character and sentence levels, combined with techniques such as convolution, self-attention, and multilayer perceptron; it can better capture semantic differences in context and solves the limitations of existing methods in feature weight allocation and the lack of explicit word boundaries. Attached Figure Description
[0052] Figure 1 This is a schematic diagram of the method flow of the present invention;
[0053] Figure 2 This is a schematic diagram of the overall framework of an embodiment of the present invention;
[0054] Figure 3 This is a schematic diagram of the character-level adaptive semantic enhancement structure according to an embodiment of the present invention;
[0055] Figure 4This is a schematic diagram of the sentence-level adaptive semantic enhancement structure according to an embodiment of the present invention. Detailed Implementation
[0056] The principles and technical solutions of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that the scope of protection of the present invention is not limited thereto.
[0057] Example 1
[0058] like Figure 2 As shown, a Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement is as follows:
[0059] Step 1: Construct character-level features, including character features, radical features, boundary features, and pinyin features, and use the ERNIE-Health pre-trained model to convert them into feature vectors. Taking the construction of boundary features as an example.
[0060] Assuming the input text is "Patients with hypertension and hyperglycemia need long-term treatment", and the matching dictionary D, the character features are shown in Table 1:
[0061] Table 1: Character Feature Table
[0062]
[0063] Step 2: Use the ERNIE-Health pre-trained model to convert it into feature vectors;
[0064] ERNIE-Health is a pre-trained model specifically designed for the medical field, based on the ERNIE (Enhanced Representation through Knowledge Integration) framework developed by Baidu. It utilizes large-scale text data from the medical field for pre-training and boasts advantages such as high accuracy, strong adaptability, and powerful knowledge integration capabilities.
[0065] Using ERNIE-Health, the four character-level features are transformed into four 50-dimensional character feature vectors e. c Boundary eigenvector e b , radical feature vector e r Pinyin feature vector e p ;
[0066] Step 3: Represent the vector C j The adaptive semantic enhancement module at the input character layer dynamically adjusts semantic weights, as shown in the framework. Figure 3 As shown.
[0067] Step 31: In the character-level adaptive semantic enhancement module, the input feature matrix X∈R 4×50Where 4 represents the sequence length and 50 represents the feature dimension. The feature matrix is as follows:
[0068]
[0069] Step 32: Compress using convolution, with a kernel size of 3 and a number of 64.
[0070]
[0071] Where X is the input feature matrix, k is the convolution kernel size, and W... conv It is the convolution kernel weight, b conv It is a bias;
[0072] Step 33: Perform non-linear transformation using a gating mechanism and the ReLU activation function to dynamically adjust semantic weights. Use a gating mechanism to select features:
[0073] G=σ(W g X conv +b g (4)
[0074] Among them, W g and b g These are the weights and biases of the gating mechanism, and σ is the sigmoid function;
[0075] The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional semantic weight matrix, as shown in the following formula:
[0076] X relu =ReLU(G·X) conv (5)
[0077] Step 34: Use a multilayer perceptron (MLP) (containing two fully connected layers, each with 128 neurons) to decompress the low-dimensional semantic weight matrix back into the high-dimensional space, generating a high-dimensional weight matrix:
[0078] X expanded =MLP(X relau (6)
[0079] Step 35: Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced character-level semantic features:
[0080]
[0081] Step 4: Input the enhanced character-level features into the sentence-level adaptive semantic enhancement module. Through compression and decompression mechanisms, it adaptively learns the contribution of different characters in the sentence. The framework is as follows: Figure 4 As shown. Specifically, it includes the following steps:
[0082] Step 41: In the sentence-level adaptive semantic enhancement module, input the feature matrix X∈R n×200 Where n is the sequence length and 200 is the feature dimension. The feature matrix is as follows:
[0083]
[0084] Step 42: Compress using convolutional layers. After convolutional compression, the 200-dimensional enhanced feature vector is transformed into a sentence-level one-dimensional feature vector:
[0085]
[0086] Step 43: Perform non-linear transformation using a gating mechanism and the ReLU activation function to dynamically adjust semantic weights. Use a gating mechanism to select features:
[0087]
[0088] Among them, W g and b g σ represents the weights and biases of the gating mechanism, and σ is the activation function.
[0089] The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional reconstructed semantic weight matrix, as shown in the following formula:
[0090]
[0091] Step 44: Use a multilayer perceptron (MLP) to decompress the low-dimensional semantic weight matrix back into a high-dimensional space to generate a high-dimensional weight matrix:
[0092] T expanded =MLP(T relu (12)
[0093] Step 45: Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced sentence-level semantic features:
[0094]
[0095] Example 2
[0096] This embodiment relates to a Chinese medical named entity recognition device based on multi-level adaptive semantic enhancement, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement of the present invention.
[0097] Example 3
[0098] This embodiment relates to a computer-readable storage medium storing a program that, when executed by a processor, implements a Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement according to the present invention.
[0099] The above description represents a preferred embodiment of the present invention. Even if some adjustments or changes are made according to the concept of the present invention, as long as the resulting new functions or effects still conform to the spirit and principles covered by the specification and drawings, these adjustments or changes should still be considered to fall within the protection scope of the present invention.
Claims
1. A Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement, characterized in that, Includes the following steps: (1) Length is n The Chinese text is represented as , construct characters Features include character features, boundary features, radical features, and pinyin features; (2) Character-level data is processed using the ERNIE-Health pre-trained model. The features are transformed into vector representations, including character feature vectors. Boundary feature vectors Radical feature vectors Pinyin feature vector ; (3) Input the four character-level features into the character layer adaptive semantic enhancement module; use convolutional layers to compress the character features, perform nonlinear transformation through gating mechanism and ReLU activation function, dynamically adjust semantic weights, and use multilayer perceptron to decompress to obtain the enhanced character-level features; (4) The enhanced character-level features are input into the sentence-level adaptive semantic enhancement module, which adaptively learns the contribution of different characters in the sentence through compression and decompression mechanisms; specifically including: (4.1) Compression is performed using convolutional layers, The feature vector of each character is represented as a feature matrix: After convolutional compression, the 200-dimensional enhanced feature vector is transformed into a sentence-level one-dimensional feature vector: (4.2) Nonlinear transformation is performed using gating mechanism and ReLU activation function to dynamically adjust semantic weights; gating mechanism is used to select features: in, and It refers to the weights and biases of the gating mechanism. It is an activation function; The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional reconstructed semantic weight matrix, as shown in the following formula: (4.3) Use a multilayer perceptron (MLP) to decompress the low-dimensional semantic weight matrix back into the high-dimensional space to generate a high-dimensional weight matrix: (4.4) Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced sentence-level semantic features: (5) Input the enhanced features obtained by the multi-level adaptive semantic enhancement module into the BiLSTM-CRF module for label prediction.
2. The Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement according to claim 1, characterized in that: Step (2) is as follows: (2.1) For character features, boundary features, radical features, and pinyin features, the ERNIE-Health pre-trained model is used to transform them into 50-dimensional feature vectors respectively. The vector representation of each character is as follows: It consists of four 50-dimensional character-level features: in, Represents the character feature vector. Represents the boundary feature vector. Represents the feature vector of a radical. This represents the feature vector of the pinyin.
3. The Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement according to claim 1, characterized in that: Step (3) Represent the vector The adaptive semantic enhancement module at the input character layer dynamically adjusts semantic weights, including: (3.1) Use convolutional layers for compression, representing the feature vectors as feature matrices: The formula for convolution operation is as follows: in, It is the input feature matrix. It is the kernel size. These are the convolution kernel weights. It is a bias; (3.2) Nonlinear transformation is performed using gating mechanism and ReLU activation function to dynamically adjust semantic weights; gating mechanism is used to select features: in, and It refers to the weights and biases of the gating mechanism. It is the sigmoid function; The gated features are non-linearly transformed using the ReLU activation function to obtain a low-dimensional reconstructed semantic weight matrix, as shown in the following formula: (3.3) Use a multilayer perceptron (MLP) to decompress the low-dimensional semantic weight matrix back into the high-dimensional space to generate a high-dimensional weight matrix: (3.4) Perform a Hadamard product operation on the original character-level semantic feature matrix and the decompressed weight matrix to obtain the enhanced character-level semantic features: 。 4. A Chinese medical named entity recognition device based on multi-level adaptive semantic enhancement, characterized in that, The device includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement as described in any one of claims 1-3.
5. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the Chinese medical named entity recognition method based on multi-level adaptive semantic enhancement as described in any one of claims 1-3.