Biomedical entity identification method, device, equipment, medium and product

Abstract

The invention discloses a biomedical entity recognition method, which comprises the following steps of: splicing an original text of a biomedical entity to be recognized with discrete prompts to form a template text; inputting the template text into a pre-training model embedded with continuous prompts, and outputting an entity label predicted by the model; and converting the entity label into an entity identification result based on a pre-constructed mapping table. The method can effectively improve the recognition efficiency, comprehensiveness and accuracy of a biomedical entity recognition task in a few-sample scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information recognition, and in particular to a biomedical entity recognition method, device, equipment, medium and product. BACKGROUND

[0002] Compared with the named entity recognition in the general field, the biomedical entity (BioNEs) recognition is much more difficult, which is usually composed of long words, an entity has multiple variant names, and is often composed of numbers and letter abbreviations, and the same letter may have different meanings. Deep learning has gradually become the mainstream method of BioNER recognition because it has a powerful feature extractor that can automatically and effectively capture the characteristics and rules of raw data.

[0003] Many existing named entity recognition methods require a large amount of labeled data, fine-tune the pre-trained model, and then predict new samples. However, the pre-trained model is mostly trained for general language model tasks and is not fully optimized for the named entity recognition task. In the general field with sufficient data, this gap may not be obvious, but in the limited biomedical field, this method faces challenges. The traditional deep learning method has limited accuracy and transferability. The "pre-training + fine-tuning" paradigm relies on a large amount of labeled data, and in the biomedical field with few samples, it is difficult to effectively reduce the gap between the pre-trained model and the fine-tuning, and it is difficult to fully extract information and complete the biomedical named entity recognition task well. SUMMARY

[0004] The embodiment of the present application provides a biomedical entity recognition method, which can effectively improve the recognition efficiency, comprehensiveness and accuracy of the biomedical entity recognition task in the few-sample scene.

[0005] In a first aspect, the embodiment of the present application provides a biomedical entity recognition method, comprising: Splicing the original text of the biomedical entity to be recognized and the discrete prompt to form a template text; Inputting the template text into a pre-trained model embedded with a continuous prompt, and outputting an entity label predicted by the model; Converting the entity label into an entity recognition result based on a pre-constructed mapping table.

[0006] Further, the discrete prompt is a fixed text sequence constructed based on the biomedical entity category, and the splicing of the original text of the biomedical entity to be recognized and the discrete prompt to form a template text comprises: Obtaining a data set type corresponding to the biomedical entity to be recognized, and generating a discrete prompt according to the data set type; Splicing the discrete prompt and the biomedical entity to be recognized to form a template text.

[0007] Further, the continuous prompt is a randomly initialized trainable continuous vector, and the continuous vector is embedded into a prefix position of each layer network of the pre-trained model.

[0008] Further, the pre-trained model optimizes parameters of the continuous prompt through a loss function during a training process. The loss function is a cross-entropy loss function, which is used to calculate a difference between an entity prediction result output by the model and a real entity category.

[0009] Further, the construction process of the mapping table includes: Obtaining a composite entity label in the biomedical field, eliminating meaningless connecting words in the composite entity label, and obtaining a basic word set; Performing similar word expansion on the basic word set to obtain a label word set; Establishing a corresponding relationship between words in the label word set and original entity categories to obtain the mapping table.

[0010] Further, the entity label is converted into an entity recognition result based on the pre-constructed mapping table, including: Extracting a label word with the highest probability from the entity label as a to-be-matched word; Obtaining a pre-constructed mapping table, searching for an original entity category corresponding to the to-be-matched word in the mapping table, and taking the original entity category as a final entity recognition result.

[0011] In a second aspect, an embodiment of the present application provides a biomedical entity recognition device, including: A prompt splicing module is configured to splice an original text of a biomedical entity to be recognized and a discrete prompt to form a template text. A model prediction module is configured to input the template text into a pre-trained model embedded with a continuous prompt, and output an entity label predicted by the model. A label conversion module is configured to convert the entity label into an entity recognition result based on a pre-constructed mapping table.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including: A memory is configured to store a computer program. A processor is configured to execute the computer program. When the processor executes the computer program, the processor implements the biomedical entity recognition method in any one of the first aspect.

[0013] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program. The computer program is executed to implement the biomedical entity recognition method according to any one of the first aspect.

[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. The computer instructions are executed by a processor to implement the biomedical entity recognition method according to any one of the first aspect.

[0015] Compared with the prior art, the biomedical entity recognition method provided by the embodiment of the present application has the beneficial effects that: the original text of the biomedical entity to be recognized is spliced with discrete prompts to form a template text; the template text is input into a pre-trained model embedded with continuous prompts, and an entity label predicted by the model is output; and the entity label is converted into an entity recognition result based on a pre-constructed mapping table. The present application can effectively improve the recognition efficiency, comprehensiveness and accuracy of the biomedical entity recognition task in a few-sample scenario. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical features of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0017] Figure 1 is a flowchart of a biomedical entity recognition method provided by an embodiment of the present application; Figure 2 is a structural diagram of a biomedical entity recognition device provided by an embodiment of the present application; Figure 3 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0019] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the description and claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing example embodiments of the present application only and is not intended to be limiting of the present application.

[0021] In a first aspect, embodiments of the present application provide a biomedical entity recognition method, see Figure 1 A flowchart of an embodiment of a biomedical entity recognition method provided by the present application.

[0022] As Figure 1 shown, the method comprises the following steps: S1: Splicing the original text of the biomedical entity to be recognized and the discrete prompt to form a template text; Step S1 splices the original text of the biomedical entity to be recognized and the selected discrete prompt, wherein the discrete prompt is a fixed text sequence containing core words strongly related to the semantic of the target entity category. By providing explicit task guidance for the pre-trained model through the discrete prompt, the model can be guided to focus on the target biomedical entity category.

[0023] S2: Inputting the template text into the pre-trained model embedded with continuous prompts, and outputting the entity label predicted by the model; A pre-trained model dedicated to the biomedical field is selected, and a randomly initialized trainable continuous vector is used as a continuous prompt. The continuous prompt vector is embedded into the prefix position of each layer of the Transformer network of the pre-trained model. The template text generated in step S1 is input into the pre-trained model to form hierarchical input features of the continuous prompt vector + template text vector. The model outputs the predicted entity label corresponding to the original text.

[0024] S3: Converting the entity label into an entity recognition result based on a pre-constructed mapping table.

[0025] Load the pre-constructed mapping table of entity labels and original entity categories. Based on the mapping table, convert the entity label into the final entity recognition result.

[0026] In summary, the present application provides explicit task guidance for the model by splicing the original text of the biomedical entity to be identified and the discrete prompt into a template text, so that the model can quickly locate the key information when processing the input text, thereby improving the recognition efficiency and accuracy; the continuous prompt of the embedded pre-training model can implicitly guide the model to capture the semantic features of the biomedical entity, helping the model better understand the context relationship and semantic connotation of the biomedical entity, thereby improving the accuracy of recognition; finally, with the help of the pre-constructed mapping table, the entity label output by the model is accurately converted into a standard entity recognition result, improving the compatibility of the vocabulary variants, and the three work together to achieve efficient and accurate recognition effect in the few-shot scenario of the biomedical entity recognition task.

[0027] In an optional implementation, the discrete prompt is a fixed text sequence constructed based on the biomedical entity category, and the splicing of the original text of the biomedical entity to be identified and the discrete prompt to form a template text comprises: obtaining a data set type corresponding to the biomedical entity to be identified, and generating a discrete prompt according to the data set type; splicing the discrete prompt and the biomedical entity to be identified to form a template text.

[0028] Specifically, the discrete prompt is a fixed text sequence constructed based on the biomedical entity to be identified, and its generation process needs to strictly correspond to the data set type to which the text to be identified belongs. First, the data set type corresponding to the original text of the biomedical entity to be identified is obtained. For example, the data set can be JNLPBA data set, NCBI-disease data set and CORD-NER data set, etc. Different data sets correspond to different core recognition entity categories: JNLPBA data set: the core recognition entity is a biological macromolecule and cell structure entity such as protein, DNA, cell, genome, etc. NCBI-disease data set: the core recognition entity is a disease entity such as disease, cancer, syndrome, etc. CORD-NER data set: the core recognition entity is a multi-category mixed entity such as genes, diseases, organizations, etc.

[0029] According to the determined data set type, a discrete prompt is generated. For example, when the data set is the JNLPBA data set, the generated discrete prompt can be "Find words related to protein, DNA, cell, etc.", when the data set is the NCBI-disease data set, the generated discrete prompt can be "Find words related to disease, cancer, syndrome, etc.", and when the data set is the CORD-NER data set, the generated discrete prompt can be "Find out about genes, diseases, organizations, etc.".

[0030] After the matching generation of the discrete prompt is completed, the discrete prompt is combined with the original text of the biomedical entity to be recognized in a prefix splicing or suffix splicing manner to generate a template text that can be directly input to the model.

[0031] The embodiment can provide more targeted and guided input information for the model by accurately generating a discrete prompt corresponding to the data set type to which the biomedical entity to be recognized belongs, and then splicing the discrete prompt and the original text into a template text, so that the model can more clearly identify the core entity category of the task, effectively improve the recognition accuracy and efficiency of the model for biomedical entities, and the different discrete prompts corresponding to different data sets enhance the adaptability and flexibility of the method, which can be widely used in entity recognition tasks in various biomedical data scenarios.

[0032] In an optional implementation, the continuous prompt is a randomly initialized trainable continuous vector, and the continuous vector is embedded into a prefix position of each layer network of the pre-training model.

[0033] Specifically, the continuous prompt is a randomly initialized trainable continuous vector, and the core role is to assist the pre-training model in accurately capturing the semantic features of biomedical entities in a small sample scenario through implicit feature guidance. The dimension parameter of the continuous prompt is matched with the hidden layer dimension of the pre-training model to ensure that the vector features are compatible with the semantic representation space of the model.

[0034] Each layer of the pre-training model includes an attention mechanism and a feedforward neural network. In the input stage of each layer of the network, the continuous prompt vector is spliced to the head of the input sequence to form a combined input feature of "continuous prompt vector + template text vector". The continuous prompt embedded in the prefix provides implicit feature guidance, while the discrete prompt spliced at the head of the template text provides explicit task guidance. The two work together to achieve the dual effects of implicit feature enhancement and explicit task focus.

[0035] The embodiment takes a randomly initialized and trainable continuous vector as a continuous prompt, embeds the pre-trained model in each layer network prefix position, cooperates with a discrete prompt that provides explicit task guidance, realizes both implicit feature enhancement and auxiliary model accurate capture of biomedical entity semantic features in a few-shot scenario, and achieves explicit task focus, effectively improving the recognition ability and performance of the model on biomedical entities.

[0036] In an optional implementation, the pre-trained model optimizes the parameters of the continuous prompt through a loss function during the training process. The loss function is a cross-entropy loss function, which is used to calculate the difference between the entity prediction result output by the model and the real entity category.

[0037] Specifically, the pre-trained model adopts a parameter freezing strategy in the training phase, and only optimizes the vector parameters of the continuous prompt through the cross-entropy loss function, so as to realize efficient training and accurate recognition in a few-shot scenario.

[0038] During the entire training process, all network parameters of the pre-trained model are fixed and not updated, avoiding model overfitting caused by few-shot data fine-tuning, while retaining the general biomedical semantic knowledge learned by the pre-trained model to provide a solid feature basis for the entity recognition task.

[0039] The cross-entropy loss function calculates the difference between the entity prediction result output by the model and the real entity category, and updates the vector parameters of the continuous prompt with the cross-entropy loss value as the optimization target.

[0040] The embodiment adopts a parameter freezing strategy to fix all network parameters of the pre-trained model during training, and only optimizes the vector parameters of the continuous prompt through the cross-entropy loss function, which not only avoids the problem of model overfitting caused by few-shot data fine-tuning, but also retains the general biomedical semantic knowledge learned by the pre-trained model to provide a reliable feature basis for entity recognition. At the same time, the cross-entropy loss value is used as the optimization target to update the continuous prompt parameters, realizing efficient training in a few-shot scenario and effectively improving the accuracy of biomedical entity recognition.

[0041] In an optional implementation, the construction process of the mapping table is as follows: Obtain a composite entity label in the biomedical field, remove meaningless connecting words in the composite entity label, and obtain a basic word set; Expand the basic word set to obtain a label word set; Establish the correspondence between the words in the label word set and the original entity categories to obtain the mapping table.

[0042] Specifically, the mapping table is a core bridge connecting the label words of the model output and the standard biomedical entity categories. In the construction process, first, the composite entity labels in the biomedical field are obtained. Such labels contain meaningless connecting words, which will interfere with the recognition accuracy of the model. The meaningless connecting words are removed, the core words with actual semantics are extracted, and the core words after removing the connecting words are combined to form a basic word set corresponding to each composite label.

[0043] To improve the compatibility of the model to entity word variants, the basic word set is expanded with semantically similar words. For example, the Embedding layer of the biomedical field special pre-training model can be used to calculate the semantic similar words of the basic words, and the label word set is expanded to form a label word set with a scale of 3-4 words. This avoids the calculation redundancy caused by an excessively large word set, and still maintains high efficient recognition performance under a small sample data.

[0044] The embodiment establishes the corresponding relationship between the label word set and the original entity category to form a mapping table, which builds a bridge for the model output and the standard entity category, and still guarantees efficient and accurate biomedical entity recognition performance under a small sample data.

[0045] In an optional implementation, the converting the entity label into an entity recognition result based on the pre-constructed mapping table comprises: extracting the label word with the highest probability from the entity label as a to-be-matched word; obtaining a pre-constructed mapping table, searching for an original entity category corresponding to the to-be-matched word in the mapping table, and taking the original entity category as the final entity recognition result.

[0046] Specifically, after the model completes the feature calculation of the template text, the entity label word corresponding to the to-be-recognized text is output, the label word with the highest probability is selected, and the label word is taken as the to-be-matched word.

[0047] The pre-constructed mapping table stores the corresponding relationship between the label word and the original entity category. The original entity category corresponding to the to-be-matched word is searched in the mapping table, and the original entity category is taken as the final entity recognition result.

[0048] The embodiment quickly and accurately finds the original entity category corresponding to the to-be-matched word as the final recognition result by means of the pre-constructed mapping table, thereby improving the accuracy and efficiency of biomedical entity recognition.

[0049] In a second aspect, the embodiment of the present application provides a biomedical entity recognition device, which is shown in Figure 2 The structure of an embodiment of the biomedical entity recognition device provided by the present application is shown in the figure.

[0050] As Figure 2As shown, the device includes: The prompt splicing module 21 is used to splice the original text of the biomedical entity to be identified with discrete prompts to form template text; The model prediction module 22 is used to input the template text into a pre-trained model embedded with continuous prompts and output the entity labels predicted by the model. The label conversion module 23 is used to convert the entity labels into entity recognition results based on a pre-built mapping table.

[0051] In one optional implementation, the discrete prompt is a fixed text sequence constructed based on biomedical entity categories, and the prompt splicing module 21 is specifically used for: Obtain the dataset type corresponding to the biomedical entity to be identified, and generate discrete prompts based on the dataset type; The discrete prompts are combined with the biomedical entity to be identified to form template text.

[0052] In one alternative implementation, the continuous cue is a randomly initialized trainable continuous vector, which is embedded into the prefix position of each layer of the pre-trained model.

[0053] In one alternative implementation, the pre-trained model optimizes the parameters of the continuous prompts using a loss function during the training process. The loss function is the cross-entropy loss function, which is used to calculate the difference between the entity prediction results output by the model and the actual entity categories.

[0054] In one optional implementation, the mapping table is constructed as follows: Obtain composite entity tags in the biomedical field, remove meaningless conjunctions from the composite entity tags, and obtain a basic word set; The basic word set is expanded with similar words to obtain a tag word set; Establish the correspondence between the vocabulary in the tag set and the original entity categories to obtain the mapping table.

[0055] In one optional implementation, the label conversion module 23 is specifically used for: Extract the tag word with the highest probability from the entity tags and use it as the word to be matched; Obtain a pre-built mapping table, find the original entity category corresponding to the word to be matched in the mapping table, and use it as the final entity recognition result.

[0056] It should be noted that the biomedical entity recognition device provided in this embodiment of the invention is used to execute all the process steps of the biomedical entity recognition method in the above embodiment. The working principle and beneficial effects of the two are one-to-one, so they will not be described again.

[0057] Thirdly, embodiments of the present invention provide an electronic device, see [link to previous document]. Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention.

[0058] like Figure 3 As shown, the device includes: Memory 31 is used to store computer programs; Processor 32 is used to execute the computer program; When the processor 32 executes the computer program, it implements the biomedical entity recognition method as described in any of the above embodiments.

[0059] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 32 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0060] The processor 32 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0061] The memory 31 can be used to store the computer programs and / or modules. The processor 32 implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory 31 and calling the data stored in the memory 31. The memory 31 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 31 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0062] It should be noted that the aforementioned electronic devices include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 3 The structural diagram is merely an example of the electronic device described above and does not constitute a limitation on the electronic device. It may include more components than shown in the diagram, or combine certain components, or use different components.

[0063] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed, implements the biomedical entity recognition method described in any of the above embodiments.

[0064] It should be understood that the implementation of all or part of the processes in the above-described biomedical entity recognition method can also be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above-described biomedical entity recognition method. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0065] Fifthly, embodiments of this application also provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the biomedical entity recognition method described in any of the above embodiments.

[0066] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. It should be noted that, for those skilled in the art, several equivalent obvious modifications and / or equivalent substitutions can be made without departing from the technical principles of the present invention, and these obvious modifications and / or equivalent substitutions should also be considered within the scope of protection of the present invention.

Claims

1. A biomedical entity recognition method, characterized in that, include: The original text of the biomedical entity to be identified is concatenated with discrete prompts to form template text; The template text is input into a pre-trained model embedded with continuous prompts, and the entity labels predicted by the model are output. Based on a pre-built mapping table, the entity labels are converted into entity recognition results.

2. The biomedical entity recognition method as described in claim 1, characterized in that, The discrete prompts are fixed text sequences constructed based on biomedical entity categories. The process of concatenating the original text of the biomedical entity to be identified with the discrete prompts to form template text includes: Obtain the dataset type corresponding to the biomedical entity to be identified, and generate discrete prompts based on the dataset type; The discrete prompts are combined with the biomedical entity to be identified to form template text.

3. The biomedical entity recognition method as described in claim 1, characterized in that, The continuous prompts are randomly initialized trainable continuous vectors, which are embedded into the prefix position of each layer of the pre-trained model.

4. The biomedical entity recognition method as described in claim 1, characterized in that, During the training process, the pre-trained model optimizes the parameters of the continuous prompts using a loss function. The loss function is the cross-entropy loss function, which is used to calculate the difference between the entity prediction results output by the model and the actual entity categories.

5. The biomedical entity recognition method as described in claim 1, characterized in that, The process of constructing the mapping table is as follows: Obtain composite entity tags in the biomedical field, remove meaningless conjunctions from the composite entity tags, and obtain a basic word set; The basic word set is expanded with similar words to obtain a tag word set; Establish the correspondence between the vocabulary in the tag set and the original entity categories to obtain the mapping table.

6. The biomedical entity recognition method as described in claim 1, characterized in that, The process of converting entity labels into entity recognition results based on a pre-built mapping table includes: Extract the tag word with the highest probability from the entity tags and use it as the word to be matched; Obtain a pre-built mapping table, find the original entity category corresponding to the word to be matched in the mapping table, and use it as the final entity recognition result.

7. A biomedical entity recognition device, characterized in that, include: The prompt concatenation module is used to concatenate the original text of the biomedical entity to be identified with discrete prompts to form template text; The model prediction module is used to input the template text into a pre-trained model embedded with continuous prompts and output the entity labels predicted by the model. The label conversion module is used to convert the entity labels into entity recognition results based on a pre-built mapping table.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program; Wherein, when the processor executes the computer program, it implements the biomedical entity recognition method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the biomedical entity recognition method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the biomedical entity recognition method as described in any one of claims 1 to 6.

Images