Model generation, entity recognition methods, devices, electronic equipment and storage media

Abstract

This invention discloses a model generation and entity recognition method, apparatus, electronic device, and storage medium. The model generation method includes: acquiring a pre-constructed initial entity recognition model, wherein the initial entity recognition model includes a language network, a sentence contrast learning network, and a character contrast learning network connected to the language network; the language network outputs text representations of sample texts; inputting the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network respectively; and adjusting the network parameters in the language network based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network to obtain a target entity recognition model containing the trained language network. The technical solution of this invention can generate a target entity recognition model with high entity recognition accuracy.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of natural language processing, and in particular to a model generation, entity recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] Entity recognition is a fundamental technology in the field of natural language processing and is now widely used in various practical engineering projects.

[0003] In the process of realizing this invention, the inventors discovered the following technical problems in the prior art: the accuracy of entity recognition needs to be improved. Summary of the Invention

[0004] This invention provides a model generation, entity recognition method, apparatus, electronic device, and storage medium to generate a target entity recognition model with high entity recognition accuracy.

[0005] According to one aspect of the present invention, a model generation method is provided, which may include:

[0006] Obtain the constructed initial entity recognition model, which includes a language network, a sentence contrast learning network and a character contrast learning network connected to the language network, and the language network is used to output the text representation of the sample text;

[0007] The text representations of multiple sample texts are input into the sentence contrast learning network and the character contrast learning network, respectively. Based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain a target entity recognition model that includes the trained language network.

[0008] According to another aspect of the present invention, an entity recognition method is provided, which may include:

[0009] Obtain the text to be recognized and the target entity recognition model generated by the model generation method provided in any embodiment of the present invention;

[0010] The text to be recognized is input into the target entity recognition model, and the entity information in the text is obtained based on the output of the target entity recognition model.

[0011] According to another aspect of the present invention, a model generation apparatus is provided, which may include:

[0012] The initial entity recognition model acquisition module is used to acquire the constructed initial entity recognition model, which includes a language network, a sentence contrast learning network and a character contrast learning network connected to the language network, and the language network is used to output the text representation of the sample text.

[0013] The target entity recognition model module is used to input the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network, respectively. Based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain the target entity recognition model containing the trained language network.

[0014] According to another aspect of the present invention, an entity recognition device is provided, which may include:

[0015] The target entity recognition model acquisition module is used to acquire the text to be recognized and the target entity recognition model generated according to the model generation method provided in any embodiment of the present invention.

[0016] The entity information acquisition module is used to input the text to be recognized into the target entity recognition model and obtain the entity information in the text to be recognized based on the output of the target entity recognition model.

[0017] According to another aspect of the present invention, an electronic device is provided, which may include:

[0018] At least one processor; and

[0019] A memory that is communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, such that when the at least one processor executes the program, it implements the model generation method or entity recognition method provided in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, having stored thereon computer instructions that cause a processor to execute and implement the model generation method or entity recognition method provided in any embodiment of the present invention.

[0022] The technical solution of this invention involves acquiring a pre-constructed initial entity recognition model. This initial model includes a language network for outputting text representations of sample texts, and a sentence contrast learning network and a character contrast learning network connected to the language network. The text representations of multiple sample texts are input into the sentence contrast learning network and the character contrast learning network, respectively. Then, based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted, thereby obtaining a target entity recognition model containing the trained language network. This technical solution, through the cooperation of the sentence contrast learning network constructed based on a sentence-level contrast learning strategy and the character contrast learning network constructed based on a character-level contrast learning strategy, fine-tunes the language network. This results in text representations output by the fine-tuned language network having better semantic quality at both the sentence and character levels, thus obtaining a language network highly relevant to the entity recognition task and ensuring the recognition accuracy of the generated target entity recognition model.

[0023] It should be understood that the description in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a model generation method provided according to an embodiment of the present invention;

[0026] Figure 2 This is a flowchart of another model generation method provided according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional example of another model generation method provided according to an embodiment of the present invention;

[0028] Figure 4 This is a flowchart of an entity recognition method provided according to an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of an optional example of an entity recognition method provided according to an embodiment of the present invention;

[0030] Figure 6This is a structural block diagram of a model generation device provided according to an embodiment of the present invention;

[0031] Figure 7 This is a structural block diagram of an entity recognition device provided according to an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the structure of an electronic device that implements the model generation method or entity recognition method of the embodiments of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Before introducing the embodiments of the present invention, an exemplary application scenario is provided: Taking the text to be identified as "Chongqing Yangtze River Bridge" as an example, existing entity recognition models may identify "Chongqing," "Mayor," and "Yangtze River Bridge," which are respectively city entities, title entities, and personal name entities. Although "Mayor" is indeed a title entity, "Chongqing" is indeed a city entity, and "Yangtze River Bridge" is also likely a personal name entity, for the text to be identified, none of these three entities should be identified. The correct entities to be identified should be "Chongqing" and "Yangtze River Bridge." Analysis suggests that the above-mentioned entity recognition error is because the entity recognition model only memorizes common, frequently occurring words during training. Therefore, when these words are present in the text to be identified, the entity recognition model is likely to consider these words as entities without considering whether these words are actually entities to be identified in the context of the text, thus failing to guarantee the accuracy of entity recognition.

[0036] Figure 1 This is a flowchart of a model generation method provided in this embodiment of the invention. This embodiment is applicable to generating target entity recognition models with high entity recognition accuracy, especially for generating target entity recognition models with high entity recognition accuracy through sentence-level contrastive learning strategies and word-level contrastive learning strategies. This method can be executed by the model generation apparatus provided in this embodiment of the invention. This apparatus can be implemented by software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0037] See Figure 1 The method of this invention specifically includes the following steps:

[0038] S110. Obtain the constructed initial entity recognition model, which includes a language network, a sentence contrast learning network and a character contrast learning network connected to the language network, and the language network is used to output the text representation of the sample text.

[0039] The initial entity recognition model can be a pre-built entity recognition model to be trained, which may include a language network and sentence contrast learning networks and character contrast learning networks connected to the language network. Specifically, the language network can be used to output the text representation of the sample text, that is, it can perform semantic modeling on the sample text input to itself, thereby outputting the corresponding semantic vector (i.e., text representation). In practical applications, the language network can also be called a language representation model, which can be represented in various ways, such as topic models, fixed representation models based on word vectors, or dynamic representation models based on word vectors. In practical applications, the Bidirectional Encoder Representations from Transformers (BERT) model, as a dynamic representation model based on word vectors, is widely used in practical engineering.

[0040] Research has shown that the quality of the language network directly impacts the accuracy of downstream entity recognition tasks. Therefore, to ensure the accuracy of the trained target entity recognition model, this invention connects a pre-constructed sentence contrastive learning network and a character contrastive learning network to the language network. These two contrastive learning networks fine-tune the language network, resulting in a language network highly relevant to the entity recognition task. Specifically, the sentence contrastive learning network can be understood as a network constructed based on a sentence-level contrastive learning strategy. It optimizes the language network at the sentence level, resulting in better semantic quality in the text representation output at that level. The character contrastive learning network can be understood as a network constructed based on a character-level contrastive learning strategy. It considers that in the entity recognition process, not only are sentence vectors (i.e., text representations) calculated, but more importantly, character vectors (i.e., character representations in the text representation) are also classified. This network is constructed by classifying each sample character in the sample text, thereby optimizing the language network at the character level and resulting in better semantic quality in the text representation output at that level. The specific construction of these two contrastive learning networks will be described in detail in the next step.

[0041] S120. Input the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network respectively. Based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, adjust the network parameters in the language network to obtain a target entity recognition model containing the trained language network.

[0042] For each sample text, the language network can output at least two text representations of the sample text, such as inputting the sample text into the language network at least twice to obtain the at least two text representations; or performing text enhancement on the sample text to obtain at least one enhanced text, and inputting the sample text and the at least one enhanced text into the language network respectively to obtain the at least two text representations; and so on, without specific limitations.

[0043] Building upon this, at least two text representations from multiple sample texts can be input into a sentence contrast learning network to obtain the sentence contrast loss value calculated by the sentence contrast loss in the sentence contrast learning network. Similarly, at least two text representations from multiple sample texts can be input into a character contrast learning network to obtain the character contrast loss value calculated by the character contrast loss in the character contrast learning network. Furthermore, the network parameters in the language network are adjusted based on the sentence contrast loss value and the character contrast loss value, either separately or jointly (e.g., based on their sum). This achieves the effect of fine-tuning the language network, resulting in a language network closely related to the entity recognition task. A target entity recognition model can then be obtained based on this language network.

[0044] It's important to note that, on one hand, the combination of sentence contrastive learning networks and character contrastive learning networks allows the model to learn differentially during training. This enables the trained entity recognition model to identify entity information in the text from a holistic perspective, moving beyond simply checking for entity presence. Instead, it identifies entities relevant to the context based on the overall meaning of the text. On the other hand, subsequent entity recognition models do not need to include sentence and character contrastive learning networks. These networks are built to help the language network learn how to output semantically better text representations. Once the language network has completed its learning (training), the tasks of these two contrastive learning networks are finished.

[0045] The technical solution of this invention involves acquiring a pre-constructed initial entity recognition model. This initial model includes a language network for outputting text representations of sample texts, and a sentence contrast learning network and a character contrast learning network connected to the language network. The text representations of multiple sample texts are input into the sentence contrast learning network and the character contrast learning network, respectively. Then, based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted, thereby obtaining a target entity recognition model containing the trained language network. This technical solution, through the cooperation of the sentence contrast learning network constructed based on a sentence-level contrast learning strategy and the character contrast learning network constructed based on a character-level contrast learning strategy, fine-tunes the language network. This results in text representations output by the fine-tuned language network having better semantic quality at both the sentence and character levels, thus obtaining a language network highly relevant to the entity recognition task and ensuring the recognition accuracy of the generated target entity recognition model.

[0046] Building upon this, an optional technical solution involves determining the sentence contrast loss value based on sentence contrast loss. Sentence contrast loss is constructed from positive sentence samples composed of different text representations of the same sample text, and negative sentence samples composed of text representations of different sample texts. Specifically, a positive sentence sample can be understood as a sentence sample composed of different text representations of the same sample text, meaning each text representation corresponds to the same sample text; a negative sentence sample can be understood as a sentence sample composed of text representations of different sample texts, meaning each text representation corresponds to a different sample text. The sentence contrast loss in the sentence contrast learning network constructed based on the aforementioned positive and negative sentence samples allows similar sample texts to be closer in the feature space, and dissimilar sample texts to be farther apart in the feature space, thereby enabling the language network to learn overall semantic information at the sentence level.

[0047] Another optional technical solution involves sample text containing sample characters. The character contrast loss value is determined based on the character contrast loss, which is constructed from positive and negative character samples composed of character representations from different text representations of the same sample text. Positive character samples correspond to sample characters of the same entity category, while negative character samples correspond to sample characters of different entity categories. Specifically, a positive character sample can be understood as a character sample composed of character representations of sample characters corresponding to the same entity category from different text representations of the same sample text; that is, the sample characters corresponding to each character representation belong to the same entity category. A negative character sample can be understood as a character sample composed of character representations of sample characters corresponding to different entity categories from different text representations of the same sample text; that is, the sample characters corresponding to each character representation belong to different entity categories. For example, suppose we perform text augmentation on the statement "I study at XX University in Beijing" (hereinafter referred to as A), resulting in "I study at XX School in Beijing" (hereinafter referred to as B). Then, the word samples formed by the word representations corresponding to XX University in A's text representation and XX School in B's text representation are positive word samples (both corresponding to the school entity). Conversely, the word samples formed by the word representations corresponding to XX University in A's text representation and Beijing's text representation in B's text representation are negative word samples (the former being the school entity, and the latter the city entity). The word contrast loss in the word contrast learning network constructed based on these positive and negative word samples allows similar word samples to be closer together in the feature space, and dissimilar word samples to be farther apart, thus enabling the language network to learn the differences and connections between different word samples.

[0048] Based on this, optionally, the above model generation method further includes: for each sample text, obtaining a first enhanced text and a second enhanced text of the sample text, and inputting the sample text, the first enhanced text, and the second enhanced text into the language network respectively to obtain the text representation of the sample text, the text representation of the first enhanced text, and the text representation of the second enhanced text; using the text representation of the first enhanced text and the text representation of the second enhanced text as the text representation of the sample text respectively to obtain different text representations of the sample text. Specifically, the text enhancement of the sample text to obtain the first enhanced text and the second enhanced text can be achieved through methods such as replacing or deleting any sample character in the sample text, or replacing any sample character with a synonym or antonym, etc., without specific limitations. Further, the first enhanced text and the second enhanced text are input into the language network respectively to obtain the text representations of these two enhanced texts, and these are used as the text representations of the sample text respectively, thus obtaining different text representations of the sample text, thereby ensuring the effective construction of subsequent sentence positive samples and character samples.

[0049] Another optional technical solution is that the initial entity recognition model also includes an entity recognition network connected to the language network. The above model generation method may further include: for each sample text, obtaining the entity annotation result of the sample text, and using the text representation and entity annotation result of the sample text as a set of training samples; inputting multiple sets of training samples into the entity recognition network to obtain the entity recognition loss value output by the entity recognition model; accordingly, adjusting the network parameters in the language network according to the sentence comparison loss value output by the sentence comparison learning network and the character comparison loss value output by the character comparison learning network to obtain the target entity recognition model containing the trained language network. This may include: adjusting the network parameters in the language network according to the entity recognition loss value, the sentence comparison loss value output by the sentence comparison learning network, and the character comparison loss value output by the character comparison learning network to obtain the target entity recognition model containing the trained language network. The entity recognition network can be used to identify entities based on the text representation output by the language network. In practical applications, it can be constructed using various methods such as Conditional Random Field (CRF), Bidirectional Long Short-Term Memory (BiLSTM), or CRF+BiLSTM. The specific construction method depends on the application scenario of the entity recognition task and is not specifically limited here. The entity annotation result can be the result of annotating the entity information in the sample text. The text representation of the sample text and the entity annotation result are used as a set of training samples. Then, multiple sets of training samples are input into the entity recognition network to obtain the entity recognition loss value output by the entity recognition model. Furthermore, the network parameters can be adjusted based on the entity recognition loss value, sentence comparison loss value, and character comparison loss value, thereby further improving the semantic quality of the text representation output by the language network.

[0050] Figure 2This is a flowchart of another model generation method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, the sentence comparison learning network includes a sentence similarity determination subnetwork and a sentence comparison loss value determination subnetwork constructed based on sentence comparison loss; inputting the text representations of multiple sample texts into the sentence comparison learning network, and adjusting the network parameters in the language network according to the sentence comparison loss value output by the sentence comparison learning network, may include: inputting the text representations of multiple sample texts into the sentence similarity determination subnetwork to obtain the sentence similarity between any two text representations of the multiple sample texts; inputting the sentence similarity between any two text representations into the sentence comparison loss value determination subnetwork, and adjusting the network parameters in the language network according to the sentence comparison loss value output by the sentence comparison loss value determination subnetwork. Alternatively, the character contrast learning network includes a character similarity determination subnetwork and a character contrast loss value determination subnetwork constructed based on character contrast loss. The text representations of multiple sample texts are input into the character contrast learning network, and the network parameters in the language network are adjusted according to the character contrast loss value output by the character contrast learning network. This includes: for each sample text, inputting different text representations of the sample text into the character similarity determination subnetwork to obtain the character similarity between any two text representations of the sample text; inputting the character similarity between any two text representations into the character contrast loss value determination subnetwork, and adjusting the network parameters in the language network according to the character contrast loss value output by the character contrast loss value determination subnetwork. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0051] See Figure 2 The method in this embodiment may specifically include the following steps:

[0052] S210. Obtain the constructed initial entity recognition model, which includes a language network, a sentence contrast learning network and a character contrast learning network connected to the language network. The language network is used to output the text representation of the sample text, which contains sample characters. The sentence contrast learning network includes a sentence similarity determination subnetwork and a sentence contrast loss value determination subnetwork constructed based on the sentence contrast loss. The sentence contrast loss is constructed based on positive sentence samples composed of different text representations of the same sample text and negative sentence samples composed of text representations of different sample texts. The character contrast learning network includes a character similarity determination subnetwork and a character contrast loss value determination subnetwork constructed based on the character contrast loss. The character contrast loss is constructed based on positive character samples and negative character samples composed of character representations in different text representations of the same sample text. The character representations in the positive character samples correspond to sample characters of the same entity category, and the character representations in the negative character samples correspond to sample characters of different entity categories.

[0053] The sentence similarity determination subnetwork can be used to determine the sentence similarity between text representations in any sentence sample. The sentence contrast loss determination subnetwork, built based on sentence contrast loss, calculates the sentence contrast loss value based on the similarity of each sentence; that is, sentence similarity represents the distance between two sample texts in the feature space. Similarly, the character similarity determination subnetwork can be used to determine the character similarity between character representations in any character sample. The character contrast loss determination subnetwork, built based on character contrast loss, calculates the character contrast loss value based on the similarity of each character; that is, character similarity represents the distance between two sample characters in the feature space.

[0054] S220. Input the text representations of multiple sample texts into the sentence similarity determination subnetwork to obtain the sentence similarity between any two text representations of the multiple sample texts.

[0055] S230. Input the sentence similarity between any two text representations into the sentence contrast loss value determination subnetwork, determine the sentence contrast loss value output by the subnetwork based on the sentence contrast loss value, and adjust the network parameters in the language network.

[0056] S240. For each sample text, input the different text representations of the sample text into the character similarity determination subnetwork to obtain the character similarity between any two text representations of the sample text.

[0057] Since the character samples are constructed based on the character representations of the sample characters corresponding to the same sample text, each sample text is processed separately.

[0058] S250. Input the word similarity between word representations in any two text representations into the word contrast loss value determination subnetwork, and determine the word contrast loss value output by the subnetwork based on the word contrast loss value, and adjust the network parameters in the language network.

[0059] S260. Based on the adjustment results of the network parameters, a target entity recognition model containing the trained language network is obtained.

[0060] The technical solution of this invention uses sentence similarity to represent the distance between two sample texts in the feature space, and uses character similarity to represent the distance between two sample characters in the feature space. Then, the network parameters are adjusted based on the contrast loss value calculated based on the similarity, thus achieving the effect of fine-tuning the language network.

[0061] One optional technical solution involves representing the sentence contrast loss using a sentence fraction. One of the numerator and denominator of the sentence fraction is correlated with the sentence similarity of positive sentences in the sentence sample, and the other is correlated with the sentence similarity of negative sentences in the sentence sample. Sentence samples are composed of any two text representations. Therefore, when the numerator is correlated with the sentence similarity of positive sentences and the denominator is correlated with the sentence similarity of negative sentences, the network parameters can be adjusted to increase the sentence contrast loss value, making similar text samples closer in the feature space and dissimilar text samples farther apart. Conversely, when the numerator is correlated with the sentence similarity of negative sentences and the denominator is correlated with the sentence similarity of positive sentences, the network parameters can be adjusted to decrease the sentence contrast loss value, achieving the same effect.

[0062] Another alternative technical approach is to represent the character contrast loss using a character fraction. One of the numerator and denominator of the character fraction is related to the character similarity between positive and negative character samples in the character sample, while the other is related to the character similarity between negative character samples in the character sample. The character samples are composed of character representations from any two text representations. The specifics of the character contrast loss are similar to those of the sentence contrast loss and will not be elaborated upon here.

[0063] To gain a comprehensive understanding of the above technical solution, specific examples are provided below for illustration. For example,... Figure 3 As shown, this is a specific example of fine-tuning a language network using a contrastive learning strategy to obtain a target entity recognition model for entity recognition. Specifically, the initial entity recognition model includes a BERT language network, an entity recognition network connected to the BERT language network, a Sentence-CL (Sentence Contrastive Learning) network, a Word-CL (Word Contrastive Learning) network, and an overall loss value determination network connected to all three networks. The entity recognition network includes a Bidirectional Short-Term Memory (BiLSTM) subnetwork, a Conditional Random Field (CRF), and an entity recognition loss value determination subnetwork. The sentence-CL subnetwork includes a sentence similarity determination subnetwork and a sentence contrast loss value determination subnetwork. The word-CL subnetwork includes a word similarity determination subnetwork and a word contrast loss value determination subnetwork. CL stands for Contrastive Learning.

[0064] For each sample text x, text augmentation is performed on x to obtain the first augmented text x. aug1 Second enhanced text x aug2 and x, x aug1 and x aug2 The inputs are fed into the BERT language network to obtain the text representations of x, e and x.aug1 The text represents e aug1 and x aug2 The text represents e aug2 The function `e` is processed using a BiLSTM for feature extraction, and an entity class probability is output for each position to represent the entity class corresponding to the sample word at that position. Finally, it is processed using a CRF, and the entity recognition loss value is obtained using the entity recognition loss and entity annotation results. Simultaneously, multiple `e` values ​​are... aug1 and e aug1 The input is fed into the Sentence-CL network to obtain the sentence contrast loss value, and the loss values ​​of multiple e are then processed. aug1 and e aug1 The input is fed into the Word-CL network to obtain the character contrast loss value. Then, based on the overall loss network, the entity recognition loss value, sentence contrast loss value, and character contrast loss value are calculated by summing them to obtain the overall loss value. Then, gradient backpropagation is performed based on the overall loss value to achieve fine-tuning of the BERT language network and train the target entity recognition model.

[0065] Figure 4 This is a flowchart of an entity recognition method provided in an embodiment of the present invention. This embodiment is applicable to entity recognition scenarios. The method can be executed by the entity recognition device provided in this embodiment of the present invention. This device can be implemented in software and / or hardware, and can be integrated into an electronic device, which can be various user terminals or servers.

[0066] See Figure 4 The method of this invention specifically includes the following steps:

[0067] S310. Obtain the text to be recognized and the target entity recognition model generated by the model generation method provided in any embodiment of the present invention.

[0068] S320. Input the text to be recognized into the target entity recognition model, and obtain the entity information in the text to be recognized based on the output of the target entity recognition model.

[0069] The technical solution of this invention, since the language network in the target entity recognition model can output text representations with good semantic quality at both the sentence and character levels, can obtain accurate entity information when performing entity recognition on the text to be recognized based on the target entity recognition model.

[0070] An optional technical solution, before acquiring the text to be recognized, may further include: acquiring the consultation question input by the consultation recipient, determining the consultation service corresponding to the consultation question, and outputting prompt information to prompt the consultation recipient to input business questions related to the consultation service; receiving the business question input by the consultation recipient, and using the business question as the text to be recognized. Here, the consultation recipient can be an object with a consultation need, the consultation question can be the question the consultation recipient needs to consult about, and the consultation service can be the service corresponding to the consultation question. After obtaining the consultation service, prompt information can be output to prompt the consultation recipient to input relevant business questions, so that the consultation recipient can further input questions that better reflect their consultation needs. Furthermore, after receiving the business question, it can be used as the text to be recognized, so that entity recognition can be performed on the text to be recognized based on the entity recognition method described above, thereby resolving the consultation recipient's consultation needs based on the recognized entity information.

[0071] It should be noted that entity recognition is used in many application scenarios, such as automatically recognizing invoice information containing personal and company names provided by users in intelligent customer service, automatically recognizing express delivery information containing names and addresses provided by users in online shopping malls, and automatically recognizing medical record information containing names and conditions provided by patients in medical systems. To more vividly illustrate the above technical solutions, the application of entity recognition in intelligent customer service will be used as an example below. For an example, see [link to example]. Figure 5 First, the user enters their inquiry question on the intelligent customer service interface (first input). Then, the dialogue system determines the specific business need the user wants to resolve based on the input question, guiding the information flow to the corresponding business node (e.g., invoice node, order node, or logistics node). Finally, at the business node, entity recognition is performed based on the user's subsequent input of a related business question (second input). For example, if a user enters "I want to modify an invoice," the dialogue system performs business intent matching based on this question, determining that the inquiry is related to invoices, and then guides the information flow to the invoice node. Upon entering the invoice node, the intelligent customer service prompts the user to enter another invoice-related business question. Then, based on the target entity recognition model, it identifies and outputs invoice-related entity information, such as names and company names. This concludes the entity recognition process in the intelligent customer service. Next, downstream tasks (such as modifying invoices, orders, or addresses) can be performed based on this entity information obtained at the business node. Accurate entity recognition helps improve the resolution rate of user needs and the user's service experience.

[0072] Figure 6This is a structural block diagram of a model generation apparatus provided in an embodiment of the present invention. This apparatus is used to execute the model generation method provided in any of the above embodiments. This apparatus and the model generation methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the model generation apparatus can be found in the embodiments of the above model generation methods. See also... Figure 6 The device may specifically include: an initial entity recognition model acquisition module 410 and a target entity recognition model acquisition module 420.

[0073] The initial entity recognition model acquisition module 410 is used to acquire the constructed initial entity recognition model, which includes a language network, a sentence contrast learning network and a character contrast learning network connected to the language network, and the language network is used to output the text representation of the sample text.

[0074] The target entity recognition model module 420 is used to input the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network, respectively. Based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain the target entity recognition model containing the trained language network.

[0075] Optionally, the sentence contrast loss value is determined based on the sentence contrast loss, which is constructed based on positive sentence samples composed of different text representations of the same sample text and negative sentence samples composed of text representations of different sample texts.

[0076] And / or,

[0077] The sample text contains sample words. The word contrast loss value is determined based on the word contrast loss, which is constructed from positive and negative word samples composed of word representations in different text representations of the same sample text. The word representations in the positive word samples correspond to sample words of the same entity category, while the word representations in the negative word samples correspond to sample words of different entity categories.

[0078] Based on this, an optional sentence comparison learning network includes a sentence similarity determination subnetwork and a sentence comparison loss value determination subnetwork constructed based on sentence comparison loss;

[0079] The target entity recognition model module 420 may include:

[0080] The sentence similarity unit is used to input the text representations of multiple sample texts into the sentence similarity determination subnetwork to obtain the sentence similarity between any two text representations of the multiple sample texts.

[0081] The first network parameter adjustment unit is used to input the sentence similarity between any two text representations into the sentence contrast loss value determination subnetwork, and determine the sentence contrast loss value output by the subnetwork based on the sentence contrast loss value, thereby adjusting the network parameters in the language network.

[0082] Optionally, sentence contrast loss can be represented by a sentence fraction, where one of the numerator and denominator is related to the sentence similarity of the positive sentence sample in the sentence sample, and the other is related to the sentence similarity of the negative sentence sample in the sentence sample. The sentence sample is composed of any two text representations.

[0083] Alternatively, the character contrast learning network includes a character similarity determination subnetwork and a character contrast loss value determination subnetwork constructed based on character contrast loss;

[0084] The target entity recognition model module 420 may include:

[0085] The character similarity unit is used to input different text representations of the sample text into the character similarity determination subnetwork for each sample text, and obtain the character similarity between any two text representations of the sample text.

[0086] The second network parameter adjustment unit is used to input the word similarity between word representations in any two text representations into the word contrast loss value determination subnetwork, determine the word contrast loss value output by the subnetwork based on the word contrast loss value, and adjust the network parameters in the language network.

[0087] Based on this, optionally, the character contrast loss is represented by a character fraction, where one of the numerator and denominator is related to the character similarity of the positive character samples in the character sample, and the other is related to the character similarity of the negative character samples in the character sample. The character samples are composed of character representations from any two text representations.

[0088] Alternatively, the above-mentioned model generation apparatus may further include:

[0089] The first text representation module is used to obtain the first and second augmented texts of each sample text, and input the sample text, the first augmented text and the second augmented text into the language network respectively to obtain the text representation of the sample text, the text representation of the first augmented text and the text representation of the second augmented text;

[0090] The second text representation module is used to take the text representations of the first enhanced text and the second enhanced text as the text representations of the sample text, respectively, to obtain different text representations of the sample text.

[0091] Optionally, the initial entity recognition model further includes an entity recognition network connected to a language network, and the aforementioned model generation device may further include:

[0092] The training sample acquisition module is used to obtain the entity annotation results of each sample text and use the text representation and entity annotation results of the sample text as a set of training samples.

[0093] The entity recognition loss value acquisition module is used to input multiple sets of training samples into the entity recognition network and obtain the entity recognition loss value output by the entity recognition model.

[0094] The target entity recognition model module 420 may include:

[0095] The target entity recognition model obtains a unit, which is used to adjust the network parameters in the language network based on the entity recognition loss value, the sentence comparison loss value output by the sentence comparison learning network, and the character comparison loss value output by the character comparison learning network, so as to obtain a target entity recognition model containing the trained language network.

[0096] The model generation apparatus provided in this embodiment of the invention acquires a pre-constructed initial entity recognition model through an initial entity recognition model acquisition module. This initial entity recognition model includes a language network for outputting text representations of sample texts, and a sentence contrast learning network and a character contrast learning network connected to the language network. A target entity recognition model acquisition module inputs the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network, respectively. Then, based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain a target entity recognition model containing the trained language network. This apparatus, through the cooperation of a sentence contrast learning network constructed based on a sentence-level contrast learning strategy and a character contrast learning network constructed based on a character-level contrast learning strategy, fine-tunes the language network. This results in text representations output by the fine-tuned language network having better semantic quality at both the sentence and character levels, thereby obtaining a language network highly relevant to the entity recognition task and ensuring the recognition accuracy of the generated target entity recognition model.

[0097] The model generation apparatus provided in this embodiment of the invention can execute the model generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0098] It is worth noting that in the embodiments of the above model generation device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0099] Figure 7 This is a structural block diagram of an entity recognition device provided in an embodiment of the present invention. This device is used to execute the entity recognition method provided in any of the above embodiments. This device and the entity recognition methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the entity recognition device can be found in the embodiments of the above entity recognition methods. See also... Figure 7 The device may specifically include: a target entity recognition model acquisition module 510 and an entity information acquisition module 520.

[0100] The target entity recognition model acquisition module 510 is used to acquire the text to be recognized and the target entity recognition model generated according to the model generation method provided in any embodiment of the present invention.

[0101] The entity information acquisition module 520 is used to input the text to be recognized into the target entity recognition model and obtain the entity information in the text to be recognized based on the output of the target entity recognition model.

[0102] Optionally, the above-mentioned entity recognition device may further include:

[0103] The prompt information output module is used to obtain the consultation question input by the consultation recipient, determine the consultation business corresponding to the consultation question, and output prompt information to prompt the consultation recipient to input business questions related to the consultation business;

[0104] The text to be recognized module is used to receive business questions input by the consulting object and use the business questions as text to be recognized.

[0105] The entity recognition device provided in this embodiment of the invention uses a target entity recognition model acquisition module and an entity information acquisition module to cooperate with each other. Since the language network in the target entity recognition model can output text representations with good semantic quality at both the sentence and word levels, accurate entity information can be obtained when performing entity recognition on the text to be recognized based on the target entity recognition model.

[0106] The entity recognition device provided in the embodiments of the present invention can execute the entity recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0107] It is worth noting that in the embodiments of the above-mentioned entity recognition device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0108] Figure 8 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0109] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0110] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0111] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as model generation methods or entity recognition methods.

[0112] In some embodiments, the model generation method or entity recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the model generation method or entity recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the model generation method or entity recognition method by any other suitable means (e.g., by means of firmware).

[0113] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0116] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0117] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0118] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0119] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0120] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A model generation method, characterized in that, include: Obtain the constructed initial entity recognition model, wherein the initial entity recognition model includes a language network, and a sentence contrast learning network and a character contrast learning network connected to the language network, wherein the language network is used to output the text representation of the sample text; The text representations of multiple sample texts are respectively input into the sentence contrast learning network and the character contrast learning network. Based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain a target entity recognition model containing the trained language network. The sentence contrast loss value is determined based on the sentence contrast loss, which is constructed based on positive sentence samples composed of different text representations of the same sample text and negative sentence samples composed of text representations of different sample texts; wherein, the positive sentence samples are sentence samples composed of different text representations of the same sample text, and the negative sentence samples are sentence samples composed of text representations of different sample texts; The sample text contains sample characters, and the character contrast loss value is determined based on the character contrast loss. The character contrast loss is constructed based on positive and negative character samples composed of character representations in different text representations of the same sample text. The character representations in the positive character samples correspond to the sample characters of the same entity category, and the character representations in the negative character samples correspond to the sample characters of different entity categories.

2. The method according to claim 1, characterized in that, The sentence comparison learning network includes a sentence similarity determination subnetwork and a sentence comparison loss value determination subnetwork constructed based on the sentence comparison loss; The step of inputting the text representations of multiple sample texts into the sentence contrast learning network, and adjusting the network parameters in the language network based on the sentence contrast loss value output by the sentence contrast learning network, includes: The text representations of multiple sample texts are input into the sentence similarity determination subnetwork to obtain the sentence similarity between any two text representations of the multiple sample texts; The sentence similarity between any two text representations is input into the sentence contrast loss determination subnetwork, and the sentence contrast loss value output by the subnetwork is determined based on the sentence contrast loss value, thereby adjusting the network parameters in the language network.

3. The method according to claim 2, characterized in that, The sentence contrast loss is represented by a sentence fraction, where one of the numerator and denominator is related to the sentence similarity of the positive sentence sample in the sentence sample, and the other is related to the sentence similarity of the negative sentence sample in the sentence sample. The sentence sample is composed of any two text representations.

4. The method according to claim 1, characterized in that, The character comparison learning network includes a character similarity determination subnetwork and a character comparison loss value determination subnetwork constructed based on the character comparison loss; The step of inputting the text representations of multiple sample texts into the character contrast learning network, and adjusting the network parameters in the language network based on the character contrast loss value output by the character contrast learning network, includes: For each sample text, different text representations of the sample text are input into the character similarity determination subnetwork to obtain the character similarity between any two text representations of the sample text; The word similarity between the word representations in any two text representations is input into the word contrast loss value determination subnetwork. The word contrast loss value output by the subnetwork is determined based on the word contrast loss value, and the network parameters in the language network are adjusted.

5. The method according to claim 4, characterized in that, The character contrast loss is represented by a character fraction, wherein one of the numerator and denominator of the character fraction is related to the character similarity of the positive character sample in the character sample, and the other is related to the character similarity of the negative character sample in the character sample, wherein the character sample is composed of character representations from any two text representations.

6. The method according to claim 1, characterized in that, Also includes: For each sample text, a first enhanced text and a second enhanced text of the sample text are obtained, and the sample text, the first enhanced text and the second enhanced text are respectively input into the language network to obtain the text representation of the sample text, the text representation of the first enhanced text and the text representation of the second enhanced text; The text representations of the first enhanced text and the second enhanced text are used as the text representations of the sample text to obtain different text representations of the sample text.

7. The method according to claim 1, characterized in that, The initial entity recognition model further includes an entity recognition network connected to the language network, and the method further includes: For each sample text, obtain the entity annotation result of the sample text, and use the text representation of the sample text and the entity annotation result as a set of training samples; Multiple sets of training samples are input into the entity recognition network to obtain the entity recognition loss value output by the entity recognition model; The step of adjusting the network parameters in the language network based on the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network to obtain a target entity recognition model containing the trained language network includes: Based on the entity recognition loss value, the sentence contrast loss value output by the sentence contrast learning network, and the character contrast loss value output by the character contrast learning network, the network parameters in the language network are adjusted to obtain a target entity recognition model that includes the trained language network.

8. An entity recognition method, characterized in that, include: Obtain the text to be recognized and the target entity recognition model generated by the model generation method according to any one of claims 1-7; The text to be identified is input into the target entity recognition model, and the entity information in the text to be identified is obtained based on the output of the target entity recognition model.

9. The method according to claim 8, characterized in that, Before obtaining the text to be recognized, the following is also included: The system obtains the consultation question input by the consultation recipient, determines the consultation service corresponding to the consultation question, and outputs a prompt message to prompt the consultation recipient to input a service-related question. The system receives the business question input by the person seeking consultation and uses the business question as the text to be identified.

10. A model generation apparatus, characterized in that, include: The initial entity recognition model acquisition module is used to acquire the constructed initial entity recognition model, wherein the initial entity recognition model includes a language network, and a sentence contrast learning network and a character contrast learning network connected to the language network, and the language network is used to output the text representation of the sample text; The target entity recognition model acquisition module is used to input the text representations of multiple sample texts into the sentence contrast learning network and the character contrast learning network respectively, and adjust the network parameters in the language network according to the sentence contrast loss value output by the sentence contrast learning network and the character contrast loss value output by the character contrast learning network, so as to obtain a target entity recognition model containing the trained language network. The sentence contrast loss value is determined based on the sentence contrast loss, which is constructed based on positive sentence samples composed of different text representations of the same sample text and negative sentence samples composed of text representations of different sample texts; wherein, the positive sentence samples are sentence samples composed of different text representations of the same sample text, and the negative sentence samples are sentence samples composed of text representations of different sample texts; The sample text contains sample characters, and the character contrast loss value is determined based on the character contrast loss. The character contrast loss is constructed based on positive and negative character samples composed of character representations in different text representations of the same sample text. The character representations in the positive character samples correspond to the sample characters of the same entity category, and the character representations in the negative character samples correspond to the sample characters of different entity categories.

11. An entity recognition device, characterized in that, include: The target entity recognition model acquisition module is used to acquire the text to be recognized and the target entity recognition model generated according to the model generation method of any one of claims 1-7. The entity information acquisition module is used to input the text to be recognized into the target entity recognition model, and obtain the entity information in the text to be recognized based on the output of the target entity recognition model.

12. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the model generation method as described in any one of claims 1-7, or the entity recognition method as described in claim 8 or 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the model generation method as described in any one of claims 1-7, or the entity recognition method as described in claim 8 or 9.

Images