Text entity recognition method and its device, equipment, medium, and product

Abstract

The present application discloses a text entity recognition method and its device, equipment, medium, and product, including: reading text information to be recognized; inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and the model scale of the student model is smaller than the model scale of the teacher model; reading the first pointer vector and the tail pointer vector output by the student model, and performing entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector; generating the recognition result of the text information according to the entity information and the entity attribute information. Since the model scale of the student model is smaller than that of the teacher model, entity recognition through the student model reduces the requirements for the deployment environment and improves the adaptability to the deployment environment.

Description

Technical Field

[0001] The present application relates to the field of text processing, and in particular to a text entity recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] Entity recognition is a natural language field that is used to identify various entity information, such as the text boundaries and types of entities such as names, times, and places. It is usually used in many fields, such as product brand name recognition and product attribute recognition in the e-commerce field.

[0003] The inventors of this application found in their research that the text entity recognition models in the prior art are all large pre-trained models. When using large pre-trained models to train entity recognition models, although the accuracy is higher than that of small models, if they are deployed in a production environment, they will face huge performance problems. Large pre-trained models require longer training time and a larger operating environment, which hinders the deployment and application of entity recognition models. Summary of the invention

[0004] The present application provides a text entity recognition method, apparatus, computer equipment and storage medium that can accelerate training and is lightweight.

[0005] In order to solve the above technical problems, a technical solution adopted by the embodiment created by the present application is: to provide a text entity recognition method, comprising:

[0006] Read the text information to be recognized;

[0007] Inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used to extract the first pointer vector and the last pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model;

[0008] Read the first pointer vector and the tail pointer vector output by the student model, and perform entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector;

[0009] A recognition result of the text information is generated according to the entity information and the entity attribute information.

[0010] Optionally, before reading the text information to be identified, the process includes:

[0011] Collect sample text to be processed;

[0012] Performing word segmentation processing on the sample text to generate sample entities;

[0013] Generate a label head pointer vector and a label tail pointer vector of the sample text according to the positions where the first character and the last character of the sample entity appear and the mapping value of the sample entity in the entity attribute mapping table;

[0014] Constructing a training sample according to the annotation head pointer vector, the annotation tail pointer vector and the sample text;

[0015] The student model and / or teacher model is trained according to the training samples.

[0016] Optionally, the training sample is used to perform model training on the teacher model, and the performing model training on the student model and / or the teacher model according to the training sample includes:

[0017] Inputting the sample text into an initial teacher model of the teacher model, wherein the initial teacher model is a non-convergent state of the teacher model;

[0018] Reading a first head pointer vector and a first tail pointer vector output by the initial teacher model;

[0019] Calculating a first loss value between the marked head pointer vector and the first head pointer vector, and a second loss value between the marked tail pointer vector and the first tail pointer vector;

[0020] The model parameters of the initial teacher model are back-corrected according to the second loss value between the first tail pointer vectors.

[0021] Optionally, performing model training on the student model and / or the teacher model according to the training sample includes:

[0022] Inputting the training samples into the teacher model and the initial student model of the student model respectively, wherein the initial student model is a non-convergent state of the student model;

[0023] Reading a teacher feature vector output by the teacher model and a student feature vector output by the initial student model;

[0024] A distillation loss is calculated between the teacher feature vector and the student feature vector.

[0025] Optionally, the teacher feature vector includes: a teacher feature encoding vector, a second first pointer vector and a second tail pointer vector, and the student feature vector includes: a student feature encoding vector, a third first pointer vector and a third tail pointer vector; and the calculating of the distillation loss between the teacher feature vector and the student feature vector includes:

[0026] Respectively calculating a first cross entropy loss value between the third first pointer vector and the marked first pointer vector, and a second cross entropy loss value between the third last pointer vector and the marked last pointer vector;

[0027] respectively calculating a first divergence loss value between the second first pointer vector and the third first pointer vector, and a second divergence loss value between the second tail pointer vector and the third tail pointer vector;

[0028] Calculating a third divergence loss value between the teacher feature coding vector and the student feature coding vector;

[0029] The distillation loss is generated according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, and the third divergence loss value.

[0030] Optionally, after calculating the distillation loss between the teacher feature vector and the student feature vector, the step further includes:

[0031] Constructing negative samples of the training samples;

[0032] Inputting the training sample into the teacher model and the initial student model to generate a first joint feature set;

[0033] Inputting the training samples into the initial student model for a second time, and combining the student model features in the first joint feature set to generate a second joint feature set;

[0034] Inputting the negative sample into the initial student model and combining it with the student model features in the first joint feature set to generate a third joint feature set;

[0035] A first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set are calculated.

[0036] Optionally, after calculating a first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set, the method further comprises:

[0037] Performing a weighted operation on the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value according to a preset parameter threshold to generate a total loss value;

[0038] The model parameters of the initial student model are back-corrected according to the total loss value.

[0039] In order to solve the above technical problems, the present application also provides a text entity recognition device, including:

[0040] A reading module, used for reading text information to be identified;

[0041] An extraction module, used for inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used for extracting a head pointer vector and a tail pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model;

[0042] A processing module, used for reading the first pointer vector and the tail pointer vector output by the student model, and performing entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector;

[0043] An execution module is used to generate a recognition result of the text information according to the entity information and entity attribute information.

[0044] Optionally, the text entity recognition device further includes:

[0045] The first collection submodule is used to collect sample text to be processed;

[0046] A first word segmentation submodule is used to perform word segmentation processing on the sample text to generate sample entities;

[0047] A first mapping submodule, configured to generate a labeling first pointer vector and a labeling last pointer vector of the sample text according to the positions of the first character and the last character of the sample entity and the mapping value of the sample entity in the entity attribute mapping table;

[0048] A first processing submodule, used for constructing a training sample according to the annotation head pointer vector, the annotation tail pointer vector and the sample text;

[0049] The first execution submodule is used to perform model training on the student model and / or teacher model according to the training samples.

[0050] Optionally, the training sample is used to perform model training on the teacher model, and the text entity recognition device further includes:

[0051] A first input submodule, used for inputting the sample text into an initial teacher model of the teacher model, wherein the initial teacher model is a non-convergent state of the teacher model;

[0052] A first reading submodule, used for reading a first head pointer vector and a first tail pointer vector output by the initial teacher model;

[0053] A first calculation submodule, used for calculating a first loss value between the marked head pointer vector and the first head pointer vector, and a second loss value between the marked tail pointer vector and the first tail pointer vector;

[0054] The first correction submodule is used to perform back-propagation correction on the model parameters of the initial teacher model according to the second loss value between the first tail pointer vectors.

[0055] Optionally, the text entity recognition device further includes:

[0056] A second input submodule is used to input the training samples into the teacher model and the initial student model of the student model respectively, wherein the initial student model is a non-convergent state of the student model;

[0057] A second reading submodule is used to read the teacher feature vector output by the teacher model and the student feature vector output by the initial student model;

[0058] The second calculation submodule is used to calculate the distillation loss between the teacher feature vector and the student feature vector.

[0059] Optionally, the teacher feature vector includes: a teacher feature coding vector, a second first pointer vector and a second last pointer vector; the student feature vector includes: a student feature coding vector, a third first pointer vector and a third last pointer vector; the text entity recognition device also includes:

[0060] A third calculation submodule, used to respectively calculate a first cross entropy loss value between the third first pointer vector and the marked first pointer vector, and a second cross entropy loss value between the third tail pointer vector and the marked tail pointer vector;

[0061] a fourth calculation submodule, configured to respectively calculate a first divergence loss value between the second first pointer vector and the third first pointer vector, and a second divergence loss value between the second tail pointer vector and the third tail pointer vector;

[0062] A fifth calculation submodule, used for calculating a third divergence loss value between the teacher feature coding vector and the student feature coding vector;

[0063] The second processing submodule is used to generate the distillation loss according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value and the third divergence loss value.

[0064] Optionally, the text entity recognition device further includes:

[0065] A first construction submodule, used to construct a negative sample of the training sample;

[0066] A third input submodule is used to input the training sample into the teacher model and the initial student model to generate a first joint feature set;

[0067] A fourth input submodule, used for inputting the training samples into the initial student model for a second time, and generating a second joint feature set by combining the student model features in the first joint feature set;

[0068] A third processing submodule, configured to input the negative sample into the initial student model, and generate a third joint feature set by combining the student model features in the first joint feature set;

[0069] The sixth calculation submodule is used to calculate a first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set.

[0070] Optionally, the text entity recognition device further includes:

[0071] a fourth processing submodule, configured to perform a weighted operation on the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value according to a preset parameter threshold to generate a total loss value;

[0072] The second execution submodule is used to perform feedback correction on the model parameters of the initial student model according to the total loss value.

[0073] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned text entity recognition method.

[0074] In order to solve the above technical problems, an embodiment of the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the above-mentioned text entity recognition method.

[0075] A computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the text entity recognition method described in any embodiment of the present application.

[0076] The beneficial effects of the present application are: when performing text information entity recognition, the student model involved in entity recognition is trained through a pre-selected teacher model. Since the teacher model has been trained to convergence during the student model training, the student model can be quickly trained to a convergence state through knowledge distillation and comparison training. Since the model size of the student model is smaller than that of the teacher model, entity recognition through the student model reduces the requirements for the deployment environment and improves the adaptability to the deployment environment. In addition, the use of the head and tail double pointer method to extract entity information can improve the accuracy of entity information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0078] Figure 1 A basic flow chart of a text entity recognition method according to a specific embodiment of the present application;

[0079] Figure 2 A schematic diagram of a flow chart of generating training samples according to a specific embodiment of the present application;

[0080] Figure 3 A training flow chart of a teacher model for a specific embodiment of the present application;

[0081] Figure 4 A schematic diagram of a process for generating distillation loss according to a specific embodiment of the present application;

[0082] Figure 5 A schematic diagram of a process for calculating distillation loss according to a specific embodiment of the present application;

[0083] Figure 6 A flowchart of comparative training for a specific embodiment of the present application is shown;

[0084] Figure 7 A schematic diagram of a process of performing back-transmission correction on an initial student model according to a specific embodiment of the present application;

[0085] Figure 8 A basic structural diagram of a text entity recognition device according to a specific embodiment of the present application;

[0086] Fig. 9 This is a basic structural block diagram of a computer device according to a specific embodiment of the present application. DETAILED DESCRIPTION

[0087] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0088] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0089] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0090] It will be understood by those skilled in the art that the term "terminal" as used herein includes both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of performing two-way communications over a two-way communication link. Such devices may include: cellular or other communication devices, which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which may combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "terminal" used here can be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, running at any other location on the earth and / or in space. The "terminal" used here can also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0091] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0092] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0093] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0094] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0095] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0096] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0097] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0098] See also Figure 1 , Figure 1 This is a basic flow chart of the text entity recognition method in a typical embodiment of the present application. Figure 1 As shown, a text entity recognition method includes:

[0099] S1100, reading text information to be identified;

[0100] The text entity recognition method in this embodiment is used to recognize entity attribute information in text information. Therefore, the text information can be text information that records commodity attribute information.

[0101] The commodity in this embodiment refers to any commodity that can be traded, including (but not limited to): food, clothing, medicine, toys, electrical appliances, virtual goods or gifts.

[0102] The text information may include one or more attribute information of the product, where the attribute information may be (but not limited to): product name, product brand, product material, product function, product style, product shape or product expiration date or implementation standard and other attribute data related to the product.

[0103] S1200, inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used to extract a head pointer vector and a tail pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model;

[0104] The text information is input into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation and comparison learning based on a preset teacher model, and is used to extract the first pointer vector and the tail pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model.

[0105] The teacher model in this embodiment is a convolutional neural network model, a deep convolutional neural network model, a recurrent neural network model, a neural network model of an encoding-decoding structure type, or a variant model of any of the above models, which is trained to a convergence state through supervised training or semi-supervised training. The teacher model trained to a convergence state can extract the first pointer vector and the last pointer vector in the input text information.

[0106] The student model in this embodiment is a transformer model, and the student model only includes the encoder component in the transformer model. Specifically, the student model includes n layers of cascaded encoders, where n is less than or equal to 12. Since the student model does not include the decoding module in the transformer model, the model scale of the student model is smaller than that of the teacher model.

[0107] Since the student model lacks a decoding module, it lacks the necessary conditions for feedback correction of the transformer model. Therefore, it is necessary to train the student model in combination with the teacher model. Specifically, the teacher model trains the student model through knowledge distillation processing and comparative training.

[0108] Knowledge distillation refers to calculating the loss between the output results of the student model and the teacher model during the student model training process. Since the teacher model is trained to convergence in advance, the loss value between the output results of the student model and the output results of the teacher model can be used to calculate the gradient of the student model through the feedback function, and correct the weight parameters of the student model to make the student model converge.

[0109] Furthermore, while the student model is being converged through knowledge distillation, positive and negative samples are constructed for training, and the loss value between the positive sample features output by the teacher model and the student model and the positive sample features and negative sample features output by the student model is calculated, and the student model is constrained to converge through the loss value. This improves the robustness and accuracy of the model.

[0110] The first pointer vector refers to a vector set consisting of the first characters of entity information in the text information extracted by the student model, and the tail pointer vector refers to a vector set consisting of the last characters of entity information in the text information extracted by the student model.

[0111] S1300, reading the first pointer vector and the tail pointer vector output by the student model, and performing entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector;

[0112] Read the first pointer vector and the tail pointer vector output by the student model. Since the first pointer vector can determine the first character position of one or more entity information in the text information, and the tail pointer vector can determine the last character position of one or more entity information in the text information, combining the two positions can determine one or more entity information in the text information.

[0113] Furthermore, since the first pointer vector and the tail pointer vector extracted by the student model during the training process include entity attribute information corresponding to the entity information, the extracted first pointer vector and the tail pointer vector can also be used to find the entity attribute information corresponding to the entity information through a preset entity attribute mapping table.

[0114] S1400: Generate a recognition result of the text information according to the entity information and entity attribute information.

[0115] According to the one-to-one correspondence between entity information and entity attribute information corresponding to the entity information, the recognition result of text information is obtained. For example, the content recorded in the text information is: a certain company's 2021 summer anti-ultraviolet ice silk loose and comfortable windbreaker. The results extracted by the student model are: ("brand", "certain company"), ("applicable season", "summer"), ("function", "anti-ultraviolet"), ("fabric", "ice silk"), ("version", "loose"), ("style", "windbreaker").

[0116] When performing text information entity recognition, the above implementation method uses a pre-selected teacher model to train the student model involved in entity recognition. Since the teacher model has been trained to convergence during the student model training, the student model can be quickly trained to convergence through knowledge distillation and comparison training. Since the model size of the student model is smaller than that of the teacher model, entity recognition through the student model reduces the requirements for the deployment environment and improves the adaptability to the deployment environment. In addition, the use of the head and tail double pointer method to extract entity information can improve the accuracy of entity information extraction.

[0117] In some implementations, before training the teacher model or the student model, it is necessary to prepare training samples for training. Figure 2 , Figure 2 A schematic diagram of the process of generating training samples in this embodiment.

[0118] like Figure 2 As shown, before S1100, it included:

[0119] S1101, collecting sample text to be processed;

[0120] Before model training, it is necessary to first collect sample texts for training. The sample texts can be obtained by requesting a professional text supply server. In some embodiments, the product data and sample texts corresponding to the product data can also be crawled from the online e-commerce platform through web crawler technology, and the original crawled sample texts can be cleaned, filtered, stored, and stored according to the pre-designed database specifications.

[0121] S1102, performing word segmentation processing on the sample text to generate sample entities;

[0122] The extracted sample text is segmented, and the segmentation method can be: standard segmentation, NLP segmentation, index segmentation, N-shortest path segmentation or fast dictionary segmentation. After the segmentation is completed, the meanings of nouns, adjectives, quantifiers, etc. that represent product attributes obtained by the segmentation are defined as sample entities. Therefore, one or more sample entities can be generated for the sample text.

[0123] S1103, generating a labeling first pointer vector and a labeling last pointer vector of the sample text according to the positions where the first character and the last character of the sample entity appear, and the mapping value of the sample entity in the entity attribute mapping table;

[0124] In this embodiment, a corresponding mapping value is set for the attribute of each sample entity, and an entity attribute mapping table is constructed based on this. For example, the entity attribute mapping table shown in Table 1 uses numbers 1-6 to map each sample entity of the fabric commodity. However, the mapping table in this embodiment is not limited to the contents listed in Table 1. According to different specific application scenarios, the contents of the entity attribute mapping table can be customized according to the requirements of the applicable scenario.

[0125] Table 1

[0126]

[0127]

[0128] According to the position of the sample entity in the sample text and the entity attribute mapping table, the first character in the sample entity is mapped to the annotation first pointer vector, and the last character in the sample entity is mapped to the annotation tail pointer vector. For example, "A certain company's 2021 summer anti-ultraviolet ice silk loose and comfortable windbreaker", the entity sample is ("brand", "a certain company"), ("applicable season", "summer"), ("function", "anti-ultraviolet"), ("fabric", "ice silk"), ("version", "loose"), ("style", "windbreaker"). According to the position of the sample entity in the sample text and the mapping value of Table 1 above, the converted annotation first pointer vector is [10000050600040200030]. Similarly, the converted annotation tail pointer vector is [01000005000604020003]. In the annotation first pointer vector and the annotation tail pointer vector, "1" and "1" are adjacent to each other, indicating that the first and second characters in the sample text represent the brand of the product. After mapping query, it can be obtained that the brand of the product is "a certain company". Similarly, the sample entity may be determined by marking the positions of the same numbers in the first pointer vector and the last pointer vector, and the attribute information of the sample entity is determined by the sample entity in the entity attribute mapping table. This mapping process is the same as the method of determining the recognition result according to the first pointer vector and the last pointer vector in S1300-S1400.

[0129] S1104, constructing a training sample according to the annotation head pointer vector, the annotation tail pointer vector and the sample text;

[0130] The first and last pointer vectors of the sample text are obtained, and the first and last pointer vectors are used as the annotation data of the sample text. At this point, the annotation process of the sample text is completed, and the first and last pointer vectors and the sample text together construct the training sample.

[0131] S1105: Perform model training on the student model and / or teacher model according to the training samples.

[0132] The training sample constructed in this implementation is only one of several training samples in the subsequent training set. In the process of constructing the training set, the generation of each training sample must go through the process of S1101-S1105.

[0133] The training samples obtained from the training can be used for training the student model, for training the teacher model, or for joint training of the teacher model and the student model.

[0134] The training samples in this embodiment use the head and tail double pointer method to extract the product attributes of the sample text. The traditional keyword method can only be listed in an exhaustive manner. For the rapidly growing e-commerce product data, this method has obvious technical limitations. For the newly added product data, the extraction result accuracy is low. The template-based method can improve the accuracy, but it can only mine results that match the design pattern, and the recall rate is low. This solution uses the head and tail double pointer method, which can improve the confidence of product attribute extraction while also improving the accuracy and recall rate.

[0135] In some implementations, when training the student model, it is first necessary to train the teacher model so that the teacher model is in a convergent state. Figure 3 , Figure 3 This is a training flow chart of the teacher model in this embodiment.

[0136] like Figure 3 The S1105 shown includes:

[0137] S1111, inputting the sample text into an initial teacher model of the teacher model, wherein the initial teacher model is in a non-convergent state of the teacher model;

[0138] In this embodiment, the training samples are used to train the teacher model. The model parameters of the model used to train the teacher model are initialized. We call the initialized model the initial teacher model. The initial teacher model is in a discrete state, that is, a non-convergent state.

[0139] The teacher model in this embodiment is a convolutional neural network model, a deep convolutional neural network model, a recurrent neural network model, or a variant model of any of the above models, which is trained to a convergence state through supervised training.

[0140] At the beginning of training, the sample text in the training sample is input into the initial teacher model for feature extraction.

[0141] S1112, reading the first head pointer vector and the first tail pointer vector output by the initial teacher model;

[0142] The initial teacher model extracts feature vectors based on the input sample text, and obtains two sets of feature vectors, namely the first head pointer vector and the first tail pointer vector.

[0143] S1113, calculating a first loss value between the marked head pointer vector and the first head pointer vector, and a second loss value between the marked tail pointer vector and the first tail pointer vector;

[0144] A teacher loss function is set in the initial teacher model to calculate a first loss value between the labeled head pointer vector and the first head pointer vector, and a second loss value between the labeled tail pointer vector and the first tail pointer vector.

[0145] The teacher loss function in this embodiment includes (but is not limited to): a 0-1 loss function, an absolute value loss function, a log loss function, a square loss function, and an exponential loss function, or a loss function that combines two or more loss functions.

[0146] S1114. Perform back-propagation correction on the model parameters of the initial teacher model according to the second loss value between the first tail pointer vectors.

[0147] After calculating the first loss value and the second loss value, the initial teacher model is back-propagated through the AdamW algorithm and the first loss value and the second loss value. By calculating the gradient value of the initial teacher function, the weight parameters in the initial teacher model are corrected so that the first loss value and the second loss value in the initial teacher model gradually tend to be smaller than the preset teacher loss threshold. Only when the first loss value and the second loss value are smaller than the preset teacher loss threshold, the teacher model training of the training sample ends.

[0148] It should be noted that a large number of training samples are required for training the initial teacher model to converge into the teacher model. S1111-S1114 only show the process of one training sample in one training round. The training of the model is iterative, so the same training sample needs to go through one or more rounds of S1111-S1114 during the training process.

[0149] When the training rounds of the initial teacher model reach the preset number of training rounds, or the accuracy of the initial teacher model reaches the preset accuracy requirement, the initial teacher model is trained to a convergence state and becomes the teacher model.

[0150] In some implementations, after the teacher model is successfully trained, the initial student model needs to be trained using the teacher model. The first step in training the initial student model is to calculate the distillation loss between the teacher model and the initial student model. Figure 4 , Figure 4 A flow diagram of the distillation losses was generated for this example.

[0151] like Figure 4 As shown, after S1114, it includes:

[0152] S1121, inputting the training samples into the teacher model and the initial student model of the student model respectively, wherein the initial student model is a non-convergent state of the student model;

[0153] When the teacher model is trained to a convergent state, the teacher model is needed to assist in training the student model. The student model in this embodiment is a transformer model, and the student model only includes the encoder component in the transformer model. Specifically, the student model includes n layers of cascaded encoders, where n is less than or equal to 12.

[0154] Before training the student model, the model parameters used to train the student model need to be initialized. We call the model with initialized parameters the initial student model. The initial student model is discrete, that is, it is in a non-convergent state.

[0155] At the beginning of training, the sample texts in the training samples are input into the teacher model and the initial student model respectively. The teacher model and the initial student model perform feature extraction and classification calculation on the sample texts respectively.

[0156] S1122, reading the teacher feature vector output by the teacher model and the student feature vector output by the initial student model;

[0157] The sample text is input into the teacher model, and the teacher model extracts the feature vector in the sample text through convolution operation to generate the teacher feature encoding vector, and then calculates the confidence of the teacher feature encoding vector through the fully connected layer or classification layer to generate the second first pointer vector and the second tail pointer vector. Among them, the teacher feature encoding vector, the second first pointer vector and the second tail pointer vector are collectively referred to as the teacher feature vector.

[0158] The sample text is input into the initial student model, which extracts the feature vector in the sample text through convolution operation and self-attention calculation to generate the student feature encoding vector, and then obtains the third first pointer vector and the third last pointer vector through feature mapping. Among them, the student feature encoding vector, the third first pointer vector and the third last pointer vector are collectively referred to as the student feature vector.

[0159] S1123. Calculate the distillation loss between the teacher feature vector and the student feature vector.

[0160] The distillation loss between the teacher feature vector and the student feature vector is calculated by the set loss function, where the distillation loss refers to the loss value between the teacher feature encoding vector and the student feature encoding vector, the loss between the second first pointer vector and the third first pointer, the loss between the second tail pointer vector and the third tail pointer vector, the loss between the third first pointer vector and the labeled first pointer vector, and the loss between the third tail pointer vector and the labeled tail pointer vector.

[0161] See also Figure 5 , Figure 5 The following is a schematic diagram of the process for calculating the distillation loss in this embodiment. Figure 5 As shown, S1123 includes:

[0162] S1131, respectively calculating a first cross entropy loss value between the third first pointer vector and the marked first pointer vector, and a second cross entropy loss value between the third last pointer vector and the marked last pointer vector;

[0163] Cross entropy can be used as a loss function in neural networks. p represents the distribution of true annotations, and q is the predicted label distribution of the trained model. The cross entropy loss function can measure the similarity between p and q.

[0164] In this embodiment, a cross entropy loss function is used to calculate a first cross entropy loss value between the third first pointer vector and the labeled first pointer vector, and a second cross entropy loss value between the third last pointer vector and the labeled last pointer vector.

[0165] For example: after training the teacher model T, input the training text into the teacher model to obtain the teacher feature encoding vector V of the teacher model t , the second pointer vector Y t，s and the second tail pointer vector Y t，e .

[0166] Input the training sample into the student model S to obtain the feature encoding vector V of the student model s , first pointer vector output Y s，s and tail pointer vector output Y s，e .

[0167] Assume that the labels of the labeled first and last pointer vectors and the labeled tail pointer vectors corresponding to the training sample X are Y s , Y e .

[0168] The calculation formula for the first cross entropy loss value and the second cross entropy loss value is:

[0169] Loss ce，s =CrossEntropy((Y s，s ,Y s )

[0170] Loss ce，e = CrossEntropy(K s，e , Y e )

[0171] S1132, respectively calculating a first divergence loss value between the second first pointer vector and the third first pointer vector, and a second divergence loss value between the second tail pointer vector and the third tail pointer vector;

[0172] KL divergence (Kullback–Leibler divergence), also known as relative entropy, is a method to describe the difference between two probability distributions P and Q.

[0173] In this implementation, a first divergence loss value between the second leading pointer vector and the third leading pointer vector, and a second divergence loss value between the second trailing pointer vector and the third trailing pointer vector are calculated using a KL divergence function.

[0174] The calculation formulas for the first divergence loss value and the second divergence loss value are as follows:

[0175] Loss kl，s =KLdivergence(Y s，s , Y t，s )

[0176] Loss kl，e =KLdivergence(Y s，e , Y t，e )

[0177] S1133, calculating a third divergence loss value between the teacher feature coding vector and the student feature coding vector;

[0178] A third divergence loss value between the teacher feature encoding vector and the student feature encoding vector is calculated using a KL divergence function.

[0179] The calculation formula of the third divergence loss value is as follows:

[0180] Losskl，v =KLdivergence(V s , Y t )

[0181] S1134. Generate the distillation loss according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, and the third divergence loss value.

[0182] After calculating the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value and the third divergence loss value, the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value and the third divergence loss value are collectively referred to as the distillation loss of the student model.

[0183] It should be pointed out that in this implementation, there is no clear order relationship between the operations of S1131-S1133, and the order of operations can be adjusted accordingly according to actual needs.

[0184] In some implementations, after the distillation loss is calculated, the initial student model needs to be trained comparatively using positive and negative samples. Figure 6 , Figure 6 Schematic diagram of the comparative training process of this embodiment.

[0185] like Figure 6 As shown, after S1134, including:

[0186] S1141, constructing negative samples of the training samples;

[0187] In this embodiment, the annotation results in the training samples constructed by S1101-S1105 are the same as the actual expression content of the sample text, so they are used as positive samples. When constructing negative samples, the training text is still used as the text of the negative sample, but the annotation content of the training sample is replaced with the annotation content of other training texts. Even if the training sample in the negative sample is inconsistent with the corresponding true standard, the annotation content adapted to the negative sample is the wrong standard content. In this way, the negative sample is constructed.

[0188] S1142, inputting the training sample into the teacher model and the initial student model to generate a first joint feature set;

[0189] The training samples are used as positive samples and input into the teacher model and the initial student model, and the output of the teacher model and the output of the initial student model are merged to generate a first joint feature set. The first joint feature set includes the first pointer vector and the tail pointer vector output by the teacher model, and the first pointer vector and the tail pointer vector output by the initial student model.

[0190] S1143, inputting the training sample into the initial student model for a second time, and combining the student model features in the first joint feature set to generate a second joint feature set;

[0191] The training samples are input into the initial student model for the second time. Since the student model is composed of transformer encoder, and the transformer encoder has its own random pooling mechanism, the initial student model outputs different results for the same input. Therefore, in order to constrain the randomness of the student model output and make the positive sample outputs consistent, the training samples need to be input for the second time.

[0192] The training sample is input into the initial student model twice to obtain the secondary output result of the initial student model, and the result of the initial student model output in the first joint feature set and the secondary output result of the initial student model output are combined to generate a second joint feature set. The second joint feature set includes the first pointer vector and the tail pointer vector output by the initial student model for the training sample for the first time, and the first pointer vector and the tail pointer vector output for the second time.

[0193] S1144, inputting the negative sample into the initial student model, and combining the student model features in the first joint feature set to generate a third joint feature set;

[0194] The negative sample is input into the initial student model to obtain the negative sample features output by the initial student model, and the output results of the training sample output by the initial student model in the first joint feature set and the negative sample features are merged to generate a third joint feature set. The third joint feature set includes: the first pointer vector and the tail pointer vector output by the initial student model for the training sample, and the first pointer vector and the tail pointer vector output by the initial student model for the negative sample.

[0195] S1145. Calculate a first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set.

[0196] After obtaining the first joint feature set, the second joint feature set and the third joint feature set, the first comparison loss value between the first joint feature set and the third joint feature set, and the second comparison loss value between the second joint feature set and the third joint feature set are calculated by using the log loss function and the conformity loss function combined with the log loss function.

[0197] For example, let the training sample be X i , after the teacher model T outputs V t，i , after the initial student model S outputs V s，i .

[0198] The training sample X i The corresponding negative sample is denoted as X j , where j∈1,2,i-1,i+1,...m, m is the number of samples of the training sample. Then the feature set of the sample set output by the teacher model T is V t，j , the feature set output by the initial student model S is V s，j .

[0199] Construct positive sample pair A1: {V t，i , V s，i}, negative sample pair B1: {(V s，i , V s，j )|j∈1,2,...i-1,i+1,...m}. Since the student model is composed of the transformer encoder, and the transformer encoder has its own random pooling mechanism, that is to say, for the same X i , input the student model S twice, and the results obtained twice are different. Here, the feature encoding vector obtained by the second input is defined as V′ s，i , then construct another positive sample pair A2: {V s，i , V′ s，i}.

[0200] The calculation formulas for the first comparison loss value and the second comparison loss value are as follows:

[0201]

[0202]

[0203]

[0204]

[0205] In this embodiment, the contrastive learning training method is used to enable the student model to learn more semantic information in addition to the entity recognition task, thereby improving the performance of the student model. In the model training stage, two positive sample pairs (the output of the teacher model and the initial student model, and the two outputs of the initial student model) and negative sample pairs (the student model output of the current training sample and the student / teacher model output of the negative sample in the same training process) are constructed, and the contrastive learning method is used to train the student model. During the training of the model, the semantic space distance between positive samples is shortened, while the semantic space distance between negative samples is widened, and the distribution is kept uniform on the parameter space hypersphere.

[0206] In some embodiments, after the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value are calculated, the student model needs to be corrected according to the above loss values. Figure 7 , Figure 7 This is a schematic diagram of the process of performing feedback correction on the initial student model in this embodiment.

[0207] like Figure 7 As shown, after S1145, including:

[0208] S1151, performing a weighted operation on the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value according to a preset parameter threshold to generate a total loss value;

[0209] After calculating the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value, it is necessary to calculate the total loss value according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value. The total loss value can be calculated by weighted calculation of the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value.

[0210] The total loss value is calculated as follows:

[0211] Loss = α1Loss ce，s +α2Loss ce，e +α3Loss kl，s +α4Loss kl，e +α5Loss kl，v +α6Loss cl，1 +α7Loss cl，2

[0212] Among them, α represents the hyperparameter, which is set before training.

[0213] S1152. Perform feedback correction on the model parameters of the initial student model according to the total loss value.

[0214] After calculating the total loss value, the initial student model is back-propagated through the AdamW algorithm and the total loss value. The weight parameters in the initial student model are corrected by calculating the gradient value of the initial student function, so that the total loss value in the initial student model gradually tends to be less than the preset student loss threshold. Only when the total loss value is less than the preset student loss threshold, the training of the student model for the training sample ends.

[0215] It should be noted that a large number of training samples are required for the training of the initial student model until it converges to the student model. S1121-S1152 only shows the process of one training sample in one training round. The training of the model is iterative, so the same training sample needs to go through one or more rounds of S1121-S1152 during the training process.

[0216] Furthermore, the calculation of the loss value of the initial student model during the training process is not limited to the calculation process of the total loss value described above. In some embodiments, the loss value of the initial student model is the cumulative value of the distillation loss obtained in S1134, that is, the weighted value of the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, and the third divergence loss value. In other embodiments, the loss value of the initial student model is the comparison loss, that is, the weighted value of the first comparison loss value and the second comparison loss value.

[0217] Please refer to Figure 8 , Figure 8 Schematic diagram of the basic structure of the text entity recognition device of this embodiment.

[0218] like Figure 8 As shown, a text entity recognition device includes: a reading module 1100, an extraction module 1200, a processing module 1300 and an execution module 1400. The reading module 1100 is used to read the text information to be recognized; the extraction module 1200 is used to input the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used to extract the first pointer vector and the tail pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model; the processing module 1300 is used to read the first pointer vector and the tail pointer vector output by the student model, and according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector, perform entity extraction on the text information to generate entity information and entity attribute information; the execution module 1400 is used to generate the recognition result of the text information according to the entity information and the entity attribute information.

[0219] When the text entity recognition device performs text information entity recognition, the student model involved in entity recognition is trained through the pre-selected teacher model. Since the teacher model has been trained to convergence during the student model training, the student model can be quickly trained to convergence through knowledge distillation and comparison training. Since the model scale of the student model is smaller than that of the teacher model, entity recognition through the student model reduces the requirements for the deployment environment and improves the adaptability to the deployment environment. In addition, the use of the head and tail double pointer method to extract entity information can improve the accuracy of entity information extraction.

[0220] Optionally, the text entity recognition device further includes: a first acquisition submodule, a first word segmentation submodule, a first mapping submodule, a first processing submodule and a first execution submodule. The first acquisition submodule is used to acquire sample text to be processed; the first word segmentation submodule is used to perform word segmentation processing on the sample text to generate sample entities; the first mapping submodule is used to generate a labeling head pointer vector and a labeling tail pointer vector of the sample text according to the positions of the first character and the last character of the sample entity and the mapping value of the sample entity in the entity attribute mapping table; the first processing submodule is used to construct a training sample according to the labeling head pointer vector, the labeling tail pointer vector and the sample text; the first execution submodule is used to perform model training on the student model and / or the teacher model according to the training sample.

[0221] Optionally, the training sample is used to perform model training on the teacher model, and the text entity recognition device further includes: a first input submodule, a first reading submodule, a first calculation submodule and a first correction submodule. Among them, the first input submodule is used to input the sample text into the initial teacher model of the teacher model, wherein the initial teacher model is a non-convergent state of the teacher model; the first reading submodule is used to read the first head pointer vector and the first tail pointer vector output by the initial teacher model; the first calculation submodule is used to calculate the first loss value between the annotated head pointer vector and the first head pointer vector, and the second loss value between the annotated tail pointer vector and the first tail pointer vector; the first correction submodule is used to perform back-transmission correction on the model parameters of the initial teacher model according to the second loss value between the first tail pointer vectors.

[0222] Optionally, the text entity recognition device further includes: a second input submodule, a second reading submodule and a second calculation submodule. The second input submodule is used to input the training samples into the teacher model and the initial student model of the student model respectively, wherein the initial student model is a non-convergent state of the student model; the second reading submodule is used to read the teacher feature vector output by the teacher model and the student feature vector output by the initial student model; the second calculation submodule is used to calculate the distillation loss between the teacher feature vector and the student feature vector.

[0223] Optionally, the teacher feature vector includes: a teacher feature encoding vector, a second first pointer vector and a second tail pointer vector, and the student feature vector includes: a student feature encoding vector, a third first pointer vector and a third tail pointer vector; the text entity recognition device also includes: a third calculation submodule, a fourth calculation submodule, a fifth calculation submodule and a second processing submodule. Among them, the third calculation submodule is used to respectively calculate the first cross entropy loss value between the third first pointer vector and the annotated first pointer vector, and the second cross entropy loss value between the third tail pointer vector and the annotated tail pointer vector; the fourth calculation submodule is used to respectively calculate the first divergence loss value between the second first pointer vector and the third first pointer vector, and the second divergence loss value between the second tail pointer vector and the third tail pointer vector; the fifth calculation submodule is used to calculate the third divergence loss value between the teacher feature encoding vector and the student feature encoding vector; the second processing submodule is used to generate the distillation loss according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value and the third divergence loss value.

[0224] Optionally, the text entity recognition device further includes: a first construction submodule, a third input submodule, a fourth input submodule, a third processing submodule and a sixth calculation submodule. Among them, the first construction submodule is used to construct a negative sample of the training sample; the third input submodule is used to input the training sample into the teacher model and the initial student model to generate a first joint feature set; the fourth input submodule is used to input the training sample into the initial student model for a second time, and generate a second joint feature set in combination with the student model features in the first joint feature set; the third processing submodule is used to input the negative sample into the initial student model, and generate a third joint feature set in combination with the student model features in the first joint feature set; the sixth calculation submodule is used to calculate a first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set.

[0225] Optionally, the text entity recognition device further includes: a fourth processing submodule and a second execution submodule. The fourth processing submodule is used to perform a weighted operation on the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value and the second comparison loss value according to a preset parameter threshold to generate a total loss value; and the second execution submodule is used to perform a feedback correction on the model parameters of the initial student model according to the total loss value.

[0226] To solve the above technical problems, the present application also provides a computer device. Fig. 9 , Fig. 9 This is a basic structural block diagram of the computer device in this embodiment.

[0227] like Fig. 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected via a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a text entity recognition method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute a text entity recognition method. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0228] In this embodiment, the processor is used to execute Figure 8 The memory stores the program codes and data required to execute the above modules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program codes and data required to execute all submodules in the text entity recognition device, and the server can call the program codes and data of the server to execute the functions of all submodules.

[0229] When the computer device performs entity recognition of text information, the student model involved in entity recognition is trained through the pre-selected teacher model. Since the teacher model has been trained to convergence during the student model training, the student model can be quickly trained to convergence through knowledge distillation and comparison training. Since the model size of the student model is smaller than that of the teacher model, entity recognition through the student model reduces the requirements for the deployment environment and improves the adaptability to the deployment environment. In addition, the use of the head and tail double pointer method to extract entity information can improve the accuracy of entity information extraction.

[0230] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the text entity recognition method in any of the above embodiments.

[0231] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0232] Those skilled in the art will appreciate that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, altered, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art that are similar to those disclosed in this application may also be alternated, altered, rearranged, decomposed, combined, or deleted.

[0233] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A text entity recognition method, characterized in that: include: Collect sample text to be processed; Performing word segmentation processing on the sample text to generate sample entities; Generate a label head pointer vector and a label tail pointer vector of the sample text according to the positions where the first character and the last character of the sample entity appear and the mapping value of the sample entity in the entity attribute mapping table; Constructing a training sample according to the annotation head pointer vector, the annotation tail pointer vector and the sample text; Performing model training on the student model and / or the teacher model according to the training samples; Read the text information to be recognized; Inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used to extract the first pointer vector and the last pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model; Read the first pointer vector and the tail pointer vector output by the student model, and perform entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector; Generate a recognition result of the text information according to the entity information and entity attribute information; The performing model training on the student model and / or the teacher model according to the training sample comprises: Inputting the training samples into the teacher model and the initial student model of the student model respectively, wherein the initial student model is a non-convergent state of the student model; Reading a teacher feature vector output by the teacher model and a student feature vector output by the initial student model, wherein the teacher feature vector includes: a teacher feature coding vector, a second first pointer vector and a second last pointer vector, and the student feature vector includes: a student feature coding vector, a third first pointer vector and a third last pointer vector; Calculating a distillation loss between the teacher feature vector and the student feature vector; The calculating the distillation loss between the teacher feature vector and the student feature vector comprises: Respectively calculating a first cross entropy loss value between the third first pointer vector and the marked first pointer vector, and a second cross entropy loss value between the third last pointer vector and the marked last pointer vector; respectively calculating a first divergence loss value between the second first pointer vector and the third first pointer vector, and a second divergence loss value between the second tail pointer vector and the third tail pointer vector; Calculating a third divergence loss value between the teacher feature coding vector and the student feature coding vector; The distillation loss is generated according to the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, and the third divergence loss value.

2. The text entity recognition method according to claim 1, characterized in that: The training samples are used to perform model training on the teacher model, and the performing model training on the student model and / or the teacher model according to the training samples includes: Inputting the sample text into an initial teacher model of the teacher model, wherein the initial teacher model is a non-convergent state of the teacher model; Reading a first head pointer vector and a first tail pointer vector output by the initial teacher model; Calculating a first loss value between the labeled head pointer vector and the first head pointer vector, and a second loss value between the labeled tail pointer vector and the first tail pointer vector; The model parameters of the initial teacher model are back-corrected according to the second loss value between the first tail pointer vectors.

3. The text entity recognition method according to claim 1, characterized in that: After calculating the distillation loss between the teacher feature vector and the student feature vector, comprising: Constructing negative samples of the training samples; Inputting the training sample into the teacher model and the initial student model to generate a first joint feature set; Inputting the training samples into the initial student model for a second time, and combining the student model features in the first joint feature set to generate a second joint feature set; Inputting the negative sample into the initial student model and combining it with the student model features in the first joint feature set to generate a third joint feature set; A first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set are calculated.

4. The text entity recognition method according to claim 3, characterized in that: After calculating a first comparison loss value between the first joint feature set and the third joint feature set, and a second comparison loss value between the second joint feature set and the third joint feature set, the method further comprises: Performing a weighted operation on the first cross entropy loss value, the first divergence loss value, the second cross entropy loss value, the second divergence loss value, the third divergence loss value, the first comparison loss value, and the second comparison loss value according to a preset parameter threshold to generate a total loss value; The model parameters of the initial student model are back-propagated and corrected according to the total loss value.

5. A text entity recognition device, characterized in that: The method for realizing text entity recognition as claimed in any one of claims 1 to 4 comprises: A reading module, used for reading text information to be identified; An extraction module, used for inputting the text information into a preset student model, wherein the student model is trained to a convergence state through knowledge distillation based on a preset teacher model, and is used for extracting a head pointer vector and a tail pointer vector of the text information, and the model scale of the student model is smaller than the model scale of the teacher model; A processing module, used for reading the first pointer vector and the tail pointer vector output by the student model, and performing entity extraction on the text information to generate entity information and entity attribute information according to the entity attribute mapping table corresponding to the first pointer vector and the tail pointer vector; An execution module is used to generate a recognition result of the text information according to the entity information and entity attribute information.

6. A computer storage medium, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the text entity recognition method according to any one of claims 1 to 4.

7. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 4 are implemented.

Images