Sequence labeling method and device, electronic equipment and storage medium

Abstract

The application provides a sequence labeling method and device, electronic equipment and a storage medium, wherein the method comprises: determining a text to be labeled; inputting the text to be labeled into a sequence labeling model to obtain a labeling result output by the sequence labeling model; the sequence labeling model is obtained by applying a teacher model to knowledge transfer of each hidden layer in a student model based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on similarity between each group of hidden layers in the teacher model and each hidden layer in the student model. The method, device, electronic equipment and storage medium provided by the application can ensure flexibility and simultaneity of selection of the teacher model and the student model, improve knowledge transfer efficiency, ensure reliability of knowledge transfer, and thus the sequence labeling model obtained can meet high reliability, low latency and low energy consumption required by an industrial scene, and help application expansion of automatic sequence labeling.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a sequence labeling method and device, electronic equipment and a storage medium. BACKGROUND

[0002] As a basic natural language processing (NLP) task, sequence labeling can be used to solve a series of character classification problems, such as word segmentation, part-of-speech tagging, named entity recognition, and relation extraction.

[0003] At present, methods for solving sequence labeling tasks include maximum entropy models, Hidden Markov Model (HMM) models, and Conditional Random Field (CRF) models. Although the above models can meet the requirement of accurate reasoning results in industrial applications, they are limited by large model parameters and complex operators, and thus cannot meet the requirement of low reasoning time delay and low energy consumption in industrial applications. SUMMARY

[0004] The present application provides a sequence labeling method, device, electronic equipment and storage medium to solve the problem of large model reasoning time delay and large calculation resource consumption in the prior art.

[0005] The present application provides a sequence labeling method, comprising:

[0006] determining a text to be labeled;

[0007] inputting the text to be labeled into a sequence labeling model to obtain a labeling result output by the sequence labeling model;

[0008] The sequence labeling model is obtained by applying a teacher model to each hidden layer in a student model for knowledge transfer based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on a similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0009] According to the sequence labeling method provided by the present application, the sequence labeling model is obtained based on the following steps:

[0010] determining a current hidden layer correspondence relationship based on a similarity between each group of hidden layers in the teacher model and each hidden layer in the student model;

[0011] Based on the current hidden layer correspondence relationship, knowledge migration is performed on each hidden layer in the student model by using the teacher model, to obtain a knowledge-migrated student model, and the knowledge-migrated student model is taken as a student model, and similarity between each group of hidden layers in the teacher model and each hidden layer in the student model is updated until knowledge migration is completed, to obtain the sequence labeling model.

[0012] According to the sequence labeling method provided by the application, the knowledge-migrated student model is taken as a student model, and similarity between each group of hidden layers in the teacher model and each hidden layer in the student model is updated until knowledge migration is completed, to obtain the sequence labeling model, which comprises the following steps:

[0013] The knowledge-migrated student model is taken as a student model, and similarity between each group of hidden layers in the teacher model and each hidden layer in the student model is updated until the current hidden layer correspondence relationship is fixed.

[0014] Based on the fixed hidden layer correspondence relationship, knowledge migration is performed on each hidden layer in the student model by using the teacher model until knowledge migration is completed, to obtain the sequence labeling model.

[0015] According to the sequence labeling method provided by the application, the knowledge migration performed on each hidden layer in the student model by using the teacher model comprises the following steps:

[0016] Based on a dimension conversion parameter, a first output of each group of hidden layers in the teacher model and a second output of each hidden layer in the student model are converted to the same dimension, and the dimension conversion parameter is determined based on a model width of the teacher model and a model width of the student model.

[0017] Based on the dimension-converted first output and second output, knowledge migration is performed on each hidden layer in the student model.

[0018] According to the sequence labeling method provided by the application, the knowledge migration performed on each hidden layer in the student model by using the teacher model comprises the following steps:

[0019] The sample text is input into the teacher model and the student model respectively, to obtain a first labeling distribution output by the teacher model and an output of each group of hidden layers in the teacher model, and a second labeling distribution output by the student model and an output of each hidden layer in the student model.

[0020] Based on a sequence labeling label of the sample text, the first labeling distribution is corrected.

[0021] The student model is knowledge transferred based on the second annotation distribution and the corrected first annotation distribution, and outputs of each group of hidden layers in the teacher model and outputs of each hidden layer in the student model.

[0022] According to the sequence labeling method provided in the application, the teacher model is obtained based on the following steps:

[0023] A pre-training model is determined.

[0024] The pre-training model is fine-tuned based on sample texts and sequence labeling labels of the sample texts, so as to obtain the teacher model.

[0025] According to the sequence labeling method provided in the application, the pre-training model is fine-tuned based on sample texts and sequence labeling labels of the sample texts, so as to obtain the teacher model, which comprises the following steps:

[0026] The pre-training model is input with the same sample text multiple times, so as to obtain prediction annotation distributions output by the pre-training model multiple times, and the pre-training model comprises a dropout layer.

[0027] The pre-training model is fine-tuned based on differences between the prediction annotation distributions output multiple times and differences between the sequence labeling labels of the sample texts and the prediction annotation distributions output multiple times, so as to obtain the teacher model.

[0028] The application further provides a sequence labeling device, which comprises:

[0029] A text determination unit is configured to determine a text to be labeled.

[0030] A sequence labeling unit is configured to input the text to be labeled into a sequence labeling model, so as to obtain an annotation result output by the sequence labeling model.

[0031] The sequence labeling model is obtained by applying a teacher model to knowledge transfer of each hidden layer in a student model based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on similarities between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0032] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements steps of the sequence labeling method according to any one of the above embodiments when executing the computer program.

[0033] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement steps of the sequence labeling method according to any one of the above embodiments.

[0034] The sequence labeling method, device, electronic equipment and storage medium provided by the present application determine the hidden layer correspondence relationship through the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model, thereby realizing the learning of the student model on the information of each group of hidden layers in the teacher model in the knowledge transfer, ensuring the flexibility and simultaneity of the selection of the teacher model and the student model, improving the knowledge transfer efficiency, and ensuring the reliability of the knowledge transfer. The sequence labeling model obtained thereby can meet the high reliability, low latency and low energy consumption required by industrial scenarios, and is helpful to the application expansion of automatic sequence labeling. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present application or prior art, the drawings needed to be used in the embodiments or prior art description will be briefly described below. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0036] Figure 1 is a flowchart of the sequence labeling method provided by the present application;

[0037] Figure 2 is a flowchart of the sequence labeling model acquisition method provided by the present application;

[0038] Figure 3 is a prediction loss diagram of the sequence labeling model provided by the present application;

[0039] Figure 4 is a knowledge transfer diagram of the sequence labeling model provided by the present application;

[0040] Figure 5 is a training diagram of the teacher model provided by the present application;

[0041] Figure 6 is a flowchart of the sequence labeling model acquisition method provided by the present application;

[0042] Figure 7 is a structural diagram of the sequence labeling device provided by the present application;

[0043] Figure 8 is a structural diagram of the electronic equipment provided by the present application. DETAILED DESCRIPTION

[0044] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0045] In industrial applications, in addition to requiring the model to output accurate inference results, the model inference is required to have as low a delay as possible, the model size is required to be as small as possible, and the model calculation energy consumption is required to be as little as possible.

[0046] At present, the model used to solve the sequence labeling task is usually obtained by fine-tuning on the basis of a pre-trained model. The application of the pre-trained model reduces the dependence on the labeled corpus to a certain extent, and good results can be achieved with only a small amount of labeled data. However, the pre-trained model has a fatal defect, that is, compared with some other traditional neural networks, the pre-trained model has more parameters and more complex operators, which makes most of the pre-trained models very resource-consuming and extremely slow in inference, and it is difficult to deploy in actual applications. How to achieve a better prediction result through a lightweight neural network model is a problem that the industry urgently needs to solve.

[0047] Knowledge distillation, as a model compression method, can well alleviate the problems of large model size and slow inference speed. Simply put, knowledge distillation is to transfer the knowledge of a complex and powerful teacher model to a simple and fast student model through an objective function, so as to achieve high accuracy while ensuring inference speed.

[0048] In most knowledge distillation schemes, the output distribution of the teacher model and the student model is usually used to calculate the distillation loss, but this cannot learn the information of the hidden layers of the teacher model. In related technologies, there are also schemes for distilling the output of some specified layers in the teacher model to transfer the hidden layer information of the teacher model to the student model. However, in order to compress the model size, the number of hidden layers of the student model is generally less than that of the teacher model, and the learning of the middle layers cannot be one-to-one corresponding on the level, and only the corresponding relationship between the specified levels can be forced, but this method will cause some information of the levels to be discarded in the distillation process, and may not be suitable for various task scenarios.

[0049] To this end, an embodiment of the present application provides a sequence labeling method for realizing low inference delay and low energy consumption of sequence labeling. Figure 1 As shown in the flowchart of the sequence labeling method provided by the present application, Figure 1 The method comprises the following steps.

[0050] In step 110, a text to be labeled is determined.

[0051] Specifically, the text to be labeled, i.e., the text requiring sequence labeling, can be directly input by a user, can be obtained by performing speech transcription on collected audio, or can be obtained by performing OCR on an image collected by an image collection device such as a scanner, a mobile phone, or a camera.

[0052] The sequence labeling task for the text to be labeled can be part-of-speech tagging of each word in the text to be labeled, or named entity recognition of the text to be labeled, or relation extraction, extractive question answering, or answer labeling in the text to be labeled, and the embodiments of the present application do not make specific limitations thereon.

[0053] In step 120, the text to be labeled is input into a sequence labeling model to obtain a labeling result output by the sequence labeling model.

[0054] The sequence labeling model is obtained by applying a teacher model to each hidden layer in a student model based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on a similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0055] Specifically, the sequence labeling of the text to be labeled can be implemented by a sequence labeling model, and the sequence labeling result obtained thereby can include the part-of-speech of each word in the text to be labeled, or include a named entity existing in the text to be labeled, or include a named entity and an entity type existing in the text to be labeled, and the embodiments of the present application do not make specific limitations thereon.

[0056] In order to meet the requirements of industrial applications for the inference delay and the calculation energy consumption of the sequence labeling model, the sequence labeling model can be obtained by knowledge distillation. In this process, the teacher model, i.e., a model capable of implementing sequence labeling, has a larger size, a more complex structure, and a better sequence labeling capability than the student model. Based on the teacher-student network idea, the knowledge in the teacher model can be migrated to the student model to improve the network performance of the student model. The process of knowledge migration is knowledge distillation. Migration of the knowledge of multiple teacher models to the student model can make the performance of the sequence labeling model obtained by knowledge migration closer to the performance of the teacher model, and because the student model itself has a smaller size and a simpler structure, the sequence labeling model obtained by knowledge migration can guarantee the inference performance while achieving the goals of smaller inference delay and lower calculation energy consumption.

[0057] In order to ensure that the selection of the student model has sufficient flexibility, the number of hidden layers of the teacher model and the number of hidden layers of the student model are different, which causes the hidden layers of the teacher model and the hidden layers of the student model to not naturally correspond one by one, in the embodiment of the application, the hidden layers in the teacher model are grouped, and based on the similarity between each group of hidden layers after grouping and each hidden layer in the student model, the correspondence relationship between the hidden layers of the teacher model and the student model is determined.

[0058] Here, the grouping restriction condition for grouping the hidden layers in the teacher model is to ensure that each hidden layer in a group of hidden layers is continuous, for example, the 1st, 2nd and 3rd layers can be grouped as a group, and the 4th, 5th and 6th layers can be grouped as a group. Under this grouping restriction condition, the hidden layers in the teacher model can be randomly grouped, or each hidden layer in the teacher model can be uniformly grouped. After grouping, the number of groups of hidden layers in the teacher model can be consistent with the number of hidden layers in the student model, or can be different from the number of hidden layers in the student model. As preferred, after grouping, the number of groups of hidden layers in the teacher model can be consistent with the number of hidden layers in the student model, for example, the teacher model has a total of 12 layers, and the student model has a total of 6 layers. The number of layers of the teacher model can be divided into an equal number of layers as the number of layers of the student model, that is, every two layers in the teacher model is a group, resulting in 1, 2 layers, 3, 4 layers, 5, 6 layers, 7, 8 layers, 9, 10 layers and 11, 12 layers, a total of 6 groups. In the case that the number of groups of hidden layers in the teacher model is consistent with the number of hidden layers in the student model after grouping, each group of hidden layers in the teacher model can construct a one-to-one corresponding hidden layer relationship with each hidden layer in the student model, so as to further ensure that the information of all hidden layers in the teacher model is not missed in the process of knowledge transfer, improve the knowledge transfer efficiency, and ensure the reliability of knowledge transfer.

[0059] After completing the grouping of the hidden layers in the teacher model, the similarity between each group of hidden layers and each hidden layer in the student model can be calculated. Here, the similarity can be calculated by calculating the similarity between the output of each group of hidden layers in the teacher model and the output of each hidden layer in the student model for the same sample text. The similarity can be specifically realized by cosine similarity, Euclidean distance, Manhattan distance and the like, which is not limited in the embodiment of the application.

[0060] On this basis, the correspondence between each group of hidden layers in the teacher model and each hidden layer in the student model, i.e., the hidden layer correspondence, can be determined based on the similarity between each group of hidden layers and each hidden layer in the student model. For example, for any hidden layer in the student model, the group of hidden layers corresponding to the maximum value in the similarity between the hidden layer and each group of hidden layers in the teacher model can be selected as the corresponding group of hidden layers of the hidden layer, so that the hidden layer can learn the output of the corresponding group of hidden layers during knowledge transfer. Compared with the manually specified correspondence, the hidden layer correspondence determined based on the similarity is more in line with the actual situation, does not require human intervention, can better guarantee the flexibility of the selection of the teacher model and the student model, and can improve the knowledge transfer efficiency and guarantee the reliability of knowledge transfer.

[0061] During knowledge transfer, the hidden layer correspondence can be determined at the beginning of knowledge transfer, and then the knowledge of each group of hidden layers in the teacher model can be transferred to the corresponding hidden layers in the student model based on the hidden layer correspondence, so that the update iteration of the parameters of each hidden layer in the student model is realized to obtain a sequence labeling model. Alternatively, the hidden layer correspondence can be updated synchronously at each transfer stage, i.e., during a transfer stage, the hidden layer correspondence can be updated based on the teacher model and the student model after the previous transfer stage, and the knowledge of each group of hidden layers in the teacher model can be transferred to the corresponding hidden layers in the student model based on the updated hidden layer correspondence, so that the update iteration of the parameters of each hidden layer in the student model at the current transfer stage is realized, and then the next transfer stage is entered to update the hidden layer correspondence and perform knowledge transfer. The present embodiment does not make a specific limitation on this.

[0062] The method provided by the present embodiment determines the hidden layer correspondence based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model, so that the student model learns the information of each group of hidden layers in the teacher model during knowledge transfer, guarantees the flexibility of the selection of the teacher model and the student model, improves the knowledge transfer efficiency, and guarantees the reliability of knowledge transfer. The sequence labeling model obtained in this way can meet the high reliability, low latency, and low energy consumption required by industrial scenarios, and is helpful for the application expansion of automatic sequence labeling.

[0063] Based on the above embodiment, Figure 2 is a flowchart of the method for obtaining the sequence labeling model provided by the present application, as Figure 2 shown, the sequence labeling model is obtained based on the following steps:

[0064] In step 210, the current hidden layer correspondence is determined based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0065] In step 220, based on the current hidden layer correspondence relationship, the teacher model is applied to perform knowledge transfer on each hidden layer in the student model to obtain a knowledge-transferred student model, and the knowledge-transferred student model is taken as the student model to update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model until the knowledge transfer is completed to obtain the sequence labeling model.

[0066] Specifically, in the process of obtaining the sequence labeling model, steps 210 and 220 are alternately executed, wherein step 210 is used to update and determine the hidden layer correspondence relationship in the current transfer stage, and step 220 is used to perform knowledge transfer under the hidden layer correspondence relationship in the current transfer stage, and update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model for the student model after the current transfer stage, so as to return to step 210 to update and determine the hidden layer correspondence relationship in the next transfer stage in the next transfer stage, thereby realizing the alternation of steps 210 and 220, that is, the knowledge transfer in each transfer stage is performed by applying the hidden layer correspondence relationship in each transfer stage, and the hidden layer correspondence relationship will change with the student model after the knowledge transfer in the previous transfer stage, so that the hidden layer correspondence relationship applied in each transfer stage is the actual situation that best fits the current student model, thereby ensuring that the hidden layer information in the teacher model can be transferred to the student model without omission and accurately.

[0067] Further, considering that the hidden layer correspondence relationship will gradually tend to be fixed as the knowledge transfer is performed, the number of times of updating the hidden layer correspondence relationship based on the similarity matching can be gradually reduced as the number of training steps increases, that is, the number of times of interleaving step 210 in step 220 can be gradually reduced as the knowledge transfer progresses, thereby further reducing the computing resources occupied by the knowledge transfer.

[0068] Based on any of the above embodiments, in step 220, the knowledge-transferred student model is taken as the student model to update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model until the knowledge transfer is completed to obtain the sequence labeling model, comprising:

[0069] The knowledge-transferred student model is taken as the student model to update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model until the current hidden layer correspondence relationship is fixed.

[0070] Based on the fixed hidden layer correspondence relationship, the teacher model is applied to perform knowledge transfer on each hidden layer in the student model until the knowledge transfer is completed to obtain the sequence labeling model.

[0071] Specifically, with the execution of the knowledge transfer, the hidden layer correspondence relationship gradually tends to be fixed, so in the alternating cycle of step 210 and step 220, for one knowledge transfer, after the knowledge transfer is completed, whether the hidden layer correspondence relationship is fixed can be determined according to the similarity between each group of hidden layers in the updated teacher model and each hidden layer in the student model.

[0072] The basis for determining whether the hidden layer correspondence relationship is fixed can be whether the current updated hidden layer correspondence relationship is consistent with the last updated hidden layer correspondence relationship, or whether the similarity of each hidden layer in the student model corresponding to each group of hidden layers in the teacher model obtained in the current calculation is greater than a preset similarity threshold, and the like, so as to determine whether the current hidden layer correspondence relationship can be changed again, that is, whether the hidden layer correspondence relationship is fixed.

[0073] After determining that the hidden layer correspondence relationship is fixed, the current hidden layer correspondence relationship can be used as the fixed hidden layer correspondence relationship, and step 210 is not returned for execution, but the fixed hidden layer correspondence relationship is directly applied to complete the knowledge transfer for the student model, thereby obtaining a sequence labeling model.

[0074] In addition, to obtain a more compact and fast student model, in addition to reducing the number of layers, the width of the model can also be reduced. The width of a certain layer in the model refers to the number of neurons of the neural network of this layer of the model, which is directly related to the output dimension of this layer of the model. If the width of the student model and the width of the teacher model are not the same, then the loss on the prediction layer between the two can only be calculated, and the loss between the hidden layers cannot be calculated, resulting in an unsatisfactory distillation effect.

[0075] To solve this problem, based on any of the above embodiments, the application of the teacher model to the knowledge transfer of each hidden layer in the student model comprises:

[0076] Converting the first output of each group of hidden layers in the teacher model and the second output of each hidden layer in the student model to the same dimension based on a dimension conversion parameter, wherein the dimension conversion parameter is determined based on the model width of the teacher model and the model width of the student model;

[0077] Based on the dimension-converted first output and second output, performing knowledge transfer on each hidden layer in the student model.

[0078] Specifically, in order to ensure that the selection of the student model has sufficient flexibility, the dimension conversion parameters are introduced in the embodiment of the present application to convert the first output of each group of hidden layers in the teacher model and the second output of each hidden layer in the student model to the same dimension, so that the loss between the hidden layers can be calculated in the case of inconsistent model widths, and the knowledge transfer based on the correspondence of the hidden layers can be performed.

[0079] Here, the first output of each group of hidden layers in the teacher model and the second output of each hidden layer in the student model can be converted to the dimension of the second output, or to the dimension of the first output, or the dimensions of the first output and the second output are converted respectively, so that the converted dimensions are consistent.

[0080] Considering that the width of the teacher model is usually greater than the width of the student model, accordingly, the dimension of the first output is greater than the dimension of the second output, in order to reduce the conversion loss, as a preferred, the second output can be converted to the dimension of the first output. Here, the dimension conversion based on the dimension conversion parameter can be expressed as follows:

[0081]

[0082] Suppose the hidden layer output of the teacher model, i.e. the first output, is with the dimension of O x T, and the hidden layer output of the student model, i.e. the second output, is with the dimension of O x S. In the formula, and are the second output before and after dimension conversion respectively, W S′ and b S′ are dimension conversion parameters. W S′ is an S x T matrix initialized as all 1s, and b s′ is a 1 x T matrix initialized as all 1s. After dimension conversion, is the same as the dimension of the hidden layer output of the teacher model, at this time, even if the width of the student model is not the same as the width of the teacher model, distillation can also be performed at the level of the hidden layers. In order to ensure that the student model learns the knowledge of the hidden layers in the teacher model better, W S′ and b s′ need to remain unchanged during the knowledge transfer process and cannot be updated together with the student model.

[0083] The method provided by the embodiment of the present application converts the hidden layer outputs of the teacher model and the student model to the same dimension through the dimension conversion parameters, so that the knowledge transfer of the hidden layers can also be performed in the case of inconsistent widths of the teacher model and the student model.

[0084] In the process of knowledge migration, not only the hidden layer information of the teacher model can be migrated to the student model, but also the prediction layer information of the teacher model can be migrated to the student model. The two migrations can be performed synchronously. The prediction layer information here refers to the probability distribution of each annotation type output by the teacher model. Although the teacher model itself is a high-precision large model, it may still have reasoning errors. In the process of migrating the prediction layer information, if the teacher model has a reasoning error, the learning effect of the student model will be affected.

[0085] To solve this problem, based on any of the above embodiments, the application of the teacher model to the knowledge migration of each hidden layer in the student model includes:

[0086] The sample text is input into the teacher model and the student model respectively to obtain the first annotation distribution output by the teacher model and the output of each group of hidden layers in the teacher model, and the second annotation distribution output by the student model and the output of each hidden layer in the student model;

[0087] Based on the sequence labeling label of the sample text, the first annotation distribution is corrected;

[0088] Based on the second annotation distribution and the corrected first annotation distribution, and the output of each group of hidden layers in the teacher model and the output of each hidden layer in the student model, the knowledge of the student model is migrated.

[0089] Specifically, the first annotation distribution output by the teacher model and the second annotation distribution output by the student model are probability distributions of the sample text belonging to various annotation types obtained by performing sequence labeling on the same sample text. In the conventional scheme, the loss of the prediction layer can be calculated based on the first annotation distribution and the second annotation distribution, so as to migrate the knowledge of the prediction layer information of the student model. However, considering that the teacher model may have reasoning errors, that is, the first annotation distribution output by the teacher model may not conform to the true situation, therefore, before migrating the knowledge based on the first annotation distribution and the second annotation distribution, the sequence labeling label of the sample text can be applied to correct the first annotation distribution.

[0090] Here, the sequence labeling label of the sample text is the sequence labeling result of the pre-labeled sample text, and the sequence labeling label is used to reflect the reliable sequence labeling result. After obtaining the first labeling distribution, it can be first judged whether the sequence labeling result indicated by the first labeling distribution is consistent with the sequence labeling label. If consistent, the first labeling distribution can be directly applied to knowledge transfer. If not consistent, it is considered that the teacher model may have reasoning errors, and the first labeling distribution at this time is unusable and needs to be corrected based on the sequence labeling label. The specific correction method can be to convert the maximum probability in the first labeling distribution to the probability of the labeling type indicated by the sequence labeling label, for example, the first labeling distribution is [0.1, 0.1, 0.6, 0.2], and the sequence labeling label is [0, 0, 0, 1], the first labeling distribution can be corrected to [0.1, 0.1, 0.2, 0.6].

[0091] The second labeling distribution and the corrected first labeling distribution are applied to the knowledge transfer of the prediction layer information, and the knowledge transfer of the hidden layer information is performed in combination with the output of each group of hidden layers in the teacher model and the output of each hidden layer in the student model, that is, the knowledge distillation for the whole teacher model is realized. Further, based on the second labeling distribution and the corrected first labeling distribution, the prediction layer loss can be determined, based on the output of each group of hidden layers in the teacher model and the output of each hidden layer in the student model, the hidden layer loss can be determined, and the total loss required for the update iteration of the student model can be obtained by fusing the prediction layer loss and the hidden layer loss, thereby realizing the update iteration of the student model. Here, the total loss can be the sum of the prediction layer loss and the hidden layer loss, or the weighted sum of the prediction layer loss and the hidden layer loss, which is not specifically limited by the embodiments of the present application.

[0092] The method provided by the embodiments of the present application can correct the first labeling distribution output by the teacher model through the sequence labeling label of the sample text, and apply the corrected first labeling distribution for knowledge transfer, which can avoid the problem that the effect of the student model is affected by the reasoning error of the teacher model, and ensure the training effect of the student model.

[0093] Based on any of the above embodiments, Figure 3 is a prediction loss diagram of the sequence labeling model provided by the present application. Figure 3 What is shown is the training of the loss on the prediction layer, which includes two branches, and T is used to distinguish between the two branches. Here, T is the temperature. The higher the temperature, the smoother the probability distribution of the output of the softmax, the greater the entropy of the distribution, and the information carried by the negative label will be relatively amplified.

[0094] Wherein, T = 1, it can be regarded as the student model itself outputting the second annotation distribution to perform softmax, so as to obtain the annotation result, and the annotation result is compared with the sequence annotation label Ground Truth Label of the sample text, so as to obtain the loss of the label on the predicted value and the true value, that is, the supervised learning loss Student Loss in the figure. The Student Loss here can be recorded as L CE , which is calculated by the following formula:

[0095]

[0096] In the formula, P s is the annotation result predicted by the student model, and P t is the sequence annotation label.

[0097] T = t (t > 1), the first annotation distribution output by the teacher model and the second annotation distribution output by the student model can be obtained after T-softmax respectively to obtain soft labels Soft Tragets, and the loss between the two is calculated, that is, the distilled loss Distilled Loss in the figure is obtained. The Distilled Loss here can be recorded as L KD , which is calculated by the following formula:

[0098]

[0099] L KD = CE (P S , P T )

[0100] Wherein, z i is the score of class i, p i is the soft label score obtained by T-softmax, and p i of each annotation type in the first annotation distribution output by the teacher model constitutes P T , and p i of each annotation type in the second annotation distribution output by the student model constitutes P S , and the two can obtain L KD through the CE (Cross Entropy, Cross Entropy) loss function.

[0101] After obtaining L CE and L KD , the two can be combined as the loss of the prediction layer through the hyperparameter a, that is:

[0102] loss pred = (1-α) L CE + αL KD

[0103] based on any one of the above embodiments, Figure 4 is a knowledge transfer schematic diagram of the sequence labeling model provided by the present application, Figure 4 In the knowledge transfer in the above formula, the loss of the prediction layer is calculated not only by KD Loss and CE Loss, but also by PT Loss. Here, KD Loss is the distillation loss in the above formula, and CE Loss is the supervised learning loss in the above formula. Figure 3 Figure 3

[0104] The PT Loss is set here to enable the student model to output feature representations at each layer that are as similar as possible to the feature representations output by each group of hidden layers indicated by the corresponding relationship between the hidden layers in the teacher model. PT Loss can apply L2 loss to measure, that is, first perform L2 normalization on the feature representations of both, and then calculate the MSE (mean square error) on the normalized feature representations as PT Loss. Figure 4 In the above formula, the Embedding_layer and Trm_layer in the teacher model and the student model are hidden layers.

[0105] In addition, in the context of deep learning, in order to achieve better inference results, a large amount of labeled data is often needed for model training. A large amount of labeled data is difficult to obtain, and a lot of manpower and material resources are often needed to find and label data. Moreover, data in different fields cannot be reused, and when the adaptation field of the task needs to be changed, new data needs to be found and labeled again, which is time-consuming and laborious. How to obtain a teacher model with excellent inference results in the case of insufficient labeled data in the industry is still a problem to be solved.

[0106] based on any one of the above embodiments, the teacher model is obtained based on the following steps:

[0107] determining a pre-trained model;

[0108] based on the sample text and its sequence labeling label, fine-tuning the pre-trained model to obtain the teacher model.

[0109] ​​The pre-training model here is a pre-trained language model (PLM), such as BERT (Bidirectional Encoder Representation from Transformers), ELMo (Embedding from language models), and the like. Compared with a parameter-initialized neural network, the pre-training model itself carries rich language knowledge, has a higher starting point, better initial performance, and a larger training slope. By applying sample text and its sequence labeling labels to fine-tune the pre-training model, the dependence on labeled corpus can be reduced to a certain extent, and good results can be achieved with only a small amount of labeled data. Moreover, during the fine-tuning of the pre-training model, the improvement speed of the inference performance of the pre-training model is steeper than that in other cases, and the asymptote is better, so the inference performance of the teacher model obtained is better than that in other cases.

[0110] The teacher model obtained in this way has fixed parameters and only updates the parameters of the student model during the knowledge transfer process, thereby avoiding the influence of the student model on the teacher model, reducing the distillation time, and accelerating the distillation process.

[0111] The method provided by the embodiment of the application obtains a teacher model by fine-tuning a pre-training model, and only a small amount of sample text and its sequence labeling labels are needed to obtain a teacher model with excellent inference ability, thereby alleviating the problem of insufficient labeled data in the industry.

[0112] However, since the pre-training model has a large number of parameters, when the amount of labeled data is small, overfitting is easily caused. To solve this problem, based on any one of the above embodiments, in the process of obtaining the teacher model, data augmentation is performed by using a contrastive learning method to improve the performance of the teacher model.

[0113] Figure 5 is a training schematic diagram of the teacher model provided by the application, as shown in Figure 5 The teacher model is obtained by fine-tuning the pre-training model based on sample text and its sequence labeling labels, including:

[0114] The same sample text is input into the pre-training model multiple times to obtain prediction annotation distribution output by the pre-training model multiple times, and the pre-training model includes a dropout layer;

[0115] The pre-training model is fine-tuned based on the difference between the prediction annotation distribution output multiple times and the difference between the sequence labeling labels of the sample text and the prediction annotation distribution output multiple times, to obtain the teacher model.

[0116] Specifically, because the pre-trained model includes a dropout layer, the same sample text x is dropped. i (correspond Figure 5 The sample text x is input into the pre-trained model multiple times. If dropout is performed multiple times in the pre-trained model, it can be assumed that the sample text x... i Several slightly different models have been tested. "Several times" here can be understood as two or more times. For ease of explanation, we will use two times as an example below. For cases with more than two times, please refer to the case with two times, which will not be elaborated here.

[0117] In the case of dropping out twice in the pre-trained model, the sample text x can be considered as... i Two slightly different models have been approved. They are denoted as follows: and Accordingly, the predicted label distributions of the pre-trained model's outputs in the two tests are denoted as follows: and Considering that the same sample text should produce the same output even after being perturbed by dropout, therefore and The predicted label distributions should be as consistent as possible, meaning the difference between the two outputs should approach zero. Furthermore, regardless of which output's predicted label distribution is used, it should be consistent with the sample text x. i Sequence label y i As consistent as possible, therefore y i and y i and All should be as consistent as possible.

[0118] At this point, the loss function of the pre-trained model during the fine-tuning phase can be divided into two parts: one part characterizes the difference between the predicted label distributions of multiple outputs, and the other part characterizes the difference between the sequence label of the sample text and the predicted label distributions of multiple outputs, i.e., the loss function L of the pre-trained model. i It can be represented as:

[0119]

[0120] In the formula, CE loss is used to characterize the difference between the sequence label distribution of the sample text and the predicted label distribution of multiple outputs. KL loss is used to characterize the difference between the predicted label distributions of multiple outputs, and α is a preset weight.

[0121] The CE loss is shown in the following formula:

[0122]

[0123] The KL loss is shown in the following formula:

[0124]

[0125] The method provided by the embodiment of the present application is based on the gap between the prediction label distribution of multiple outputs and the gap between the sequence label of the sample text and the prediction label distribution of multiple outputs, and the pre-training model is fine-tuned, so that the teacher model obtained under a small amount of sample text fine-tuning can still achieve better inference effect.

[0126] Based on any of the above embodiments, considering that it is difficult to obtain sample texts labeled with sequence labels in domain tasks, such as medical, biological and other fields, and that domain experts need to manually label high-quality labeled data, it is time-consuming and laborious. To solve this problem, based on fine-tuning the pre-training model based on a small amount of sample texts labeled with sequence labels, a high-accuracy teacher model is obtained, and a large amount of unlabeled sample texts can be input into the teacher model to obtain the sequence labeling result predicted by the teacher model for the sample texts. At this time, the sequence labeling result can be used as the sequence labeling label of the sample text and applied to the distillation process of the student model, thereby realizing supervised knowledge transfer to alleviate the problem of insufficient labeled data.

[0127] Based on any of the above embodiments, Figure 6 is a flowchart of the method for obtaining the sequence labeling model provided by the present application, as Figure 6 shown, in order to obtain a sequence labeling model with good inference effect, low inference delay and low resource consumption, the selection of the teacher model and the student model is required.

[0128] Here, the pre-training model can be selected as the basis of the teacher model, and the pre-training model can be fine-tuned to obtain a teacher model with excellent inference effect. In addition, in order to ensure that the inference speed of the student model meets the requirements, LSTM (Long Short-Term Memory) and CNN (Convolutional Neural Network) can be selected as the student model.

[0129] The fine-tuning of the teacher model, i.e. Figure 6 as shown in the pre-training, can be realized by a small amount of sample texts with sequence labels, i.e. labeled data. After fine-tuning, the teacher model can be applied to predict the sequence labeling result of the sample text without carrying the sequence labeling label, i.e. unlabeled data, to realize the labeling of this part of the sample.

[0130] In addition, for a student model that can be different from the teacher model in both the number of layers and the width, for the difference in the number of layers, the hidden layers in the teacher model can be grouped based on the number of hidden layers of the student model, and a hidden layer correspondence between the teacher model and the student model can be determined based on the similarity between each group of hidden layers after grouping and each hidden layer in the student model, and the hidden layer correspondence is applied to the learning of the information of each hidden layer in the knowledge transfer. For the difference in the width, a dimension conversion parameter can be introduced to convert the first output of each group of hidden layers in the teacher model and the second output of each hidden layer in the student model to the same dimension, so as to facilitate the knowledge transfer based on the hidden layer correspondence to be performed in the case of different model widths.

[0131] On the basis of the known hidden layer correspondence and the dimension conversion parameter, based on the sample text with the sequence labeling label, the target loss containing the prediction layer loss and the hidden layer loss is applied to transfer the knowledge of the teacher model to the student model, and the student model after the transfer is completed is used as a sequence labeling model.

[0132] Based on any of the above embodiments, Figure 7 is a structural schematic diagram of a sequence labeling device provided by the present application, as Figure 7 shown, the device comprises:

[0133] The text determination unit 710 is configured to determine the text to be labeled.

[0134] The sequence labeling unit 720 is configured to input the text to be labeled into a sequence labeling model to obtain a labeling result output by the sequence labeling model.

[0135] The sequence labeling model is obtained by applying the teacher model to the knowledge transfer of each hidden layer in the student model based on the hidden layer correspondence between the teacher model and the student model, and the hidden layer correspondence is determined based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0136] The device provided by the embodiment of the present application determines the hidden layer correspondence through the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model, thereby realizing the learning of the information of each group of hidden layers in the teacher model by the student model in the knowledge transfer, ensuring the flexibility of the selection of the teacher model and the student model at the same time, improving the knowledge transfer efficiency, and ensuring the reliability of the knowledge transfer. The sequence labeling model obtained thereby can meet the high reliability, low latency and low energy consumption required by industrial scenarios, and is helpful to the application expansion of automatic sequence labeling.

[0137] Based on any of the above embodiments, the device further comprises a knowledge transfer unit configured to:

[0138] determine a current hidden layer correspondence relationship between the groups of hidden layers in the teacher model and the hidden layers in the student model based on similarities between the groups of hidden layers in the teacher model and the hidden layers in the student model;

[0139] based on the current hidden layer correspondence relationship, apply the teacher model to perform knowledge transfer on the hidden layers in the student model to obtain a knowledge-transferred student model, and update the similarities between the groups of hidden layers in the teacher model and the hidden layers in the student model based on the knowledge-transferred student model as the student model until knowledge transfer is completed to obtain the sequence labeling model.

[0140] based on any of the above embodiments, the knowledge transfer unit is configured to:

[0141] update the similarities between the groups of hidden layers in the teacher model and the hidden layers in the student model based on the knowledge-transferred student model as the student model until the current hidden layer correspondence relationship is fixed;

[0142] based on the fixed hidden layer correspondence relationship, apply the teacher model to perform knowledge transfer on the hidden layers in the student model until knowledge transfer is completed to obtain the sequence labeling model.

[0143] based on any of the above embodiments, the knowledge transfer unit is configured to:

[0144] based on a dimension conversion parameter, convert the first outputs of the groups of hidden layers in the teacher model and the second outputs of the hidden layers in the student model to the same dimension, the dimension conversion parameter being determined based on a model width of the teacher model and a model width of the student model;

[0145] based on the dimension-converted first outputs and second outputs, perform knowledge transfer on the hidden layers in the student model.

[0146] based on any of the above embodiments, the knowledge transfer unit is configured to:

[0147] input sample text into the teacher model and the student model respectively to obtain first annotation distribution output by the teacher model and outputs of the groups of hidden layers in the teacher model, and second annotation distribution output by the student model and outputs of the hidden layers in the student model;

[0148] based on sequence labeling labels of the sample text, correct the first annotation distribution;

[0149] based on the second annotation distribution and the corrected first annotation distribution, and the outputs of the groups of hidden layers in the teacher model and the outputs of the hidden layers in the student model, perform knowledge transfer on the student model.

[0150] Based on any of the above embodiments, the device further includes a teacher model determination unit, used for:

[0151] Determine the pre-trained model;

[0152] The pre-trained model is fine-tuned based on the sample text and its sequence label to obtain the teacher model.

[0153] Based on any of the above embodiments, the teacher model determination unit is used for:

[0154] The same sample text is input into the pre-trained model multiple times to obtain the predicted label distribution of the pre-trained model multiple times. The pre-trained model includes a dropout layer.

[0155] Based on the differences between the predicted label distributions of the multiple outputs, and the differences between the sequence label of the sample text and the predicted label distributions of the multiple outputs, the pre-trained model is fine-tuned to obtain the teacher model.

[0156] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a sequence labeling method, which includes:

[0157] Identify the text to be annotated;

[0158] The text to be labeled is input into the sequence labeling model to obtain the labeling results output by the sequence labeling model;

[0159] The sequence labeling model is obtained by applying the teacher model to transfer knowledge from each hidden layer in the student model based on the hidden layer correspondence between the teacher model and the student model. The hidden layer correspondence is determined based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0160] Further, the logic instructions in the memory 830 described above can be implemented in the form of software functional units and sold or used as standalone products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or partially contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0161] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, when the program instructions are executed by a computer, the computer can execute the sequence labeling method provided by the above-mentioned method, and the method comprises:

[0162] determining a text to be labeled;

[0163] inputting the text to be labeled into a sequence labeling model to obtain a labeling result output by the sequence labeling model;

[0164] The sequence labeling model is obtained by applying a teacher model to each hidden layer in a student model for knowledge transfer based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on a similarity between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0165] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the sequence labeling method provided by the above-mentioned method, and the method comprises:

[0166] determining a text to be labeled;

[0167] inputting the text to be labeled into a sequence labeling model to obtain a labeling result output by the sequence labeling model;

[0168] The sequence labeling model is obtained by applying a teacher model to each hidden layer in a student model for knowledge transfer based on a hidden layer correspondence relationship between the teacher model and the student model, and the hidden layer correspondence relationship is determined based on similarities between each group of hidden layers in the teacher model and each hidden layer in the student model.

[0169] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0170] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus necessary universal hardware platforms, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0171] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A sequence labeling method, characterized in that, include: Identify the text to be annotated; The text to be labeled is input into the sequence labeling model to obtain the labeling results output by the sequence labeling model; The sequence labeling model is obtained by applying the teacher model to transfer knowledge from each hidden layer in the student model based on the hidden layer correspondence between the teacher model and the student model. The hidden layer correspondence is determined based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model. Each group of hidden layers is obtained by grouping the hidden layers in the teacher model. The grouping condition is that the hidden layers in each group are consecutive, and the number of hidden layer groups in the teacher model is the same as the number of hidden layers in the student model after grouping. The sequence labeling model is obtained based on the following steps: Based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model, the current hidden layer correspondence is determined; Based on the current hidden layer correspondence, the teacher model is applied to transfer knowledge to each hidden layer in the student model to obtain the knowledge-transferred student model. The knowledge-transferred student model is then used as the student model to update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model until the knowledge transfer is completed, thus obtaining the sequence labeling model.

2. The sequence labeling method according to claim 1, characterized in that, The process of using the knowledge-transferred student model as the student model, updating the similarity between each hidden layer in the teacher model and each hidden layer in the student model, until knowledge transfer is complete, to obtain the sequence labeling model includes: The student model after knowledge transfer is used as the student model. The similarity between each group of hidden layers in the teacher model and each hidden layer in the student model is updated until the current hidden layer correspondence is fixed. Based on the fixed correspondence of hidden layers, the teacher model is used to transfer knowledge to each hidden layer in the student model until the knowledge transfer is completed, thus obtaining the sequence labeling model.

3. The sequence labeling method according to claim 1 or 2, characterized in that, The application of the teacher model to perform knowledge transfer on each hidden layer of the student model includes: Based on the dimension transformation parameter, the first output of each hidden layer in the teacher model and the second output of each hidden layer in the student model are transformed to the same dimension. The dimension transformation parameter is determined based on the model width of the teacher model and the model width of the student model. Based on the first and second outputs after dimensional transformation, knowledge transfer is performed on each hidden layer in the student model.

4. The sequence labeling method according to claim 1 or 2, characterized in that, The application of the teacher model to perform knowledge transfer on each hidden layer of the student model includes: The sample text is input into the teacher model and the student model respectively to obtain the first label distribution output by the teacher model and the output of each hidden layer in the teacher model, as well as the second label distribution output by the student model and the output of each hidden layer in the student model. Based on the sequence labeling tags of the sample text, the first labeling distribution is corrected; Based on the second annotation distribution and the corrected first annotation distribution, as well as the outputs of each hidden layer in the teacher model and the outputs of each hidden layer in the student model, knowledge transfer is performed on the student model.

5. The sequence labeling method according to claim 1 or 2, characterized in that, The teacher model was obtained based on the following steps: Determine the pre-trained model; The pre-trained model is fine-tuned based on the sample text and its sequence label to obtain the teacher model.

6. The sequence labeling method according to claim 5, characterized in that, The process of fine-tuning the pre-trained model based on sample text and its sequence labeling to obtain the teacher model includes: The same sample text is input into the pre-trained model multiple times to obtain the predicted label distribution of the pre-trained model multiple times. The pre-trained model includes a dropout layer. Based on the differences between the predicted label distributions of the multiple outputs, and the differences between the sequence label of the sample text and the predicted label distributions of the multiple outputs, the pre-trained model is fine-tuned to obtain the teacher model.

7. A sequence labeling device, characterized in that, include: The text determination unit is used to determine the text to be labeled; A sequence labeling unit is used to input the text to be labeled into a sequence labeling model and obtain the labeling results output by the sequence labeling model; The sequence labeling model is obtained by applying the teacher model to transfer knowledge from each hidden layer in the student model based on the hidden layer correspondence between the teacher model and the student model. The hidden layer correspondence is determined based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model. Each group of hidden layers is obtained by grouping the hidden layers in the teacher model. The grouping condition is that the hidden layers in each group are consecutive, and the number of hidden layer groups in the teacher model is the same as the number of hidden layers in the student model after grouping. The sequence labeling model is obtained based on the following steps: Based on the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model, the current hidden layer correspondence is determined; Based on the current hidden layer correspondence, the teacher model is applied to transfer knowledge to each hidden layer in the student model to obtain the knowledge-transferred student model. The knowledge-transferred student model is then used as the student model to update the similarity between each group of hidden layers in the teacher model and each hidden layer in the student model until the knowledge transfer is completed, thus obtaining the sequence labeling model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the sequence labeling method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the sequence labeling method as described in any one of claims 1 to 6.

Images