Named entity recognition method and named entity recognition device

Abstract

The present application discloses a method and device for named entity recognition. The method comprises: parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text; obtaining a target dependency tree structure corresponding to the input text by inference from the obtained plurality of candidate dependency tree structures; inputting the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a named entity recognition method and a named entity recognition device. Background Art

[0002] Named Entity Recognition (NER) refers to the identification of entities with specific meanings in texts, mainly including names of people, places, institutions, proper nouns, etc. Named entity recognition is an important basic tool in application fields such as information extraction, question-answering systems, syntactic analysis, machine translation, and metadata annotation for the Semantic Web. It plays an important role in the process of natural language processing technology becoming practical.

[0003] Named entity recognition is widely used in practical scenarios, but currently most of the annotation data exists in English, Chinese and other common languages. Some languages ​​in the embodiments of the present invention have only a small amount of named entity annotations, resulting in unsatisfactory results of the trained model.

[0004] In addition, named entity recognition (NER) requires a large amount of annotated data to achieve state-of-the-art performance. In fact, it is challenging to provide such a large amount of annotations for named entity recognition (NER) datasets for low-resource languages ​​or datasets in new domains such as e-commerce domain. Annotating such data without certain domain knowledge is not an easy task.

[0005] Previous research has shown that the linguistic dependency tree structure plays a significant role in improving the named entity recognition effect, especially in some data with relatively little training data. However, this method has the following problems:

[0006] 1. The dependency tree structure relies on a large number of dependency annotations. In actual use, there are usually no dependency annotations available for named entity recognition datasets for new languages ​​and fields.

[0007] 2. The effect of the dependency tree structure on improving named entity recognition is only for a specific language or domain of a specific dependency tree. If it is migrated to another language or domain, the recognition performance will drop significantly.

[0008] 3. For the target language, if the dependency tree of the target language is obtained from an external parser and then named entity recognition is performed, such a step-by-step model is prone to cascading errors, resulting in suboptimal or even harmful named entity recognition performance. Summary of the invention

[0009] In view of the above problems, an embodiment of the present invention provides a method and a device for named entity recognition to solve the problems existing in the prior art.

[0010] In order to solve the above problem, an embodiment of the present application discloses a method for named entity recognition, which is used to perform named entity recognition on an input text containing at least two words. The method comprises the following steps:

[0011] Parsing the input text to obtain multiple candidate dependency tree structures corresponding to the input text;

[0012] Obtaining a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0013] The input text and the target dependency tree structure corresponding to the input text are input into a named entity recognition model component to obtain a named entity sequence corresponding to the input text.

[0014] In order to solve the above problem, an embodiment of the present application discloses a named entity recognition device, including:

[0015] A parsing module, used to parse the input text to obtain multiple candidate dependency tree structures corresponding to the input text;

[0016] A target dependency tree acquisition module, used to obtain a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0017] The named entity sequence acquisition module is used to input the input text and the target dependency tree structure corresponding to the input text into the named entity recognition model component to obtain the named entity sequence corresponding to the input text.

[0018] In order to solve the above problem, an embodiment of the present application further discloses an electronic device, which includes:

[0019] A memory for storing a computer readable program;

[0020] The processor is configured to read the computer-readable program in the memory, and the electronic device performs the following operations:

[0021] Parsing the input text to obtain multiple candidate dependency tree structures corresponding to the input text;

[0022] Obtaining a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0023] The input text and the target dependency tree structure corresponding to the input text are input into a named entity recognition model component to obtain a named entity sequence corresponding to the input text.

[0024] In order to solve the above problem, an embodiment of the present application further discloses a terminal device, including:

[0025] one or more processors; and

[0026] One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, enable the terminal device to perform the above method.

[0027] An embodiment of the present application further discloses one or more machine-readable media having instructions stored thereon, which, when executed by one or more processors, enable a terminal device to execute the above method.

[0028] As can be seen from the above, the method and device for named entity recognition proposed in the embodiments of the present application have at least the following advantages:

[0029] The named entity recognition method and device proposed in the embodiment of the present invention trains the named entity recognition model proposed in the present invention through the dependency tree data in the general dependency tree library, uses the dependency tree as a potential variable, trains a named entity recognition model component based on the dependency tree, and then inputs the input text into the named entity recognition model component to obtain a named entity recognition annotation sequence. In low-resource named entity annotation, the named entity recognition annotation of the present invention has high accuracy, which significantly improves the traditional named entity recognition model, and does not require a large amount of annotation to train a named entity recognition model component with superior performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a schematic diagram of the named entity sequence model of the first embodiment of the present application.

[0032] Figure 2 It is a flowchart of the named entity recognition method of the first embodiment of the present application.

[0033] Figure 3 It is a flowchart of the named entity recognition method of the second embodiment of the present application.

[0034] Figure 4It is a flowchart of the sub-steps of step S101 of the second embodiment of the present application.

[0035] Figure 5 It is a block diagram of a named entity recognition device according to the third embodiment of the present application.

[0036] Figure 6 It is a block diagram of a named entity recognition device according to the third embodiment of the present application.

[0037] Figure 7 A block diagram schematically shows a terminal device for executing the method according to the present invention; and

[0038] Figure 8 A storage unit for holding or carrying a program code for implementing the method according to the present invention is schematically shown. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0040] First embodiment

[0041] Named Entity Recognition (NER) can be widely used in various fields such as name recognition, place name recognition, address recognition, organization name recognition, proper nouns, new retail, tourism, etc. It refers to identifying entities with specific meanings in text. For example, the named entity recognition model component can identify entities from various texts such as "Xihu District, Hangzhou City, Zhejiang Province", "I love science", "The earth revolves around the sun", etc. for subsequent use.

[0042] Figure 1 FIG. 1 is a schematic diagram of a named entity recognition model component according to an embodiment of the present invention. Figure 1 As shown, the dependency tree-based named entity recognition model component 30 mainly includes two models, one is a dependency parser 10, and the other is a named entity recognition model 20 for labeling. The dependency parser 10 can be a Biaffine dependency parser for performing dependency parsing; the named entity recognition model 20 can be a traditional LSTM-CRF neural network model, such as a dependency-oriented LSTM-CRF neural network model for performing sequence labeling.

[0043] like Figure 1As shown, in the model application process represented by the solid line with arrows, the input text 40 is parsed by the dependency parser 10 to generate a target dependency tree structure, which is used to characterize the relationship between the words in the input text; after passing through the dependency parser 10, a target dependency tree structure among multiple candidate dependency tree structures for the input text 40 can be obtained, for example, the best dependency tree structure. The input text and the target dependency tree structure corresponding to the input text are input into the named entity recognition model to generate a named entity recognition sequence. Further, Figure 1 In the model training process shown by the dotted line with an arrow, the input text 40 is input into the dependency parser and the named entity recognition model respectively, and both can be trained at the same time.

[0044] A first embodiment of the present invention provides a method for named entity recognition. Figure 2 FIG. 1 is a flowchart of the steps of the named entity recognition method according to the first embodiment of the present invention. Figure 2 As shown, the named entity recognition method of the embodiment of the present invention is used to perform named entity recognition on an input text containing at least two words. The input text is, for example, a sentence, which includes multiple words (words), where "words" can be English words, Chinese characters, etc., and the present invention is not particularly limited.

[0045] The named entity recognition method proposed in the first embodiment of the present invention may include the following steps:

[0046] S101, parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text;

[0047] S102, obtaining a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0048] S103, inputting the input information including the target dependency tree into the named entity recognition model to obtain a named entity sequence corresponding to the input text.

[0049] Step S101, parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text;

[0050] In this step, for example, after the user inputs text into the machine learning model, the execution entity (such as a server) can obtain multiple dependency tree structures corresponding to the input text based on the user's input text through a parsing step. These dependency tree structures can be latent dependency trees.

[0051] The named entity recognition model component proposed in the embodiment of the present invention may include a dependency parser. The dependency parser is, for example, a latent Biaffine dependency tree parser. The dependency parser can parse the user's input text into a dependency tree. There may be multiple dependency trees parsed by the dependency parser, and the best dependency tree will be obtained from the multiple dependency trees later by, for example, structured inference. The multiple dependency trees initially parsed may be called candidate dependency trees, and the dependency tree finally selected may be called the target dependency tree.

[0052] In step S101, the execution subject may first obtain word embedding information and contextualized representation information corresponding to the input text, and obtain word representation information corresponding to the input text according to the word embedding information and the contextualized representation information.

[0053] Word embedding is the process of converting text data that cannot be directly processed by existing machine learning methods into numerical data. If a word is regarded as the smallest unit of input text, word embedding can be understood as a mapping process, which maps or embeds a word in the text space into another numerical vector space through a certain method.

[0054] The input of word embedding is a set of non-overlapping words in the original text. These words can be placed in a dictionary, and this dictionary can be regarded as an input of word embedding. The output of word embedding is the vector representation of each word. For the original input above, assuming the simplest one hot encoding method is used, each word corresponds to a numerical representation. For example, for the text sentence "I love science", the vector corresponding to I is [1,0,0], and the vector corresponding to science is [0,0,1]. Various machine learning applications can build their own models based on the numerical representation of this word.

[0055] The contextualized representation information is obtained through a deep bidirectional LSTM model to obtain word representation based on contextual meaning. It is a correction to the word embedding information and dynamically updates the word embedding information based on the context.

[0056] Word embedding is to represent words as vectors (word2vec), which is essentially a static model. After each word is trained using word2vec, the representation of the word is fixed, and when used later, the word embedding information will not change with the context scenario. In an embodiment of the present invention, a corresponding context component processor, such as ELMo (Deepcontextualized word representation), can be used to solve the context problem. ELMo is a new type of deep contextualized word representation that can model complex features of words (such as syntax and semantics) and changes in words in the language context (i.e., modeling polysemous words). The specific approach is to use a language model to learn the word embedding information of a good word on a large corpus in advance, and then use the training data with the labels removed to fine-tune the pre-trained ELMO model, and the word embedding information of the word can be dynamically updated when in use.

[0057] By using the input text, through the corresponding word2vec and contextual representation processor (such as ELMo), we can obtain the word embedding information and contextualized representation information corresponding to the input text, that is, the vectorized word representation information related to the contextual meaning.

[0058] In this step, the word representation information may be input into a dependency parser to obtain a plurality of candidate dependency trees, each of which corresponds to a dependency marginal score.

[0059] Since the dependency tree is potential, in this step, the dependency parser needs to enumerate all possible dependency trees, and the probability of these dependency trees being the target dependency tree (e.g., the best dependency tree) varies, some with higher probabilities and some with lower probabilities. After enumerating a large number of dependency trees, the best dependency tree (target dependency tree) can be found in the subsequent inference process.

[0060] The main purpose of inference is to estimate the value of hidden variables when the value of observed variables is known. For example, if you observe the "leaves" of the generative model, you can try to infer the hidden causes (diagnosis). Conversely, if you observe the "roots" of the generative model, you can try to predict its nodes (predicition). Structural inference is the operation of inferring the potential structure based on the observable parts.

[0061] In this embodiment, since the number of dependency item trees is exponentially large, the target dependency tree can be obtained by performing a structured inference method. In the scheme proposed in the embodiment of the present invention, the sum of the marginalized implicit structures on the dependency arc can be estimated, so as to determine the target dependency tree by scoring the sum of the implicit structures.

[0062] Since there are a huge number of potential candidate dependency trees, an approximate estimation method can be used in the embodiments of the present invention. For example, the probability distribution of the parent item of each word can be calculated, and then the approximate estimation result of the probability distribution can be used to enumerate all dependency structures.

[0063] The aforementioned word representation information is sent to the dependency parser, and the dependency marginal score can be obtained in the following way: For example, taking the input text "I love science" as an example, the word representation information of this input text is passed through the dependency parser, and the following information can be obtained: The probability that the parent of the word I is love is 0.8, and the probability that the parent is science is 0.2. The probability that the parent of the word science is love is 0.9, and the probability that it is I is 0.1. After the dependency parser, the different probabilities of different parents of each word can be obtained.

[0064] The specific method of estimating the sum of the marginalized implicit structures on the dependency arc can be performed, for example, in the following way: since the distribution probability of the parent of each word has been obtained, the distribution probability of each word can be multiplied and merged with the representation information of the word in this step, and the sum of the distribution probabilities of the word can be calculated. For example, in the sentence "I love science" mentioned above, if the parent of the word I is determined to be love, it is directly merged with the representation information of love. However, the probability that its parent is love is 0.8, and the probability that its parent is the parent of science is 0.2. Here, it can be obtained in the form of probability summation, for example, through: weighted sum: 0.8*representation information of love + 0.2*representation information of science, and then merged with I to obtain the dependency marginal score.

[0065] Through the above-mentioned method, different candidate dependency trees can be obtained, and each candidate dependency tree corresponds to a dependency margin score, which is related to the probability that the candidate dependency tree is the target dependency tree. In one embodiment, if the score is high, it can be considered that the probability of it being the target dependency tree is high, and vice versa.

[0066] After executing step S101, step S102 may be executed as follows:

[0067] S102, obtaining a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0068] In this step, since the number of the aforementioned potential dependency trees is very large, in this embodiment, a structured inference method can be used to obtain a target dependency tree.

[0069] Since the sum of the marginalized implicit structures on the dependency arcs has been obtained in the previous step, the best dependency tree can be selected in this step by the sum of the implicit structures. For example, the dependency tree with the highest probability of the sum of the implicit structures calculated in the previous step is the best dependency tree. The best dependency tree structure is used as the target dependency tree structure corresponding to the input text.

[0070] The structured inference method can be implemented, for example, in the following manner: scoring the representation information of every two words. For example, a sentence has three words, and the representation information of the first word and the second word is scored, and the representation information of the second and third words is scored. After the softmax function is applied, the probabilities of the first word and the second word, and the probabilities of the second word and the third word can be obtained. By selecting the word combination with the highest probability for reverse deduction, an approximate best dependency tree can be obtained, which is the target dependency tree structure corresponding to the input text.

[0071] The method of obtaining the target dependency tree structure corresponding to the input text by structured inference is only one embodiment of the present invention. In other embodiments, latent variable inference can also be performed by SPIGOT or direct estimator. For example, the SPIGOT program can be used to directly obtain the dependency tree structure from the input sentence itself, and then the existing dependency-guided LSTM-CRF work can be directly used through the dependency tree structure.

[0072] After executing step S102, step S103 may be executed as follows:

[0073] S103: Input the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text.

[0074] In this step, the target dependency tree can be used as the input of the named entity recognition model to obtain the named entity sequence of the input text. The target dependency tree structure can be used as part of the input of the aforementioned LSTM-CRF neural network model, and the input text is passed through the target dependency tree structure and the LSTM-CRF neural network model to obtain the named entity sequence of the text. The target dependency tree structure is, for example, the best dependency tree structure for the input text. After the target dependency tree structure is determined in the aforementioned steps, the dependency relationship between the words of the input text is determined. This dependency relationship can be represented by tree structure data, for example. For example, in the input text "I love science", the aforementioned steps parse love as the root node root, the relationship between love to I is a subject relationship, and the relationship between love to science is an object relationship. Then the relationship between the three words (for example, the subject is I and the object is science) and the attributes (for example, love as the root node root) can be recorded, and the data representing the tree relationship can be used as part of the named entity model and input into the named entity model together with the input text.

[0075] This step uses the sentence dependency tree structure to optimize the traditional named entity recognition model. It is worth noting that the target dependency tree structure and the LSTM-CRF model in the embodiment of the present invention are not in a traditional step-by-step superposition relationship, but the target dependency tree structure determined by inference is used as the input of the named entity recognition model, so that the named entity recognition model containing the target dependency tree structure is fully differentiable without breakpoints. Since the optimized named entity recognition model is fully differentiable, the named entity recognition model training failure and low precision will not occur during training due to cascade error transmission, and a high-precision named entity recognition model based on the target dependency tree structure can be trained using limited training data.

[0076] Optionally, before step S102, a step of training a dependency parser using a universal dependency tree in a dependency tree library may be included, as follows:

[0077] S100, training the dependency parser using universal dependency tree data.

[0078] This step may be performed before or after the aforementioned step S101, and may provide general dependency tree data to train a better potential dependency parser.

[0079] The data in the Universal Dependency Tree Database can provide consistent annotations of grammar, such as part-of-speech tags and dependencies across many different languages. For example, in English grammar, nn is used to represent nouns and vb is used to represent verbs. In Spanish, n and c are used to represent them. The Universal Dependency Tree Database will mark the parts of speech and dependency relationships of different languages ​​in a unified way. The Universal Dependency Tree Database has included more and more languages ​​in this tree bank.

[0080] Therefore, although it is impossible to obtain named annotation data for texts in less annotated fields, the general dependency tree library contains a large amount of (sentence text, dependency tree) data. In other words, these data are data that characterize the relationship between sentences and dependency trees. In the solution proposed in the embodiment of the present invention, these (sentence text, dependency tree) data can be used to train a dependency parser, such as the aforementioned potential Biaffine dependency tree parser, so that it can feedback better parsing dependency trees.

[0081] In addition, as a preferred embodiment, before step S102, that is, before the step of obtaining the target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures, the method further includes:

[0082] S1001, using general dependency tree data and named entity recognition data, simultaneously training the dependency parser and the named entity recognition model in a multi-task learning environment.

[0083] The meaning of multi-tasking here is: train the dependency tree parsing module and the named entity recognition model at the same time; during the training process, some data with only dependencies can be used to train the dependency tree parsing module; these data with only dependencies do not participate in the training of the named entity recognition model, and other named entity recognition data, such as historical data, can participate at the same time Figure 1 The dependency parser 10 and the named entity recognition model 20 shown are involved in the training of the entire named entity recognition model component 30 .

[0084] In one embodiment, the step S100, i.e., the step of training the dependency parser using universal dependency tree data, may include: inputting the universal dependency tree data and the loss data of the dependency parser into the dependency parser to train the dependency parser.

[0085] For example, after training the dependency tree parsing module, the loss data generated by the training will be obtained, that is, the error between the training result and the actual result. In this step, the loss can be incorporated into the dependency tree parsing module training, that is, the general dependency tree data and the aforementioned loss data are used as a dimension of the dependency tree parsing module, and the dependency parser module is continuously trained in a loop.

[0086] The dependency tree parsing module trained by the combined loss method can obtain the desired accuracy more quickly, thereby improving the training efficiency. Since the named entity recognition model component in the solution proposed in the embodiment of the present invention is a continuously differentiable model, it has good transferability and will not produce cascade errors.

[0087] It can be seen from the above that the named entity recognition method proposed in the first embodiment of the present invention has at least the following technical effects:

[0088] The solution proposed in the embodiment of the present invention is to train the named entity recognition model proposed in the present invention through the dependency tree data in the general dependency tree library, use the dependency tree structure as a potential variable, train a named entity recognition model component based on the dependency tree structure, and then input the input text into the named entity recognition model component to obtain a named entity recognition annotation sequence. In low-resource named entity annotation, the named entity recognition annotation of the present invention is highly accurate, which significantly improves the traditional named entity recognition model, and does not require a large amount of annotation to train a model with superior performance.

[0089] Second embodiment

[0090] A second embodiment of the present invention provides a method for named entity recognition. Figure 3 FIG. 2 is a flowchart of the steps of the named entity recognition method according to the second embodiment of the present invention. Figure 3 As shown, the named entity recognition method of the embodiment of the present invention comprises the following steps:

[0091] S201, parsing the input text using a dependency parser to obtain a plurality of candidate dependency tree structures corresponding to the input text;

[0092] S202, obtaining a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0093] S203: Input the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model component to obtain a named entity sequence corresponding to the input text.

[0094] The above steps S202 to S203 are the same or similar to the steps S102 to S103 of the previous embodiment, and are not described in detail here. This embodiment focuses on the differences from the previous embodiment.

[0095] In step S201, the input text is parsed using a dependency parser, such as a Biaffine dependency tree parser, which is used to parse the text and generate multiple possible dependency trees.

[0096] Since the dependency tree is potential, in this step, the existing Biaffine dependency tree parser can be used to enumerate all possible dependency trees to find the best dependency tree in the subsequent inference process, which is the target dependency tree. Since the number of dependency trees is exponentially large, it is necessary to find the correct method to perform structured inference. In the scheme proposed in the embodiment of the present invention, the sum of the marginalized implicit structures on the dependency arc can be estimated. For example, the probability distribution of each word can be calculated for its parent. The parent of the word may be the first word or the second word. These two possible probabilities are multiplied and summed with the representation information of the word respectively. The most likely tree structure can be determined by the final score, so as to reversely obtain the best dependency tree and determine the target dependency tree.

[0097] Optionally, before step S202, a step of training a dependency parser using the general dependency tree data in the dependency tree library may be included as follows:

[0098] S200, training a dependency parser using general dependency tree data.

[0099] This step may be performed before or after step S201 and may provide general dependency tree data to train a better potential dependency parser.

[0100] The data in the Universal Dependency Tree Database can provide consistent annotations of grammar, such as part-of-speech tags and dependencies across many different languages. For example, in English grammar, nn is used to represent nouns and vb is used to represent verbs. In Spanish, n and c are used to represent them. The Universal Dependency Tree Database will mark the parts of speech and dependency relationships of different languages ​​in a unified way. The Universal Dependency Tree Database has included more and more languages ​​in this tree bank.

[0101] Therefore, although it is impossible to obtain named annotation data for texts in less annotated fields, the general dependency tree library contains a large amount of (sentence text, dependency tree) data. In other words, these data are data that characterize the relationship between sentences and dependency trees. In the solution proposed in the embodiment of the present invention, these (sentence text, dependency tree) data can be used to train a dependency parser, such as the aforementioned potential Biaffine dependency tree parser, so that it can feedback better parsing dependency trees.

[0102] Optionally, step S201 may include the following sub-steps:

[0103] S201a, obtaining word embedding information and context representation information corresponding to the input text; S201b, obtaining word representation information corresponding to the input text according to the word embedding information and the context representation information; S201c, inputting the word representation information into a dependency parser to obtain multiple candidate dependency trees;

[0104] The multiple candidate dependency trees correspond to dependency marginal scores respectively.

[0105] The execution subject may first obtain word embedding information and contextualized representation information corresponding to the input text, and obtain word representation information corresponding to the input text according to the word embedding information and the contextualized representation information.

[0106] Word embedding is the process of converting text data that cannot be directly processed by existing machine learning methods into numerical data. If a word is regarded as the smallest unit of input text, word embedding can be understood as a mapping process, which maps or embeds a word in the text space into another numerical vector space through a certain method.

[0107] The input of word embedding is a set of non-overlapping words in the original text. These words can be placed in a dictionary, and this dictionary can be regarded as an input of word embedding. The output of word embedding is the vector representation of each word. For the original input above, assuming the simplest one hot encoding method is used, each word corresponds to a numerical representation. For example, for the text sentence "I love science", the vector corresponding to I is [1,0,0], and the vector corresponding to science is [0,0,1]. Various machine learning applications can build their own models based on the numerical representation of this word.

[0108] The contextualized representation information is obtained through a deep bidirectional LSTM model to obtain word representation based on contextual meaning. It is a correction to the word embedding information and dynamically updates the word embedding information based on the context.

[0109] Word embedding is to represent words as vectors (word2vec), which is essentially a static model. After each word is trained using word2vec, the representation of the word is fixed, and when used later, the word embedding information will not change with the context scenario. In an embodiment of the present invention, a corresponding context component processor, such as ELMo (Deepcontextualized word representation), can be used to solve the context problem. ELMo is a new type of deep contextualized word representation that can model complex features of words (such as syntax and semantics) and changes in words in the language context (i.e., modeling polysemous words). The specific approach is to use a language model to learn the word embedding information of a good word on a large corpus in advance, and then use the training data with the labels removed to fine-tune the pre-trained ELMO model, and the word embedding information of the word can be dynamically updated when in use.

[0110] By using the input text, through the corresponding word2vec and contextual representation processor (such as ELMo), we can obtain the word embedding information and contextualized representation information corresponding to the input text, that is, the vectorized word representation information related to the contextual meaning.

[0111] In this step, the word representation information may be input into a dependency parser to obtain a plurality of candidate dependency trees, each of which corresponds to a dependency marginal score.

[0112] Since the dependency tree is potential, in this step, the dependency parser needs to enumerate all possible dependency trees, and the probability of these dependency trees being the target dependency tree (e.g., the best dependency tree) varies, some with higher probabilities and some with lower probabilities. After enumerating a large number of dependency trees, the best dependency tree (target dependency tree) can be found in the subsequent inference process.

[0113] The main purpose of inference is to estimate the value of hidden variables when the value of observed variables is known. For example, if you observe the "leaves" of the generative model, you can try to infer the hidden causes (diagnosis). Conversely, if you observe the "roots" of the generative model, you can try to predict its nodes (predicition). Structural inference is the operation of inferring the potential structure based on the observable parts.

[0114] In this embodiment, since the number of dependency item trees is exponentially large, the target dependency tree can be obtained by performing a structured inference method. In the scheme proposed in the embodiment of the present invention, the sum of the marginalized implicit structures on the dependency arc can be estimated, so as to determine the target dependency tree by scoring the sum of the implicit structures.

[0115] Since there are a huge number of potential candidate dependency trees, an approximate estimation method can be used in the embodiments of the present invention. For example, the probability distribution of the parent item of each word can be calculated, and then the approximate estimation result of the probability distribution can be used to enumerate all dependency structures.

[0116] The aforementioned word representation information is sent to the dependency parser, and the dependency marginal score can be obtained in the following way: For example, taking the input text "I love science" as an example, the word representation information of this input text is passed through the dependency parser, and the following information can be obtained: The probability that the parent of the word I is love is 0.8, and the probability that the parent is science is 0.2. The probability that the parent of the word science is love is 0.9, and the probability that it is I is 0.1. After the dependency parser, the different probabilities of different parents of each word can be obtained.

[0117] The specific method of estimating the sum of the marginalized implicit structures on the dependency arc can be performed, for example, in the following way: since the distribution probability of the parent of each word has been obtained, the distribution probability of each word can be multiplied and merged with the representation information of the word in this step, and the sum of the distribution probabilities of the word can be calculated. For example, in the sentence "I love science" mentioned above, if the parent of the word I is determined to be love, it is directly merged with the representation information of love. However, the probability that its parent is love is 0.8, and the probability that its parent is the parent of science is 0.2. Here, it can be obtained in the form of probability summation, for example, through: weighted sum: 0.8*representation information of love + 0.2*representation information of science, and then merged with I to obtain the dependency marginal score.

[0118] Through the above-mentioned method, different candidate dependency trees can be obtained, and each candidate dependency tree corresponds to a dependency margin score, which is related to the probability that the candidate dependency tree is the target dependency tree. In one embodiment, if the score is high, it can be considered that the probability of it being the target dependency tree is high, and vice versa.

[0119] In addition, as a preferred embodiment, before step S202, i.e., the step of obtaining the target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures, the step of training the dependency parser by using the general dependency tree data in the dependency tree library and the named entity recognition data may be included as follows:

[0120] The dependency parser and the named entity recognition model included in the named entity recognition component are trained simultaneously in a multi-task learning environment using the general dependency tree data and the named entity recognition data.

[0121] The meaning of multi-tasking here is: train the dependency tree parsing module and the named entity recognition model at the same time; during the training process, some data with only dependencies can be used to train the dependency tree parsing module; these data with only dependencies do not participate in the training of the named entity recognition model, and other named entity recognition data, such as historical data, can participate at the same time Figure 1 The dependency parser 10 and the named entity recognition model 20 shown are involved in the training of the entire named entity recognition model component 30 .

[0122] In one embodiment, the step S200, i.e., the step of training the dependency parser using universal dependency tree data, may include: inputting the universal dependency tree data and the loss data of the dependency parser into the dependency parser to train the dependency parser.

[0123] For example, after training the dependency tree parsing module, the loss data generated by the training will be obtained, that is, the error between the training result and the actual result. In this step, the loss can be incorporated into the dependency tree parsing module training, that is, the general dependency tree data and the aforementioned loss data are used as a dimension of the dependency tree parsing module, and the dependency parser module is continuously trained in a loop.

[0124] The dependency tree parsing module trained by the combined loss method can obtain the desired accuracy more quickly, thereby improving the training efficiency. Since the named entity recognition model component in the solution proposed in the embodiment of the present invention is a continuously differentiable model, it has good transferability and will not produce cascade errors.

[0125] In summary, the named entity recognition method proposed in the second embodiment of the present invention has at least the following technical effects:

[0126] The solution proposed in the embodiment of the present invention is to train the named entity recognition model proposed in the present invention through the dependency tree data in the general dependency tree library, use the dependency tree structure as a potential variable, train a named entity recognition model component based on the dependency tree structure, and then input the input text into the named entity recognition model component to obtain a named entity recognition annotation sequence. In low-resource named entity annotation, the named entity recognition annotation of the present invention is highly accurate, which significantly improves the traditional named entity recognition model, and does not require a large amount of annotation to train a model with superior performance.

[0127] In an optional embodiment, the named entity recognition method proposed in this embodiment has at least the following advantages:

[0128] 1. The embodiment of the present invention proposes a latent variable model to understand the latent dependency structure in the named entity recognition (NER) task. The embodiment of the present invention can further perform multi-task learning to merge dependency data from a general dependency tree library. The model of the present invention is differentiable, which means there are no breakpoints throughout the process. Compared with the existing step-by-step model, the model of the present invention does not pass the error of the previous step model to the next step, thereby improving the accuracy of the model.

[0129] 2. Compared with the standard LSTM-CRF model, the embodiments of the present invention have been extensively experimented on multiple languages ​​and achieved excellent performance. The analysis of the embodiments of the present invention shows that during multi-task learning, a small amount of dependency data can significantly improve the model, and the learned latent dependencies can partially mimic the golden dependency structure.

[0130] 3. The method of the embodiment of the present invention can be further extended to large-scale experiments with more training data, which may lead to better potential dependencies.

[0131] Third embodiment

[0132] The third embodiment of the present invention provides a named entity recognition device, such as Figure 5 As shown, the device comprises:

[0133] A parsing module 301 is used to parse the input text and obtain multiple candidate dependency tree structures corresponding to the input text;

[0134] A target dependency tree acquisition module 302 is used to obtain a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0135] The named entity sequence acquisition module 303 is used to input the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text.

[0136] In summary, the named entity recognition device proposed in this embodiment has at least the following advantages:

[0137] The solution proposed in the embodiment of the present invention is to train the named entity recognition model proposed in the present invention through the dependency tree data in the general dependency tree library, use the dependency tree as a potential variable, train a named entity recognition model component based on the dependency tree, and then input the input text into the named entity recognition model component to obtain a named entity recognition annotation sequence. In low-resource named entity annotation, the named entity recognition annotation of the present invention is highly accurate, which significantly improves the traditional named entity recognition model, and does not require a large amount of annotation to train a model with superior performance.

[0138] Fourth embodiment

[0139] A fourth embodiment of the present invention provides a named entity recognition device 500, such as Figure 6 As shown, the device comprises:

[0140] A parsing module 401 is used to parse the input text and obtain multiple candidate dependency tree structures corresponding to the input text;

[0141] A target dependency tree acquisition module 402 is used to obtain a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures;

[0142] The named entity sequence acquisition module 403 is used to input the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text.

[0143] In some optional embodiments, the parsing module 401 includes:

[0144] A word vector acquisition module 401a is used to obtain word embedding information and context representation information corresponding to the input text;

[0145] A word representation information acquisition module 401b, configured to obtain word representation information corresponding to an input text according to the word embedding information and the context representation information;

[0146] The candidate dependency tree acquisition module 401c is used to input the word representation information into the dependency relationship parser to obtain dependency marginal scores corresponding to multiple dependency trees.

[0147] In some optional embodiments, the word representation information is input into a dependency parser to obtain dependency marginal scores corresponding to a plurality of dependency trees, and the dependency parser includes a potential Biaffine dependency tree parser.

[0148] In some optional embodiments, the target dependency tree acquisition module 402 is used to obtain a target dependency tree structure corresponding to the input text according to the dependency marginal scores respectively corresponding to the multiple candidate dependency trees.

[0149] In some optional embodiments, the named entity recognition model is a dependency-oriented LSTM-CRF model, and the named entity sequence acquisition module 403 is used to:

[0150] The target dependency tree is superimposed on the LSTM-CRF model, and the input text is passed through the LSTM-CRF model after the target dependency tree is superimposed to obtain a named entity sequence of the text.

[0151] In some optional embodiments, the device further comprises:

[0152] The training module 400 is used to train the dependency parser using the universal dependency tree data. Alternatively, the training module 400 is used to simultaneously train the dependency parser and the named entity recognition model included in the named entity recognition component in a multi-task learning environment using the universal dependency tree data and the named entity recognition data.

[0153] In some optional embodiments, the general dependency tree data includes correspondence data between text and dependency tree structure.

[0154] In some optional embodiments, the training module 400 is further used to: input the general dependency tree data and the loss data of the dependency parser into the dependency parser to train the dependency parser.

[0155] In summary, the named entity recognition device proposed in the fourth embodiment of the present invention has at least the following technical effects:

[0156] The solution proposed in the embodiment of the present invention is to train the named entity recognition model proposed in the present invention through the dependency tree data in the general dependency tree library, use the dependency tree as a potential variable, train a named entity recognition model component based on the dependency tree, and then input the input text into the named entity recognition model component to obtain a named entity recognition annotation sequence. In low-resource named entity annotation, the named entity recognition annotation of the present invention is highly accurate, which significantly improves the traditional named entity recognition model, and does not require a large amount of annotation to train a model with superior performance.

[0157] In an optional embodiment, the named entity recognition device proposed in this embodiment has at least the following advantages:

[0158] 1. The embodiment of the present invention proposes a latent variable model to understand the latent dependency structure in the named entity recognition (NER) task. The embodiment of the present invention can further perform multi-task learning to merge dependency data from a general dependency tree library. The model of the present invention is differentiable, which means there are no breakpoints throughout the process. Compared with the existing step-by-step model, the model of the present invention does not pass the error of the previous step model to the next step, thereby improving the accuracy of the model.

[0159] 2. Compared with the standard LSTM-CRF model, the embodiments of the present invention have been extensively experimented on multiple languages ​​and achieved excellent performance. The analysis of the embodiments of the present invention shows that during multi-task learning, a small amount of dependency data can significantly improve the model, and the learned latent dependencies can partially mimic the golden dependency structure.

[0160] 3. The method of the embodiment of the present invention can be further extended to large-scale experiments with more training data, which may lead to better potential dependencies.

[0161] Figure 7 This is a schematic diagram of the hardware structure of a terminal device provided in one embodiment of the present application. Figure 7 As shown, the terminal device may include an input device 90, a processor 91, an output device 92, a memory 93 and at least one communication bus 94. The communication bus 94 is used to realize the communication connection between the components. The memory 93 may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory. Various programs may be stored in the memory 93 to complete various processing functions and implement the method steps of this embodiment.

[0162] Optionally, the processor 91 may be implemented as a central processing unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and the processor 91 is coupled to the input device 90 and the output device 92 via a wired or wireless connection.

[0163] Optionally, the input device 90 may include multiple input devices, for example, it may include at least one of a user interface for users, a device interface for devices, a programmable interface for software, a camera, and a sensor. Optionally, the device interface for devices may be a wired interface for data transmission between devices, or a hardware insertion interface for data transmission between devices (such as a USB interface, a serial port, etc.); Optionally, the user interface for users may be, for example, a control button for users, a voice input device for receiving voice input, and a touch sensing device for users to receive user touch input (such as a touch screen with a touch sensing function, a touchpad, etc.); Optionally, the programmable interface for the software may be, for example, an entry for users to edit or modify programs, such as an input pin interface or an input interface of a chip; Optionally, the transceiver may be a radio frequency transceiver chip with communication function, a baseband processing chip, and a transceiver antenna, etc. Audio input devices such as microphones may receive voice data. The output device 92 may include output devices such as displays and speakers.

[0164] In this embodiment, the processor of the terminal device includes functions for executing each module of the data processing device in each device. The specific functions and technical effects can be referred to the above embodiments and will not be repeated here.

[0165] Figure 8 A schematic diagram of the hardware structure of a terminal device provided in another embodiment of the present application. Figure 8 Yes Figure 7 A specific embodiment in the implementation process. Figure 8 As shown, the terminal device of this embodiment includes a processor 101 and a memory 102 .

[0166] The processor 101 executes the computer program code stored in the memory 102 to implement the above-mentioned embodiment. Figures 2 to 4 Named entity recognition method.

[0167] The memory 102 is configured to store various types of data to support operations on the terminal device. Examples of such data include instructions for any application or method used to operate on the terminal device, such as messages, pictures, videos, etc. The memory 102 may include a random access memory (RAM) and may also include a non-volatile memory, such as at least one disk storage.

[0168] Optionally, the processor 101 is provided in the processing component 100. The terminal device may further include: a communication component 103, a power component 104, a multimedia component 105, an audio component 106, an input / output interface 107 and / or a sensor component 108. The specific components included in the terminal device are set according to actual needs, and this embodiment does not limit this.

[0169] The processing component 100 generally controls the overall operation of the terminal device. The processing component 100 may include one or more processors 101 to execute instructions to complete the above Figures 2 to 4 In addition, the processing component 100 may include one or more modules to facilitate the interaction between the processing component 100 and other components. For example, the processing component 100 may include a multimedia module to facilitate the interaction between the multimedia component 105 and the processing component 100.

[0170] The power supply component 104 provides power to various components of the terminal device. The power supply component 104 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device.

[0171] The multimedia component 105 includes a display screen that provides an output interface between the terminal device and the user. In some embodiments, the display screen may include a liquid crystal display (LCD) and a touch panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0172] The audio component 106 is configured to output and / or input audio signals. For example, the audio component 106 includes a microphone (MIC), and when the terminal device is in an operating mode, such as a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 102 or sent via the communication component 103. In some embodiments, the audio component 106 also includes a speaker for outputting audio signals.

[0173] The input / output interface 107 provides an interface between the processing component 100 and a peripheral interface module, which may be a click wheel, a button, etc. These buttons may include but are not limited to: a volume button, a start button, and a lock button.

[0174] The sensor component 108 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor component 108 can detect the open / closed state of the terminal device, the relative positioning of the components, and the presence or absence of contact between the user and the terminal device. The sensor component 108 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor component 108 may also include a camera, etc.

[0175] The communication component 103 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot for inserting a SIM card, so that the terminal device can log in to the GPRS network and establish communication with the server through the Internet.

[0176] From the above, we can see that Figure 8 The communication component 103, the audio component 106, the input / output interface 107, and the sensor component 108 involved in the embodiment can all be used as Figure 7 Implementation method of the input device in the embodiment.

[0177] An embodiment of the present application provides a terminal device, including: one or more processors; and one or more machine-readable media having instructions stored thereon, which, when executed by the one or more processors, enable the terminal device to execute the method for generating a video summary as described in one or more of the embodiments of the present application.

[0178] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0179] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0180] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0181] The above is a detailed introduction to a named entity recognition method and a named entity recognition device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for named entity recognition, for performing named entity recognition on an input text containing at least two words, characterized in that: The method comprises: Parsing the input text to obtain multiple candidate dependency tree structures corresponding to the input text; The target dependency tree structure corresponding to the input text is obtained by inferring from the obtained multiple candidate dependency tree structures; the target dependency tree is obtained by executing a structured inference method; the structured inference is an operation of inferring a potential structure based on an observed part; the structured inference is implemented by scoring the representation information of every two words, selecting the word combination with the highest probability for reverse inference, and obtaining an approximate estimated best dependency tree as the target dependency tree structure; Inputting the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text; The step of obtaining the target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures includes: According to the dependency marginal scores corresponding to the multiple candidate dependency trees, the target dependency tree structure corresponding to the input text is obtained; each candidate dependency tree corresponds to a dependency marginal score, and the dependency marginal score is related to the probability that the candidate dependency tree is the target dependency tree.

2. The method for named entity recognition according to claim 1, characterized in that: The step of parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text comprises: Obtain word embedding information and context representation information corresponding to the input text; Obtain word representation information corresponding to the input text according to the word embedding information and the context representation information; Inputting the word representation information into a dependency parser to obtain multiple candidate dependency trees; The multiple candidate dependency trees correspond to dependency marginal scores respectively.

3. The method for named entity recognition according to claim 2, characterized in that: In the step of inputting the word representation information into a dependency parser to obtain a plurality of candidate dependency trees, the dependency parser includes a potential Biaffine dependency tree parser.

4. The method for named entity recognition according to claim 1, characterized in that: The step of parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text is performed by a dependency parser. Before the step of obtaining the target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures, the method further includes: Training a dependency parser using general dependency tree data.

5. The method for named entity recognition according to claim 1, characterized in that: The step of parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text is performed by a dependency parser. Before the step of obtaining the target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures, the method further includes: Using the general dependency tree data and the named entity recognition data, the dependency parser and the named entity recognition model are trained simultaneously in a multi-task learning environment.

6. The method for named entity recognition according to claim 4 or 5, characterized in that: The general dependency tree data includes correspondence data between text and dependency tree structure.

7. The method for named entity recognition according to claim 6, characterized in that: The step of training the dependency parser using universal dependency tree data comprises: The general dependency tree data and the loss data of the dependency parser are input into the dependency parser to train the dependency parser.

8. A named entity recognition device, characterized in that: include: A parsing module, used to parse an input text and obtain a plurality of candidate dependency tree structures corresponding to the input text; A target dependency tree acquisition module, used to obtain a target dependency tree structure corresponding to the input text by inference from the obtained multiple candidate dependency tree structures; The target dependency tree is obtained by performing a structured inference method; the structured inference is an operation of inferring a potential structure based on an observed part; the structured inference is implemented in the following manner: scoring the representation information of every two words, selecting the word combination with the highest probability for reverse inference, and obtaining an approximate estimated best dependency tree as the target dependency tree structure; A named entity sequence acquisition module, used to input the input text and the target dependency tree structure corresponding to the input text into a named entity recognition model to obtain a named entity sequence corresponding to the input text; The target dependency tree acquisition module is used to obtain the target dependency tree structure corresponding to the input text according to the dependency marginal scores corresponding to the multiple candidate dependency trees respectively; Each candidate dependency tree corresponds to a dependency margin score, and the dependency margin score is related to the probability that the candidate dependency tree is the target dependency tree.

9. The named entity recognition device according to claim 8, characterized in that: The parsing module comprises: A word vector acquisition module, used to obtain word embedding information and context representation information corresponding to the input text; A word representation information acquisition module, used to obtain word representation information corresponding to an input text according to the word embedding information and the context representation information; A candidate dependency tree acquisition module, used for inputting the word representation information into a dependency parser to obtain multiple candidate dependency trees; The multiple candidate dependency trees respectively correspond to dependency marginal scores.

10. The named entity recognition device according to claim 9, characterized in that: The word representation information is input into a dependency parser to obtain multiple candidate dependency trees, and the dependency parser includes a potential Biaffine dependency tree parser.

11. The named entity recognition device according to claim 8, characterized in that: The operation of parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text is performed by a dependency relation parser. The device also includes: A training module is used to train the dependency parser using general dependency tree data.

12. The named entity recognition device according to claim 8, characterized in that: The operation of parsing the input text to obtain a plurality of candidate dependency tree structures corresponding to the input text is performed by a dependency relation parser. The device also includes: A training module is used to simultaneously train the dependency parser and the named entity recognition model in a multi-task learning environment using the general dependency tree data and the named entity recognition data.

13. The named entity recognition device according to claim 11 or 12, characterized in that: The general dependency tree data includes correspondence data between text and dependency tree structure.

14. The named entity recognition device according to claim 13, characterized in that: The training module is used to: input the general dependency tree data and the loss data of the dependency parser into the dependency parser to train the dependency parser.

15. A terminal device, characterized in that: include: one or more processors; and One or more machine-readable media having instructions stored thereon, when executed by the one or more processors, cause the terminal device to perform the method as claimed in any one of claims 1 to 7.

16. A machine-readable medium having instructions stored thereon, which, when executed by one or more processors, causes a terminal device to execute the method according to any one of claims 1 to 7.

Images