A method, apparatus, device, and storage medium for model training
By initially labeling samples with the prediction model and manually labeling samples that are prone to misidentification, and dividing them into positive and negative sample sets, the prediction model is trained, which solves the problem of high training cost and improves the training effect of the model.
Patent Information
- Application Number
- CN202310117049.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-14
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-02-14
AI Technical Summary
In existing technologies, the training cost of prediction models is high, mainly because a large number of sample texts need to be labeled.
The sample set is initially labeled by the prediction model, and samples that are easy to misidentify are handed over to labelers for manual labeling, and divided into positive sample set and negative sample set. The prediction model is trained using these sample sets with the goal of minimizing the deviation between the labeling results and the actual results.
This reduces the number of manually labeled samples, lowers training costs, and improves the training performance of the prediction model.
Smart Images

Figure CN116029298B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to the field of natural language processing technology, and in particular to a method, apparatus, device, and storage medium for model training. Background Technology
[0002] Currently, in natural language processing technology, semantic relationships between entities contained in text are usually extracted for tasks such as machine translation, sentiment analysis, and question answering systems. For example, the semantic relationship between entities in the text "Paris is a city in France" contains two entities, "Paris" and "France", and the semantic relationship between these two entities is an inclusion relationship.
[0003] In existing technologies, predictive models are typically used to predict whether semantic relationships exist between entities in text. Different tasks can then be performed based on the prediction results. However, training these predictive models requires pre-labeling a large amount of sample text, which significantly increases the training cost of the predictive models.
[0004] Therefore, how to reduce the training cost of prediction models is an urgent problem to be solved. Summary of the Invention
[0005] This specification provides a method, apparatus, device, and storage medium for model training, in order to partially solve the aforementioned problems existing in the prior art.
[0006] The following technical solution is adopted in this specification:
[0007] This specification provides a method for model training, the method comprising:
[0008] Obtain the sets of samples to be labeled;
[0009] For each set of samples to be labeled, each sample to be labeled contained in the set is input into the prediction model, and the labeling result for the set of samples to be labeled is obtained based on the output of the prediction model. The labeling result is used to indicate whether the set of samples to be labeled contains samples with entity relationships.
[0010] If the labeling result is determined to be the specified labeling result, then each sample to be labeled in the sample set to be labeled is sent to the device used by the labeler, and the actual labeling result for each sample to be labeled in the sample set to be labeled is determined according to the return result returned by the device. The actual labeling result is used to indicate whether there is an entity relationship between the entities contained in the sample to be labeled.
[0011] Based on the actual labeling results, the sample set to be labeled is divided into a positive sample set and a negative sample set. The positive sample set contains samples with entity relationships, and the negative sample set contains samples without entity relationships.
[0012] Each sample in the positive sample set is input into the prediction model, and a labeling result for the positive sample set is obtained based on the output of the prediction model. Similarly, each sample in the negative sample set is input into the prediction model, and a labeling result for the negative sample set is obtained based on the output of the prediction model.
[0013] The prediction model is trained with the training objective of minimizing the deviation between the labeling results of the prediction model for the positive sample set and the labeling results for the negative sample set, and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
[0014] Optionally, obtain each set of samples to be labeled, specifically including:
[0015] Get each untagged text;
[0016] Based on the entities contained in each unlabeled text, the unlabeled texts are divided into sets of unlabeled texts, and at least a portion of the unlabeled texts are selected from each set of unlabeled texts as sets of samples to be labeled.
[0017] Optionally, at least a portion of the unlabeled text sets are selected from each of the unlabeled text sets, specifically including:
[0018] For each unlabeled text set, the contribution of the unlabeled text set is determined by a specified sampling method. The specified sampling method includes at least one of the following: sampling method based on Fisher information matrix, sampling method based on uncertainty, and sampling method based on diversity. The contribution is used to indicate the degree of improvement in the performance of the prediction model after labeling the unlabeled sample set and using it to train the prediction model.
[0019] Based on the contribution of each unlabeled text set, at least a portion of the unlabeled text sets are selected from each unlabeled text set.
[0020] Optionally, before inputting each unlabeled sample in each unlabeled sample set into the prediction model and obtaining the labeling result for that unlabeled sample set based on the output of the prediction model, the method further includes:
[0021] Obtain the labeled sample set;
[0022] Each sample in the labeled sample set is input into the prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in the sample for each sample in the labeled sample set, and determines the uncertainty value corresponding to the sample and the labeling result of the sample based on the determined probability value of whether there is an entity relationship between the entities contained in the sample.
[0023] Based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample, the labeling result of the prediction model for the labeled sample set is obtained;
[0024] The prediction model is initially trained with the training objective of minimizing the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set according to the output of the prediction model, and the deviation between the labeling result for the labeled sample set and the actual labeling result of the labeled sample set.
[0025] Optionally, before sending each sample in the set of samples to be labeled to the device used by the labeler, the method further includes:
[0026] Based on the output of the prediction model, determine the probability that each sample in the set of samples to be labeled is a positive sample, and determine the confidence level corresponding to each sample based on the probability that each sample is a positive sample.
[0027] Based on the confidence level, each sample to be labeled is sorted to obtain a sorted set of samples to be labeled;
[0028] Each sample in the set of samples to be labeled is sent to the device used by the labeler, specifically including:
[0029] Each sample to be labeled in the sorted sample set is sent to the device used by the labeler.
[0030] Optionally, based on the returned result from the device, the actual labeling result for each sample in the set of samples to be labeled is determined, specifically including:
[0031] For each sample to be labeled in the set of samples to be labeled, the initial labeling result for each labeler is determined based on the return result received from the device.
[0032] Based on the preset weight of each labeler, the initial labeling results of each labeler for the sample to be labeled are merged to determine the actual labeling result for the sample to be labeled.
[0033] This specification provides a method for predicting entity relationships, the method comprising:
[0034] Get the text set to be labeled;
[0035] The text set to be labeled is input into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled, and obtains the labeling result of the text set to be labeled and the labeling result of each text in the text set to be labeled based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled. The prediction model is trained by the above model training method.
[0036] The labeling results of the text set to be labeled, and the labeling results of each text in the text set to be labeled, are sent to the downstream nodes to execute the corresponding tasks.
[0037] This specification provides a model training apparatus, comprising:
[0038] The acquisition module is used to acquire each set of samples to be labeled;
[0039] The first labeling module is used to input each sample to be labeled contained in each sample set to be labeled into the prediction model, and obtain the labeling result for the sample set to be labeled based on the output of the prediction model. The labeling result is used to indicate whether the sample set to be labeled contains samples with entity relationships.
[0040] The second marking module is used to send each sample to be marked in the sample set to be marked to the device used by the marking personnel if the marking result is determined to be a specified marking result, and to determine the actual marking result for each sample to be marked in the sample set to be marked based on the return result returned by the device. The actual marking result is used to indicate whether there is an entity relationship between the entities contained in the sample to be marked.
[0041] The splitting module is used to divide the sample set to be labeled according to the actual labeling result to obtain a positive sample set and a negative sample set, wherein the positive sample set contains samples with entity relationships and the negative sample set contains samples without entity relationships.
[0042] The prediction module is used to input each sample contained in the positive sample set into the prediction model and obtain the labeling result for the positive sample set based on the output result of the prediction model; and to input each sample contained in the negative sample set into the prediction model and obtain the labeling result for the negative sample set based on the output result of the prediction model.
[0043] The training module is used to train the prediction model with the training objective of minimizing the deviation between the labeling results of the prediction model output for the positive sample set and the labeling results for the negative sample set and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
[0044] Optionally, the acquisition module is specifically used to: acquire each unlabeled text; divide each unlabeled text into sets of unlabeled texts according to the entities contained in each unlabeled text; and select at least a portion of the unlabeled text sets from each set of unlabeled texts as each set of samples to be labeled.
[0045] Optionally, the acquisition module is specifically used to determine the contribution of each unlabeled text set by using a specified sampling method. The specified sampling method includes at least one of the following: a sampling method based on the Fisher information matrix, a sampling method based on uncertainty, and a sampling method based on diversity. The contribution is used to represent the degree of performance improvement of the prediction model after labeling the unlabeled sample set for training the prediction model. Based on the contribution of each unlabeled text set, at least a portion of the unlabeled text sets are selected from each unlabeled text set.
[0046] Optionally, the training module is further configured to: acquire a labeled sample set; input each sample in the labeled sample set into a prediction model, so that the prediction model determines, for each sample in the labeled sample set, the probability value of whether there is an entity relationship between the entities contained in the sample, and determines the uncertainty value corresponding to the sample and the labeling result of the sample based on the determined probability value of whether there is an entity relationship between the entities contained in the sample; obtain the labeling result of the prediction model for the labeled sample set based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample; and perform initial training on the prediction model with the training objective of minimizing the deviation between the labeling result for the labeled sample set obtained based on the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set output by the prediction model and the actual labeling result of the labeled sample set.
[0047] Optionally, the first labeling module is further configured to: determine the probability that each sample to be labeled in the sample set to be labeled is a positive sample based on the output of the prediction model; determine the confidence level corresponding to each sample to be labeled based on the probability that each sample to be labeled is a positive sample; and sort the samples to be labeled according to the confidence level to obtain a sorted sample set to be labeled.
[0048] The second labeling module is specifically used to send each sample to be labeled in the sorted sample set to the device used by the labeler.
[0049] Optionally, the second labeling module is specifically used to: for each sample to be labeled in the set of samples to be labeled, determine the initial labeling result of each labeler for the sample to be labeled based on the return result returned by the device; and merge the initial labeling results of each labeler for the sample to be labeled according to the preset weight of each labeler to determine the actual labeling result for the sample to be labeled.
[0050] This specification provides an entity relationship prediction device, including:
[0051] The text acquisition module is used to acquire the text set to be labeled;
[0052] The text tagging module is used to input the text set to be tagged into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged, and obtains the tagging result of the text set to be tagged and the tagging result of each text in the text set to be tagged based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged. The prediction model is trained by the above model training method.
[0053] The task execution module is used to send the marking results of the text set to be marked, as well as the marking results of each text in the text set to be marked, to downstream nodes to execute the corresponding tasks.
[0054] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for training the model.
[0055] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for training the aforementioned model.
[0056] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:
[0057] The model training method provided in this specification first obtains each set of samples to be labeled. For each set, the samples to be labeled are input into a prediction model. Based on the output of the prediction model, a labeling result is obtained for that set. This labeling result indicates whether the set contains samples with entity relationships. If the labeling result is determined to be the specified result, each sample in the set is sent to the device used by the labeler. Based on the returned result from the device, the actual labeling result for each sample in the set is determined. This actual labeling result indicates whether entity relationships exist between the entities contained in the sample. Based on the actual labeling result, the set is then further processed. The system partitions the data into positive and negative sample sets. The positive sample set contains samples with entity relationships, while the negative sample set contains samples without entity relationships. Each sample in the positive sample set is input into the prediction model, and the model's output is used to obtain the labeling results for the positive and negative sample sets. The training objective is to minimize the deviation between the predicted labeling results for the positive and negative sample sets and the actual labeling results for both sets. The prediction model is then trained to label the received text to be labeled, obtaining the labeled text, and then sending the labeled text to downstream nodes to perform corresponding tasks.
[0058] As can be seen from the above method, each set of samples to be labeled can be labeled first through the prediction model, and the samples to be labeled in each set of samples to be labeled with the specified labeling result can be screened out and handed over to labelers for manual labeling. Here, the specified labeling result refers to the labeling result of a class of samples that the prediction model is prone to misidentification in the pre-set. Thus, based on the labeling result of the labelers for the class of samples that the prediction model is prone to misidentification, the set of samples to be labeled used to train the prediction model can be split and re-aggregated to obtain a new set of samples. The prediction model can then be trained using the new set of samples. In this way, the training effect of the prediction model can be improved while reducing the number of samples that need to be manually labeled to reduce the training cost of the prediction model. Attached Figure Description
[0059] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:
[0060] Figure 1 This is a flowchart illustrating a model training method provided in this specification.
[0061] Figure 2 This is a flowchart illustrating an entity relationship prediction method provided in this specification.
[0062] Figure 3 A schematic diagram of a model training apparatus provided in this specification;
[0063] Figure 4 This is a schematic diagram of an entity relationship prediction device provided in this specification;
[0064] Figure 5 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation
[0065] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0066] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0067] Figure 1 This is a flowchart illustrating a model training method provided in this specification, including the following steps:
[0068] S101: Obtain the sample sets to be labeled.
[0069] In this specification, the business platform can obtain each unlabeled text, divide each unlabeled text into sets based on the entities contained in each unlabeled text, and select at least a portion of the unlabeled text sets from each set as each set of samples to be labeled.
[0070] Among them, the business platform can divide several unlabeled texts containing the same entity into an unlabeled text set for each unlabeled text.
[0071] For example, suppose the texts are "Paris is in France", "Paris is France's capital", and "Paris is France in miniature". The entities contained in these three texts are "Paris" and "France". Therefore, these three unlabeled texts can be grouped into one text set.
[0072] In addition, the business platform can also divide several unlabeled texts containing similar entities into a set of unlabeled texts according to actual needs.
[0073] For example, in the texts “Paris is in France” and “Cannes is in France”, the entities contained in these three texts are “Paris”, “Cannes”, and “France”. Among them, “Paris” and “Cannes” are both cities in France. Therefore, the two unlabeled texts containing similar entities can be grouped into one text set.
[0074] Furthermore, after obtaining each unlabeled text set, the contribution of each unlabeled text set can be determined by specifying a sampling method. The specified sampling method includes at least one of the following: sampling method based on Fisher information matrix, sampling method based on uncertainty, and sampling method based on diversity. The contribution is used to indicate the degree to which labeling the unlabeled sample set improves the performance of the prediction model when used to train the prediction model.
[0075] To illustrate the above in detail, the following section uses the Fisher information matrix sampling method as an example to explain the method for determining the contribution of the unlabeled text set. Please refer to the following formula for details:
[0076]
[0077] In the above formula, contribution(x) is the contribution of the unlabeled text set, Tr(*) is the trace of the Fisher information matrix, and x is the unlabeled text set.
[0078] As can be seen from the above formula, the information content of each unlabeled text set when it is labeled as a positive sample set and a negative sample set can be determined by using the trace of the Fisher information matrix, and the contribution of each unlabeled text set can be determined based on the information content of each unlabeled text set when it is labeled as a positive sample set and a negative sample set.
[0079] The labeling results of the unlabeled text set include: a positive sample set and a negative sample set. The positive sample set is used to indicate that the unlabeled text set contains samples with entity relationships, and the negative sample set is used to indicate that the unlabeled text set does not contain samples with entity relationships. Here, entity relationships refer to semantic relationships between entities.
[0080] In this specification, the execution entity used to implement the model training method can refer to a designated device set up on the business platform, such as a server, or a device such as a laptop or desktop computer. For ease of description, the following description will only use a server as the execution entity to illustrate the model training method provided in this specification.
[0081] S102: For each set of samples to be labeled, each sample to be labeled contained in the set is input into the prediction model, and a labeling result for the set of samples to be labeled is obtained based on the output of the prediction model. The labeling result is used to indicate whether the set of samples to be labeled contains samples with entity relationships.
[0082] After obtaining the sample set to be labeled, the server can input each sample in the sample set into the prediction model and obtain the labeling result for the sample set based on the output of the prediction model. The labeling result includes two types: positive sample set and negative sample set, which are used to indicate whether the sample set contains samples with entity relationships.
[0083] It should be noted that before the server inputs each unlabeled sample contained in the unlabeled sample set into the prediction model, it also needs to perform initial training on the prediction model, so that each unlabeled sample contained in the unlabeled sample set can be input into the prediction model after initial training.
[0084] The method for the server to initially train the prediction model can be as follows: A labeled sample set is obtained, and each sample in the labeled sample set is input into the prediction model. For each sample in the labeled sample set, the prediction model determines the probability value of whether there is an entity relationship between the entities contained in that sample. Based on the determined probability value, the uncertainty value and labeling result of that sample are determined. Based on the labeling result and the uncertainty value of each sample in the labeled sample set, the labeling result output by the prediction model for the labeled sample set is obtained. The training objective is to minimize the deviation between the labeling result obtained from the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set output by the prediction model and the actual labeling result of the labeled sample set. This process is used to initially train the prediction model.
[0085] In the above content, the method by which the server obtains the labeling result of the prediction model for the labeled sample set based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample can be to obtain the labeling result of the prediction model for the labeled sample set based on the labeling result of each sample in the labeled sample set whose uncertainty value is higher than a preset threshold.
[0086] S103: If the marking result is determined to be the specified marking result, then each sample to be marked in the sample set to be marked is sent to the device used by the marking personnel, and the actual marking result for each sample to be marked in the sample set to be marked is determined according to the return result returned by the device. The actual marking result here includes two types: positive sample and negative sample. A positive marking result indicates that there is an entity relationship between the entities contained in the sample to be marked, and a negative marking result indicates that there is no entity relationship between the entities contained in the sample to be marked.
[0087] Furthermore, the server can determine whether the labeling result of the prediction model for the sample set to be labeled is a positive sample set. If so, it can send each sample to be labeled in the sample set to be labeled to the device used by the labeler, and determine the actual labeling result for each sample to be labeled in the sample set to be labeled based on the return result returned by the device.
[0088] Specifically, if the server determines that the labeling result of the prediction model for the sample set to be labeled is a positive sample set, it can determine the probability that each sample in the sample set to be labeled is a positive sample based on the output of the prediction model, determine the confidence level corresponding to each sample to be labeled based on the probability that each sample to be labeled is a positive sample, and sort the samples to be labeled based on the confidence level corresponding to each sample to be labeled to obtain a sorted sample set to be labeled. Then, each sample to be labeled in the sorted sample set to be labeled can be sent to the device used by the labeler to obtain the initial labeling result of the labeler for each sample to be labeled in the sorted sample set to be labeled.
[0089] It should be noted that by having labelers manually label each sample in the sorted set of samples to be labeled, the labeling efficiency of labelers can be increased, thereby reducing the training cost of the prediction model.
[0090] It is worth noting that when the server manually labels each sample in the sample set to be labeled by the labelers, each labeler can simultaneously manually label each sample in the sample set to be labeled, thereby obtaining the initial labeling results of each labeler for each sample in the sample set to be labeled.
[0091] For each sample to be labeled, the initial labeling results of each labeler for that sample can be merged according to the preset weights of each labeler to determine the actual labeling result for that sample. The specific formula is as follows:
[0092]
[0093] In the above formula, l ij For the initial labeling result of labeler j for sample i to be labeled, c k For positive or negative samples, w is the number of labeled personnel, and r is the number of labeled personnel. j I(l) represents the weight of the person being labeled. ij =c k The parentheses are indicator functions; the result is 1 if the condition in the parentheses is met, and 0 otherwise. The result of manual labeling for sample i to be labeled.
[0094] It should be noted that the server can adjust the weight of each labeler based on the deviation between the manual labeling result of each sample to be labeled and the labeler's initial labeling result.
[0095] In addition, when the server manually labels each sample in the sample set, it can sort the labelers according to their weight from smallest to largest. Then, the sample set can be sent to the device of the first labeler in the sorted list to obtain the initial labeling results of the first labeler for each sample in the sample set, as well as the labeling reason text of the first labeler for at least some of the samples. The obtained initial labeling results and labeling reason text of the first labeler for each sample in the sample set can then be sent to the device of the second labeler in the sorted list, and so on, until the initial labeling results of the last labeler for each sample in the sample set are obtained. The initial labeling results of the last labeler are then used as the manual labeling results of the samples.
[0096] It should be noted that the weight of the taggers in the above content can also be determined based on the relevant information of the taggers obtained. This relevant information may include at least one of the tagger's ability level, reputation, and domain matching degree.
[0097] The method by which the server determines the weight of a marker based on their relevant information can be referenced by the following formula:
[0098]
[0099] In the above formula, r j To assign weights to personnel j, Indicates the ability level of the marked personnel. This indicates the credibility of the person being tagged. This indicates the degree of matching between the labeler and the domain of the labeled sample.
[0100] S104: Based on the actual labeling results, the sample set to be labeled is divided into a positive sample set and a negative sample set, wherein the positive sample set contains samples with entity relationships and the negative sample set contains samples without entity relationships.
[0101] Furthermore, after obtaining the actual labeling result of each sample in the sample set to be labeled, the server can divide the sample set to be labeled into a positive sample set and a negative sample set based on the actual labeling result of each sample in the sample set to be labeled. The positive sample set contains positive samples with entity relationships, and the negative sample set contains negative samples without entity relationships.
[0102] S105: Input each sample contained in the positive sample set into the prediction model, and obtain the labeling result for the positive sample set according to the output result of the prediction model; input each sample contained in the negative sample set into the prediction model, and obtain the labeling result for the negative sample set according to the output result of the prediction model.
[0103] S106: The prediction model is trained with the training objective of minimizing the deviation between the labeling results of the prediction model output for the positive sample set and the labeling results of the negative sample set and the actual labeling results of the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
[0104] After obtaining each positive sample set and each negative sample set, the server can input each sample contained in each positive or negative sample set into the prediction model, and obtain the labeling result for that positive or negative sample set based on the output of the prediction model.
[0105] Furthermore, the server can train the prediction model with the training objective of minimizing the deviation between the labeling results of the prediction model output for the positive sample set and the labeling results for the negative sample set and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
[0106] The downstream nodes mentioned above can be models such as machine translation, question answering systems, knowledge graphs, and sentiment analysis, or they can refer to databases. The corresponding tasks can be, for example, machine translation based on tagged text, or saving the tagged text to a database.
[0107] In practical applications, when training the prediction model, it may be necessary to repeat the training several times using the model training method described above. After each training, the server can determine whether the termination condition is met. If so, the final trained prediction model is obtained.
[0108] The termination condition mentioned above can be at least one of the following: the performance of the trained prediction model no longer improves, the performance of the prediction model has reached the expected set value, or the cost of training the prediction model has reached the budget limit.
[0109] As can be seen from the above, the server can first label each set of samples to be labeled using a prediction model, and then filter out the samples in each set whose labeling results are specified, and hand them over to labelers for manual labeling. Here, the specified labeling results refer to the labeling results of a class of samples that the prediction model is prone to misidentifying, as pre-set. Therefore, based on the labeling results of the labelers for this class of samples that the prediction model is prone to misidentifying, the set of samples to be labeled used to train the prediction model can be split and re-aggregated to obtain a new set of samples. This new set of samples can then be used to train the prediction model, thereby reducing the number of samples requiring manual labeling, thus lowering the training cost of the prediction model, while improving the training effect of the prediction model.
[0110] To provide a detailed explanation of the above, this specification also provides a method for predicting entity relationships in text using the prediction model trained by the above method, as detailed below. Figure 2 As shown.
[0111] Figure 2 This is a flowchart illustrating an entity relationship prediction method provided in this specification, including the following steps:
[0112] S201: Get the text set to be labeled.
[0113] S202: Input the text set to be labeled into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled, and obtains the labeling result of the text set to be labeled and the labeling result of each text in the text set to be labeled based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled. The prediction model is trained by the above model training method.
[0114] S203: Send the labeling results of the text set to be labeled, and the labeling results of each text in the text set to be labeled, to the downstream node to execute the corresponding task.
[0115] A server with a prediction model can obtain a set of texts to be labeled, and then input the set of texts to be labeled into a pre-trained prediction model. The prediction model can then determine the probability value of whether there is an entity relationship between the entities contained in each text in the set of texts to be labeled, and obtain the labeling result of the set of texts to be labeled and the labeling result of each text in the set of texts to be labeled based on the probability value of whether there is an entity relationship between the entities contained in each text in the set of texts to be labeled. The labeling result of the set of texts to be labeled and the labeling result of each text in the set of texts to be labeled are then sent to downstream nodes to execute the corresponding tasks.
[0116] The prediction model is trained using the aforementioned prediction model training method.
[0117] The tasks mentioned above can include machine translation, question answering systems, knowledge graphs, and sentiment analysis.
[0118] The above describes a model training method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides corresponding model training apparatus and entity relationship prediction apparatus, such as... Figure 3 , Figure 4 As shown.
[0119] Figure 3 A schematic diagram of a model training apparatus provided in this specification includes:
[0120] The acquisition module 301 is used to acquire each set of samples to be labeled;
[0121] The first labeling module 302 is used to input each sample to be labeled contained in the sample set to be labeled into the prediction model for each sample set to be labeled, and obtain the labeling result for the sample set to be labeled according to the output result of the prediction model. The labeling result is used to indicate whether the sample set to be labeled contains samples with entity relationships.
[0122] The second marking module 303 is used to send each sample to be marked in the sample set to be marked to the device used by the marking personnel if the marking result is determined to be a specified marking result, and to determine the actual marking result for each sample to be marked in the sample set to be marked based on the return result returned by the device. The actual marking result is used to indicate whether there is an entity relationship between the entities contained in the sample to be marked.
[0123] The splitting module 304 is used to divide the sample set to be labeled according to the actual labeling result to obtain a positive sample set and a negative sample set, wherein the positive sample set contains samples with entity relationships and the negative sample set contains samples without entity relationships.
[0124] The prediction module 305 is used to input each sample contained in the positive sample set into the prediction model and obtain the labeling result for the positive sample set according to the output result of the prediction model; and to input each sample contained in the negative sample set into the prediction model and obtain the labeling result for the negative sample set according to the output result of the prediction model.
[0125] The training module 306 is used to train the prediction model with the training objective of minimizing the deviation between the labeling results of the prediction model output for the positive sample set and the labeling results for the negative sample set and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
[0126] Optionally, the acquisition module 301 is specifically used to acquire each unlabeled text; divide each unlabeled text into sets of unlabeled texts according to the entities contained in each unlabeled text, and select at least a portion of the unlabeled text sets from each set of unlabeled texts as each set of samples to be labeled.
[0127] Optionally, the acquisition module 301 is specifically used to determine the contribution of each unlabeled text set by a specified sampling method, wherein the specified sampling method includes at least one of the following: a sampling method based on the Fisher information matrix, a sampling method based on uncertainty, and a sampling method based on diversity. The contribution is used to represent the degree of improvement in the performance of the prediction model after labeling the unlabeled sample set for training the prediction model. Based on the contribution of each unlabeled text set, at least a portion of the unlabeled text sets are selected from each unlabeled text set.
[0128] Optionally, the training module 306 is further configured to: acquire a labeled sample set; input each sample in the labeled sample set into a prediction model, so that the prediction model determines, for each sample in the labeled sample set, the probability value of whether there is an entity relationship between the entities contained in the sample, and determines the uncertainty value corresponding to the sample and the labeling result of the sample based on the determined probability value of whether there is an entity relationship between the entities contained in the sample; obtain the labeling result of the prediction model for the labeled sample set based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample; and perform initial training on the prediction model with the training objective of minimizing the deviation between the labeling result for the labeled sample set obtained based on the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set output by the prediction model and the actual labeling result of the labeled sample set.
[0129] Optionally, the first labeling module 302 is further configured to: determine the probability that each sample to be labeled in the sample set to be labeled is a positive sample based on the output of the prediction model; determine the confidence level corresponding to each sample to be labeled based on the probability that each sample to be labeled is a positive sample; and sort the samples to be labeled according to the confidence level to obtain a sorted sample set to be labeled.
[0130] The second labeling module 303 is specifically used to send each sample to be labeled in the sorted sample set to the device used by the labeler.
[0131] Optionally, the second labeling module 303 is specifically used to: for each sample to be labeled in the sample set to be labeled, determine the initial labeling result of each labeler for the sample to be labeled based on the return result returned by the device; and merge the initial labeling results of each labeler for the sample to be labeled according to the preset weight of each labeler to determine the actual labeling result for the sample to be labeled.
[0132] Figure 4 A schematic diagram of a model training apparatus provided in this specification includes:
[0133] Text acquisition module 401 is used to acquire the text set to be labeled;
[0134] The text tagging module 402 is used to input the text set to be tagged into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged, and obtains the tagging result of the text set to be tagged and the tagging result of each text in the text set to be tagged based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged. The prediction model is trained by the above model training method.
[0135] The task execution module 403 is used to send the marking results of the text set to be marked, and the marking results of each text in the text set to be marked, to the downstream node to execute the corresponding task.
[0136] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 One method provided.
[0137] This instruction manual also provides Figure 5 The one shown corresponds to Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 5 As shown, at the hardware level, this electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above. Figure 1 The method described.
[0138] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.
[0139] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.
[0140] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0141] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.
[0142] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.
[0143] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0144] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0147] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0148] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0149] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0150] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0151] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0152] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0153] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0154] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.
Claims
1. A method for model training, the method comprising: The method includes: Obtain each unlabeled text; divide each unlabeled text into sets based on the entities contained in each unlabeled text, and determine each set of samples to be labeled from each set of unlabeled texts; For each set of samples to be labeled, each sample in the set is input into a prediction model. Based on the output of the prediction model, a labeling result is obtained for the set. A positive labeling result indicates that the set contains samples with entity relationships; a negative labeling result indicates that the set does not contain samples with entity relationships. Entity relationships refer to semantic relationships between entities. If the labeling result is determined to be a positive sample set, then each sample to be labeled in the sample set is sent to the device used by the labeler, and the actual labeling result for each sample to be labeled in the sample set is determined based on the return result returned by the device. The actual labeling result is used to indicate whether there is an entity relationship between the entities contained in the sample to be labeled. Based on the actual labeling results, the sample set to be labeled is divided into a positive sample set and a negative sample set. The positive sample set contains samples with entity relationships, and the negative sample set contains samples without entity relationships. Each sample in the positive sample set is input into the prediction model, and a labeling result for the positive sample set is obtained based on the output of the prediction model. Similarly, each sample in the negative sample set is input into the prediction model, and a labeling result for the negative sample set is obtained based on the output of the prediction model. The prediction model is trained with the training objective of minimizing the deviation between the labeling results of the prediction model for the positive sample set and the labeling results for the negative sample set, and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
2. The method of claim 1, wherein, Determining each set of samples to be labeled from each set of unlabeled texts specifically includes: For each unlabeled text set, the contribution of the unlabeled text set is determined by a specified sampling method. The specified sampling method includes at least one of the following: a sampling method based on the Fisher information matrix, a sampling method based on uncertainty, and a sampling method based on diversity. The contribution is used to indicate the degree of improvement in the performance of the prediction model after labeling the unlabeled sample set and using it to train the prediction model. Based on the contribution of each unlabeled text set, at least a portion of the unlabeled text sets are selected from each unlabeled text set as each unlabeled sample set to be labeled.
3. The method of claim 1, wherein, Before inputting each unlabeled sample set into a prediction model and obtaining the labeling result for that unlabeled sample set based on the output of the prediction model, the method further includes: Obtain the labeled sample set; Each sample in the labeled sample set is input into the prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in the sample for each sample in the labeled sample set, and determines the uncertainty value corresponding to the sample and the labeling result of the sample based on the determined probability value of whether there is an entity relationship between the entities contained in the sample. Based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample, the labeling result of the prediction model for the labeled sample set is obtained; The prediction model is initially trained with the training objective of minimizing the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set according to the output of the prediction model, and the deviation between the labeling result for the labeled sample set and the actual labeling result of the labeled sample set.
4. The method of claim 1, wherein, Before sending each sample in the set of samples to be labeled to the device used by the labeler, the method further includes: Based on the output of the prediction model, determine the probability that each unlabeled sample in the unlabeled sample set belongs to a positive sample, and determine the confidence level corresponding to each unlabeled sample based on the probability that each unlabeled sample belongs to a positive sample. Based on the confidence level, each sample to be labeled is sorted to obtain a sorted set of samples to be labeled; Each sample in the set of samples to be labeled is sent to the device used by the labeler, specifically including: Each sample to be labeled in the sorted sample set is sent to the device used by the labeler.
5. The method as described in claim 1, characterized in that, Based on the returned result from the device, determine the actual labeling result for each sample in the set of samples to be labeled, specifically including: For each sample to be labeled in the set of samples to be labeled, the initial labeling result for each labeler is determined based on the return result received from the device. Based on the preset weight of each labeler, the initial labeling results of each labeler for the sample to be labeled are merged to determine the actual labeling result for the sample to be labeled.
6. A method for predicting entity relationships, characterized in that, The method includes: Get the text set to be labeled; The text set to be labeled is input into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled, and obtains the labeling result of the text set to be labeled and the labeling result of each text in the text set to be labeled based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be labeled. The prediction model is trained by the method described in any one of claims 1 to 5 above. The labeling results of the text set to be labeled, and the labeling results of each text in the text set to be labeled, are sent to the downstream nodes to execute the corresponding tasks.
7. A device for model training, characterized in that, include: The acquisition module is used to acquire each untagged text. Based on the entities contained in each unlabeled text, the unlabeled texts are divided into sets of unlabeled texts, and sets of samples to be labeled are determined from the sets of unlabeled texts. The first labeling module is used to input each sample to be labeled contained in each sample set to be labeled into the prediction model, and obtain the labeling result for the sample set to be labeled based on the output of the prediction model. The labeling result is a positive sample set to indicate that the sample set to be labeled contains samples with entity relationships; the labeling result is a negative sample set to indicate that the sample set to be labeled does not contain samples with entity relationships; the entity relationship refers to the semantic relationship between entities. The second labeling module is used to send each sample to be labeled in the sample set to be labeled to the device used by the labeler if the labeling result is determined to be a positive sample set, and to determine the actual labeling result for each sample to be labeled in the sample set to be labeled based on the return result returned by the device. The actual labeling result is used to indicate whether there is an entity relationship between the entities contained in the sample to be labeled. The splitting module is used to divide the sample set to be labeled according to the actual labeling result to obtain a positive sample set and a negative sample set, wherein the positive sample set contains samples with entity relationships and the negative sample set contains samples without entity relationships. The prediction module is used to input each sample contained in the positive sample set into the prediction model and obtain the labeling result for the positive sample set based on the output result of the prediction model; and to input each sample contained in the negative sample set into the prediction model and obtain the labeling result for the negative sample set based on the output result of the prediction model. The training module is used to train the prediction model with the training objective of minimizing the deviation between the labeling results of the prediction model output for the positive sample set and the labeling results for the negative sample set and the actual labeling results for the positive sample set and the negative sample set. The prediction model is used to label the received text to be labeled, obtain the labeled text, and send the labeled text to the downstream node to perform the corresponding task.
8. The apparatus as claimed in claim 7, characterized in that, The acquisition module is specifically used to determine the contribution of each unlabeled text set by using a specified sampling method. The specified sampling method includes at least one of the following: a sampling method based on the Fisher information matrix, a sampling method based on uncertainty, and a sampling method based on diversity. The contribution is used to represent the degree to which the performance of the prediction model is improved after the unlabeled sample set is labeled and used to train the prediction model. Based on the contribution of each unlabeled text set, at least a portion of the unlabeled text sets are selected from each unlabeled text set as each unlabeled sample set to be labeled.
9. The apparatus as claimed in claim 7, characterized in that, The training module is further configured to: acquire a labeled sample set; input each sample in the labeled sample set into a prediction model, so that the prediction model determines, for each sample in the labeled sample set, the probability value of whether there is an entity relationship between the entities contained in the sample, and determines the uncertainty value corresponding to the sample and the labeling result of the sample based on the determined probability value of whether there is an entity relationship between the entities contained in the sample; and obtain the labeling result of the prediction model for the labeled sample set based on the labeling result of each sample in the labeled sample set and the uncertainty value corresponding to each sample. The prediction model is initially trained with the training objective of minimizing the probability value of whether there is an entity relationship between the entities contained in each sample in the labeled sample set according to the output of the prediction model, and the deviation between the labeling result for the labeled sample set and the actual labeling result of the labeled sample set.
10. The apparatus as claimed in claim 7, characterized in that, The first labeling module is further configured to: determine the probability that each sample to be labeled in the sample set to be labeled is a positive sample based on the output of the prediction model; determine the confidence level corresponding to each sample to be labeled based on the probability that each sample to be labeled is a positive sample; and sort the samples to be labeled according to the confidence level to obtain a sorted sample set to be labeled. The second labeling module is specifically used to send each sample to be labeled in the sorted sample set to the device used by the labeler.
11. The apparatus as claimed in claim 7, characterized in that, The second labeling module is specifically used to determine, for each sample to be labeled in the set of samples to be labeled, the initial labeling result for each labeler for that sample based on the return result received from the device; Based on the preset weight of each labeler, the initial labeling results of each labeler for the sample to be labeled are merged to determine the actual labeling result for the sample to be labeled.
12. An entity relationship prediction device, characterized in that, include: The text acquisition module is used to acquire the text set to be labeled; A text tagging module is used to input the text set to be tagged into a pre-trained prediction model, so that the prediction model determines the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged, and obtains the tagging result of the text set to be tagged and the tagging result of each text in the text set to be tagged based on the probability value of whether there is an entity relationship between the entities contained in each text in the text set to be tagged. The prediction model is trained by the method described in any one of claims 1 to 5 above. The task execution module is used to send the marking results of the text set to be marked, as well as the marking results of each text in the text set to be marked, to downstream nodes to execute the corresponding tasks.
13. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.
14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 6.
